Task scheduling method based on timeline modeling and adaptive task division

Abstract

The invention discloses a task scheduling method based on timeline modeling and self-adaptive task division, relates to the field of distributed task scheduling, and aims to reflect the availability of a memory in real time by constructing an accurate virtual machine memory timeline model, ensure that a task can strictly comply with memory constraints during allocation, and improve the task scheduling efficiency. And meanwhile, the scale of each batch of tasks is maximized, so that the processing efficiency of the virtual machine is improved. Through an intelligent search mechanism, the heterogeneous characteristics can be fully considered in a scheduling decision, and the most suitable virtual machine resources are selected for each user task, so that the overall efficiency of a heterogeneous cluster is played to the maximum extent. According to the method, by introducing a fine-grained timeline modeling technology, a large task or a task facing a timeout risk is allowed to be intelligently decomposed into smaller subtasks, the subtasks can be executed in parallel on a plurality of virtual machines, the response capability of a system to a burst task load is improved, and the system performance is improved. Therefore, the task timeout problem and the resource utilization efficiency are fundamentally improved.

Description

Technical Field

[0001] This invention relates to the field of distributed task scheduling, specifically a task scheduling method based on timeline modeling and adaptive task partitioning. Background Technology

[0002] In the field of distributed task scheduling, traditional scheduling methods mainly include First-Come, First-Served (FCFS) and Shortest Job First (SJF) strategies. FCFS is a direct and easy-to-implement non-preemptive scheduling algorithm. Its core idea is to strictly process tasks according to the order in which they arrive at the system, without considering task priority, length, or specific resource requirements such as memory or processing time. This method is often applied to order-sensitive or batch processing scenarios due to its simplicity and intuitive fairness. However, the main drawback of FCFS is that it can lead to a "badge effect," where a longer-timed task entering the system first and occupying resources may block all subsequent shorter or more urgent tasks for an extended period, significantly reducing the overall system's response speed and resource utilization efficiency. SJF, on the other hand, is a scheduling strategy aimed at achieving high efficiency. Its core idea is to prioritize and execute the task with the shortest expected completion time. This method aims to minimize the average waiting time and average turnaround time of tasks, theoretically providing optimal average response performance. SJF can flexibly adopt preemptive or non-preemptive modes. For example, when a shorter task is expected to arrive, the currently executing long task can be interrupted under certain conditions, thereby improving the response speed of the short task.

[0003] Besides traditional scheduling methods, metaheuristic methods such as Genetic Algorithms (GA) and Particle Swarm Optimization (PSO) are also widely used in distributed task scheduling. Genetic Algorithms are optimization search algorithms that simulate biological evolution and natural selection mechanisms. They gradually approach the optimal solution to a problem by simulating the evolutionary process of a population. In scheduling scenarios, GA typically encodes possible scheduling schemes as "chromosomes" and continuously improves the "fitness" of these schemes by iteratively applying genetic operations such as selection, crossover, and mutation to optimize multiple objectives such as minimizing task completion time, maximizing resource utilization, or balancing system load. Particle Swarm Optimization (PSO) is a computational intelligent optimization technique inspired by the foraging behavior of flocks of birds. In PSO, each "particle" represents a potential solution in the problem space and adjusts its flight trajectory in the search space by tracking its own best position (individual best) and the best position found by the entire swarm (global best). The particles update their velocity and position using simple mathematical formulas, thus iteratively moving in the search space until they find the optimal or near-optimal solution. Metaheuristic methods typically have strong global search capabilities, effectively avoiding getting trapped in local optima. They also exhibit good adaptability and robustness for highly complex, nonlinear multi-objective optimization problems, enabling them to integrate various complex constraints such as task dependencies, heterogeneous resource characteristics, and communication overhead.

[0004] Traditional scheduling methods, such as First-Come, First-Served (FCFS) and Shortest Job First (SJF), have significant drawbacks in complex distributed task scheduling scenarios. While FCFS is straightforward to implement, its non-preemptive mechanism, which strictly processes tasks according to their arrival order, cannot effectively handle task heterogeneity—the vast differences in resource requirements such as virtual machine memory and processing speed among different tasks. This can lead to a severe "protection effect," where a memory-intensive, time-consuming task entering the system first and occupying critical virtual machine resources can block the execution of all subsequent urgent tasks with low memory requirements and short processing times for an extended period, significantly reducing the overall system throughput and response speed. FCFS also completely ignores users' strict constraints on task completion time—the requirement for tasks to complete within a specific time window—which can lead to numerous task timeouts, directly impacting user satisfaction and system service quality. While SJF aims to minimize average waiting time and average turnaround time by prioritizing the execution of tasks with the shortest expected completion time, its core flaw lies in its extremely high requirement for accurate prediction of task execution time. In real-world distributed environments, the actual execution time of tasks is influenced by a complex array of dynamic factors, such as data transmission latency in network communication, dynamic fluctuations in the load within virtual machines, and the unpredictability of external I / O operations. This makes accurately predicting task execution time virtually impossible. Inaccurate predictions can lead to severe biases in scheduling decisions, even causing long-running tasks to remain unexecuted for extended periods due to the continuous arrival of new, shorter tasks, resulting in a phenomenon known as "starvation." Furthermore, both traditional methods lack the ability to flexibly split or dynamically adjust batch sizes for large-scale tasks, and they cannot effectively utilize parallel processing across multiple virtual machines to accelerate task completion. They fail to provide effective solutions, especially when a single virtual machine cannot meet the user-defined time window requirements.

[0005] Metaheuristic methods, such as Genetic Algorithms (GA) and Particle Swarm Optimization (PSO), also face challenges in solving distributed task scheduling problems. While these methods possess powerful global search capabilities, finding near-optimal solutions in complex search spaces and handling multi-objective optimization problems, their computational overhead is a major limiting factor. For large-scale tasks and numerous heterogeneous virtual machines, GA requires evaluating and operating on a large number of scheduling schemes in each iteration, leading to significant computational complexity and long convergence times. Although PSO is relatively simple to implement and typically converges quickly, it is prone to getting trapped in local optima in complex problem spaces and is sensitive to initial parameter settings, affecting the quality of the final solution. In practical applications requiring real-time or near-real-time scheduling decisions, these metaheuristic methods may fail to provide satisfactory scheduling results within a finite timeframe. Furthermore, their performance is highly sensitive to parameter settings; different parameter combinations can lead to drastically different scheduling outcomes, requiring extensive experimentation and domain knowledge for tuning, which increases the difficulty of deployment and maintenance. These algorithms also struggle to directly handle dynamically changing system states and task requests. When new tasks arrive or the virtual machine state changes, the algorithm may need to be rerun, leading to low scheduling efficiency. Furthermore, their solutions are poorly interpretable, making it difficult to intuitively understand why a particular scheduling result occurs, thus increasing the difficulty of analyzing and optimizing scheduling behavior. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the present invention aims to propose a task scheduling method based on timeline modeling and adaptive task partitioning, comprising: Construct timelines for multiple virtual machines. Each virtual machine timeline includes multiple time units, each time unit corresponding to an available memory size. Receive multiple user requests. Each user request includes a target task, the target task's start time (startTime), end time (endTime), and the target task's size (size). On the timeline of each virtual machine, obtain the actual execution interval of each batch of the target task and determine the actual completion time of the target task. Then determine the actual completion time on each virtual machine and take the minimum value among all actual completion times as the first completion time. Determine whether the first completion time exceeds the end time of the target task. If the first completion time does not exceed the end time of the target task, then the virtual machine corresponding to the first completion time is taken as the best virtual machine for the target task, and an optimal scheduling scheme containing the actual execution interval of each batch of the target task is generated. If the first completion time exceeds the end time of the target task, the target task is divided into multiple subtasks according to the target task size according to a preset ratio. For each subtask, the actual execution interval of each batch of the subtask is obtained on the virtual machine timeline, and a multi-virtual machine optimal scheduling scheme containing the actual execution interval of each batch of each subtask of the target task and its corresponding virtual machine is generated.

[0007] Optionally, on the timeline of each virtual machine, obtain the actual execution interval of each batch of the target task, determine the actual completion time of the target task, and then determine the actual completion time on each virtual machine. The minimum value among all actual completion times is then taken as the first completion time, including: For each user request, the target task is divided into multiple batches, with the first batch being the current batch; For the current batch, calculate the earliest start time minStartTime and the minimum batch size minBatchSize. Based on the relationship between the task volume and memory usage of the virtual machine, the minimum memory usage minMemorySize is calculated according to the minimum batch size minBatchSize. Based on the relationship between the task volume and processing time of the virtual machine, the minimum processing time minSovleTime is calculated according to the minimum batch size minBatchSize. Based on the minimum processing time (minSovleTime), minimum memory usage (minMemorySize), and earliest start time (minStartTime), the actual execution interval of the current batch is determined on the timeline of a single virtual machine. The left endpoint of the actual execution interval is taken as the actual start time of the current batch, and the right endpoint of the actual execution interval is taken as the actual end time of the current batch, actualEndTime. The actual batch size (actualBatchSize) is determined based on the total processing time and available memory size of the actual execution interval. Subtract the delayTime sent by the user to the virtual machine from the actualStartTime to obtain the actualSendTime of the batch. Update the earliest send time minSendTime and the completion time lastFinishTime of the previous batch, get the next batch as the current batch, and return to execute: calculate the earliest start time minStartTime of the current batch until the last batch of the target task. The actual end time of the last batch is the actual end time of the target task. Repeat the above steps to calculate the actual completion time of the target task on each virtual machine, and take the minimum value among all actual completion times as the first completion time.

[0008] Optionally, calculate the earliest start time minStartTime for the current batch, specifically using the following formula: minStartTime=max(minSendTime+ delayTime,lastFinishTime)+1; Among them, minSendTime is the earliest sending time. When the current batch is the first batch of the target task, the earliest sending time is the start time of the target task, startTime. delayTime is the delay time that the user sends to the virtual machine, and lastFinishTime is the completion time of the previous batch.

[0009] Optionally, the minimum batch size minBatchSize for the current batch can be calculated using the following formula: minBatchSize=min(leftTaskSize, minAcceptBatchSize); Where leftTaskSize is the difference between the target task size and the batch size after processing, and minAcceptBatchSize represents the pre-set minimum acceptable task size.

[0010] Optionally, based on the minimum processing time (minSovleTime), minimum memory usage (minMemorySize), and earliest start time (minStartTime), determine the actual execution interval of the current batch on the timeline of a single virtual machine, including: On the timeline of a single virtual machine, obtain multiple time intervals with a length of minimum processing time (minSovleTime) and a height not less than minimum memory usage (minMemorySize). Among all time intervals, obtain the time interval with the start time closest to the earliest start time (minStartTime) and use it as the initial time interval. Extend the end time of the initial time interval while ensuring that the available memory size of the extended interval is not less than the minimum memory usage (minMemorySize) to obtain the final time interval. Use the final time interval as the actual execution interval for the current batch.

[0011] Optionally, the actual batch size (actualBatchSize) is determined based on the total processing time and available memory size of the actual execution interval, including: Based on the relationship between the number of tasks and processing time corresponding to the virtual machine, the first batch size is calculated according to the total processing time of the actual execution interval. Based on the relationship between the number of tasks and memory usage corresponding to the virtual machine, the second batch size is calculated according to the available memory size of the actual execution interval. The minimum value between the first batch size and the second batch size is obtained and used as the actual batch size (actualBatchSize).

[0012] Optionally, during the process of obtaining the actual execution range of each batch of subtasks, if the last batch of the current subtask is smaller than the minimum acceptable task size (except for the last subtask), then the last batch is added to the next subtask and processed in the next subtask.

[0013] An electronic device includes: one or more processors, and a memory for storing instructions that, when executed by the one or more processors, cause the one or more processors to perform a task scheduling method based on timeline modeling and adaptive task partitioning.

[0014] A computer-readable storage medium storing executable instructions that, when executed, cause a processor to perform the task scheduling method based on timeline modeling and adaptive task partitioning.

[0015] A computer program product includes a computer program or instructions that, when executed by a processor, implement the aforementioned task scheduling method based on timeline modeling and adaptive task partitioning.

[0016] The beneficial effects of adopting the above technical solution are as follows: This invention constructs a precise virtual machine memory timeline model to reflect memory availability in real time, ensuring that tasks strictly adhere to memory constraints during allocation while maximizing the size of each batch of tasks, thereby improving virtual machine processing efficiency. This fine-grained resource management, combined with dynamic batch size adjustment, allows the system to maximize the amount of tasks completed per unit time while satisfying all constraints, effectively increasing the overall system throughput and meeting the computing power needs of multiple users. Through its intelligent search mechanism, this invention fully considers these heterogeneous characteristics in scheduling decisions, selecting the most suitable virtual machine resources for each user task, thus maximizing the overall performance of the heterogeneous cluster. This flexibility ensures that the scheduling scheme can dynamically adapt to differences in the underlying hardware environment, further improving the system's performance and efficiency in highly complex scenarios. This invention introduces fine-grained timeline modeling technology to precisely manage the future resource availability of virtual machines, allowing large tasks or tasks facing timeout risks to be intelligently decomposed into smaller subtasks, which can be executed in parallel on multiple virtual machines. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a task scheduling method based on timeline modeling and adaptive task partitioning in an embodiment of the present invention. Figure 2 This is a flowchart illustrating another task scheduling method based on timeline modeling and adaptive task partitioning in an embodiment of the present invention. Figure 3 This is a schematic diagram of segment tree interval management in an embodiment of the present invention; Figure 4 This is a schematic diagram of virtual machine timeline modeling in an embodiment of the present invention; Figure 5 This is a schematic diagram of task division in an embodiment of the present invention. Detailed Implementation

[0018] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0019] This invention aims to solve the distributed task scheduling problem in multi-user shared computing resource scenarios. In this context, multiple users submit tasks of varying sizes at different times, while the available servers are heterogeneous, each containing multiple virtual machines (VMs), each with different processing capabilities and memory sizes. This complexity poses a significant challenge to task scheduling. Traditional scheduling methods often struggle to effectively handle asynchronous task startup times, significant differences in performance requirements, and the complexity brought about by server heterogeneity, resulting in low resource utilization and an inability to fully meet user throughput demands. Specifically, the core problem facing the system is how to efficiently allocate user-submitted tasks to appropriate VMs for execution on limited and heterogeneous computing resources. This includes determining the batch size of tasks, selecting the server and VMs to execute the tasks, and determining the task sending time, while also considering communication latency and memory usage. Furthermore, the execution order of tasks on the VMs follows specific queue management rules: requests arriving at the server earlier are placed at the head of the queue, and among requests arriving at the same time, requests with smaller user IDs are placed at the head of the queue, with the queue being processed every millisecond. Each time the queue is processed, the tasks received at the current moment are enqueued, and all tasks in the queue are sorted according to the aforementioned rules. The queue is scanned from beginning to end, and tasks that can be accommodated in the virtual machine are sent to the virtual machine for execution. In this dynamic and constrained environment, the key problem that this invention needs to solve is how to maximize the overall task completion efficiency and quantity while ensuring smooth task execution and not exceeding resource limits, so as to meet the throughput requirements of multiple users as much as possible.

[0020] To address the problems of existing technologies, this invention provides a task scheduling method based on timeline modeling and adaptive task partitioning, aiming to efficiently allocate and execute tasks submitted by multiple users in complex heterogeneous computing environments. The core of this scheme lies in establishing and maintaining a refined timeline model for each virtual machine in the cluster. This model can accurately reflect and predict the memory resource availability of the virtual machine at different points in time. Based on this, the method intelligently searches for the optimal task execution time period and virtual machine, and adaptively adjusts the task batch size and flexibly splits tasks according to the user's task characteristics and the real-time system status to address the challenge of a single task not being able to complete within a specified time. The entire scheduling process emphasizes the real-time requirements of tasks and the effective utilization of resources. Through forward-looking planning and dynamic adjustment, it strives to optimize the overall system's operating efficiency while meeting user throughput requirements, and ensures the legality and stability of the scheduling scheme.

[0021] This invention provides a task scheduling method based on timeline modeling and adaptive task partitioning, combining... Figure 1 and Figure 2 This may include the following steps: Step 1: Build timelines for multiple virtual machines. Each virtual machine timeline includes multiple time units, each time unit corresponds to an available memory size, and receive multiple user requests. Each user request includes a target task, the start time of the target task, the end time of the target task, and the size of the target task. In the specific implementation process, the global timeline of the virtual machine is first initialized. Each virtual machine's timeline is constructed using a segment tree, which efficiently manages resource states within a time interval. Figure 3 This precisely records the available memory of the virtual machine at different points in time. It is agreed that a preset unit of time is used as the smallest granularity of task execution time, and the available memory size within a unit of time is used as the height of the unit time interval, such as... Figure 4 The height of a time interval is the minimum height of each unit of time within the time interval.

[0022] This invention queues user requests within a fixed window. To improve overall scheduling efficiency, received user requests undergo preprocessing and sorting. Each user request contains three attributes: startTime, endTime, and size. startTime represents the task's ready-to-send time (i.e., the start time of the target task), endTime represents the task's deadline (the end time of the target task), and size represents the size of the target task. The specific sorting strategy prioritizes user requests with earlier deadlines, followed by those with earlier ready-to-send times, and finally, user requests based on task size. This sorting method aims to effectively reduce the risk of subsequent task timeouts by completing tasks as early as possible and releasing resources.

[0023] Furthermore, it checks if the queue is empty. If the queue is empty, the process ends. If the queue is not empty, the user request at the head of the queue is retrieved, and the best scheduling scheme is found for the current user request. This is explained in detail in step 2. The best scheme here is not the final scheme, but refers to the actual execution interval of each batch of the target task in step 2.

[0024] Step 2: On the timeline of each virtual machine, obtain the actual execution interval of each batch of the target task and determine the actual completion time of the target task. Then determine the actual completion time on each virtual machine and take the minimum value among all actual completion times as the first completion time. Step 2.1: For each user request, divide the target task into multiple batches, and take the first batch as the current batch; Step 2.2: For the current batch, calculate the earliest start time minStartTime, specifically using the following formula: minStartTime=max(minSendTime+ delayTime,lastFinishTime)+1; Among them, minSendTime is the earliest sending time. When the current batch is the first batch of the target task, the earliest sending time is the start time of the target task, startTime. delayTime is the delay sent by the user to the virtual machine. lastFinishTime is the completion time of the previous batch. When the current batch is the first batch of the target task, the value is -1. Step 2.3: Calculate the minimum batch size (minBatchSize) for the current batch, specifically using the following formula: minBatchSize=min(leftTaskSize, minAcceptBatchSize); Where leftTaskSize is the difference between the target task size and the size of the batch after processing, and minAcceptBatchSize represents the minimum acceptable task size set in advance, which can also be understood as the minimum size of each batch that needs to be sent. Step 2.4: Based on the relationship between the number of tasks and memory usage corresponding to the virtual machine, calculate the minimum memory usage minMemorySize according to the minimum batch size minBatchSize; where the batch size is also the number of tasks.

[0025] Step 2.5: Based on the relationship between the task volume and processing time of the virtual machine, calculate the minimum processing time minSovleTime according to the minimum batch size minBatchSize; Step 2.6: Based on the minimum processing time (minSovleTime), minimum memory usage (minMemorySize), and earliest start time (minStartTime), determine the actual execution interval of the current batch on the timeline of a single virtual machine; Specifically, on the timeline of a single virtual machine, multiple time intervals with a length of minimum processing time (minSovleTime) and a height not less than minimum memory usage (minMemorySize) are obtained. Among all time intervals, the time interval with the start time closest to the earliest start time (minStartTime) is obtained and used as the initial time interval. The end time of the initial time interval is extended, while ensuring that the available memory size of the extended interval is not less than the minimum memory usage (minMemorySize), to obtain the final time interval. The final time interval is used as the actual execution interval for the current batch. Step 2.7: Take the left endpoint of the actual execution interval as the actual start time (actualStartTime) of the current batch, and the right endpoint of the actual execution interval as the actual end time (actualEndTime) of the current batch. Step 2.8: Determine the actual batch size (actualBatchSize) based on the total processing time and available memory size of the actual execution interval. Based on the relationship between the number of tasks and processing time corresponding to the virtual machine, the first batch size is calculated according to the total processing time of the actual execution interval. Based on the relationship between the number of tasks and memory usage corresponding to the virtual machine, the second batch size is calculated according to the available memory size of the actual execution interval. The minimum value between the first batch size and the second batch size is obtained and used as the actual batch size (actualBatchSize). Step 2.9: Subtract the delayTime sent by the user to the virtual machine from the actualStartTime to obtain the actualSendTime of the batch. Step 2.10: Update the earliest send time (minSendTime) and the last batch completion time (lastFinishTime), specifically using the following formula: minSendTime=max(actualSendTime+delayTime,actualEndTime)+1; lastFinishTime=actualEndTime; Get the next batch as the current batch, and return to step 2.2: calculate the earliest start time minStartTime of the current batch until the last batch of the target task. The actual end time of the last batch is the actual end time of the target task. Step 2.11: Repeat the above operation to calculate the actual end time of the target task on each virtual machine, obtain the minimum value among all actual completion times, and take it as the first completion time; Step 3: Determine if the first completion time exceeds the target task's end time (i.e., ...). Figure 2 In the process of determining whether the optimal scheduling scheme has timed out, if the first completion time does not exceed the end time of the target task, then the virtual machine corresponding to the first completion time is taken as the optimal virtual machine for the target task, and an optimal scheduling scheme containing the actual execution interval of each batch of the target task is generated (i.e., Figure 2 (If the first completion time exceeds the target task's end time, then proceed to step 4.) Step 4: Divide the target task into portions according to the target task size using a preset ratio, such as... Figure 5 Multiple subtasks are obtained. For each subtask, the actual execution interval of each batch of the subtask is obtained on the timeline of the virtual machine. Specifically, step 2 is executed to replace the target task with the subtask, so that the actual execution interval of each batch of the subtask can be obtained, and the optimal scheduling scheme of multiple virtual machines containing the actual execution interval of each batch of each subtask of the target task and its corresponding virtual machine is generated. After dividing the system into multiple subtasks, an optimal single-virtual machine scheduling scheme is sought for each subtask using the aforementioned method. During the search for the optimal virtual machine, already used virtual machines are recorded to avoid reusing the same virtual machine within the same partition, thereby indirectly reducing unnecessary task migrations. Simultaneously, the system considers a "tail-drop" strategy: except for the last subtask, if the last batch size for the current subtask does not meet the minimum batch size requirement, the last batch can be reserved for the next subtask. After finding the optimal single-virtual machine scheduling scheme for each subtask, the optimal single-virtual machine scheduling schemes for all subtasks are merged to obtain the optimal multi-virtual machine scheduling scheme for the current task within the current partition. The optimal multi-virtual machine scheduling schemes corresponding to all partition schemes are compared to obtain the optimal multi-virtual machine scheduling scheme for the current task.

[0026] In another implementation, combining Figure 2 The task can be divided into multiple partitioning schemes, and the best partitioning scheme is determined by trying them. Then, it is determined whether the best partitioning scheme times out. If it does not time out, the scheduling scheme corresponding to the best partitioning scheme is adopted; if it times out, the best scheduling scheme is adopted.

[0027] In the process of obtaining the actual execution range of each batch of subtasks, if the last batch of the current subtask is smaller than the minimum acceptable task size (except for the last subtask), then the last batch is added to the next subtask and processed in the next subtask. This embodiment proposes an electronic device, including: one or more processors, and a memory, wherein the memory is used to store instructions, and when the instructions are executed by the one or more processors, the one or more processors execute the task scheduling method based on timeline modeling and adaptive task partitioning.

[0028] The electronic device may be a mobile phone, computer, or tablet computer, etc., and includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements a task scheduling method based on timeline modeling and adaptive task partitioning as described in the embodiments. It is understood that the electronic device may also include input / output (I / O) interfaces and communication components.

[0029] The processor is used to execute all or part of the steps in the task scheduling method based on timeline modeling and adaptive task partitioning as described in the above embodiments. The memory is used to store various types of data, which may include, for example, instructions for any application or method in the electronic device, as well as application-related data.

[0030] The processor can be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic components, and is used to execute the task scheduling method based on timeline modeling and adaptive task partitioning described in the above embodiments.

[0031] This embodiment proposes a computer-readable storage medium that stores executable instructions. When these instructions are executed, if they are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0032] The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the task scheduling method based on timeline modeling and adaptive task partitioning described in various embodiments of this application.

[0033] The aforementioned storage media include: flash memory, hard disks, multimedia cards, card-type memory (e.g., SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory), random access memory (RAM), static random-access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, disks, optical discs, servers, APP (Application) application stores, and other media capable of storing program verification codes. These media store computer programs, which, when executed by a processor, can implement the various steps of the task scheduling method based on timeline modeling and adaptive task partitioning described above.

[0034] This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the aforementioned task scheduling method based on timeline modeling and adaptive task partitioning.

[0035] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0036] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0037] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of this invention and its equivalents, then the intent of this disclosure also includes these modifications and variations.

[0038] In terms of technical implementation, this invention ensures the accuracy and efficiency of scheduling through a series of innovative mechanisms, specifically including: Virtual machine memory management based on fine-grained timeline modeling: Each virtual machine is equipped with an independent memory timeline model, which precisely manages its memory usage over time using a data structure called a segment tree. Each logical node in this data structure stores the minimum available memory height for its represented time interval and a lazy tag for batch updates. During initialization, the timeline model establishes and sets the initial available memory height of the virtual machine. When a task consumes memory, the model reduces the memory height of the specified time interval and efficiently propagates updates through the lazy tag mechanism, avoiding frequent low-level operations. The core search function allows for efficient searching and returning a time interval that meets the conditions (start time, end time, actual minimum memory height) given a start time and minimum memory requirements. This search process combines binary search and recursive query to ensure that the found interval meets memory requirements while having the longest possible length (corresponding to the time required to execute the task), thereby maximizing the batch size. In addition, the system uses a series of conversion functions to flexibly implement the mutual conversion between batch size, task execution time, and memory usage, enabling the scheduler to dynamically adapt to the processing speed characteristics and memory usage models of different virtual machines.

[0039] Dynamic Batch Size Adjustment and Adaptive Task Splitting: When attempting to schedule user tasks to virtual machines, the system dynamically calculates and determines the actual batch size of the current task based on the user's remaining task size, the current virtual machine's processing power, and the resources available on the virtual machine's timeline. This actual batch size is determined by comprehensively considering the maximum batch size supported by the virtual machine's timeline, the user's current remaining task volume, and the preset minimum batch size. When selecting the optimal virtual machine, the scheduler not only considers the task completion time but also introduces the "drop tail" parameter (indicating whether to allow tasks not in the last batch to not be forced to meet the minimum batch size requirement, i.e., leaving the remaining portion for later processing) and the "send as early as possible" parameter (indicating whether to send requests as early as possible) to fine-tune the scheduling strategy and improve its flexibility. When the user's total task needs to be processed, the system can adaptively split it into multiple independent subtasks. Each subtask can be independently scheduled to different virtual machines for parallel execution, and the system maintains its independent global remaining task volume and the number of requests sent for each subtask. This task splitting capability allows for effective reduction of the overall completion time through parallelization when a single virtual machine cannot complete the task within the specified time.

[0040] Timeout Handling and Exploration of Multiple Partition Schemes: The scheduler can detect whether the current scheduling scheme will cause user tasks to time out and accurately calculate the size of the timed-out tasks. Once a timeout is detected, this invention does not simply accept it, but actively generates a variety of task partitioning schemes. These schemes may include handling the timed-out task parts separately, or splitting the total task into two, three, or even more parts according to different proportions. The system will traverse and evaluate these different partitioning schemes, finding the best virtual machine for scheduling the subtasks in each partitioning scheme. Finally, the overall solution that results in the earliest total completion time is selected. This strategy of exploring and comparing multiple partitioning schemes significantly enhances the ability and flexibility of this invention to solve the task timeout problem.

[0041] Optimization of communication latency and transmission time: The task request data structure clearly distinguishes between the transmission time, the moment the server receives and begins executing the task, and the moment the task completes. During scheduling, the actual transmission time calculation fully considers the communication latency between the user and the specific server. Furthermore, the system provides an optimization function that can adjust the transmission time of requests. This function calculates the transmission time of each request backward from the start time of the last request's task execution, ensuring that requests are sent as late as possible without affecting their task execution start time. This optimization strategy helps reduce the dwell time of requests in the system, thereby making more efficient use of valuable computing resources.

[0042] The core objective of this invention is to overcome the inherent shortcomings of existing distributed task scheduling methods in complex heterogeneous environments. Traditional scheduling strategies, due to their fixed execution logic, cannot effectively address the heterogeneity of tasks in current scenarios where multiple users share heterogeneous computing resources, i.e., the differentiated requirements of different tasks for virtual machine memory, processing speed, and other resources. Similarly, while metaheuristic methods possess global search capabilities, their high computational overhead and high sensitivity to parameter tuning make it difficult to provide effective real-time or near-real-time scheduling in real-world dynamic environments. This rigid or computationally expensive scheduling approach often leads to low resource utilization, failing to fully leverage the potential of heterogeneous clusters and failing to meet users' strict requirements for task completion time windows, resulting in numerous task timeouts. This invention aims to overcome these limitations by providing an intelligent scheduling scheme that can dynamically adapt to changes in resource characteristics and task requirements, ensuring maximum utilization of computing resources while significantly improving the on-time completion rate of user tasks.

[0043] Optimizing task scheduling strategies can significantly reduce task timeout rates and avoid resource waste. Existing methods, when handling large-scale or high-concurrency tasks, often lack flexible task decomposition and parallel processing capabilities, causing individual virtual machines to become performance bottlenecks. This leads to prolonged task blocking and even a large number of tasks exceeding their expected completion time. This inefficient resource allocation not only directly affects user experience and system service quality but also signifies idle or inefficient use of computing resources. This invention introduces fine-grained timeline modeling technology to precisely manage the future resource availability of virtual machines. Combined with an adaptive task decomposition strategy, it allows large tasks or tasks at risk of timeout to be intelligently decomposed into smaller subtasks, which can be executed in parallel on multiple virtual machines. This cooperative scheduling mechanism aims to minimize task completion time and improve the system's responsiveness to sudden task loads, thereby fundamentally improving task timeout issues and resource utilization efficiency.

[0044] This invention aims to achieve precise management and efficient utilization of virtual machine memory resources, thereby improving the overall system throughput. In a multi-user resource-sharing environment, accurately understanding the memory usage of each virtual machine at different points in time is crucial for efficient scheduling. Traditional scheduling methods often lack fine-grained awareness of memory resources over time, easily leading to memory overruns or fragmentation, thus limiting the ability of virtual machines to handle more tasks. This invention constructs a precise virtual machine memory timeline model to reflect memory availability in real time, ensuring that tasks are allocated strictly according to memory constraints while maximizing the size of each batch of tasks, thereby improving the processing efficiency of virtual machines. This fine-grained resource management, combined with dynamic batch size adjustment, allows the system to maximize the amount of tasks completed per unit time while satisfying all constraints, effectively improving the overall system throughput and meeting the computing power needs of multiple users.

[0045] This invention provides flexible scheduling capabilities to adapt to heterogeneous server environments. A significant characteristic of current distributed computing environments is the heterogeneity of servers and virtual machines. Virtual machines on different servers may have different processing speeds and memory capacities, and communication latency between users and different servers varies. Traditional scheduling methods often struggle to effectively handle this complex heterogeneity, potentially leading to uneven task allocation or insufficient resource utilization. This invention, through its intelligent search mechanism, fully considers these heterogeneous characteristics in scheduling decisions, selecting the most suitable virtual machine resources for each user task, thereby maximizing the overall performance of the heterogeneous cluster. This flexibility ensures that the scheduling scheme can dynamically adapt to differences in the underlying hardware environment, further improving the system's performance and efficiency in highly complex scenarios.

[0046] This invention can significantly reduce the timeout rate and severity of user tasks. By introducing task partitioning and multi-virtual machine parallel scheduling mechanisms, especially when user tasks face the risk of timeout, the system can effectively distribute tasks across multiple virtual machines for parallel processing, thereby greatly shortening the overall completion time. For example, when the initial scheduling scheme causes a user task to exceed its preset end time, by reasonably partitioning and scheduling the task across multiple virtual machines, the completion time can be effectively brought forward, thereby reducing the number of timeout users and the timeout ratio of individual users, and significantly improving the final task completion efficiency.

[0047] This invention improves virtual machine resource utilization. Through virtual machine memory management based on fine-grained timeline modeling, the scheduler can more accurately identify idle time periods that meet memory and processing time requirements, avoiding resource fragmentation and underutilization that can occur with simple sequential scheduling. Simultaneously, dynamic batch size adjustment ensures that each virtual machine can process as many tasks as possible in each execution, thereby fully leveraging the maximum throughput of heterogeneous computing resources.

[0048] This invention is capable of adapting to heterogeneous server environments. The algorithm fully considers the processing speed coefficients and memory sizes of the virtual machines contained in different servers, as well as the communication latency between different users and different servers, in its scheduling decisions. When searching for the optimal virtual machine, this invention comprehensively considers these heterogeneity parameters, enabling it to select the most suitable resources for each user task and fully leverage the overall performance of the heterogeneous cluster.

[0049] This invention optimizes the overall performance of task completion. Through timeout handling and a multi-partitioning scheme exploration mechanism, it proactively identifies and addresses situations that may prevent tasks from completing on time. By intelligently searching and comparing multiple partitioning schemes, the system can optimize task completion time while meeting throughput requirements. Although the logic for directly optimizing the number of migrations is not detailed in the technical solution, scheduling tasks on unoccupied virtual machines as much as possible after partitioning can indirectly control the number of migrations between different virtual machines.

[0050] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A task scheduling method based on timeline modeling and adaptive task partitioning, characterized in that, The method comprises the following steps: constructing a timeline of a plurality of virtual machines, the timeline of the virtual machines comprising a plurality of unit time periods, each unit time period corresponding to an available memory size, receiving a plurality of user requests, each user request containing a target task, a start time startTime of the target task, an end time endTime of the target task, and a size size of the target task; on the timeline of each virtual machine, obtaining an actual execution interval of each batch of the target task, and determining an actual completion time of the target task, and then determining the actual completion time on each virtual machine, and obtaining a minimum value among all the actual completion times as a first completion time; determining whether the first completion time exceeds the end time endTime of the target task, if the first completion time does not exceed the end time endTime of the target task, taking the virtual machine corresponding to the first completion time as the best virtual machine of the target task, and generating a best scheduling scheme containing the actual execution interval of each batch of the target task; if the first completion time exceeds the end time endTime of the target task, dividing the target task into a plurality of subtasks according to a preset ratio according to the size size of the target task, for each subtask, obtaining an actual execution interval of each batch of the subtask on the timeline of the virtual machine, and generating a best scheduling scheme containing the actual execution interval of each batch of each subtask of the target task and the corresponding virtual machine.

2. The task scheduling method based on timeline modeling and adaptive task partitioning according to claim 1, characterized in that, on the timeline of each virtual machine, obtaining an actual execution interval of each batch of the target task, and determining an actual completion time of the target task, and then determining the actual completion time on each virtual machine, and obtaining a minimum value among all the actual completion times as a first completion time, comprising: for each user request, dividing the target task into a plurality of batches, and taking the first batch as a current batch; for the current batch, calculating a earliest start time minStartTime of the current batch, and calculating a minimum batch size minBatchSize of the current batch; based on the relationship between the task amount and the memory occupation of the virtual machine, the minimum batch size minBatchSize is calculated to obtain a minimum memory occupation minMemorySize; based on the relationship between the task amount and the processing time of the virtual machine, the minimum batch size minBatchSize is calculated to obtain a minimum processing time minSovleTime; based on the minimum processing time minSovleTime, the minimum memory occupation minMemorySize and the earliest start time minStartTime, the actual execution interval of the current batch is determined on the timeline of a single virtual machine; the left end point of the actual execution interval is taken as the actual start time actualStartTime of the current batch, and the right end point of the actual execution interval is taken as the actual end time actualEndTime of the current batch. Determine a batch actual size actualBatchSize based on a total processing time of an actual execution interval and an available memory size of the actual execution interval; Subtract a delay time delayTime sent by a user to a virtual machine from an actual start time actualStartTime to obtain a batch actual sending time actualSendTime; Update an earliest sending time minSendTime and a last batch completion time lastFinishTime, obtain a next batch as a current batch, and return execution: calculate an earliest start time minStartTime of the current batch until a last batch of the target task, and an actual end time of the last batch is an actual end time of the target task; Repeat the above operation to calculate an actual end time of the target task on each virtual machine, and obtain a minimum value of all actual completion times as a first completion time.

3. The task scheduling method based on timeline modeling and adaptive task partitioning according to claim 2, characterized in that, Calculate the earliest start time minStartTime of the current batch, which is specifically implemented by the following formula: minStartTime=max(minSendTime+delayTime,lastFinishTime)+1; Wherein, minSendTime is the earliest sending time, when the current batch is the first batch of the target task, the earliest sending time is a start time startTime of the target task, delayTime is the delay time sent by the user to the virtual machine, and lastFinishTime is the completion time of the last batch.

4. The task scheduling method based on timeline modeling and adaptive task partitioning according to claim 2, characterized in that, Calculate a minimum batch size minBatchSize of the current batch, which is specifically implemented by the following formula: minBatchSize=min(leftTaskSize,minAcceptBatchSize); Wherein, leftTaskSize is a difference between the target task size and a processed batch size, and minAcceptBatchSize represents a pre-set minimum acceptable task size.

5. The task scheduling method based on timeline modeling and adaptive task partitioning according to claim 2, characterized in that, Based on the minimum processing time minSovleTime, the minimum memory size minMemorySize and the earliest start time minStartTime, determine an actual execution interval of the current batch on a timeline of a single virtual machine, including: On the timeline of the single virtual machine, obtain a plurality of time intervals with a length of the minimum processing time minSovleTime and a height not less than the minimum memory size minMemorySize, obtain a time interval with a start time closest to the earliest start time minStartTime from all the time intervals, and take the time interval as an initial time interval; expand an end time of the initial time interval, and simultaneously make an available memory size of the expanded interval not less than the minimum memory size minMemorySize to obtain a final time interval, and take the final time interval as the actual execution interval of the current batch.

6. The task scheduling method based on timeline modeling and adaptive task partitioning according to claim 2, characterized in that, Determine a batch actual size actualBatchSize based on a total processing time of the actual execution interval and an available memory size of the actual execution interval, including: Determine a first batch size based on a relationship between a task amount and a processing time of the virtual machine and a second batch size based on a relationship between the task amount and a memory occupation of the virtual machine, and obtain a minimum value of the first batch size and the second batch size as the batch actual size actualBatchSize.

7. The task scheduling method based on timeline modeling and adaptive task partitioning according to claim 1, characterized in that, In the process of obtaining the actual execution interval of each batch of the subtask, if the last batch of the current subtask is less than the minimum acceptable task size except for the last subtask, the last batch is added to the next subtask for processing.

8. An electronic device, comprising: One or more processors, and a memory for storing instructions that, when executed by the one or more processors, cause the one or more processors to perform the timeline-based modeling and adaptive task partitioning method. A storage has executable instructions that, when executed, cause a processor to perform the timeline-based modeling and adaptive task partitioning method.

9. A computer-readable storage medium, characterized in that, A computer program or instructions that, when executed by a processor, implement the timeline-based modeling and adaptive task partitioning method.

10. A computer program product, characterised in that, ​

Images