Task scheduling

Abstract

One or more embodiments of the present description relate to the technical field of computers. Provided are a task scheduling method and apparatus. The method comprises: determining a target task queue among a plurality of pre-constructed task queues, wherein the plurality of task queues are respectively used for storing tasks of different priorities, each task queue is configured with a scheduling weight, and the scheduling weight is used for controlling the probability of the task queue being determined as the target task queue; and scheduling target tasks in the target task queue to a specified thread in a thread pool so as to execute the target tasks. By means of the embodiments of the present description, the execution of tasks of different priorities can be rationally coordinated, thereby avoiding resource contention between the tasks of different priorities in a task scheduling process.

Description

Task Scheduling Technical Field

[0001] One or more embodiments of this specification relate to the field of computer technology, and more particularly, to a task scheduling method and apparatus. Background Art

[0002] Task scheduling is the process of assigning pending tasks to available threads in a thread pool based on a pre-defined resource allocation strategy. By properly scheduling tasks within a system, the system can efficiently execute various tasks while ensuring stability, responsiveness, and performance while meeting task execution requirements.

[0003] However, in the process of task scheduling, related technologies cannot coordinate the execution order of tasks of different priorities well, which easily leads to contention of system resources among tasks of different priorities.

[0004] Summary of the Invention

[0005] In view of this, one or more embodiments of this specification provide a task scheduling method and apparatus.

[0006] To achieve the above objectives, one or more embodiments of this specification provide the following technical solutions.

[0007] According to a first aspect of one or more embodiments of the present specification, a task scheduling method is proposed, comprising: determining a target task queue among a plurality of pre-constructed task queues, wherein the plurality of task queues are respectively used to store tasks of different priorities, and each task queue is configured with a scheduling weight, and the scheduling weight is used to control the probability of a task queue being determined as a target task queue; scheduling a target task in the target task queue to a designated thread in a thread pool so as to execute the target task.

[0008] According to the second aspect of one or more embodiments of the present specification, a task scheduling device is proposed, comprising: a determination module for determining a target task queue among a plurality of pre-constructed task queues, wherein the plurality of task queues are respectively used to store tasks of different priorities, and each task queue is configured with a scheduling weight, and the scheduling weight is used to control the probability of a task queue being determined as a target task queue; a scheduling module for scheduling a target task in the target task queue to a specified thread in a thread pool so as to execute the target task.

[0009] According to a third aspect of one or more embodiments of this specification, an electronic device is proposed, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor implements the method of the first aspect by running the executable instructions.

[0010] According to a fourth aspect of one or more embodiments of this specification, a computer-readable storage medium is provided, on which computer instructions are stored. When the instructions are executed by a processor, the steps of the method of the first aspect are implemented.

[0011] The method that this specification provides is pre-built with a plurality of task queues, wherein, a plurality of task queues are respectively used for storing tasks of different priorities, and each task queue is configured with a scheduling weight, and the scheduling weight is used to control the probability that the task queue is determined as the target task queue.By determining different task queues as the target task queue each time, the scheduling of tasks of different priorities can be realized.Because the probability that each task queue is determined as the target task queue can be controlled by the scheduling weight, therefore can control the frequency that tasks in each task queue are scheduled by configuring the scheduling weight, thereby the tasks of different priorities are carried out the scheduling of different frequencies.Thus, this specification embodiment can reasonably coordinate the execution of tasks of different priorities, thereby avoid resource contention for the tasks of different priorities in the task scheduling process. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] FIG1 is a flowchart of a task scheduling method provided by an exemplary embodiment.

[0013] FIG2 is a flow chart of a method for determining the total number of threads provided by an exemplary embodiment.

[0014] FIG3 is a schematic structural diagram of a device provided by an exemplary embodiment.

[0015] FIG4 is a schematic structural diagram of a task scheduling device provided by an exemplary embodiment. DETAILED DESCRIPTION

[0016] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The implementations described in the following exemplary embodiments are not intended to represent all implementations consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of one or more embodiments of this specification, as detailed in the appended claims.

[0017] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than those described in this specification. In addition, a single step described in this specification may be broken down into multiple steps for description in other embodiments, and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0018] With the advancement of computer technology, task scheduling has become ubiquitous in various computer systems. For example, in database systems based on log-structured merge trees (LSM-Trees), compaction tasks are a typical task that requires scheduling. Compaction tasks can be specifically divided into dump tasks and merge tasks, both of which are resource-intensive tasks in database systems. Therefore, when executing dump and / or merge tasks, it is necessary to consider both the resource requirements of the tasks and the impact of the allocated resources on tasks in other modules of the system. Therefore, it is necessary to configure appropriate task scheduling strategies to manage resources.

[0019] In related technologies, due to the varying data processing volumes and characteristics of different types of tasks, allocating resources solely based on system resource utilization can lead to waste and impact resource usage for tasks in other modules. Furthermore, for tasks with varying priorities, related technologies fail to effectively balance the execution of tasks of varying priorities. When priorities are considered, low-priority tasks can be put on hold for extended periods, while when priorities are ignored, high-priority tasks are prevented from being prioritized, leading to contention for resources among tasks of varying priorities.

[0020] In view of this, this specification sheet embodiment is pre-built with a plurality of task queues, wherein, a plurality of task queues are respectively used for storing the tasks of different priorities, and each task queue is configured with a scheduling weight, and the scheduling weight is used to control the probability that the task queue is determined as the target task queue.By determining different task queues as the target task queue at each time, can realize the scheduling of tasks of different priorities.Because the probability that each task queue is determined as the target task queue can be controlled by the scheduling weight, therefore can control the frequency that tasks in each task queue are scheduled by configuring the scheduling weight, thereby the tasks of different priorities are carried out the scheduling of different frequencies.Thus, this specification sheet embodiment can reasonably coordinate the execution of tasks of different priorities, thereby avoid resource contention taking place in the tasks of different priorities in the task scheduling process.

[0021] For ease of understanding, the following will take the compaction task scheduling in a distributed database as an example to first introduce the application scenario of the embodiments of this specification.

[0022] As mentioned above, the compaction task is a typical task in an LSM-Tree-based database system. An LSM-Tree-based database system typically includes a memory table (Memtable) and a sorted string table (SSTable).

[0023] Memtable is an in-memory data structure within the LSM tree. Because the Memtable is stored in memory, data read and write operations in the Memtable can be performed efficiently. When a write operation occurs in the database, the data to be written is first written to the Memtable to maintain fast write performance. However, due to limited memory capacity, when the size of the Memtable reaches a certain threshold, the data stored in the Memtable will be dumped to the SSTable on the hard disk to free up storage space in memory for new data writing.

[0024] An SSTable is a disk data structure within an LSM tree that persists data. For example, when data in a memtable is flushed to disk, it is written to an SSTable. When multiple SSTables reach a certain number or size, they can be merged into a larger SSTable file to reduce disk space usage and optimize read performance.

[0025] The dump and merge tasks mentioned above are different types of compaction tasks. Since dumping can free up occupied memory in the database system and the dump process may affect the execution of database foreground SQL statements, the priority of the dump task is usually higher than that of the merge task. The task scheduling method provided in the embodiments of this specification can effectively coordinate the execution of the dump and merge tasks, ensuring that the dump task is executed first while preventing the merge task from being shelved for a long time.

[0026] Next, the exemplary embodiments of this specification will be described in detail in conjunction with the above application scenarios.

[0027] First, the embodiments of this specification provide a task scheduling method, which can be executed by any electronic device.

[0028] FIG1 is a flowchart of a task scheduling method provided by an exemplary embodiment. As shown in FIG1 , the task scheduling method provided by the embodiment of this specification includes the following steps.

[0029] S101: Determine a target task queue among multiple pre-built task queues.

[0030] It should be noted that a task queue is a container for storing tasks. This specification pre-configures multiple task queues, wherein the multiple task queues are respectively used to store tasks of different priorities.

[0031] For example, in the above application scenario, the compaction task can be divided into two task types with different priorities: dump and merge. Since the dump task has a higher priority than the merge task, the subtasks of the dump task and the merge task can be stored in different task queues.

[0032] In addition, each task queue is assigned a scheduling weight, which controls the probability of a task queue being designated as the target task queue. By adjusting the scheduling weight, you can increase the probability that a task queue containing high-priority tasks will be designated as the target task queue, or decrease the probability that a task queue containing low-priority tasks will be designated as the target task queue.

[0033] Exemplarily, after the scheduling weights are configured, the probability of each task queue being determined as the target task queue is positively correlated with the priority of the tasks stored in the task queue.

[0034] Since the scheduler schedules the tasks in the target task queue to the corresponding thread each time, by controlling the probability of each task queue being determined as the target task queue, the frequency with which tasks of different priorities are scheduled can be indirectly controlled.

[0035] In some embodiments, the scheduling weight is shown by a weight coefficient interval, where the size of the weight coefficient interval corresponding to each task queue is positively correlated with the priority of the task stored in the task queue, that is, the higher the priority of the task stored in the task queue, the larger the weight coefficient interval corresponding to the task queue.

[0036] After configuring a weight coefficient interval for each task queue, a random weight coefficient may be generated, and the task queue corresponding to the weight coefficient interval into which the random weight coefficient falls may be determined as the target task queue.

[0037] For example, assume that there are four task queues A, B, C, and D with different priorities, and the priorities of the four task queues are: task queue A > task queue B > task queue C > task queue D. In this case, the weight coefficient intervals corresponding to the four task queues A, B, C, and D can be configured as [1, 4], [5, 7], [8, 9], and [10, 10] respectively.

[0038] When determining the target task queue, by generating a random number between 1 and 10, the task queue corresponding to the weight coefficient interval into which the random number falls can be determined as the target task queue.

[0039] At this point, the embodiments of this specification can effectively control the frequency with which tasks of different priorities are scheduled by storing tasks of different priorities in different task queues respectively, and assigning different scheduling weights to each task queue according to the priority of the task, thereby ensuring that high-priority tasks are scheduled first and avoiding low-priority tasks from being shelved for a long time.

[0040] S102, dispatching the target task in the target task queue to a designated thread in the thread pool so as to execute the target task.

[0041] It should be noted that the target task is a task in the target task queue that does not depend on the execution results of other tasks. Exemplarily, the tasks in each task queue can be pre-constructed into a directed acyclic graph (DAG), which is used to illustrate the dependency relationship between each task in the task queue.

[0042] If there are multiple tasks that meet the criteria in the target task queue, they can be scheduled according to the "first in, first out" rule in the queue. In other words, the task that was stored in the task queue earlier will be scheduled first.

[0043] For example, let's still take the above-mentioned Compaction task as an example. During the execution of the Compaction task, several subtasks need to be scheduled. Since the subtasks of the same Compaction task have the same priority, these subtasks are stored in the same task queue. There may be three types of subtasks in the subtasks of the Compaction task. Among them, the first type of subtask is used to select the SSTable to be merged, the second type of subtask is used to organize the SSTable to be merged and form a new macroblock, and the third type of subtask is used to generate the new macroblock into an SSTable structure. Among the above three types of subtasks, the first type of subtask does not depend on the execution results of other tasks and can be executed independently. The second type of subtask depends on the execution results of the first type of subtask, and the third type of subtask depends on the execution results of the second type of subtask.

[0044] Therefore, the DAG corresponding to the task queue can be pre-built according to the dependency relationship between the above three types of subtasks, so that the tasks in the task queue are scheduled in the order of the first type of subtasks, the second type of subtasks, and the third type of subtasks.

[0045] It's important to note that a thread pool is a thread management mechanism that maintains a collection of reusable threads. After a task is dispatched to a thread in the thread pool, that thread executes the scheduled task and, after completing it, waits for the next scheduled task. However, the same thread cannot execute multiple tasks simultaneously.

[0046] As mentioned above, since high-priority tasks have a greater probability of being scheduled, in order to avoid high-priority tasks being continuously scheduled and occupying too many system resources (for example, threads and memory), the embodiments of this specification also differentiate the threads in the thread pool and constrain specific threads to be used only to execute tasks of specific priorities.

[0047] Specifically, in the embodiments of this specification, threads in the thread pool are divided into at least a first thread and a second thread, wherein the first thread and the second thread are respectively used to execute tasks of different priorities. Of course, in order to finely control the resources that can be occupied by tasks of different priorities, the threads in the thread pool can be divided more finely, for example, into a first thread, a second thread, a third thread, etc., and different threads are respectively used to execute tasks of different priorities, but this embodiment of this specification does not limit this.

[0048] When the threads are divided into the first thread and the second thread, the scheduling of low-priority tasks and the simplicity of the thread division process can be effectively taken into account. In this case, the first thread is used to execute the highest-priority task stored in the multiple task queues, and the second thread is used to execute tasks of other priorities stored in the multiple task queues except the highest priority.

[0049] That is, the embodiment of this specification uses the first thread as the exclusive thread for the highest priority task, while retaining the second thread as the exclusive thread for tasks of other priorities, thereby preventing tasks of a certain priority from being scheduled in large quantities and preempting resources of tasks of other priorities.

[0050] In some embodiments, the number of first threads is less than or equal to the number of second threads. Considering that the first thread is a dedicated thread for executing the highest priority task, and the highest priority tasks are generally small in number and scheduled frequently, the number of threads reserved for the highest priority task can be less than or equal to the number of threads used to execute tasks of other priorities.

[0051] Exemplarily, the number of first threads may be 40% of the total number of threads in the thread pool, and the number of second threads may be 60% of the total number of threads in the thread pool, and the number of first threads and the number of second threads are at least 1 to avoid the absence of threads in the thread pool that can execute tasks of a specific priority.

[0052] Therefore, in S102 , the target task may be scheduled to the first thread or the second thread according to the priority of the target task.

[0053] In some embodiments, after determining the total number of threads in the thread pool, at least half of the threads in the thread pool may be used as second threads, and the remaining threads may be used as first threads, thereby completing the configuration of threads in the thread pool.

[0054] For ease of understanding, the method for determining the total number of threads in the embodiment of this specification will be described in detail below with reference to FIG. 2 .

[0055] FIG2 is a flow chart of a method for determining the total number of threads provided by an exemplary embodiment. As shown in FIG2 , the method for determining the total number of threads provided by an embodiment of this specification includes the following steps.

[0056] S201, preliminarily determine the total number of threads.

[0057] In some embodiments, the total number of threads can be preliminarily determined based on system resources. For example, since the number of threads is directly related to CPU performance, the total number of threads in the thread pool can be preliminarily determined based on the number of CPU cores currently available to the user, so that the total number of threads is positively correlated with the number of CPU cores.

[0058] S202 : predicting the total memory usage of the tasks stored in the plurality of task queues during execution based on the preliminarily determined total number of threads.

[0059] In some embodiments, since each thread needs to occupy a certain amount of memory when executing a task, after preliminarily determining the total number of threads, the total memory usage of tasks stored in multiple task queues during execution can be predicted based on the memory occupied by each thread in the actual task.

[0060] It is worth noting that due to the uncertainty of the task types stored in the task queue, these tasks may occupy other memory in addition to the memory occupied by each thread during execution. When predicting the total memory usage, all the memory that may be occupied by the task during execution should be considered.

[0061] S203: If the total memory usage exceeds the preset memory limit, reduce the total number of threads until the predicted total memory usage meets the preset memory limit.

[0062] In some embodiments, the preset memory limit can be a preset memory percentage. For example, the total memory usage is limited to no more than 60% of the currently available free memory. If the memory percentage is exceeded, the total number of threads can be reduced and the corresponding total memory usage can be re-estimated until the predicted total memory usage meets the preset memory limit.

[0063] The following uses the execution of multiple compaction tasks as an example to illustrate the relationship between total memory usage and the total number of threads. In this case, multiple task queues store the subtasks of each compaction task.

[0064] It can be understood that the memory occupied during the compaction task is mainly divided into the following four parts.

[0065] 1. Macroblock Buffer Memory (buffer_mem): A compaction task requires nine 2MB memory blocks from the global memory pool. These memory blocks are used to store data needed during the compaction task. When the total number of memory blocks requested by all compaction tasks exceeds the number of free blocks, the doubling mechanism doubles the memory pool space. Therefore, assuming the total number of compaction tasks is C, the estimated buffer_mem usage is 18MB × C.

[0066] 2. Macroblock metadata memory (meta_mem): During the execution of the compaction task, when a macroblock is written to disk, a deep copy of the macroblock's metadata is required. The number of copies is equal to the number of macroblocks written. Macroblock metadata contains basic information (fixed size) and row keys (Rowkey). The more macroblocks there are (i.e., more data), or the more Rowkey columns there are, the more memory is occupied. Assuming that the basic information of the macroblock metadata is fixed in size, mem_size, the number of macroblocks is N, the number of RowKey columns is M, and the average size of each column is S, the estimated usage of meta_mem is N × (meta_size + M × S).

[0067] 3. Deep copy memory (rowkey_mem): During the compaction task, a deep copy of the RowKey is required when writing row data to a macroblock. This memory is reused each time a macroblock is flushed, so the actual memory consumption is related to the number of rows in the macroblock. Assuming the number of RowKey columns is M, the macroblock contains N rows, and the average size of each RowKey column is S, the estimated rowkey_mem usage is N × (M × S).

[0068] 4. Pre-stored Arrays (static_mem): During the compaction task, some large fixed-length arrays (typically 8MB) and dynamic arrays are pre-stored, which consume a certain amount of memory. Assuming R is the estimated memory for dynamic arrays, the estimated static_mem usage is 8MB + R.

[0069] During the actual execution of the Compaction task, each Compaction task will use a copy of buffer_mem and a copy of meta_mem, and each thread used to execute the subtask of the Compaction task will use a copy of rowkey_mem and a copy of staitc_mem.

[0070] Therefore, assuming that there are C compaction tasks and the total number of threads is K, the total memory usage of the compaction task is total_mem = K×(rowkey_mem+static_mem)+C×(meta_mem+buffer_mem).

[0071] If the total memory usage exceeds the above preset memory limit, the total number of threads K can be appropriately reduced so that the total memory usage meets the preset memory limit.

[0072] That is, when the subtasks of each compaction task are stored separately in multiple task queues, S202 described above may include the following steps: separately predicting the macroblock buffer memory usage, macroblock meta-information memory usage, deep copy memory usage, and pre-stored array memory usage generated during the execution of the compaction task. Multiplying the sum of the macroblock buffer memory usage and the macroblock meta-information memory usage by the number of compaction tasks to obtain a first product. Multiplying the sum of the deep copy memory usage and the pre-stored array memory usage by the preliminarily determined total number of threads to obtain a second product. Adding the first and second products to obtain the total memory usage.

[0073] In some embodiments, the total number of threads that can be fully utilized under the preset memory limit of total memory usage total_mem can also be reversed, and the smaller value between the total number of threads reversed according to the preset memory limit and the total number of threads preliminarily determined in S201 can be used as the final total number of threads to avoid the total number of threads exceeding the maximum number of threads actually available during task execution, thereby causing waste of resources.

[0074] The embodiments of this specification can reasonably determine the resources required during task execution and allocate and occupy these resources. This ensures that high-priority tasks are scheduled first, preventing other priority tasks from being put on hold for long periods of time or running out of resources. Furthermore, the embodiments of this specification establish a relationship between threads and memory usage, enabling full utilization of system resources and avoiding resource waste caused by configuring too many threads or occupying too much memory.

[0075] FIG3 is a schematic diagram of the structure of a device provided by an exemplary embodiment. Referring to FIG3 , at the hardware level, the device includes a processor 302, an internal bus 304, a network interface 306, a memory 308, and a non-volatile memory 310, and may also include hardware required for other functions. One or more embodiments of this specification can be implemented based on software, such as the processor 302 reading the corresponding computer program from the non-volatile memory 310 into the memory 308 and then running it. Of course, in addition to software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0076] Please refer to Figure 4, which provides a task scheduling device 400 that can be applied to the device shown in Figure 3 to implement the technical solution of this specification. Specifically, the task scheduling device 400 may include the following modules.

[0077] The determination module 401 is used to determine a target task queue among a plurality of pre-built task queues, wherein the plurality of task queues are respectively used to store tasks of different priorities, and each task queue is configured with a scheduling weight, which is used to control the probability of a task queue being determined as a target task queue.

[0078] The scheduling module 402 is used to schedule the target task in the target task queue to a designated thread in the thread pool so as to execute the target task.

[0079] In some embodiments, the thread pool includes a first thread and a second thread, and the first thread and the second thread are respectively used to execute tasks of different priorities. The scheduling module 402 is specifically used to schedule the target task to the first thread or the second thread according to the priority of the target task.

[0080] In some embodiments, the first thread is used to execute the highest priority task stored in multiple task queues, and the second thread is used to execute tasks of other priorities stored in multiple task queues except the highest priority, and the number of first threads is less than or equal to the number of second threads.

[0081] In some embodiments, the task scheduling device 400 further includes a thread determination module (not shown in the figure) configured to determine the total number of threads in the thread pool and select threads that are at least half of the total number of threads in the thread pool as second threads.

[0082] In some embodiments, the thread determination module is also used to preliminarily determine the total number of threads; based on the preliminarily determined total number of threads, predict the total memory usage of tasks stored in multiple task queues during execution; if the total memory usage exceeds the preset memory limit, reduce the total number of threads until the predicted total memory usage meets the preset memory limit.

[0083] In some embodiments, the tasks stored in the multiple task queues are subtasks of the Compaction task. The thread determination module is further configured to respectively predict the macroblock buffer memory usage, macroblock meta-information memory usage, deep copy memory usage, and pre-stored array memory usage generated during the execution of the Compaction task; multiply the sum of the macroblock buffer memory usage and the macroblock meta-information memory usage by the number of Compaction tasks to obtain a first product; multiply the sum of the deep copy memory usage and the pre-stored array memory usage by the preliminarily determined total number of threads to obtain a second product; and add the first product and the second product to obtain the total memory usage.

[0084] In some embodiments, the probability of each task queue being determined as the target task queue is positively correlated with the priority of the tasks stored in the task queue.

[0085] In some embodiments, the scheduling weight is represented by a weight coefficient interval, and the size of the weight coefficient interval corresponding to each task queue is positively correlated with the priority of the tasks stored in the task queue. Determination module 401 is specifically configured to generate a random weight coefficient; and determine the task queue corresponding to the weight coefficient interval in which the random weight coefficient falls as the target task queue.

[0086] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or any combination of these devices.

[0087] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0088] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0089] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0090] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0091] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0092] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a," "an," "the," and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0093] It should be understood that although the terms first, second, third, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when..." or "when..." or "in response to determining."

[0094] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of this specification shall be included in the scope of protection of one or more embodiments of this specification.

Claims

1. A task scheduling method, comprising: Determine a target task queue from a plurality of pre-built task queues, wherein the plurality of task queues are respectively used to store tasks of different priorities, and each task queue is configured with a scheduling weight, and the scheduling weight is used to control the probability of a task queue being determined as a target task queue; The target task in the target task queue is dispatched to a designated thread in a thread pool so as to execute the target task.

2. The method according to claim 1, wherein the thread pool comprises a first thread and a second thread, and the first thread and the second thread are respectively used to execute tasks of different priorities; The step of scheduling the target task in the target task queue to a designated thread in a thread pool includes: The target task is scheduled to the first thread or the second thread according to the priority of the target task.

3. According to the method according to claim 2, the first thread is used to execute the task with the highest priority stored in the multiple task queues, and the second thread is used to execute tasks of other priorities stored in the multiple task queues except the highest priority, and the number of the first threads is less than or equal to the number of the second threads.

4. The method according to claim 2, further comprising: Determining the total number of threads in the thread pool; At least half of the total number of threads in the thread pool is used as the second thread.

5. The method according to claim 4, wherein determining the total number of threads in the thread pool comprises: Preliminarily determine the total number of threads; Predicting the total memory usage of the tasks stored in the plurality of task queues during execution according to the initially determined total number of threads; If the total memory usage exceeds the preset memory limit, the total number of threads is reduced until the predicted total memory usage meets the preset memory limit.

6. According to the method of claim 5, the tasks stored in the multiple task queues are subtasks of the compaction task; The predicting, based on the initially determined total number of threads, the total memory usage of the tasks stored in the multiple task queues during the execution process includes: respectively predicting the memory usage of the macroblock buffer, the memory usage of the macroblock meta information, the memory usage of the deep copy, and the memory usage of the pre-stored array generated during the execution of the compaction task; Multiplying the sum of the memory usage of the macroblock buffer and the memory usage of the macroblock meta information by the number of compaction tasks to obtain a first product; Multiplying the sum of the deep copy memory usage and the pre-stored array memory usage by the initially determined total number of threads to obtain a second product; The first product and the second product are added together to obtain the total memory usage. 7 . According to the method of claim 1 , the probability of each task queue being determined as the target task queue is positively correlated with the priority of the tasks stored in the task queue.

8. The method according to claim 1, wherein the scheduling weight is represented by a weight coefficient interval, and the size of the weight coefficient interval corresponding to each task queue is positively correlated with the priority of the task stored in the task queue; Determining a target task queue among the pre-built multiple task queues includes: Generate random weight coefficients; The task queue corresponding to the weight coefficient interval into which the random weight coefficient falls is determined as the target task queue.

9. A task scheduling device, comprising: A determination module, used to determine a target task queue from a plurality of pre-built task queues, wherein the plurality of task queues are respectively used to store tasks of different priorities, each task queue is configured with a scheduling weight, and the scheduling weight is used to control the probability of a task queue being determined as a target task queue; The scheduling module is used to schedule the target task in the target task queue to a specified thread in the thread pool so as to execute the target task.

10. An electronic device comprising: processor; a memory for storing processor-executable instructions; The processor implements the method according to any one of claims 1 to 8 by running the executable instructions.

11. A computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 8.

Images