Task management method and device

Abstract

The present disclosure relates to a task management method and device, wherein the method comprises: receiving a task to be processed, determining the resource isolation level and computing resource requirement corresponding to the task to be processed based on the indication information contained in the task request, wherein the resource isolation level is used to indicate the CPU isolation strategy corresponding to the task to be processed, and the computing resource requirement includes the requirement of the task to be processed for the CPU core; allocating a target CPU core to the task to be processed according to the resource isolation level, the computing resource requirement, and the CPU core allocation on each computing node, binding the target CPU core to the task process corresponding to the task to be processed, and the task process uses the target CPU core to execute the task to be processed. By binding the task process to a fixed CPU core, and different tasks corresponding to different resource isolation levels to realize binding different tasks to different CPU cores, interference between computing tasks is reduced, resource isolation is effectively realized, and resource utilization is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a task management method and device. Background Art

[0002] Service systems usually support multiple types of computing tasks, and various complex calculations have different requirements for the operating environment. For example, online services and real-time computing have extremely high requirements for stability and latency requirements at the millisecond level, offline computing has high requirements for throughput and can ignore latency, and heterogeneous computing includes two types of computing: those with high latency requirements and those with high throughput requirements, but they rely on heterogeneous devices.

[0003] Currently, multiple clusters are often deployed to run different types of computing tasks respectively, so as to meet the differentiated requirements of various types of computing tasks for the operating environment. However, this method will cause serious waste of resources. Summary of the invention

[0004] In order to solve the above technical problems, the present disclosure provides a task management method and device.

[0005] In a first aspect, the present disclosure provides a task management method, comprising:

[0006] Receive a task request, and determine the resource isolation level and computing resource requirement corresponding to the task to be processed based on the indication information contained in the task request; wherein the resource isolation level is used to indicate the central processing unit CPU isolation strategy corresponding to the task to be processed; the computing resource requirement includes the requirement of the task to be processed for the CPU core;

[0007] Allocate a corresponding target CPU core to the task to be processed according to the resource isolation level, the computing resource requirements, and the CPU core allocation on each computing node;

[0008] The task process corresponding to the task to be processed is started, the target CPU core is bound to the task process, and the task process corresponding to the task to be processed uses the target CPU core to execute the task to be processed.

[0009] In some embodiments, allocating a corresponding target CPU core to the task to be processed according to the resource isolation level, the computing resource requirement, and the CPU core allocation on each computing node includes:

[0010] Based on the CPU core allocation on each computing node, determine the computing node whose allocatable CPU core resources meet the computing resource requirements as the target computing node corresponding to the task to be processed;

[0011] Determine the target CPU core from the CPU cores included in the target computing node according to the resource isolation level, the computing resource demand, and the CPU core allocation corresponding to the target computing node;

[0012] Adding the assigned label and the label of the task to be processed to the target CPU core to assign the target CPU core to the task to be processed.

[0013] In some embodiments, determining the target CPU core from the CPU cores included in the target computing node according to the resource isolation level, the computing resource requirement, and the CPU core allocation corresponding to the target computing node includes:

[0014] If the task to be processed is at the first resource isolation level, determining the target CPU core from the idle CPU cores of the target computing node; wherein the task to be processed exclusively occupies the target CPU core;

[0015] If the task to be processed is at the second resource isolation level, the target CPU core is determined from the target computing node including idle CPU cores or CPU cores bound to tasks at the specified resource isolation level; the task to be processed exclusively occupies the target CPU core or shares the target CPU core with the tasks at the specified resource isolation level;

[0016] If the task to be processed corresponds to the third resource isolation level, determining the target CPU core from the CPU cores included in the target computing node that are not occupied by tasks at the first resource isolation level and the second resource isolation level; wherein the target CPU core allows binding of other computing tasks at the third resource isolation level;

[0017] If the task to be processed corresponds to the fourth resource isolation level, the target CPU core is determined from the CPU cores included in the target computing node that are not occupied by tasks at the first resource isolation level; wherein the target CPU core allows the binding of other computing tasks at the second to fourth resource isolation levels to share the target CPU core.

[0018] In some embodiments, after allocating the corresponding target CPU core to the task to be processed, the method further includes:

[0019] Configure one or more of the CPU core computing power usage weight, CPU usage limit, memory usage limit, disk usage limit, and network throughput usage limit corresponding to the task to be processed, so that the CPU core computing power, memory resources, disk resources, and network throughput scheduled when the task process uses the target CPU core to execute the task to be processed meet the corresponding restrictions.

[0020] In some embodiments, before determining the target computing node corresponding to the task to be processed according to the computing resource requirements corresponding to the task to be processed and the CPU core information of each computing node, the method further includes:

[0021] Receive the CPU core allocation status reported by each computing node.

[0022] In some embodiments, it also includes:

[0023] When the pending task corresponds to the lowest resource isolation level, based on the CPU core occupancy of each computing node, the pending task is migrated to a CPU core whose load is less than the load of the target CPU core for execution, and the load is used to indicate the number of tasks bound to the CPU core.

[0024] In some embodiments, it also includes:

[0025] After the execution of the task to be processed is completed, the target CPU core is unbound from the task process corresponding to the task to be processed, and the target CPU core is released.

[0026] In a second aspect, the present disclosure provides a task management device, comprising:

[0027] A receiving module, used for receiving a task request;

[0028] A resource management module, used to determine the resource isolation level and computing resource requirements corresponding to the task to be processed based on the indication information contained in the task request; wherein the resource isolation level is used to indicate the central processing unit CPU isolation strategy corresponding to the task to be processed; the computing resource requirements include the requirements of the task to be processed for the CPU core;

[0029] The resource management module is used to allocate a corresponding target CPU core to the task to be processed according to the resource isolation level, the computing resource requirements and the CPU core allocation on each computing node; and to start the task process corresponding to the task to be processed, bind the target CPU core to the task process, and the task process corresponding to the task to be processed uses the target CPU core to execute the task to be processed.

[0030] In a third aspect, the present disclosure provides an electronic device, including: a memory and a processor;

[0031] The memory is configured to store computer program instructions;

[0032] The processor is configured to execute the computer program instructions so that the electronic device implements the task management method as described in the first aspect and any one of the first aspects.

[0033] In a fourth aspect, the present disclosure provides a readable storage medium, comprising: computer program instructions; at least one processor of an electronic device executes the computer program instructions, so that the electronic device implements the task management method as described in the first aspect and any one of the first aspects.

[0034] In a fifth aspect, the present disclosure provides a computer program product, and an electronic device executes the computer program product so that the electronic device implements the task management method as described in the first aspect and any one of the first aspects.

[0035] The present disclosure provides a task management method and device, wherein the method includes: receiving a task request, determining the resource isolation level and computing resource requirements corresponding to the task to be processed based on the indication information contained in the task request, wherein the resource isolation level is used to indicate the CPU isolation strategy corresponding to the task to be processed, and the computing resource requirements include the requirements of the task to be processed for the CPU core; according to the resource isolation level, the computing resource requirements and the CPU core allocation on each computing node, the corresponding target CPU core is allocated to the task to be processed, the target CPU core is bound to the task process corresponding to the task to be processed, and the task process corresponding to the task to be processed uses the target CPU core to execute the task to be processed. The task process is bound to a fixed CPU core by task binding, and different tasks correspond to different resource isolation levels, so that different tasks can be bound to different CPU cores, reducing interference between computing tasks, effectively realizing resource isolation, and improving resource utilization; in addition, core binding can reduce the migration of task processes between CPU cores, reduce context switching overhead, improve cache hit rate, reduce memory access loss, and reduce latency. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0038] Figure 1 A schematic diagram of a scenario architecture of a task management method provided in an embodiment of the present disclosure;

[0039] Figure 2 A flowchart of a task management method provided by an embodiment of the present disclosure;

[0040] Figure 3 A schematic diagram of binding computing tasks with different resource isolation levels to CPU cores provided in an embodiment of the present disclosure;

[0041] Figure 4 A flowchart of a task management method provided by an embodiment of the present disclosure;

[0042] Figure 5 A schematic diagram of computing task migration provided by an embodiment of the present disclosure;

[0043] Figure 6 A schematic diagram of the structure of a task management device provided in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0044] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0045] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0046] As user needs continue to expand, different types of computing tasks have emerged. For example, online computing, real-time computing, offline computing, heterogeneous computing, interactive computing, graph computing, etc. Among them, online computing mainly refers to online services, real-time computing mainly refers to big data streaming computing tasks represented by the Flink engine, offline computing mainly refers to big data batch computing tasks represented by the Mapredouce engine and the Spark engine, and heterogeneous computing mainly refers to computing tasks on heterogeneous devices such as CPU and FPGA. Various complex computing tasks have different requirements for the operating environment. For example, online computing has extremely high requirements for stability and latency requirements at the millisecond level; real-time computing tasks have high requirements for stability and latency requirements at the millisecond level; offline tasks do not pay attention to latency, but have high requirements for throughput; heterogeneous computing includes two types of computing that care about latency and throughput, but rely on heterogeneous devices.

[0047] The operating environments of various complex calculations are contradictory and difficult to run on the same computing node. For example, on the one hand, tasks such as model training and big data batch computing have high requirements for throughput and hope to utilize the CPU of the computing node as much as possible; on the other hand, online services cannot respond to clients in time when the CPU of the entire machine is fully utilized.

[0048] Currently, multiple clusters are deployed to run different types of computing tasks respectively to meet the differentiated requirements of various complex tasks for the operating environment. However, this will cause serious waste of resources. In addition, the cluster management system will also isolate different computing tasks on a single machine by means of resource isolation. Resources such as memory, storage, and network can be better isolated by limiting usage, but CPU resources are more complex, and limiting CPU usage is far from enough.

[0049] In order to better isolate the CPU, the present disclosure is implemented through the method of "task hierarchical core binding", wherein task core binding refers to binding the task process to a fixed CPU core, and will not transfer it to other CPU cores, thereby ensuring that different tasks use different CPU cores, and different tasks are bound to different cores, so as not to interfere with each other, and achieve CPU isolation. Among them, in addition to achieving CPU resource isolation, task core binding can also improve computing performance. Specifically, it can reduce the migration overhead of running threads between cores, reduce context switching overhead, improve cache hit rate, reduce memory access loss, reduce the delay of large data streaming tasks, and improve the running time of offline tasks.

[0050] In addition, the present disclosure classifies computing tasks by fully considering the characteristics of various types of computing tasks on the basis of task binding, and configures different resource isolation levels for computing tasks of different categories, wherein different resource isolation levels correspond to different CPU isolation strategies. The CPU isolation strategy can indicate the usage strategy of the computing task for the allocated CPU core, such as the CPU isolation strategy can indicate whether the computing task is exclusive to the CPU core, or shared with other computing tasks when the specified conditions are met, and the CPU isolation strategy can also indicate the CPU core computing power usage weight corresponding to the task to be processed.

[0051] When receiving a computing task initiated by a client, a CPU core is allocated to it based on the resource isolation level corresponding to the computing task and the corresponding computing resource requirements, and the task process corresponding to the computing task is bound to the allocated CPU core, and the computing task is processed by executing the task process through the CPU core.

[0052] Next, the task management method provided by the present disclosure will be described in detail through the following examples combined with scenarios and related drawings.

[0053] Figure 1 This is a schematic diagram of an application scenario architecture of a task management method provided by an embodiment of the present disclosure. Figure 1 As shown, the scenario includes: a central service 101 and multiple stand-alone services 102. The central service 101 is responsible for cluster node management, global task management, resource management and scheduling; the stand-alone service 102 is responsible for resource and task management of specific computing nodes.

[0054] Among them, the central service 101 records the CPU core information reported by all stand-alone services 102. When receiving a task initiated by the client, it allocates a specific computing node to the task based on the CPU core information of each computing node, and notifies the stand-alone service 102 in the computing node that a new computing task is assigned.

[0055] The CPU core information may include one or more of the following: the number of CPU logical cores included in the computing node, information about the physical cores to which each CPU logical core belongs, information about the NUMA to which the CPU logical core belongs and the Socket to which the CPU logical core belongs, and information about the tasks bound to the CPU logical core. NUMA (Non Uniform Memory Access) stands for non-uniform memory access, which is a memory body design for multiple processors. The memory access time depends on the memory location of the processor.

[0056] When allocating CPU cores for tasks to be processed, the central service 101 can determine the CPU core allocation on the computing node based on the CPU core information, and allocate computing tasks to the CPU logical cores belonging to the same Socket and the same NUMA in the computing node as much as possible from the available CPU core resources, so as to minimize the switching of computing tasks between different Sockets and different NUMAs.

[0057] After the stand-alone service 102 senses that the task is assigned to the computing node, it starts the task process and configures resource isolation. After the task is completed, the stand-alone service 102 releases the CPU core bound to the task.

[0058] When configuring CPU cores for tasks and releasing CPU cores bound to tasks, stand-alone service 102 will report changes in CPU cores on computing nodes to central service 101, so that central service 101 can accurately and timely grasp the CPU core allocation of each computing node and perform reasonable resource scheduling.

[0059] Figure 2 A flowchart of a task management method provided by an embodiment of the present disclosure. Figure 2 As shown, the method of this embodiment includes:

[0060] S201. Receive a task request, and determine the resource isolation level and computing resource requirements corresponding to the task to be processed based on the indication information contained in the task request, wherein the resource isolation level is used to indicate the CPU isolation strategy corresponding to the processing task; the computing resource requirements include the requirements of the task to be processed for the CPU core.

[0061] Combination Figure 1In the scenario shown, the central node in the cluster management system can receive task requests sent by users through the client. The task request can be used to request the execution of a specified pending task. The pending task can be any type of computing task, such as an online computing task, a real-time computing task, an offline computing task, a heterogeneous computing task, etc. The user can select the pending task to be triggered by operating the user interface provided by the client, input the computing resource requirements and resource isolation level to the client, and send the task request to the central node through the client. When submitting the task request, the resource isolation level and computing resource requirements corresponding to the pending task can be specified.

[0062] The information indicating the resource isolation level may be a level name, a level identifier or a corresponding digital number corresponding to the resource isolation level, etc. The information indicating the computing resource requirement may include the number of required CPU cores and the type of CPU cores specified by the user through the client, and may also include the memory requirement, GPU requirement, network throughput requirement, etc. of the task to be processed.

[0063] S202: Allocate corresponding target CPU cores for the tasks to be processed according to the resource isolation level, computing resource requirements, and CPU core allocation on each computing node.

[0064] The CPU core allocation on the computing node can reflect whether the CPU core resources available for allocation on the computing node can meet the computing resource requirements of the task to be processed. Among them, the central service can determine the CPU resource allocation on the computing node based on the CPU core information reported by each stand-alone service, and then compare whether the CPU core resources available for allocation meet the computing resource requirements (such as comparing whether the number of CPU cores available for allocation is greater than or equal to the required number of CPU cores specified by the user through the client), and determine one or more computing nodes from the computing nodes whose CPU core resources available for allocation meet the computing resource requirements as the target computing nodes corresponding to the task to be processed. After that, the central service can send a notification message to the stand-alone service in the target computing node, and the notification message may include information indicating the task allocation and the resource isolation level and computing resource requirements of the task to be processed. After receiving the notification message, the stand-alone service can determine the target CPU core for the task to be processed based on the resource isolation level and the computing resource requirements, and add an "allocated" label and a label of the task to be processed to the target CPU core.

[0065] The label of the task to be processed may be, but is not limited to, an ID, a name, etc. corresponding to the task to be processed.

[0066] S203: Start a task process corresponding to the task to be processed, bind the target CPU core to the task process, and the task process corresponding to the task to be processed uses the target CPU core to execute the task to be processed.

[0067] When the target computing node receives the notification message, it perceives the task allocation, starts the task process for the allocated pending task and configures the CPU core resources; after the target CPU core is allocated, the task process is bound to the target CPU core, and then the task process can use the target CPU core to execute the pending tasks submitted by the client.

[0068] Among them, one or more task processes can be started for the task to be processed, and one or more threads can be executed under each task process. If multiple task processes are started, each task process can be bound to the target CPU core.

[0069] In this embodiment, the task process is bound to a fixed CPU core by binding the task to the core, and tasks of different levels are bound to CPU cores in different ranges, thereby reducing interference between computing tasks, effectively realizing resource isolation, and improving resource utilization. In addition, core binding can reduce the migration of task processes between CPU cores, reduce context switching overhead, improve cache hit rate, reduce memory access loss, and reduce latency.

[0070] In a specific embodiment, it is assumed that the cluster supports computing tasks at four resource isolation levels, each resource isolation level may include one or more types of computing tasks, and isolation strategies such as task core binding and CPU core computing power usage weights are defined through the four resource isolation levels, thereby ensuring the operating environment of different types of computing tasks on the same cluster and the same computing node, and taking into account the performance requirements of different types of computing tasks and cluster utilization requirements.

[0071] Among them, the four resource isolation levels are exemplified as follows:

[0072] 1. The first resource isolation level (also called the ALONE level): the CPU core is exclusively allocated and is not shared with any other computing tasks.

[0073] 2. The second resource isolation level (also called FIXED level): Exclusively allocated CPU cores can share CPU cores with computing tasks at the specified resource isolation level.

[0074] 3. The third resource isolation level (also called the NORMAL level): all CPU cores except those occupied by computing tasks at the first and second resource isolation levels are shared with each other.

[0075] 4. The fourth resource isolation level (also called the MIXED level): all CPU cores except the CPU cores occupied by computing tasks at the first resource isolation level are shared with each other.

[0076] Among them, the CPU core computing power usage weights corresponding to the computing tasks of the first to third resource isolation levels are respectively proportional (multiple) to the CPU core requirements corresponding to the computing tasks, and the computing tasks of the fourth resource isolation level have the lowest CPU core computing power usage weights. Among them, when multiple computing tasks share a CPU core, the CPU core computing power usage weights corresponding to the computing tasks can be calculated according to the number of CPU cores applied for by each computing task, wherein the CPU core computing power usage weights are proportional (multiple) to the CPU cores applied for. For example, a user submits a job, and a job includes multiple tasks. Task 1 applies for 1 CPU core, and Task 2 applies for 2 CPU cores. Task 1 and Task 2 share a CPU core. Then the CPU core computing power usage weight corresponding to Task 1 is 1 / (1+2), and the CPU core computing power usage weight corresponding to Task 2 is 2 / (1+2). If there are more tasks sharing the core, the CPU core computing power usage weights corresponding to each computing task are calculated according to the CPU core requirements corresponding to each computing task.

[0077] Therefore, computing tasks at the fourth resource isolation level are positioned as low-priority mixed deployment tasks, using the remaining CPU resources of other computing tasks. When multiple computing tasks use the same CPU core, the CPU core computing power will be preferentially allocated to different computing tasks based on usage weights. The CPU core computing power usage weight of computing tasks at the fourth resource isolation level is the lowest. When they are mixed with computing tasks at the second and / or third resource isolation levels, they can obtain less CPU core computing power, which will not affect the mixed computing tasks at higher resource isolation levels.

[0078] Figure 3 A schematic diagram of the structure of task binding cores provided in an embodiment of the present disclosure. Assume that the user starts computing tasks 1 to 11, where computing tasks 1 and 2 belong to the first resource isolation level, computing tasks 3 to 5 belong to the second resource isolation level, and computing tasks 6 to 8 belong to the third resource isolation level. There are 16 CPU cores available for allocation. Please refer to Figure 3 As shown, computing tasks 1 and 2 are bound to CPU cores 1 and 2 respectively, and they exclusively occupy their corresponding CPU cores; similarly, computing tasks 3 and 4 are bound to CPU cores 3 and 4 respectively, and they exclusively occupy their corresponding CPU cores. The number of CPU cores required for computing task 5 is 2. Therefore, computing task 5 is bound to CPU cores 5 and 6 at the same time, and CPU cores 5 and 6 jointly execute computing task 5; computing tasks 6 to 8 share CPU cores 7 to 12 with each other. Figure 3In the illustrated embodiment, computing tasks 6 to 8 do not occupy resources of CPU cores 13 to 16, so CPU cores 13 to 16 can be considered as unallocated. Assuming that computing tasks 9 to 11 of the MIXED level are started, computing tasks 9 to 11 can share resources on CPU cores 3 to 16 with computing tasks 3 to 8. It should be understood that the CPU core computing power usage weight of computing tasks 9 to 11 for CPU cores 3 to 16 is lower than that of mixed running computing tasks, so as not to affect the execution of other mixed running computing tasks as much as possible.

[0079] In order to make the meaning of "sharing" CPU cores involved in the present disclosure clearer, an example is given in which computing tasks 6 to 8 share CPU cores 7 to 12 with each other in this embodiment. It is assumed that computing tasks 6 to 8 start a task process respectively, and the task process corresponding to computing task 6 is bound to CPU cores 7 to 12 respectively, and the task process corresponding to computing task 7 is bound to CPU cores 7 to 12 respectively, and the task process corresponding to computing task 8 is bound to CPU cores 7 to 12 respectively. Through scheduling, different computing tasks can use the same CPU core in a time-sharing manner. The CPU core computing power usage weights corresponding to computing tasks 6 to 8 are calculated based on the method of the above example and the number of CPU cores applied for by each computing task.

[0080] exist Figure 3 In the illustrated embodiment, the number of all CPU cores participating in the sharing is equal to the total number of CPU cores required for the corresponding computing tasks. For example, computing tasks 6 to 8 share CPU cores 7 to 12 with each other. Assuming that the computing resource requirement of computing task 6 is 1 CPU core, the computing resource requirement of computing task 7 is 2 CPU cores, and the computing resource requirement of computing task 8 is 3 CPU cores, the total computing resource requirement is 6 cores. Therefore, 6 cores are allocated to be shared by computing tasks 6 to 8. If the number of CPU cores participating in the sharing is less than the total number of CPU cores required for the corresponding computing tasks, insufficient resources may occur, resulting in low computing task processing efficiency. Therefore, when allocating CPU core resources, it is necessary to ensure that the number of all CPU cores participating in the sharing is equal to the total number of CPU cores required for the corresponding computing tasks.

[0081] Combination Figure 3 It should be understood that the illustrated embodiment performs task management by means of the above-mentioned task classification and core binding method, which can ensure that multiple types of computing tasks can be run simultaneously on the same computing node, which can not only meet the performance requirements of the computing tasks, but also prevent the computing tasks from interfering with each other, and is conducive to improving the overall utilization of cluster resources.

[0082] Assume that the cluster supports online services, big data streaming computing tasks, model training tasks, and big data batch computing tasks with three priorities: high, medium, and low. The cluster also supports four resource isolation levels (as shown above).

[0083] Among them, ALONE-level computing tasks are bound to specific CPU cores and exclusively occupy the CPU cores, which are suitable for computing tasks with extremely high requirements for stability, such as online services.

[0084] Among them, FIXED-level computing tasks are bound to specific CPU cores and can be mixed with MIXED-level computing tasks. They are suitable for other computing tasks that hope to improve performance by binding tasks to cores, but have relatively lower stability requirements than ALONE-level computing tasks, such as big data streaming computing tasks, model training tasks, high-priority big data batch computing tasks, etc. When the FIXED-level computing tasks use low-peak CPU resources, the mixed MIXED-level computing tasks fill the resource gap, thereby ensuring the overall resource utilization of the cluster.

[0085] NORMAL-level computing tasks are not bound to specific CPU cores and are suitable for ordinary computing tasks that do not have too many requirements for the operating environment but need to ensure the CPU core computing power, such as medium and low-priority big data batch computing tasks.

[0086] Figure 4 This is a flowchart of a task management method provided by another embodiment of the present disclosure. Figure 4 As shown, including:

[0087] S401. Receive a task request, and determine the resource isolation level and computing resource requirements corresponding to the task to be processed based on the indication information contained in the task request, wherein the resource isolation level is used to indicate the CPU isolation strategy corresponding to the processing task; the computing resource requirements include the requirements of the task to be processed for the CPU core.

[0088] S402: Allocate corresponding target CPU cores for the tasks to be processed according to the resource isolation level, computing resource requirements, and CPU core allocation on each computing node.

[0089] S403: Start a task process corresponding to the task to be processed, and bind the target CPU core to the task process.

[0090] In this embodiment, steps S401 to S403 are respectively Figure 2 Steps S201 to S203 of the embodiment shown are similar, and can be referred to Figure 2 For the sake of brevity, the detailed description of the illustrated embodiment will not be repeated here.

[0091] S404: configure one or more of the CPU core computing power usage weight, CPU usage limit, memory usage limit, disk usage limit, and network throughput usage limit corresponding to the task to be processed.

[0092] In addition to task-level core binding, different types of computing tasks also need to limit CPU core computing power usage weights, CPU usage, memory usage, disk usage, network throughput usage, etc., so as to achieve comprehensive resource isolation and better ensure that different types of computing tasks can run simultaneously on computing nodes.

[0093] The CPU core computing power usage weight, CPU usage limit, memory usage limit, disk usage limit and network throughput usage limit information corresponding to the pending task can be specified by the user through the client when submitting the task request, or can be automatically determined by the cluster system according to the resource isolation level of the pending task, wherein the automatic determination can be achieved but is not limited to by querying a data table, and the data table can record the mapping relationship between different resource isolation levels and the CPU core computing power usage weight, CPU usage limit, memory usage limit, disk usage limit and network throughput usage limit information.

[0094] S405. The task process corresponding to the task to be processed uses the target CPU core to execute the task to be processed, and the CPU core computing power, memory resources, disk resources and network throughput scheduled when the target CPU core executes the task to be processed meet corresponding restrictions.

[0095] By configuring the above-mentioned restriction conditions, this embodiment can better ensure resource isolation when multiple computing tasks are running in the same computing node, ensure that each computing task is executed reliably, and maximize the resource utilization of the computing node.

[0096] It should be noted that the steps can be executed in the order shown in this embodiment, or step S404 can be executed first and then step S403, or, after allocating the target CPU core to the task to be processed, step S403 and step S404 can be executed in parallel. The present disclosure does not limit the order of execution.

[0097] Optionally, in Figure 4 Based on the embodiment, it also includes:

[0098] S406: After the execution of the pending task is completed, the target CPU core is unbound from the task process corresponding to the pending task, and the target CPU core is released.

[0099] After the pending task is executed, the label of the target CPU core is changed from assigned label to unassigned label through the stand-alone service, and the label of the pending task is deleted, thereby releasing the target CPU core. By releasing resources in time after the task is executed, the effective utilization of cluster resources can be ensured.

[0100] In addition, after the stand-alone service releases the target CPU core, it can also report the latest CPU core information to the central service in a timely manner to ensure that the CPU core information of each computing node recorded by the central service is consistent with the actual situation.

[0101] In the aforementioned Figures 2 to 4 On the basis of the illustrated embodiment, the cluster also supports the migration of computing tasks of a specified resource isolation level between CPU cores, thereby fully utilizing the computing resources of the computing nodes in the cluster.

[0102] In some embodiments, when the central service determines that the migration conditions are met based on the CPU core information of the target computing node, it can send a migration instruction to the stand-alone service, instructing the stand-alone service to migrate computing tasks of the specified resource isolation level to a CPU core with a load less than the currently bound one.

[0103] In some other embodiments, the computing node may determine based on its own CPU core information that the computing task is migrated when the migration condition is met. The migration information of the stand-alone service to the central service after the migration is executed in any of the above methods, and the migration information may include the identifier of the CPU core after the migration and the identifier of the computing task.

[0104] Exemplarily, the designated resource isolation may be the lowest resource isolation level, such as the MIXED level. Since the lowest resource isolation level has the lowest CPU core right usage weight, flexible migration can ensure that as many computing resources as possible are scheduled for computing tasks at the lowest resource isolation level.

[0105] In addition, during scheduling, the CPU cores that can be migrated can be sorted in order from low to high load, and computing tasks are preferentially migrated to the CPU core with the lowest load. Taking MIXED-level computing tasks as an example, computing tasks are preferentially migrated to idle CPU cores, followed by running them mixed with NORMAL-level computing tasks, and then mixed with FIXED-level computing tasks. For example, refer to Figure 5 In the illustrated embodiment, computing task 5 and computing task 9 are mixed running in CPU cores 5 and 6, and computing tasks 6 to 8 and computing tasks 10 and 11 are mixed running in CPU cores 7 to 12. Then, computing task 9 and computing tasks 10 and 11 are migrated to idle CPU cores 13 to 16, and computing tasks 9 to 11 share CPU cores 13 to 16, thereby improving the utilization rate of idle CPU resources and the processing efficiency of computing tasks.

[0106] Figure 6 This is a schematic diagram of the structure of a task management device provided by an embodiment of the present disclosure. Figure 6 As shown, the device 600 of this embodiment includes:

[0107] The receiving module 601 is used to receive a task request.

[0108] The resource management module 602 is used to determine the resource isolation level and computing resource requirements corresponding to the task to be processed based on the indication information included in the task request; wherein the resource isolation level is used to indicate the CPU isolation policy corresponding to the task to be processed; and the computing resource requirements include the requirements of the task to be processed for the CPU core.

[0109] The resource management module 602 is used to allocate a corresponding target CPU core to the task to be processed according to the resource isolation level, the computing resource requirements and the CPU core allocation on each computing node; and to start the task process corresponding to the task to be processed, bind the target CPU core to the task process, and the task process corresponding to the task to be processed uses the target CPU core to execute the task to be processed.

[0110] In some embodiments, the resource management module 602 is specifically used to determine, based on the CPU core allocation on each computing node, a computing node whose allocatable CPU core resources meet the computing resource requirements as the target computing node corresponding to the task to be processed; and control the target computing node to determine, based on the resource isolation level, the computing resource requirements and the CPU core allocation corresponding to the target computing node, the target CPU core that meets the corresponding CPU isolation policy and computing resource requirements from the CPU cores included in the target computing node; and add an allocated label and a label corresponding to the task to be processed to the target CPU core to allocate the target CPU core to the task to be processed.

[0111] In some embodiments, the resource management module 602 is specifically configured to:

[0112] If the task to be processed is at the first resource isolation level, determining the target CPU core from the idle CPU cores of the target computing node; wherein the task to be processed exclusively occupies the target CPU core;

[0113] If the task to be processed is at the second resource isolation level, the target CPU core is determined from the target computing node including idle CPU cores or CPU cores bound to tasks at the specified resource isolation level; the task to be processed exclusively occupies the target CPU core or shares the target CPU core with the tasks at the specified resource isolation level;

[0114] If the task to be processed corresponds to the third resource isolation level, determining the target CPU core from the CPU cores included in the target computing node that are not occupied by tasks at the first resource isolation level and the second resource isolation level; wherein the target CPU core allows binding of other computing tasks at the third resource isolation level;

[0115] If the task to be processed corresponds to the fourth resource isolation level, the target CPU core is determined from the CPU cores included in the target computing node that are not occupied by tasks at the first resource isolation level; wherein the target CPU core allows the binding of other computing tasks at the second to fourth resource isolation levels to share the target CPU core.

[0116] In some embodiments, after allocating a corresponding target CPU core to the task to be processed, the resource management module 602 is also used to configure one or more of the CPU core computing power usage weight, CPU usage limit, memory usage limit, disk usage limit and network throughput usage limit for the task to be processed according to the resource isolation level corresponding to the task to be processed, so that the CPU core computing power, memory resources, disk resources and network throughput scheduled when the task process uses the target CPU core to execute the task to be processed meet the corresponding restrictions.

[0117] In some embodiments, the resource management module 602 is further used to obtain the CPU core allocation status reported by each computing node.

[0118] In some embodiments, the resource management module 602 is also used to: when the task to be processed corresponds to the lowest resource isolation level, based on the CPU core allocation of each computing node, migrate the task to be processed to a CPU core whose load is less than the load of the target CPU core for execution, and the load is used to indicate the number of tasks bound to the CPU core.

[0119] In some embodiments, the resource management module 602 is further used to unbind the target CPU core from the task process corresponding to the task to be processed after the execution of the task to be processed is completed, thereby releasing the target CPU core.

[0120] The task management device provided in this embodiment is used to execute the technical solution of any of the aforementioned method embodiments. Its implementation method is similar to that of the aforementioned method embodiments. Please refer to the detailed description of the aforementioned method embodiments. For the sake of brevity, it will not be repeated here.

[0121] Exemplarily, the present disclosure provides an electronic device, comprising: one or more processors; a memory; and one or more computer programs; wherein the one or more computer programs are stored in the memory; when the one or more processors execute the one or more computer programs, the electronic device implements the task management method of the foregoing embodiments.

[0122] Exemplarily, the present disclosure provides a chip system, which is applied to an electronic device including a memory and a sensor; the chip system includes: a processor; when the processor executes the task management method of the above embodiment.

[0123] Exemplarily, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor in an electronic device, the task management method of the foregoing embodiment is implemented.

[0124] Exemplarily, the present disclosure provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the task management method of the foregoing embodiment.

[0125] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0126] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A task management method, characterized in that: include: Receive a task request, and determine the resource isolation level and computing resource requirement corresponding to the task to be processed based on the indication information contained in the task request; wherein the resource isolation level is used to indicate the central processing unit CPU isolation strategy corresponding to the task to be processed; the computing resource requirement includes the requirement of the task to be processed for the CPU core; Allocate a corresponding target CPU core to the task to be processed according to the resource isolation level, the computing resource requirements, and the CPU core allocation on each computing node; Starting a task process corresponding to the task to be processed, binding the target CPU core to the task process, and allowing the task process corresponding to the task to be processed to use the target CPU core to execute the task to be processed; The allocating a corresponding target CPU core to the task to be processed according to the resource isolation level, the computing resource requirement, and the CPU core allocation status on each computing node includes: If the task to be processed is at the first resource isolation level, determining the target CPU core from the idle CPU cores of the target computing node; wherein the task to be processed exclusively occupies the target CPU core; If the task to be processed is at the second resource isolation level, the target CPU core is determined from the target computing node including idle CPU cores or CPU cores bound to tasks at the specified resource isolation level; the task to be processed exclusively occupies the target CPU core or shares the target CPU core with the tasks at the specified resource isolation level; If the task to be processed corresponds to the third resource isolation level, determining the target CPU core from the CPU cores included in the target computing node that are not occupied by tasks at the first resource isolation level and the second resource isolation level; wherein the target CPU core allows binding of other computing tasks at the third resource isolation level; If the task to be processed corresponds to the fourth resource isolation level, the target CPU core is determined from the CPU cores included in the target computing node that are not occupied by tasks at the first resource isolation level; wherein the target CPU core allows the binding of other computing tasks at the second to fourth resource isolation levels to share the target CPU core.

2. The method according to claim 1, characterized in that The allocating a corresponding target CPU core to the task to be processed according to the resource isolation level, the computing resource requirement, and the CPU core allocation status on each computing node includes: Based on the CPU core allocation on each computing node, determine the computing node whose allocatable CPU core resources meet the computing resource requirements as the target computing node corresponding to the task to be processed; Determine the target CPU core from the CPU cores included in the target computing node according to the resource isolation level, the computing resource demand, and the CPU core allocation corresponding to the target computing node; Adding the assigned label and the label of the task to be processed to the target CPU core to assign the target CPU core to the task to be processed.

3. The method according to claim 1, characterized in that After allocating the corresponding target CPU core to the task to be processed, the method further includes: Configure one or more of the CPU core computing power usage weight, CPU usage limit, memory usage limit, disk usage limit, and network throughput usage limit corresponding to the task to be processed, so that the CPU core computing power, memory resources, disk resources, and network throughput scheduled when the task process uses the target CPU core to execute the task to be processed meet the corresponding restrictions.

4. The method according to claim 2, characterized in that: Before determining the target computing node corresponding to the task to be processed according to the computing resource requirements corresponding to the task to be processed and the CPU core information of each computing node, the method further includes: Receive the CPU core allocation status reported by each computing node.

5. The method according to claim 1, characterized in that Also includes: When the pending task corresponds to the lowest resource isolation level, based on the CPU core allocation of each computing node, the pending task is migrated to a CPU core whose load is less than the load of the target CPU core for execution, and the load is used to indicate the number of tasks bound to the CPU core.

6. The method according to any one of claims 1 to 5, characterized in that: Also includes: After the execution of the task to be processed is completed, the target CPU core is unbound from the task process corresponding to the task to be processed, and the target CPU core is released.

7. A task management device, characterized in that: include: A receiving module, used for receiving a task request; A resource management module, used to determine the resource isolation level and computing resource requirements corresponding to the task to be processed based on the indication information contained in the task request; wherein the resource isolation level is used to indicate the central processing unit CPU isolation strategy corresponding to the task to be processed; the computing resource requirements include the requirements of the task to be processed for the CPU core; The resource management module is used to allocate a corresponding target CPU core to the task to be processed according to the resource isolation level, the computing resource demand and the CPU core allocation on each computing node; and start the task process corresponding to the task to be processed, bind the target CPU core to the task process, and the task process corresponding to the task to be processed uses the target CPU core to execute the task to be processed; The resource management module is specifically used for: If the task to be processed is at the first resource isolation level, determining the target CPU core from the idle CPU cores of the target computing node; wherein the task to be processed exclusively occupies the target CPU core; If the task to be processed is at the second resource isolation level, the target CPU core is determined from the target computing node including idle CPU cores or CPU cores bound to tasks at the specified resource isolation level; the task to be processed exclusively occupies the target CPU core or shares the target CPU core with the tasks at the specified resource isolation level; If the task to be processed corresponds to the third resource isolation level, determining the target CPU core from the CPU cores included in the target computing node that are not occupied by tasks at the first resource isolation level and the second resource isolation level; wherein the target CPU core allows binding of other computing tasks at the third resource isolation level; If the task to be processed corresponds to the fourth resource isolation level, the target CPU core is determined from the CPU cores included in the target computing node that are not occupied by tasks at the first resource isolation level; wherein the target CPU core allows the binding of other computing tasks at the second to fourth resource isolation levels to share the target CPU core.

8. An electronic device, characterized in that: include: Memory and processor; The memory is configured to store computer program instructions; The processor is configured to execute the computer program instructions so that the electronic device implements the task management method according to any one of claims 1 to 6.

9. A readable storage medium, characterized in that: include: Computer program instructions; The electronic device executes the computer program instructions so that the electronic device implements the task management method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that The electronic device executes the computer program product so that the electronic device implements the task management method according to any one of claims 1 to 6.

Images