Computing cluster task management method and device

Abstract

The invention discloses a computing cluster task management method and a computing cluster task management device, relates to the field of computers, and realizes task scheduling which is wide in application scene, small in transmission data volume, not limited by network bandwidth and capable of reducing tail delay. The method can be applied to a computing cluster comprising a plurality of computing nodes and a shared memory, and the computing nodes are used for executing a plurality of tasks during application running and can access the shared memory. The method can comprise the steps that a first computing node in a plurality of computing nodes in a computing cluster queries global task information in a shared memory after executing a task of an application, determines a second computing node to be subjected to the task, and sends an execution request to the second computing node; and determining a to-be-executed first task by the second computing node, sending metadata of data corresponding to the first task to the first computing node, and executing the first task by the first computing node.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to a method and device for managing computing cluster tasks. Background Art

[0002] In parallel computing environments, task scheduling is often used to balance tasks across different queues to achieve load balancing. Work stealing is a load balancing algorithm used in task scheduling. In a parallel computing environment, each processor or thread has its own task queue. When a thread completes all tasks in its task queue, it can "steal" tasks from other task queues to execute. This allows for dynamic workload balancing, improving overall system performance and efficiency while preventing tail latency (delayed responses that are higher than the average).

[0003] In a multi-threaded parallel scenario, each thread can be considered a worker, which pulls jobs from its own job queue for execution. When a worker's job queue is empty, it can "steal" jobs from another worker's queue. This process requires a unified memory view (used to record the data information corresponding to the tasks executed by each thread). Each thread uses this unified memory view to obtain data information from other threads to implement task scheduling between threads. Therefore, this task scheduling process can only be applied within a process running on a single node, that is, task scheduling between threads within a process, limiting its application scenarios.

[0004] In distributed parallel computing scenarios, task scheduling is implemented for a wide range of applications, enabling tasks to be scheduled between compute nodes. Multiple workers can be launched based on the number of nodes. Each worker can have its own job queue. When a worker's job queue is empty, it initiates work theft from workers on other nodes and transfers data from the remote node (the stolen node) to its own node for processing. However, due to the large amount of data required to be transmitted between nodes, the effectiveness of task scheduling is limited by network bandwidth. This can lead to network transmission overhead exceeding computational overhead, data skew, and stagnant nodes causing tail latency. Summary of the Invention

[0005] The present application provides a computing cluster task management method and device, which realizes task scheduling with wide application scenarios, small data transmission volume, no network bandwidth limitation, and reduced tail delay.

[0006] To achieve the above objectives, the present invention adopts the following technical solutions:

[0007] In a first aspect, a computing cluster task management method is provided. The method can be applied to a computing cluster, wherein the computing cluster includes multiple computing nodes and shared memory. The multiple computing nodes are used to execute multiple tasks during application runtime and can all access the shared memory. The shared memory stores the data and global task information required by each computing node to execute multiple tasks during application runtime. The global task information records the metadata of the data corresponding to the application tasks executed by each computing node. The method may include: after executing the task of the application, a first computing node among the multiple computing nodes in the computing cluster queries the global task information in the shared memory to determine a second computing node that has a task to execute; then, the first computing node sends an execution request to the second computing node; the second computing node determines the first task to be executed, obtains the metadata of the data corresponding to the first task from the global task information, and sends the metadata to the first computing node; then, the first computing node obtains the data corresponding to the first task from the shared memory based on the metadata of the data corresponding to the first task received, and executes the first task based on the obtained data.

[0008] The solution provided by this application enables task scheduling between nodes, as multiple computing nodes in a computing cluster can access the shared memory storing the data required to execute tasks. This allows the solution to be applied in a wide range of scenarios, without being limited to a single node. Furthermore, when scheduling tasks between computing nodes, metadata about the data corresponding to the task is transmitted, replacing the data itself. This significantly reduces the amount of data transmitted when retrieving tasks, freeing task scheduling from network bandwidth constraints. This ensures that network transmission overhead is significantly less than computing overhead, reducing the occurrence of tail latency.

[0009] In one possible implementation, the method provided by this application may further include: after the second computing node sends the metadata of the data corresponding to the first task to the first computing node, deleting or identifying the metadata of the data corresponding to the first task from the metadata of the data corresponding to the task executed by the second computing node in the global task information; and after the first computing node receives the metadata of the data corresponding to the first task, adding the metadata of the data corresponding to the first task to the metadata of the data corresponding to the task executed by the first computing node in the global task information. This ensures that the data of a task is accessed by only one computing node at a time, thereby ensuring the accuracy of task processing.

[0010] In another possible implementation, the method provided in this application may further include: after receiving the execution request, the second computing node evaluates the time it takes for the second computing node to execute the first task, and when the time is greater than a preset value, sending metadata of the data corresponding to the first task to the first computing node. If the second computing node evaluates that the first task will continue to be processed within the node for too long, the metadata of the data corresponding to the first task will be sent to the first computing node, and the first computing node will execute the first task, thereby improving the efficiency of executing the first task and further reducing the probability of tail delay in the computing cluster.

[0011] In another possible implementation, the second computing node determining the first task to be executed can be specifically implemented as follows: the second computing node determining the second task to be executed; and splitting the first task from the second task according to a preset strategy. This reduces the granularity of task scheduling, further reduces the transmission overhead caused by task scheduling, and improves the effectiveness of task scheduling.

[0012] In another possible implementation, the second task to be executed may be the task that comes first in a preset order among the tasks to be executed by the second computing node.

[0013] In another possible implementation, the preset order may be an order from high priority to low priority, or an order from low priority to high priority.

[0014] In another possible implementation, the preset order may be an order from large to small corresponding data amounts, or an order from small to large corresponding data amounts.

[0015] In another possible implementation, the preset strategy may be: splitting a task into N subtasks evenly, where N is greater than or equal to 2.

[0016] In another possible implementation, the above N may be the number of first computing nodes, that is, N is the number of computing nodes that initiate the execution request.

[0017] Another possible implementation is to use the metadata in the form of handles. Transferring data in the form of handles, i.e., moving handles instead of moving data, reduces data copying and redundant data, enhances data reuse, reduces data movement, avoids frequent memory requests, and reduces the impact of memory operations. It also eliminates serialization and deserialization, reducing I / O overhead. Because the granularity of tasks and their data is much larger than handles, network transmission overhead is much smaller than computational overhead during task scheduling, avoiding data skew and reducing tail latency.

[0018] In another possible implementation, a handle can include one or more descriptors, which indicate the characteristics of a storage location in shared memory. The characteristics indicated by the descriptors in the handle identify a unique area in shared memory. The data stored in this area is the data corresponding to the task to which the handle belongs.

[0019] The descriptor may be address information in the shared memory, the size of the memory area, or other information.

[0020] In another possible implementation, data is stored in the shared memory using a persistent data structure to increase information density and reduce memory usage.

[0021] In a second aspect, a computing cluster task management device is provided. The device is applied to a computing cluster, wherein the computing cluster includes multiple computing nodes and shared memory. The multiple computing nodes are used to execute multiple tasks during application runtime and can all access the shared memory. The shared memory stores the data and global task information required by each computing node to execute multiple tasks during application runtime. The global task information records the metadata of the data corresponding to the application tasks executed by each computing node. The device may include: a determination unit, a sending unit, a first processing unit, and a second processing unit. Among them:

[0022] The determining unit is used to query global task information after the first computing node completes executing the application task, and determine the second computing node that has yet to execute the task.

[0023] A sending unit is used to send an execution request to the second computing node.

[0024] The first processing unit is configured to determine a first task to be executed, obtain metadata of data corresponding to the first task from global task information, and send the metadata to the first computing node.

[0025] The second processing unit is configured to obtain data corresponding to the first task from the shared memory according to the metadata, and execute the first task according to the data.

[0026] In another possible implementation, the apparatus further includes a configuration unit configured to: after the first processing unit sends the metadata of the data corresponding to the first task to the first computing node, delete or identify the metadata of the data corresponding to the first task from the metadata of the data corresponding to the task executed by the second computing node in the global task information. After the second processing unit receives the metadata of the data corresponding to the first task, add the metadata of the data corresponding to the first task to the metadata of the data corresponding to the task executed by the first computing node in the global task information.

[0027] In another possible implementation, the apparatus may further include an evaluation unit configured to, upon receiving the execution request, evaluate a time for the second computing node to execute the first task. The first processing unit may also be configured to, when the time is greater than a preset value, send metadata of the data corresponding to the first task to the first computing node.

[0028] In another possible implementation, the first processing unit is specifically configured to: determine a second task to be executed; and separate the first task from the second task according to a preset strategy.

[0029] In a third aspect, a computing cluster task management system is provided, comprising multiple computing nodes and shared memory. The multiple computing nodes are used to execute multiple tasks during application runtime and can all access the shared memory. The shared memory stores data required by each computing node to execute multiple tasks during application runtime, as well as global task information. The global task information records metadata of the data corresponding to the application tasks executed by each computing node. Specifically:

[0030] The first computing node among the multiple computing nodes is used to: after executing the application task, query global task information to determine the second computing node that has tasks to be executed; and send an execution request to the second computing node.

[0031] The second computing node is used to: determine the first task to be executed, obtain metadata of data corresponding to the first task from the global task information, and send the metadata of the data corresponding to the first task to the first computing node.

[0032] The first computing node is further configured to obtain data corresponding to the first task from the shared memory according to metadata of the data corresponding to the first task, and execute the first task according to the data.

[0033] In one possible implementation, the second computing node is further configured to: after sending the metadata of the data corresponding to the first task to the first computing node, delete or identify the metadata of the data corresponding to the first task from the metadata of the data corresponding to the task executed by the second computing node in the global task information. The first computing node is further configured to: after receiving the metadata of the data corresponding to the first task, add the metadata of the data corresponding to the first task to the metadata of the data corresponding to the task executed by the first computing node in the global task information.

[0034] In another possible implementation, the second computing node is further used to evaluate the time it takes for the second computing node to execute the first task after receiving the execution request, and when the time is greater than a preset value, send metadata of the data corresponding to the first task to the first computing node.

[0035] In another possible implementation, the second computing node is specifically configured to: determine a second task to be executed; and split the first task from the second task according to a preset strategy.

[0036] In a fourth aspect, a computing device is provided, which includes a memory and a processor, the memory being used to store a set of computer instructions; when the processor executes the set of computer instructions, the processor executes the operating steps of the first computing node or the second computing node in the method described in the first aspect or any possible implementation method.

[0037] In a fifth aspect, a computer-readable storage medium is provided, comprising: computer software instructions; when the computer software instructions are executed in a processor, the processor executes the operating steps of the method described in the first aspect or any possible implementation method.

[0038] In a sixth aspect, a computer program product is provided, comprising: when the computer program product is run on a computer, causing the computer to execute the operating steps of the method described in the first aspect or any possible implementation method.

[0039] The solutions provided in the second to sixth aspects above are used to implement the method provided in the first aspect above or any possible implementation method, and their specific implementations will not be described in detail one by one.

[0040] It should be noted that various possible implementations of any of the above aspects can be combined under the premise that the solutions are not contradictory. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A schematic diagram of a work-stealing technique for a multi-threaded computing model;

[0042] Figure 2A schematic diagram of the structure of a computing cluster system provided in an embodiment of the present application;

[0043] Figure 3 A flowchart of a computing cluster task management method provided in an embodiment of the present application;

[0044] Figure 4 A schematic diagram of global task information provided in an embodiment of the present application;

[0045] Figure 5 A flowchart of another computing cluster task management method provided in an embodiment of the present application;

[0046] Figure 6 A schematic diagram of an application scenario of a task scheduling method provided in an embodiment of the present application;

[0047] Figure 7 A schematic diagram comparing different data transmission processes of different transmission modes when transmitting the same set of data provided in an embodiment of the present application;

[0048] Figure 8 A schematic diagram of the structure of a computing cluster task management device provided in an embodiment of the present application;

[0049] Figure 9 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In the embodiments of the present application, in order to clearly describe the technical solutions of the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity or execution order, and words such as "first" and "second" do not necessarily mean different. There is no order of precedence or priority between the technical features described by "first" and "second".

[0051] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner to facilitate understanding.

[0052] In the embodiments of the present application, at least one can also be described as one or more, and multiple can be two, three, four or more, which is not limited in this application.

[0053] In addition, the network architecture and scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0054] To facilitate understanding, the terms involved in the embodiments of this application are first explained.

[0055] A distributed cluster is a system consisting of multiple computer nodes that communicate and collaborate over a network to complete a task or provide a service. Distributed clusters offer advantages such as high availability, high performance, and scalability, making them widely used in large-scale computing, storage, and processing scenarios.

[0056] A job refers to a big data business submitted by a user. A job can be divided into multiple tasks, which are executed in parallel by multiple compute nodes. Each task is a piece of execution logic. Depending on the complexity of the execution logic, the execution time varies. Tasks may have logical or data dependencies, and a task may generate subtasks during execution. A task is generally a process for processing a portion of data or a stage within a job. All tasks are scheduled and completed in parallel or serially.

[0057] Work stealing is a dynamic task scheduling mechanism that steals tasks from the queues associated with non-idle processor cores when the queue associated with a processor core is empty. A processor core can act as the owner of its associated queue. A processor core or the threads running on a processor core are called workers, and workers can play different roles when performing different operations.

[0058] Shared memory, also known as global memory or the global memory pool, refers to a persistent memory space that is accessible and usable by all threads, processes, or kernels during program execution. For example, the local storage media within compute nodes in a computer cluster can form shared memory.

[0059] As mentioned above, work stealing is a load balancing algorithm for task scheduling. Current industry-related technologies include work stealing for multi-core or multi-threaded computing models, and work stealing for distributed parallel computing (data grids or online transaction processing (OLTP) databases). These are briefly described below.

[0060] Work stealing techniques in multi-threaded parallel computing models are as follows Figure 1 As shown in the schematic scenario, each thread can be considered a worker. Each worker has its own task queue (job queue). Workers pull jobs from their own job queues for execution. When a worker's job queue is empty (e.g. Figure 1 After worker2 in , the worker (such as Figure 1 Worker2 in the Figure 1 steal jobs from the queue of worker1 (e.g. Figure 1 Job3) in is executed.

[0061] exist Figure 1 In the illustrated scenario, a unified memory view is required (for recording the data information corresponding to the tasks executed by each thread). Each thread obtains the data information of other threads through this unified memory view to implement task scheduling between threads. Therefore, this technology can usually only be used for task scheduling between multiple threads within a process deployed on a single node (for example, in scenarios with non-uniform memory access (NUMA)). The application scenarios are limited. Figure 1 The illustrated solution is generally not applicable to distributed scenarios such as online analytical processing (OLAP) or OLTP databases.

[0062] Work-stealing technology in distributed parallel computing scenarios enables task scheduling in a wide range of application scenarios and can schedule tasks between computing nodes. However, a large amount of data needs to be transmitted between nodes, and the effectiveness of task scheduling is limited by network bandwidth. There may be problems such as network transmission overhead exceeding computing overhead, data skew still exists, and some nodes are still in a stagnant state, causing tail delays.

[0063] Based on this, the present application provides a method for managing computing cluster tasks, in which multiple computing nodes in the computing cluster can access the shared memory for storing the data required to execute tasks. When scheduling tasks, the metadata of the data corresponding to the task is transmitted, and the task scheduling can be realized between nodes, making the application scenarios of the solution wide and not limited to a single node. At the same time, when scheduling tasks between computing nodes, the metadata of the data corresponding to the task is transmitted, replacing the transmitted data itself, greatly reducing the amount of data transmitted when calling tasks, so that task scheduling can be unrestricted by network bandwidth, ensuring that the network transmission overhead is much less than the computing overhead, and reducing the occurrence of tail delay.

[0064] The solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0065] The solution provided in this application can be applied to Figure 2 In the illustrated computing cluster system. Figure 2 As shown, the computing cluster system includes multiple computing nodes 201 , a shared memory 202 and a distributed file system 203 .

[0066] A user communicates with computing node 201 via a network. For example, a user sends a service request to computing node 201 via the network, requesting that computing node 201 perform distributed processing on the service data included in the service request. The network can be an internal enterprise network (e.g., a local area network (LAN)) or the Internet.

[0067] The computing node 201 is a computing device, such as a server, a desktop computer, or a controller of a storage array, etc. In some embodiments, each computing node 201 may be an independent physical device, and the computing node 201 may be referred to as a computing device or a data node.

[0068] The data corresponding to the task (the data required to execute the task) is stored in the shared memory 202 , and the computing node 201 executes the task by accessing the data in the shared memory 202 . Figure 2 In the computing cluster system shown, all computing nodes 201 can access the shared memory pool ( Figure 2 The specific hardware details of the shared memory pool are not specifically limited in this embodiment of the application. For example, the shared memory 202 can be based on Intel TM It is implemented using technologies such as CXL (Compute Express Link).

[0069] Figure 2 The illustrated computing cluster system supports running applications such as big data, databases, high-performance computing, artificial intelligence, distributed storage, and cloud native. For example, it can be applied to general computing fields with high concurrency. For example, Figure 2 The illustrated computing cluster system may be a distributed computing cluster based on engines such as Apache Spark and Apache Ignite. The embodiment of the present application does not limit the implementation method of the computing cluster.

[0070] On the one hand, the embodiment of the present application provides a computing cluster task management method, which is applied to a computing cluster, wherein the computing cluster includes multiple computing nodes and shared memory. For example, the computing cluster system can be Figure 2 Schematic diagram of a computing cluster system.

[0071] In a computing cluster, applications provide external functionality. Multiple applications can run within a computing cluster, providing diverse functionality. Specifically, applications are divided into multiple tasks, which are distributed and executed by the compute nodes in the computing cluster. Each node in the computing cluster can execute a different number of tasks, and the performance of each compute node can also vary. To ensure that tasks assigned to an application are executed as quickly as possible, thereby shortening application execution time, tasks of the same application executed by different compute nodes can be scheduled.

[0072] Each compute node in the computing cluster has its own task queue, which contains the tasks that the compute node is about to execute. Multiple compute nodes are used to execute multiple tasks (i.e., the tasks into which the application is divided) during the runtime of an application. Each compute node has access to shared memory, which stores the data each compute node needs to execute multiple tasks during the runtime of the application, as well as global task information. This global task information records the metadata of the data corresponding to the application tasks executed by each compute node.

[0073] The shared memory may include the storage medium of a computing node in a computing cluster system. The storage medium of a computing node includes at least one of a local storage medium within the computing node and an extended storage medium connected to the computing node. The embodiments of the present application do not limit the implementation form of the shared memory.

[0074] The operator on each computing node, also called the computing operator, has its own task queue, from which the operator obtains tasks for execution.

[0075] The operator described here can be an implementation form of the aforementioned worker. The operator can be a thread, process, or other, which can be understood as the granularity of task scheduling. The granularity can be configured by the user and is not limited in this embodiment of the application.

[0076] A unified memory management module is deployed in shared memory to manage global data and task distribution. For example, the unified memory management module can be defined as a unified memory driver. Every compute node in the compute cluster can call and execute this unified memory management module. Each operator (worker) in a compute node is deployed with a thread to execute the logic of the unified memory management module. In subsequent memory, threads within a compute node that execute the logic of the unified memory management module are referred to as unified memory management threads.

[0077] like Figure 3 As shown, the computing cluster task management method provided in the embodiment of the present application may include:

[0078] S301: After executing an application task, a first computing node among multiple computing nodes queries global task information to determine a second computing node that has yet to execute a task.

[0079] Specifically, the process of S301 may be performed by a unified memory management thread in the first computing node.

[0080] Among them, after the application tasks are executed, it can be understood that the task queue is empty.

[0081] Specifically, the first computing node may be a computing node to which an operator that has completed executing an application task belongs in a computing cluster system. The number of first computing nodes is not limited.

[0082] Illustratively, there may be one or more first computing nodes.

[0083] Specifically, in S301, the first computing node can query and manage global task information to determine which nodes still have tasks to be executed, that is, determine which computing nodes still have metadata of data corresponding to the tasks of the executed applications; and then select the second computing node that still has tasks to be executed.

[0084] In one possible implementation, the second computing node may be a computing node with the largest number of tasks to be executed, or the second computing node may be a computing node with the largest amount of data corresponding to the tasks to be executed.

[0085] In another possible implementation, the second computing node may be a plurality of computing nodes that have tasks to be executed.

[0086] The embodiment of the present application does not limit the selection principle of the second computing node.

[0087] In another possible implementation, the metadata may be in the form of a handle.

[0088] A handle consists of one or more descriptors, which indicate the characteristics of a storage location in shared memory. The characteristics indicated by the descriptors in a handle identify a unique area in shared memory. The data vector stored in this area is the data corresponding to the task to which the handle belongs.

[0089] For example, a handle stores descriptors for multiple sets of data, and operators access data through descriptors. The handle determines the granularity of the task.

[0090] Exemplarily, data is stored in the shared memory in a persistent data structure.

[0091] Furthermore, according to the content of the global task information record, it is indicated that a computing node only has the authority to read the metadata of the data corresponding to the task of the application executed by it.

[0092] In a possible implementation form, when the global task information records metadata of data corresponding to the task of the application executed by each computing node, it can be in the form of a table.

[0093] For example, Table 1 illustrates global task information. The metadata corresponding to a compute node in Table 1 is the metadata for the data corresponding to the application task executed by that compute node. For example, the metadata for the data corresponding to the application task executed by compute node 1 includes metadata a and metadata b. The metadata for the data corresponding to the application task executed by compute node 2 includes metadata c, metadata d, and metadata e.

[0094] Table 1

[0095]

[0096] In another possible implementation, the global task information may record metadata of data corresponding to the task of the application executed by each computing node in the form of a task queue. Figure 4 This diagram illustrates global task information, including the task queues of each compute node. The metadata included in a compute node's task queue is the metadata for the data corresponding to the application task executed by that compute node. For example, the metadata for the data corresponding to the application task executed by compute node 1 includes metadata a and metadata b. The metadata for the data corresponding to the application task executed by compute node 2 includes metadata c, metadata d, and metadata e.

[0097] Exemplarily, when the global task information records metadata of data corresponding to the application tasks executed by each computing node in the form of a task queue, the first computing node completing the application tasks in S301 may mean that the task queue of the first computing node is empty.

[0098] S302: The first computing node sends an execution request to the second computing node.

[0099] Specifically, the process of S302 may be performed by a unified memory management thread in the first computing node.

[0100] The execution request is used to schedule the task from the second computing node. The embodiment of the present application does not limit the content and form of the execution request.

[0101] Correspondingly, the second computing node receives the execution request.

[0102] S303: The second computing node determines the first task to be executed, obtains metadata of data corresponding to the first task from the global task information, and sends the metadata to the first computing node.

[0103] Specifically, the process of S303 may be performed by a unified memory management thread in the first computing node.

[0104] In a possible implementation, the second computing node may determine the first task to be executed according to a task scheduling policy.

[0105] Among them, the task scheduling strategy is a configuration rule, and the content of the task scheduling strategy can be configured according to actual needs, which is not limited in the embodiments of the present application.

[0106] For example, the task scheduling strategy may calculate the task that is first in a preset order among the tasks to be executed by the node.

[0107] In a possible implementation, the preset order may be an order from high priority to low priority, or an order from low priority to high priority.

[0108] In another possible implementation, the preset order may be an order from large to small corresponding data amounts, or an order from small to large corresponding data amounts.

[0109] In another possible implementation, the second computing node determines a second task to be executed; and then, splits the first task from the second task according to a preset strategy.

[0110] The second computing node may determine the second task to be executed from the tasks to be executed of the second computing node according to the aforementioned task scheduling strategy.

[0111] In a possible implementation, the preset strategy may be: splitting a task into N subtasks evenly, where N is greater than or equal to 2, and the first task is one of the subtasks.

[0112] In another possible implementation, the above N may be the number of first computing nodes, that is, N is the number of computing nodes that initiate the execution request.

[0113] In another possible implementation, N is 2, and the first task is half of the second task, that is, half of the second task is scheduled to the first computing node, and the other half remains in the second computing node for further execution.

[0114] S304: The first computing node obtains the data corresponding to the first task from the shared memory according to the received metadata of the data corresponding to the first task, and executes the first task according to the obtained data.

[0115] The solution provided by this application enables task scheduling between nodes, as multiple computing nodes in a computing cluster can access the shared memory storing the data required to execute tasks. This allows the solution to be applied in a wide range of scenarios, without being limited to a single node. Furthermore, when scheduling tasks between computing nodes, metadata about the data corresponding to the task is transmitted, replacing the data itself. This significantly reduces the amount of data transmitted when retrieving tasks, freeing task scheduling from network bandwidth constraints. This ensures that network transmission overhead is significantly less than computing overhead, reducing the occurrence of tail latency.

[0116] Furthermore, since the global task information records the metadata of the data corresponding to the task of the application executed by each computing node, the global task information needs to be updated after the task scheduling is performed, such as Figure 5 As shown, the computing cluster task management method provided in the embodiment of the present application may further include processes S305 and S306.

[0117] S305. After sending the metadata of the data corresponding to the first task to the first computing node, the second computing node deletes or identifies the metadata of the data corresponding to the first task from the metadata of the data corresponding to the task executed by the second computing node in the global task information.

[0118] Among them, identifying the metadata of the data corresponding to the first task means setting the status of the metadata of the data corresponding to the first task to unavailable, so that the second computing node no longer accesses the metadata of the data corresponding to the first task. The specific content of the identification can be configured according to actual needs, and the embodiment of the present application is not limited to this.

[0119] S306. After receiving the metadata of the data corresponding to the first task, the first computing node adds the metadata of the data corresponding to the first task to the metadata of the data corresponding to the task executed by the first computing node in the global task information.

[0120] For example, Figure 4 Taking the global task information shown in as an example, after scheduling the task to which the data corresponding to metadata e belongs to computing node p for execution, computing node 2 can delete metadata e from the task queue of computing node 2 in the global task information, and computing node p can add metadata e to the task queue of computing node p in the global task information.

[0121] Furthermore, in order to ensure the latency of the computing cluster, the task can be scheduled to other computing nodes for execution if the task takes too long to be processed in the current computing node. The computing cluster task management method provided in the embodiment of the present application may also include: after the second computing node receives the execution request, the time for the second computing node to execute the first task is evaluated.

[0122] When the time is greater than a preset value, the above-mentioned process of S303 is executed to send metadata of the data corresponding to the first task to the first computing node.

[0123] Among them, the preset value is the time threshold value for task scheduling, which can be configured according to actual needs and is not limited in this embodiment of the present application.

[0124] In a possible implementation, the second computing node may evaluate the time required for the second computing node to execute the first task based on the remaining data amount of the first task.

[0125] Exemplarily, the second computing node divides the remaining data volume of the first task by the data volume processed per unit time to obtain the time for the second computing node to execute the first task.

[0126] In another possible implementation, the second computing node may evaluate the time required for the second computing node to execute the first task based on the amount of remaining data between the currently processed task and the first task in the task queue.

[0127] Exemplarily, the second computing node divides the remaining amount of data between the currently processed task and the first task in the task queue by the amount of data processed per unit time to obtain the time it takes for the second computing node to execute the first task.

[0128] Of course, the specific implementation of the second computing node evaluating the time for executing the first task can be configured according to actual needs, and the embodiment of the present application is not limited to this.

[0129] Figure 6 This illustrates an application scenario of the solution provided by this application. Figure 6 The schematic scenario illustrates work stealing in big data scheduling algorithms.

[0130] like Figure 6As shown, the distributed computing cluster system includes three computing nodes (Node 0 to Node 2). The data corresponding to the tasks executed by the computing nodes during application runtime is stored in a shared memory pool, and all computing nodes have a unified understanding of the data in the shared memory. The operators in the computing nodes pull tasks from their own task queues for calculation. The unified memory management thread in each computing node implements the logic of the unified memory driver (Unified Memory Driver) to manage global data distribution and task distribution. The shared memory also stores global task information, recording the handles of the data corresponding to the tasks executed by each computing node during application runtime. The operators on Node 0, Node 1, and Node 2 have task queues. The operators obtain handles from the task queues for execution. The operator's task queue constitutes the aforementioned global task information. The handle stores descriptors for multiple sets of data, and the operator accesses the data through the descriptors. When the operator task queues of Node 0 and Node 2 are empty, Node 0 and Node 2 actively execute the logic of the UnifiedMemory Driver, determine that Node 1 still has tasks to execute, and initiate a work stealing request (the aforementioned execution request) to Node 1, dispatching the handle ( Figure 6 handle1) to Node 0 and Node 2.

[0131] Further, such as Figure 6 As shown in the figure, Node 1 splits handle1 into handle1' and handle1", schedules handle1' to the task queue of Node 0's operator, and schedules handle1" to the task queue of Node 2's operator. Handle1 scheduled on Node 1 is split into two new handles (handle1' and handle1"), maximizing the parallelism of the computation.

[0132] At the same time, Node 1 deletes handle1 from the metadata of the data corresponding to the task executed by Node 1 in the global task information, Node 0 adds handle1' to the metadata of the data corresponding to the task executed by Node 0 in the global task information, and Node 2 adds handle1" to the metadata of the data corresponding to the task executed by Node 2 in the global task information.

[0133] Distributing computing tasks at the handle granularity level supports task decomposition and reorganization, avoiding tail delays caused by the inability to decompose large-scale computing tasks and improving computing resource utilization. It also supports low-latency distributed work stealing and eliminates issues such as idle computing resources and low concurrency caused by data skew.

[0134] The effect of replacing data transmission with metadata transmission is described below. Figure 7 The figure illustrates the comparison of different data transmission processes of different transmission methods when transmitting the same set of data.

[0135] Figure 7 Figure (a) illustrates the traditional volcano model data transfer process. This process involves a large amount of data copying and redundancy. Each operator requires its own copy of the data (in the dashed box in the figure), and I / O consumes a large amount of computing and network resources.

[0136] Figure 7 (b) in the figure illustrates the process of transmitting data in the form of metadata. The upstream operator (collection operator (Collect)) initiates a next() request to the downstream operator, which is then passed layer by layer (through the aggregation operator (Aggregate) and projection operator (Project)) to the most downstream Scan operator. The Scan operator copies the data to the shared memory pool and obtains the data metadata. The Scan operator uses the data metadata as output and returns the next() request of the upstream operator. The operator processes the data metadata and returns the processing results to the upstream operator in the form of metadata. This process only transmits the data metadata, replacing the transmission of the data itself. There is no need to transmit large data copies, which reduces data copying and redundant data, enables operator memory sharing, and reduces the throughput bottleneck and latency caused by network I / O.

[0137] It is understood that in order to implement the functions in the above embodiments, the computing nodes include hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily appreciate that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, this application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in hardware or in a computer software-driven hardware manner depends on the specific application scenario and design constraints of the technical solution.

[0138] The following will be combined Figure 8 , describing the apparatus provided by this application. These apparatuses can be used to implement the functions of the first computing node or the second computing node in the above method embodiment, and thus can also achieve the beneficial effects possessed by the above method embodiment. In this embodiment, the apparatus can be Figure 2 The computing cluster shown in can also be a module (such as a chip) applied to a computer device.

[0139] like Figure 8 As shown, the computing cluster task management device 80 includes a determining unit 801 , a sending unit 802 , a first processing unit 803 and a second processing unit 804 .

[0140] The computing cluster task management device 80 is used to implement the functions of the first computing node and the second computing node in the above method embodiment.

[0141] Exemplarily, the determining unit 801 is used to implement the above Figure 3 or Figure 5 The sending unit 802 is used to implement the above process S301. Figure 3 or Figure 5 The first processing unit 803 is used to implement the above Figure 3 or Figure 5 The second processing unit 804 is used to implement the above Figure 3 or Figure 5 In process S304.

[0142] It should be understood that the computing cluster task management device 80 of the embodiment of the present application can be implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), and the PLD can be a complex programmable logical device (CPLD), a field programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof. It can also be implemented by software. Figure 3 or Figure 5 When the method is shown, its various modules may also be software modules, and the computing cluster task management device 80 and its various modules may also be software modules.

[0143] The computing cluster task management device 80 according to the embodiment of the present application may correspond to executing the method described in the embodiment of the present application, and the above and other operations and / or functions of each unit in the computing cluster task management device 80 are respectively to realize Figure 3 or Figure 5 For the sake of brevity, the corresponding processes of each method in are not repeated here.

[0144] Figure 9 This is a schematic diagram of the structure of a computing device 90 provided in this application. Figure 9As shown, computing device 90 includes processor 910 , bus 920 , memory 930 , communication interface 940 , and main memory 950 (also referred to as a main memory unit). Processor 910 , memory 930 , main memory 950 , and communication interface 940 are connected via bus 920 .

[0145] It should be understood that in this embodiment, the processor 910 may be a CPU, but may also be other general-purpose processors, digital signal processors (DSP), ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0146] The memories 950 of the multiple computing devices 90 may constitute the above-mentioned shared memory, which may be used to store the above-mentioned global task information.

[0147] The computing device 90 may also include a graphics processing unit (GPU), a neural network processing unit (NPU), a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the present application.

[0148] The communication interface 940 is used to implement communication between the computing device 90 and external devices or components.

[0149] The bus 920 may include a path for transmitting information between the above-mentioned components (such as the processor 910, the memory 950, and the storage 930). In addition to the data bus, the bus 920 may also include a power bus, a control bus, and a status signal bus. However, for the sake of clarity, various buses are labeled as bus 920 in the figure. The bus 920 may be a Peripheral Component Interconnect Express (PCIe) bus, an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc. The bus 920 can be divided into an address bus, a data bus, a control bus, etc.

[0150] As an example, computing device 90 may include multiple processors. The processor may be a multi-core (multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and / or computing units for processing data (e.g., computer program instructions).

[0151] It is worth mentioning that Figure 9 In this example, a computing device 90 including one processor 910 and one memory 930 is used. Here, processor 910 and memory 930 are respectively used to indicate a type of device or equipment. In specific embodiments, the number of each type of device or equipment can be determined based on business requirements. For example, computing device 90 may include multiple GPUs or multiple NPUs.

[0152] Memory 950 may be a volatile memory pool or a non-volatile memory pool, or may include both volatile and non-volatile memory. Non-volatile memory may be read-only memory (ROM), programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0153] The computing device 90 may be a general-purpose device or a dedicated device. For example, the computing device 90 may be a server or other device with computing capabilities.

[0154] It should be understood that the computing device 90 according to this embodiment may execute Figure 3 or Figure 5 For the sake of brevity, the functions of the first computing node or the second computing node in the illustrated method embodiment are not described in detail here.

[0155] The embodiment of the present application also provides a computing cluster system, including multiple computing nodes and shared memory, wherein the multiple computing nodes are used to execute multiple tasks when the application is running, and all of them can access the above-mentioned shared memory, which stores the data and global task information required by each computing node to execute multiple tasks when the application is running. The global task information records the metadata of the data corresponding to the task of the application executed by each computing node. A computing node can be Figure 9 Schematic computing device 90 .

[0156] As another form of this embodiment, a computer-readable storage medium is provided, on which instructions are stored. When the instructions are executed, the operating steps of the method in the above method embodiment are performed.

[0157] As another form of this embodiment, a computer program product containing instructions is provided. When the computer program product is run on a computer, the computer is caused to perform the method operation steps in the above method embodiment.

[0158] The method steps in this embodiment can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and storage medium can be located in an ASIC. In addition, the ASIC can be located in a computing device. Of course, the processor and storage medium can also exist as discrete components in a computing device.

[0159] In the above embodiments, they can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, they can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is executed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program or instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired or wireless means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, a hard disk, or a tape; it can also be an optical medium, such as a digital video disc (DVD); it can also be a semiconductor medium, such as a solid state drive (SSD). The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A computing cluster task management method, characterized in that: Applied to a computing cluster, the computing cluster includes multiple computing nodes and shared memory, the multiple computing nodes are used to execute multiple tasks when an application is running, and all of them can access the shared memory, the shared memory stores data required by each computing node to execute the multiple tasks and global task information, the global task information records metadata of data corresponding to the application tasks executed by each computing node; the method includes: After executing the task of the application, the first computing node among the plurality of computing nodes queries the global task information to determine a second computing node that has yet to execute a task; The first computing node sends an execution request to the second computing node; The second computing node determines a first task to be executed, obtains metadata of data corresponding to the first task from the global task information, and sends the metadata to the first computing node; The first computing node obtains data corresponding to the first task from the shared memory according to the metadata, and executes the first task according to the data.

2. The method according to claim 1, characterized in that The method further comprises: After sending the metadata of the data corresponding to the first task to the first computing node, the second computing node deletes or identifies the metadata of the data corresponding to the first task from the metadata of the data corresponding to the task executed by the second computing node in the global task information; After receiving the metadata of the data corresponding to the first task, the first computing node adds the metadata of the data corresponding to the first task to the metadata of the data corresponding to the task executed by the first computing node in the global task information.

3. The method according to claim 1, characterized in that The method further comprises: After receiving the execution request, the second computing node evaluates the time it takes for the second computing node to execute the first task, and when the time is greater than a preset value, sends metadata of the data corresponding to the first task to the first computing node.

4. The method according to claim 1, wherein The second computing node determines the first task to be executed includes: The second computing node determines a second task to be executed; The first task is separated from the second task according to a preset strategy.

5. A computing cluster task management device, characterized in that: Applied to a computing cluster, the computing cluster includes multiple computing nodes and shared memory, the multiple computing nodes are used to execute multiple tasks when an application is running, and all of them can access the shared memory, the shared memory stores data required by each computing node to execute the multiple tasks and global task information, the global task information records metadata of data corresponding to the application tasks executed by each computing node; the device includes: a determining unit, configured to query the global task information after the first computing node completes executing the task of the application, and determine a second computing node that has yet to execute a task; a sending unit, configured to send an execution request to the second computing node; a first processing unit, configured to determine a first task to be executed, obtain metadata of data corresponding to the first task from the global task information, and send the metadata to the first computing node; The second processing unit is configured to obtain data corresponding to the first task from the shared memory according to the metadata, and execute the first task according to the data.

6. The device according to claim 5, characterized in that The device further comprises a configuration unit, configured to: After the first processing unit sends the metadata of the data corresponding to the first task to the first computing node, delete or identify the metadata of the data corresponding to the first task from the metadata of the data corresponding to the task executed by the second computing node in the global task information; After the second processing unit receives the metadata of the data corresponding to the first task, the metadata of the data corresponding to the first task is added to the metadata of the data corresponding to the task executed by the first computing node in the global task information.

7. The device according to claim 5, characterized in that The apparatus further includes an evaluation unit configured to evaluate the time for the second computing node to execute the first task after receiving the execution request; The first processing unit is further configured to, when the time is greater than a preset value, send metadata of the data corresponding to the first task to the first computing node.

8. The device according to claim 7, characterized in that The first processing unit is specifically configured to: determining a second task to be performed; The first task is separated from the second task according to a preset strategy.

9. A computing cluster task management system, characterized in that: The system includes multiple computing nodes and shared memory. The multiple computing nodes are used to execute multiple tasks when the application is running, and all of them can access the shared memory. The shared memory stores data required by each computing node to execute the multiple tasks and global task information. The global task information records metadata of data corresponding to the tasks of the application executed by each computing node. The first computing node among the plurality of computing nodes is configured to: after executing the task of the application, query the global task information to determine a second computing node on which a task is to be executed; and send an execution request to the second computing node; The second computing node is configured to: determine a first task to be executed, obtain metadata corresponding to the first task from the global task information, and send the metadata to the first computing node; The first computing node is further configured to obtain data corresponding to the first task from the shared memory according to the metadata, and execute the first task according to the data.

10. The system according to claim 9, characterized in that The second computing node is further configured to: after sending the metadata of the data corresponding to the first task to the first computing node, delete or identify the metadata of the data corresponding to the first task from the metadata of the data corresponding to the task executed by the second computing node in the global task information; The first computing node is further configured to, after receiving the metadata of the data corresponding to the first task, add the metadata of the data corresponding to the first task to the metadata of the data corresponding to the task executed by the first computing node in the global task information.

11. The system according to claim 9, wherein: The second computing node is further configured to, after receiving the execution request, evaluate the time it takes for the second computing node to execute the first task, and when the time is greater than a preset value, send metadata of the data corresponding to the first task to the first computing node.

12. The system according to claim 9, wherein: The second computing node is specifically configured to: The second computing node determines a second task to be executed; The first task is separated from the second task according to a preset strategy.

13. A computer-readable storage medium, characterized in that include: Computer software instructions; when the computer software instructions are executed in a processor, the processor is caused to execute the operating steps of the method described in any one of claims 1 to 4.

14. A computer program product, characterized in that include: When the computer program product is run on a computer, the computer is caused to execute the operation steps of any one of the methods according to claims 1 to 4.

Images