Task execution method, apparatus and system

Abstract

A task execution method, apparatus, and system belong to the field of computer technology. In this method, a resource manager manages computing cards in a computing cluster. Based on the AI ​​resource requirements of a first task carried in a resource request sent by a task agent, the resource manager allocates a first computing card in the computing cluster that meets the AI ​​resource requirements of the first task and sends the service address of the first computing card to the task agent. The task agent invokes the first computing card to execute the AI ​​operation corresponding to the first task based on the service address of the first computing card. The node where the first computing card resides is a different node from the node running the task agent. This application can improve the resource utilization rate of computing cards.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a task execution method, apparatus and system. Background Technology

[0002] A computing cluster typically includes a scheduler and multiple compute nodes. The scheduler allocates tasks (such as deep learning tasks) from tenants to compute nodes for execution. Specifically, a compute node includes a central processing unit (CPU) and compute cards. After the scheduler assigns a task to any compute node, the CPU on that node calls upon the compute card on that node to execute the task. In current technology, a CPU can only call upon the compute card on the compute node where the CPU resides to execute tasks, resulting in low resource utilization of the compute cards. Summary of the Invention

[0003] This application provides a task execution method, apparatus, and system. The technical solution of this application is as follows.

[0004] Firstly, a task execution method is provided, comprising: a task agent sending a resource request to a resource manager, the resource request carrying the artificial intelligence (AI) resource requirements of a first task; the resource manager managing computing cards in a computing cluster, the computing cluster including at least one computing card; the task agent receiving a first service address sent by the resource manager, the first service address being the service address of a first computing card in the computing cluster, the first computing card satisfying the AI ​​resource requirements of the first task; the task agent, based on the first service address, invoking the first computing card to execute the AI ​​operation corresponding to the first task, wherein the node where the first computing card resides is a different node from the node running the task agent. The first computing card is allocated to the first task by the resource manager according to the AI ​​resource requirements of the first task.

[0005] The technical solution provided in this application allocates a first computing card within the managed computing cluster to meet the AI ​​resource requirements of the first task. The node where the first computing card resides is a different node from the node running the task agent. Therefore, the task agent can invoke the first computing card outside the node where the task agent resides to execute the AI ​​operation corresponding to the first task, which helps improve the resource utilization of the first computing card. Furthermore, in this application, the resource manager can allocate any computing card within the computing cluster that meets the AI ​​resource requirements of the first task. This allows the task agent to invoke any computing card outside the node where the task agent resides to execute the AI ​​operation corresponding to the first task. In other words, the resource manager can flexibly allocate computing cards to the first task, and the task agent can flexibly invoke computing cards outside the node where the task agent resides to execute the AI ​​operation corresponding to the first task. This not only improves the resource utilization of the computing cards but also enhances the flexibility of computing card invocation, achieving a flexible mapping between the first task and the computing card.

[0006] Since the task agent can invoke computing cards outside the node where it resides to execute the AI ​​operations corresponding to the task, this application can solve the problem of external computing card fragmentation. For example, if the task agent runs on a CPU, this application can enable the CPU to invoke computing cards outside the node where the CPU resides to execute the AI ​​operations corresponding to the task. If a node has idle computing card resources but its CPU resources are exhausted, CPUs in other nodes can invoke computing cards in that node to execute tasks. This extends the service scope of the computing card from a single node to the entire cluster, freeing the computing card from the availability of the local CPU and solving the problem of external computing card fragmentation.

[0007] Optionally, the AI ​​resource requirements of the first task include a first requirement and a second requirement. The first computing card meets the first requirement. The method further includes: the task agent receiving a second service address sent by the resource manager. The second service address is the service address of the second computing card in the computing cluster. The second computing card meets the second requirement. Based on the second service address, the task agent calls the second computing card to execute the AI ​​operation corresponding to the first task.

[0008] The technical solution provided in this application allows the task agent to call a first computing card and a second computing card that meet the AI ​​resource requirements of the first task to execute the AI ​​operation corresponding to the first task. For example, the task agent can call multiple computing cards that meet the AI ​​resource requirements of the first task to execute the AI ​​operation corresponding to the first task, thereby enabling the technical solution provided in this application to support large-scale tasks.

[0009] Optionally, the first computing card and the second computing card are located on the same node; or, the first computing card and the second computing card are located on different nodes.

[0010] Optionally, the node where the second computing card is located is a different node from the node running the task agent; or, the node where the second computing card is located is the same node as the node running the task agent. For example, if the first computing card and the second computing card are located on the same node, the node where the second computing card is located is a different node from the node running the task agent; if the first computing card and the second computing card are located on different nodes, the node where the second computing card is located may be a different node from the node running the task agent, or it may be the same node.

[0011] In the first implementation of the first aspect, the first requirement includes a first identifier and a first AI resource requirement, and the second requirement includes a second identifier and a second AI resource requirement. A first computing card satisfies the first AI resource requirement, and a second computing card satisfies the second AI resource requirement. The task agent receives a first service address and a second service address sent by a resource manager, including: the task agent receiving a resource response sent by the resource manager, the resource response carrying a service address table, the service address table including the correspondence between the first identifier and the first service address, and the correspondence between the second identifier and the second service address. After receiving the resource response, the task agent can obtain and store the service address table carried by the resource response; then, the task agent determines the first service address based on the first identifier and the stored service address table, and calls the first computing card to execute the AI ​​operation corresponding to the first task based on the first service address; and the task agent determines the second service address based on the second identifier and the stored service address table, and calls the second computing card to execute the AI ​​operation corresponding to the first task based on the second service address.

[0012] Since the first requirement of the first task includes a first identifier, and the second requirement of the first task includes a second identifier, both the first and second identifiers can be identifiers related to the first task. The first service address is the service address of the first computing card assigned to the first task by the resource manager, and the second service address is the service address of the second computing card assigned to the first task by the resource manager. The service address table includes the correspondence between the first identifier and the first service address, as well as the correspondence between the second identifier and the second service address. Therefore, the mapping between the first task and the computing card assigned to the first task can be achieved through this service address table. Furthermore, since the resource manager can flexibly assign computing cards to the first task, this service address table can flexibly record the service addresses of the computing cards assigned to the first task, achieving flexible mapping between the first task and the computing cards.

[0013] In an optional implementation of the first implementation method, the task agent includes a first agent module and a second agent module; the task agent sends a resource request to the resource manager, including: the first agent module sending the resource request to the resource manager; the task agent receives a resource response from the resource manager, including: the first agent module receiving the resource response from the resource manager; the method further includes: the first agent module storing the service address table in a shared memory region, the shared memory region being a memory region shared by the first agent module and the second agent module.

[0014] The first proxy module stores the service address table in the shared memory area, which facilitates access to the service address table by both the first and second proxy modules. In an optional implementation of the first method, the method further includes: the first proxy module determining a first service address based on a first identifier and the service address table stored in the shared memory area; the second proxy module determining a second service address based on a second identifier and the service address table stored in the shared memory area; and the task proxy calling the first computing card to execute the AI ​​operation corresponding to the first task based on the first service address, including: the first proxy module calling the first computing card to execute the AI ​​operation corresponding to the first task based on the first service address; and the task proxy calling the second computing card to execute the AI ​​operation corresponding to the first task based on the second service address, including: the second proxy module calling the second computing card to execute the AI ​​operation corresponding to the first task based on the second service address.

[0015] In an optional implementation of the first approach, the resource manager receives resource requests sent by the task agent, obtains a service address table based on the AI ​​resource requirements of the first task carried in the resource request, and sends a resource response carrying the service address table to the task agent.

[0016] In an optional implementation of the first approach, the resource manager includes a resource proxy and a resource controller. The resource controller manages the computing cards in the computing cluster. The resource proxy receives resource requests sent by the task proxy and sends the resource request to the resource controller. The resource controller obtains the service address table based on the AI ​​resource requirements of the first task carried in the resource request and sends a resource response carrying the service address table to the resource proxy. The resource proxy sends the resource response to the task proxy.

[0017] In the second implementation of the first aspect, the first requirement includes a first identifier and a first AI resource requirement, and the second requirement includes a second identifier and a second AI resource requirement. The first computing card satisfies the first AI resource requirement, and the second computing card satisfies the second AI resource requirement. The method further includes: a task agent sending a first query request and a second query request to a resource manager, the first query request carrying the first identifier and the second query request carrying the second identifier; the task agent receiving a first service address sent by the resource manager, including: the task agent receiving a first query response sent by the resource manager, the first query response carrying the first service address; and the task agent receiving a second service address sent by the resource manager, including: the task agent receiving a second query response sent by the resource manager, the second query response carrying the second service address.

[0018] In an optional implementation of the second approach, the task proxy includes a first proxy module and a second proxy module. The task proxy sends a first query request to the resource manager, including: the first proxy module sending the first query request to the resource manager; the task proxy receives a first query response from the resource manager, including: the first proxy module receiving the first query response from the resource manager; the task proxy invokes a first computing card to execute the AI ​​operation corresponding to the first task based on a first service address, including: the first proxy module invoking the first computing card to execute the AI ​​operation corresponding to the first task based on the first service address; the task proxy sends a second query request to the resource manager, including: the second proxy module sending the second query request to the resource manager; the task proxy receives a second query response from the resource manager, including: the second proxy module receiving the second query response from the resource manager; and the task proxy invokes a second computing card to execute the AI ​​operation corresponding to the first task based on a second service address, including: the second proxy module invoking the second computing card to execute the AI ​​operation corresponding to the first task based on the second service address.

[0019] In an optional implementation of the second approach, the resource manager receives resource requests from the task agent, obtains a service address table based on the AI ​​resource requirements of the first task carried in the resource request, stores the service address table, and sends a resource response to the task agent. This response indicates successful resource allocation and does not carry the service address table. The service address table includes a mapping between a first identifier and a first service address, and a mapping between a second identifier and a second service address. After receiving the resource response, the task agent sends a first query request and a second query request to the resource manager. The resource manager also receives the first query request from the task agent (specifically, the first agent module), determines the first service address based on the service address table and the first identifier carried in the first query request, and sends a first query response to the task agent (specifically, the first agent module); and receives the second query request from the task agent (specifically, the second agent module), determines the second service address based on the service address table and the second identifier carried in the second query request, and sends a second query response to the task agent (specifically, the second agent module).

[0020] In an optional implementation of the second approach, the resource manager includes a resource proxy and a resource controller. The resource controller manages the computing cards in the computing cluster. The resource proxy receives the resource request sent by the task proxy and sends the request to the resource controller. The resource controller obtains a service address table based on the AI ​​resource requirements of the first task carried in the resource request and sends the service address table to the resource proxy. The resource proxy stores the service address table and sends a resource response to the task proxy. Furthermore, the resource proxy receives a first query request sent by the task proxy (specifically, the first proxy module), determines a first service address based on the service address table and a first identifier carried in the first query request, and sends a first query response to the task proxy (specifically, the first proxy module). The resource proxy also receives a second query request sent by the task proxy (specifically, the second proxy module), determines a second service address based on the service address table and a second identifier carried in the second query request, and sends a second query response to the task proxy (specifically, the second proxy module).

[0021] Optionally, the task agent invokes the first computing card to execute the AI ​​operation corresponding to the first task based on the first service address, including: the task agent establishing a first connection based on the first service address; and the task agent invoking the first computing card to execute the AI ​​operation corresponding to the first task based on the first connection.

[0022] Optionally, the task agent establishes a first connection based on a first service address, including: the task agent establishes a first connection with a first service process based on the first service address, the first service process being used to manage a first computing card; the task agent invokes the first computing card to execute the AI ​​operation corresponding to the first task based on the first connection, including: the task agent sends a call request for the first task to the first service process through the first connection, thereby triggering the first service process to invoke the first computing card to process the call request for the first task in order to execute the AI ​​operation corresponding to the first task.

[0023] Optionally, the task agent establishes a first connection with the first service process based on the first service address, including: the task agent sends a connection establishment request (e.g., a first connection establishment request) to the first service process based on the first service address, the connection establishment request (i.e., the first connection establishment request) being used by the task agent to request the establishment of a first connection with the first service process.

[0024] Optionally, the task agent invokes the second computing card to execute the AI ​​operation corresponding to the first task based on the second service address, including: the task agent establishing a second connection based on the second service address; and the task agent invoking the second computing card to execute the AI ​​operation corresponding to the first task based on the second connection.

[0025] Optionally, the task agent establishes a second connection based on the second service address, including: the task agent establishes a second connection with a second service process based on the second service address, the second service process being used to manage the second computing card; the task agent invokes the second computing card to execute the AI ​​operation corresponding to the first task based on the second connection, including: the task agent sends an invocation request for the first task to the second service process through the second connection, thereby triggering the second service process to invoke the second computing card to process the invocation request for the first task in order to execute the AI ​​operation corresponding to the first task.

[0026] Optionally, the task agent establishes a second connection with the second service process based on the second service address, including: the task agent sends a connection establishment request (e.g., a second connection establishment request) to the second service process based on the second service address, the connection establishment request (i.e., the second connection establishment request) being used by the task agent to request the establishment of a second connection with the second service process.

[0027] For ease of distinction, the resource request described above in the first aspect is referred to as the first resource request. Optionally, the method further includes: in response to the first connection being disconnected (e.g., in response to the first connection and / or the second connection being disconnected), the task agent sends a second resource request to the resource manager. The second resource request carries the AI ​​resource requirements of the first task. The task agent receives a third service address sent by the resource manager. The third service address is the service address of a third computing card in the computing cluster. The third computing card meets the AI ​​resource requirements of the first task. Based on the third service address, the task agent calls the third computing card to execute the AI ​​operation corresponding to the first task. The node where the third computing card is located is a different node from the node running the task agent.

[0028] The first service process can disconnect from the task agent if the first computing card meets the disconnection conditions. These conditions include, for example, a malfunction in the first computing card or the first computing card's load exceeding a preset limit. In other words, the first service process disconnects from the task agent when the first computing card malfunctions or its load exceeds a preset limit. Consequently, the task agent, upon disconnection, resends a resource request (e.g., a second resource request) to the resource manager to request AI resources for the first task. The resource manager can then reallocate computing cards to the first task based on the resent resource request, thus preventing computing card malfunctions or excessive load from affecting the task's operation and ensuring its smooth execution.

[0029] The second service process can disconnect from the task agent if the second computing card meets certain disconnection conditions. These conditions include, for example, a malfunction in the second computing card or the second computing card's load exceeding a preset limit. In other words, the second service process disconnects from the task agent when the second computing card malfunctions or its load exceeds a preset limit. Consequently, the task agent, upon disconnection, resends a resource request (e.g., a second resource request) to the resource manager to request AI resources for the first task. The resource manager can then reallocate computing cards to the first task based on the resent resource request, thus preventing computing card malfunctions or excessive load from affecting the task's execution and ensuring its smooth operation.

[0030] In other words, after the first task is assigned a computing card (e.g., the first computing card or the second computing card mentioned above), if the computing card corresponding to the first task malfunctions or is overloaded, the solution provided in this application can flexibly and dynamically replace the computing card for the first task (reassign the computing card) even during the operation of the first task, realizing dynamic and flexible mapping between the first task and any computing card, which not only ensures the smooth operation of the first task, but also improves the overall utilization rate of the computing card.

[0031] Secondly, a task execution method is provided, comprising: a resource manager receiving a resource request sent by a task agent, the resource request carrying the AI ​​resource requirements of a first task; the resource manager managing computing cards in a computing cluster, the computing cluster including at least one computing card; the resource manager sending a first service address to the task agent, the first service address being the service address of a first computing card in the computing cluster, the first computing card meeting the AI ​​resource requirements of the first task, the node where the first computing card resides being a different node from the node running the task agent; and the first service address being used by the task agent to invoke the first computing card to execute the AI ​​operation corresponding to the first task. For example, the resource manager allocates a first computing card to the first task according to the AI ​​resource requirements of the first task, and then sends the first service address to the task agent.

[0032] The technical solution provided in this application allocates a first computing card within the managed computing cluster to meet the AI ​​resource requirements of the first task. The node where the first computing card resides is a different node from the node running the task agent. Therefore, the task agent can invoke the first computing card outside the node where the task agent resides to execute the AI ​​operation corresponding to the first task, which helps improve the resource utilization of the first computing card. Furthermore, in this application, the resource manager can allocate any computing card within the computing cluster that meets the AI ​​resource requirements of the first task. This allows the task agent to invoke any computing card outside the node where the task agent resides to execute the AI ​​operation corresponding to the first task. In other words, the resource manager can flexibly allocate computing cards to the first task, and the task agent can flexibly invoke computing cards outside the node where the task agent resides to execute the AI ​​operation corresponding to the first task. This not only improves the resource utilization of the computing cards but also enhances the flexibility of computing card invocation, achieving a flexible mapping between the first task and the computing card.

[0033] Optionally, the AI ​​resource requirements of the first task include a first requirement and a second requirement. The first computing card satisfies the first requirement. The method further includes: the resource manager sending a second service address to the task agent. The second service address is the service address of a second computing card in the computing cluster. The second computing card satisfies the second requirement. The second service address is used by the task agent to invoke the second computing card to execute the AI ​​operation corresponding to the first task. For example, the resource manager allocates a second computing card to the first task according to its AI resource requirements and then sends the second service address to the task agent.

[0034] The technical solution provided in this application allocates a first computing card and a second computing card in the computing cluster it manages to meet the AI ​​resource requirements of the first task, so that the task agent can call the first computing card and the second computing card to execute the AI ​​operation corresponding to the first task, thereby enabling the technical solution provided in this application to support large-scale tasks.

[0035] Optionally, the first computing card and the second computing card are located on the same node; or, the first computing card and the second computing card are located on different nodes.

[0036] Optionally, the node where the second computing card is located is a different node from the node running the task agent; or, the node where the second computing card is located is the same node as the node running the task agent.

[0037] In the first implementation of the second aspect, the first requirement includes a first identifier and a first AI resource requirement, and the second requirement includes a second identifier and a second AI resource requirement. The first computing card satisfies the first AI resource requirement, and the second computing card satisfies the second AI resource requirement. The method further includes: a resource manager obtaining a service address table based on the AI ​​resource requirements of the first task, the service address table including the correspondence between the first identifier and the first service address, and the correspondence between the second identifier and the second service address; the resource manager sending the first service address and the second service address to the task agent, including: the resource manager sending a resource response to the task agent, the resource response carrying the service address table.

[0038] Since the first requirement of the first task includes a first identifier, and the second requirement of the first task includes a second identifier, both the first and second identifiers can be identifiers related to the first task. The first service address is the service address of the first computing card assigned to the first task by the resource manager, and the second service address is the service address of the second computing card assigned to the first task by the resource manager. The service address table includes the correspondence between the first identifier and the first service address, as well as the correspondence between the second identifier and the second service address. Therefore, the mapping between the first task and the computing card assigned to the first task can be achieved through this service address table. Furthermore, since the resource manager can flexibly assign computing cards to the first task, this service address table can flexibly record the service addresses of the computing cards assigned to the first task, achieving flexible mapping between the first task and the computing cards.

[0039] In an optional implementation of the first approach, the resource manager includes a resource proxy and a resource controller. The resource controller manages the computing cards in the computing cluster. The resource manager receives the resource request sent by the task proxy, including: the resource proxy receiving the resource request sent by the task proxy; the method further includes: the resource proxy sending the resource request to the resource controller; the resource manager obtains a service address table based on the AI ​​resource requirements of the first task, including: the resource controller obtaining the service address table based on the AI ​​resource requirements of the first task carried in the resource request; the method further includes: the resource controller sending the resource response to the resource proxy; the resource manager sending the resource response to the task proxy, including: the resource proxy sending the resource response to the task proxy.

[0040] In an optional implementation of the first implementation method, the task agent includes a first agent module and a second agent module. The resource manager receives resource requests sent by the task agent, including: the resource manager receives resource requests sent by the first agent module; the resource manager sends resource responses to the task agent, including: the resource manager sends resource responses to the first agent module.

[0041] In the second implementation of the second aspect, the first requirement includes a first identifier and a first AI resource requirement, and the second requirement includes a second identifier and a second AI resource requirement. A first computing card satisfies the first AI resource requirement, and a second computing card satisfies the second AI resource requirement. The method further includes: a resource manager obtaining and storing a service address table based on the AI ​​resource requirements of the first task; the service address table includes a correspondence between the first identifier and the first service address, and a correspondence between the second identifier and the second service address; the resource manager receiving a first query request and a second query request sent by a task agent, the first query request carrying the first identifier and the second query request carrying the second identifier; the resource manager determining the first service address based on the service address table and the first identifier carried in the first query request; the resource manager determining the second service address based on the service address table and the second identifier carried in the second query request; the resource manager sending the first service address to the task agent includes: the resource manager sending a first query response to the task agent, the first query response carrying the first service address; and the resource manager sending the second service address to the task agent includes: the resource manager sending a second query response to the task agent, the second query response carrying the second service address.

[0042] Since the first requirement of the first task includes a first identifier, and the second requirement of the first task includes a second identifier, both the first and second identifiers can be identifiers related to the first task. The first service address is the service address of the first computing card assigned to the first task by the resource manager, and the second service address is the service address of the second computing card assigned to the first task by the resource manager. The service address table includes the correspondence between the first identifier and the first service address, as well as the correspondence between the second identifier and the second service address. Therefore, the mapping between the first task and the computing card assigned to the first task can be achieved through this service address table. Furthermore, since the resource manager can flexibly assign computing cards to the first task, this service address table can flexibly record the service addresses of the computing cards assigned to the first task, achieving flexible mapping between the first task and the computing cards.

[0043] In an optional implementation of the second approach, the resource manager includes a resource broker and a resource controller. The resource controller manages the computing cards in the computing cluster. The resource manager receives resource requests sent by the task broker, including: the resource broker receiving the resource request sent by the task broker; the method further includes: the resource broker sending the resource request to the resource controller; the resource manager obtaining a service address table based on the AI ​​resource requirements of the first task, including: the resource controller obtaining the service address table based on the AI ​​resource requirements of the first task carried in the resource request; the method further includes: the resource controller sending the service address table to the resource broker; the resource broker storing the service address table; and the resource manager receiving a first query request and a second query request sent by the task broker, including: [further details needed]. The source proxy receives a first query request and a second query request sent by the task proxy; the resource manager determines a first service address based on the service address table and a first identifier carried in the first query request, including: the resource proxy obtaining the first service address based on the service address table and the first identifier carried in the first query request; the resource manager determines a second service address based on the service address table and a second identifier carried in the second query request, including: the resource proxy obtaining the second service address based on the service address table and the second identifier carried in the second query request; the resource manager sends a first query response to the task proxy, including: the resource proxy sending a first query response to the task proxy; the resource manager sends a second query response to the task proxy, including: the resource proxy sending a second query response to the task proxy.

[0044] In an optional implementation of the second approach, the task proxy includes a first proxy module and a second proxy module. The resource manager receives resource requests from the task proxy, including: the resource manager receiving a resource request from the first proxy module; the resource manager sends a resource response to the task proxy, including: the resource manager sending a resource response to the first proxy module. The resource manager receives a first query request from the task proxy, including: the resource manager receiving a first query request from the first proxy module; the resource manager sends a first query response to the task proxy, including: the resource manager sending a first query response to the first proxy module. The resource manager receives a second query request from the task proxy, including: the resource manager receiving a second query request from the second proxy module; the resource manager sends a second query response to the task proxy, including: the resource manager sending a second query response to the second proxy module.

[0045] For ease of distinction, the resource request described above in the second aspect is referred to as the first resource request. Optionally, the method further includes: the resource manager receiving a second resource request sent by the task agent, the second resource request carrying the AI ​​resource requirements of the first task; the resource manager sending a third service address to the task agent, the third service address being the service address of a third computing card in the computing cluster, the third computing card meeting the AI ​​resource requirements of the first task, the node where the third computing card is located being a different node from the node running the task agent, and the third service address being used by the task agent to invoke the third computing card to execute the AI ​​operation corresponding to the first task. For example, the resource manager allocates a third computing card to the first task according to the AI ​​resource requirements of the first task, and then sends the third service address to the task agent.

[0046] In this application, the task agent sends a second resource request to the resource manager when the first connection corresponding to the first computing card and / or the second connection corresponding to the second computing card is disconnected. The first connection disconnection may be due to a malfunction of the first computing card, or the first computing card's load exceeding a preset load. The second connection disconnection may be due to a malfunction of the second computing card, or the second computing card's load exceeding a preset load. Therefore, when the first and / or second connections are disconnected, the task agent resends a resource request (e.g., a second resource request) to the resource manager to request AI resources for the first task. The resource manager can then reallocate computing cards to the first task based on the resent resource request. This avoids disruptions to the task's operation due to computing card malfunctions or excessive computing card load, ensuring the smooth operation of the first task. In other words, the solution provided in this application allows for flexible and dynamic replacement (reallocation) of computing cards for the first task even during its operation, ensuring the smooth operation of the first task and improving the overall utilization rate of the computing cards.

[0047] Thirdly, a task execution method is provided, comprising: a first service process and a task agent establishing a first connection based on a first service address, wherein the first service address is the service address of a first computing card in a computing cluster; the first service process managing the first computing card; the first computing card meeting the AI ​​resource requirements of a first task; and the node where the first computing card resides being a different node from the node running the task agent; the first service process receiving a call request for the first task sent by the task agent through the first connection; and the first service process calling the first computing card to process the call request for the first task to execute the AI ​​operation corresponding to the first task. The first service process and the first computing card reside on the same node.

[0048] The technical solution provided in this application allows the task agent to call the first computing card outside the node where the task agent is located to execute the AI ​​operation corresponding to the first task through the first service process used to manage the first computing card, which helps to improve the resource utilization of the first computing card.

[0049] Optionally, the first service process and the task agent establish a first connection based on the first service address, including: the first service process receiving a connection establishment request sent by the task agent based on the first service address; and the first service process responding to the connection establishment request by establishing a first connection with the task agent.

[0050] Optionally, the method further includes: a first service process caching a call request for a first task in a first request queue, the first request queue corresponding to the first task; the first service process calling a first computing card to process the call request for the first task, including: the first service process calling the first computing card to process the call request in the first request queue.

[0051] Optionally, the connection establishment request carries the task identifier of the first task, and the method further includes: in response to the connection establishment request, the first service process creates a first request queue. The creation of the first request queue by the first service process facilitates caching of invocation requests for the first task.

[0052] Optionally, the connection establishment request also carries the priority of the first task, and the method further includes: in response to the priority of the first task satisfying the first priority, the first service process caches the task identifier of the first task in the first task queue, the first task queue corresponding to the first priority, and the first task queue is used to cache the task identifier of the task whose priority satisfies the first priority.

[0053] Optionally, the first service process is used to schedule task identifiers in the first task queue in a time-slice round-robin manner. The first service process calls the first computing card to process call requests in the first request queue, including: when the minimum interference level corresponding to the first task is not less than a threshold and the first service process schedules the task identifier of the first task in the first task queue, the first service process calls the first computing card to process call requests in the first request queue within the first time slice corresponding to the first task, so that the first task exclusively occupies the first computing card. The minimum interference level corresponding to the first task is the interference level of the second task on the first task. The second task is the unfinished task with the least interference level on the first task among the tasks whose task identifiers are cached in the second task queue. The second task queue corresponds to a second priority, and the first priority is higher than the second priority. The second task queue is used to cache task identifiers of tasks whose priority meets the second priority. When the minimum interference level corresponding to the first task is less than the threshold and the first service process schedules the task identifier of the first task in the first task queue, the first service process calls the first computing card to process call requests in the first request queue and call requests in the second request queue within the first time slice, so that the first task and the second task share the first computing card in a space-division multiplexing manner. The second request queue corresponds to the second task.

[0054] The technical solution provided in this application addresses the issue that the minimum interference level corresponding to the first task is the interference level of the second task on the first task, and the second task is the incomplete task with the lowest interference level among the tasks whose task identifiers are cached in the second task queue. Therefore, if the minimum interference level corresponding to the first task is not less than a threshold, it indicates that the interference level of each task identifier in the second task queue on the first task is not less than the threshold, and the interference level of each task identifier in the second task queue on the first task is relatively large. In this case, if the tasks indicated by the task identifiers in the first and second task queues share the first computing card in a space-division multiplexing manner, it is likely to have a significant impact on the first task, such as severely affecting its execution efficiency. Therefore, if the minimum interference level corresponding to the first task is not less than the threshold, the first service process ensures that the first task exclusively occupies the first computing card within the first time slice corresponding to the first task, thereby guaranteeing the execution efficiency of the first task. Conversely, if the minimum interference level corresponding to the first task is less than the threshold, it indicates that at least the interference level of the second task on the first task is relatively small. In this case, if the first task exclusively occupies the first computing card, it will cause a waste of resources on the first computing card and trigger internal fragmentation problems. Therefore, when the minimum interference level corresponding to the first task is less than the threshold, the first service process enables the first task and the second task to share the first computing card in a space-division multiplexing manner within the first time slice corresponding to the first task. This can ensure the execution efficiency of the first task on the one hand, and the resource utilization of the first computing card on the other hand, thus solving the internal fragmentation problem of the first computing card.

[0055] Optionally, if the interference of the second task on the first task is less than a threshold, and the second task is completed within the first time slice, the first service process calls the first computing card to process the call requests in the first request queue and the call requests in the third request queue during the remaining time period of the first time slice. This allows the first and third tasks to share the first computing card in a space-division multiplexing manner. The third request queue corresponds to the third task, which is the unfinished task with the least interference to the first task, excluding the second task, among the tasks whose task identifiers are cached in the second task queue. In this way, internal fragmentation of the first computing card can be avoided during the remaining time period of the first time slice, improving resource utilization.

[0056] Optionally, the method further includes: if the first task is not completed by the end of the first time slice, the first service process moves the task identifier of the first task from the head to the tail of the first task queue. This facilitates the first service process in scheduling the next task identifier in the first task queue and calling the first computing card to process the call request for the task indicated by that task identifier.

[0057] Optionally, the method further includes: if the first task is completed within the first time slice, the first service process removes the task identifier of the first task from the first task queue. This avoids the task identifier of a completed task occupying queue resources in the first task queue.

[0058] Optionally, the method further includes: when the first service process is scheduled to the task identifier of the first task in the first task queue, the first service process determines the minimum interference level corresponding to the first task within the second time slice corresponding to the first task, wherein the second time slice is located before the first time slice.

[0059] The technical solution provided in this application allows the first service process to determine the minimum interference level corresponding to the first task. This facilitates the first service process in determining the occupation method of the first task on the first computing card based on the minimum interference level corresponding to the first task (the first task exclusively occupies the first computing card or shares the first computing card with other tasks in a space-division multiplexing manner). This ensures the resource utilization rate of the first computing card while guaranteeing the execution efficiency of the first task.

[0060] Optionally, the second task queue caches task identifiers for at least one task. The first service process determines the minimum interference level corresponding to the first task within the second time slice, including: within the second time slice: first, the first service process calls the first computing card to process m call requests in the first request queue, allowing the first task to exclusively use the first computing card, where m is a positive integer; then, the first service process sequentially schedules at least one task group, each task group corresponding to the at least one task, and each task group includes the first task and the task corresponding to that task group; if the first service process schedules to any task group in the at least one task group: the first service process calls the first computing card to process the m call requests in the first request queue and the call requests in the request queue of the task corresponding to that task group, allowing the first task and the task corresponding to that task group to share the first computing card in a space-division multiplexing manner; the first service process determines the minimum interference level corresponding to the first task based on the period during which the first task exclusively uses the first computing card. During this period, the first computing card processes m call requests in the first request queue to determine the exclusive throughput of the first task. This exclusive throughput is the throughput of the first task when it exclusively uses the first computing card. The first service process determines the shared throughput of the first task with any task group based on the time taken for the first computing card to process m call requests in the first request queue during the period when the first task and tasks corresponding to any task group share the first computing card. This shared throughput is the throughput of the first task when it shares the first computing card with tasks corresponding to any task group. The first service process determines at least one level of interference for the first task based on its exclusive throughput and at least one shared throughput. This at least one shared throughput corresponds one-to-one with the at least one task group, and the at least one level of interference corresponds one-to-one with the at least one shared throughput. The first service process determines the minimum value among the at least one level of interference as the minimum level of interference for the first task. The throughput of the first task is its performance indicator.

[0061] The technical solution provided in this application allows a first service process to detect the throughput during the execution of a first task and determine the minimum interference level corresponding to the first task based on the throughput. Then, based on the minimum interference level, the process determines how the first task occupies the first computing card. Therefore, if the minimum interference level corresponding to the first task is not less than a threshold, the first service process can ensure that the first task exclusively occupies the first computing card within its corresponding time slice, guaranteeing the execution efficiency of the first task. Conversely, if the minimum interference level corresponding to the first task is less than the threshold, the first service process can allow the first task to share the first computing card with other tasks in a space-division multiplexing manner within its corresponding time slice. This ensures both the execution efficiency of the first task and the resource utilization of the first computing card.

[0062] Optionally, the method further includes: if there are no unfinished task identifiers in the first task queue, the first service process calls the first computing card to process the call requests in the request queues of at least two tasks corresponding to at least two task identifiers in the second task queue, so that the at least two tasks share the first computing card in a space-division multiplexing manner, and the at least two tasks are unfinished tasks.

[0063] The technical solution provided in this application allows the first service process to enable the tasks indicated by the task identifiers in the second task queue to share the first computing card in a space-division multiplexing manner when there are no unfinished task identifiers in the first task queue. This can ensure the resource utilization of the first computing card and solve the internal fragmentation problem of the first computing card.

[0064] Optionally, the first service process includes at least one thread, which corresponds one-to-one with at least one task, which includes the first task; each of the at least one thread is used to call the first computing card to process the call request of the corresponding task.

[0065] Optionally, the method further includes: the first service process disconnecting the first connection in response to the first computing card meeting a disconnection condition. For example, the disconnection condition includes the first computing card malfunctioning, the first computing card's load exceeding a preset load, etc. That is, the first service process disconnects the first connection from the task agent when the first computing card malfunctions, the first computing card's load exceeds a preset load, etc.

[0066] The technical solution provided in this application allows the first service process to disconnect the first connection when the first computing card meets the disconnection conditions. When the task agent detects that the first connection has been disconnected, it can re-apply for AI resources for the first task. This can avoid the task running process being affected by computing card failure or excessive computing card load, thus ensuring the smooth operation of the first task.

[0067] Fourthly, a task execution apparatus is provided for use in a task agent. The task execution apparatus includes at least one functional unit for performing the method provided by the first aspect or any optional implementation thereof. The at least one functional unit can be implemented in software or a combination of software and hardware, and can be arbitrarily combined or divided based on implementation.

[0068] Specifically, the task execution device includes a first sending unit, a first receiving unit, and a first invocation unit. The first sending unit is used to send a resource request to a resource manager, the resource request carrying the AI ​​resource requirements of a first task. The resource manager manages computing cards in a computing cluster, the computing cluster including at least one computing card. The first receiving unit is used to receive a first service address sent by the resource manager, the first service address being the service address of a first computing card in the computing cluster, the first computing card satisfying the AI ​​resource requirements of the first task. The first invocation unit is used to invoke the first computing card to execute the AI ​​operation corresponding to the first task based on the first service address, wherein the node where the first computing card is located is a different node from the node running the task agent.

[0069] Optionally, the AI ​​resource requirements of the first task include a first requirement and a second requirement. The first computing card meets the first requirement, and the task execution device further includes a second receiving unit and a second calling unit. The second receiving unit is used to receive a second service address sent by the resource manager. The second service address is the service address of the second computing card in the computing cluster, and the second computing card meets the second requirement. The second calling unit is used to call the second computing card to execute the AI ​​operation corresponding to the first task based on the second service address.

[0070] Optionally, the first computing card and the second computing card are located on the same node; or, the first computing card and the second computing card are located on different nodes.

[0071] Optionally, the node where the second computing card is located is a different node from the node running the task agent; or, the node where the second computing card is located is the same node as the node running the task agent.

[0072] In the first implementation of the fourth aspect, the first requirement includes a first identifier and a first AI resource requirement, the second requirement includes a second identifier and a second AI resource requirement, the first computing card satisfies the first AI resource requirement, the second computing card satisfies the second AI resource requirement, the first receiving unit and the second receiving unit are the same receiving unit, the first receiving unit is used to receive the resource response sent by the resource manager, the resource response carries a service address table, the service address table includes the correspondence between the first identifier and the first service address, and the correspondence between the second identifier and the second service address.

[0073] In an optional implementation of the first approach, the task agent includes a first agent module and a second agent module; a first sending unit for the first agent module to send the resource request to the resource manager; a first receiving unit for the first agent module to receive the resource response sent by the resource manager; and a task execution device further includes a storage unit for the first agent module to store the service address table in a shared memory region, wherein the shared memory region is a memory region shared by the first agent module and the second agent module. In this optional implementation, the first sending unit, the first receiving unit, and the storage unit are all located within the first agent module.

[0074] In an optional implementation of the first approach, the task execution device further includes a first determining unit and a second determining unit. The first determining unit is used by the first proxy module to determine the first service address based on the first identifier and the service address table stored in the shared memory region. The second determining unit is used by the second proxy module to determine the second service address based on the second identifier and the service address table stored in the shared memory region. The first calling unit is used by the first proxy module to call the first computing card to execute the AI ​​operation corresponding to the first task based on the first service address. The second calling unit is used by the second proxy module to call the second computing card to execute the AI ​​operation corresponding to the first task based on the second service address. In this optional implementation, both the first determining unit and the first calling unit are located within the first proxy module, and both the second determining unit and the second calling unit are located within the second proxy module.

[0075] In an optional implementation of the first approach, the resource manager receives the resource request sent by the task agent, obtains the service address table based on the AI ​​resource requirements of the first task carried in the resource request, and sends the resource response to the task agent.

[0076] In an optional implementation of the first approach, the resource manager includes a resource proxy and a resource controller. The resource controller manages the computing cards in the computing cluster. The resource proxy receives the resource request sent by the task proxy and sends the resource request to the resource controller. The resource controller obtains the service address table based on the AI ​​resource requirements of the first task carried in the resource request and sends the resource response to the resource proxy. The resource proxy sends the resource response to the task proxy.

[0077] In the second implementation of the fourth aspect, the first requirement includes a first identifier and a first AI resource requirement, the second requirement includes a second identifier and a second AI resource requirement, the first computing card satisfies the first AI resource requirement, the second computing card satisfies the second AI resource requirement, and the task execution device further includes a second sending unit. The first sending unit is further configured to send a first query request to the resource manager, the first query request carrying the first identifier; the second sending unit is configured to send a second query request to the resource manager, the second query request carrying the second identifier; the first receiving unit is configured to receive a first query response sent by the resource manager, the first query response carrying the first service address; the second receiving unit is configured to receive a second query response sent by the resource manager, the second query response carrying the second service address.

[0078] In an optional implementation of the second approach, the task proxy includes a first proxy module and a second proxy module; a first sending unit for sending the resource request to the resource manager; a first receiving unit for sending the first query request to the resource manager; a first invoking unit for receiving the first query response from the resource manager; a first invoking unit for invoking the first computing card to execute the AI ​​operation corresponding to the first task based on the first service address; a second sending unit for sending the second query request to the resource manager; a second receiving unit for receiving the second query response from the resource manager; and a second invoking unit for invoking the second computing card to execute the AI ​​operation corresponding to the first task based on the second service address. In this optional implementation, the first sending unit, the first receiving unit, and the first invoking unit are all located in the first proxy module, and the second sending unit, the second receiving unit, and the second invoking unit are all located in the second proxy module.

[0079] In an optional implementation of the second approach, the resource manager receives the resource request sent by the task agent, obtains a service address table based on the AI ​​resource requirements of the first task carried in the resource request, stores the service address table, and sends a resource response to the task agent. The resource response indicates successful resource allocation and does not carry the service address table. The service address table includes the correspondence between the first identifier and the first service address, and the correspondence between the second identifier and the second service address. The resource manager also receives the first query request sent by the task agent (specifically, the first agent module), determines the first service address based on the service address table and the first identifier carried in the first query request, and sends the first query response to the task agent (specifically, the first agent module). Furthermore, the resource manager receives the second query request sent by the task agent (specifically, the second agent module), determines the second service address based on the service address table and the second identifier carried in the second query request, and sends the second query response to the task agent (specifically, the second agent module).

[0080] In an optional implementation of the second approach, the resource manager includes a resource proxy and a resource controller. The resource controller manages the computing cards in the computing cluster. The resource proxy receives resource requests from the task proxy and sends the resource requests to the resource controller. The resource controller obtains a service address table based on the AI ​​resource requirements of the first task carried in the resource request and sends the service address table to the resource proxy. The resource proxy stores the service address table and sends the resource response to the task proxy. The resource proxy also receives a first query request from the task proxy (specifically, a first proxy module), determines the first service address based on the service address table and the first identifier carried in the first query request, and sends the first query response to the task proxy (specifically, the first proxy module). Furthermore, the resource proxy receives a second query request from the task proxy (specifically, a second proxy module), determines the second service address based on the service address table and the second identifier carried in the second query request, and sends the second query response to the task proxy (specifically, the second proxy module).

[0081] Optionally, the first invocation unit is configured to: establish a first connection based on the first service address; and invoke the first computing card to execute the AI ​​operation corresponding to the first task based on the first connection.

[0082] Optionally, the first calling unit is configured to: establish the first connection with the first service process based on the first service address, wherein the first service process is used to manage the first computing card; and use the first sending unit to send a call request for the first task to the first service process through the first connection, so as to trigger the first service process to call the first computing card to process the call request for the first task to execute the AI ​​operation corresponding to the first task.

[0083] Optionally, the first calling unit is configured to: use the first sending unit to send a connection establishment request to the first service process based on the first service address, wherein the connection establishment request is used by the task agent to request the establishment of the first connection with the first service process.

[0084] Optionally, the second calling unit is configured to: establish a second connection based on the second service address; and call the second computing card to execute the AI ​​operation corresponding to the first task based on the second connection.

[0085] Optionally, the second calling unit is configured to: establish a second connection with a second service process based on the second service address, the second service process being used to manage the second computing card; and send a call request for the first task to the second service process through the second connection to trigger the second service process to call the second computing card to process the call request for the first task in order to execute the AI ​​operation corresponding to the first task.

[0086] Optionally, the second calling unit is configured to: use the second sending unit to send a connection establishment request to the second service process based on the second service address, the connection establishment request being used by the task agent to request the establishment of the second connection with the second service process.

[0087] For ease of distinction, the resource request described above in the fourth aspect is referred to as the first resource request. Optionally, the first sending unit is further configured to send a second resource request to the resource manager in response to the first connection being disconnected, the second resource request carrying the AI ​​resource requirements of the first task; the first receiving unit is further configured to receive a third service address sent by the resource manager, the third service address being the service address of a third computing card in the computing cluster, the third computing card satisfying the AI ​​resource requirements of the first task; the first calling unit is further configured to call the third computing card to execute the AI ​​operation corresponding to the first task based on the third service address, the node where the third computing card is located being a different node from the node running the task agent.

[0088] Fifthly, a task execution apparatus is provided for use in a resource manager. The task execution apparatus includes at least one functional unit for performing the method provided by the second aspect or any alternative implementation thereof. The at least one functional unit may be implemented in software or a combination of software and hardware, and may be arbitrarily combined or divided based on implementation.

[0089] Specifically, the task execution device includes a first receiving unit and a first sending unit. The first receiving unit is used to receive a resource request sent by a task agent, the resource request carrying the AI ​​resource requirements of a first task. The resource manager is used to manage computing cards in a computing cluster, the computing cluster including at least one computing card. The first sending unit is used to send a first service address to the task agent. The first service address is the service address of a first computing card in the computing cluster. The first computing card meets the AI ​​resource requirements of the first task. The node where the first computing card is located is a different node from the node running the task agent. The task agent uses the first service address to invoke the first computing card to execute the AI ​​operation corresponding to the first task.

[0090] Optionally, the AI ​​resource requirements of the first task include a first requirement and a second requirement. The first computing card satisfies the first requirement. The first sending unit is further configured to send a second service address to the task agent. The second service address is the service address of the second computing card in the computing cluster. The second computing card satisfies the second requirement. The second service address is used by the task agent to call the second computing card to execute the AI ​​operation corresponding to the first task.

[0091] Optionally, the first computing card and the second computing card are located on the same node; or, the first computing card and the second computing card are located on different nodes.

[0092] Optionally, the node where the second computing card is located is a different node from the node running the task agent; or, the node where the second computing card is located is the same node as the node running the task agent.

[0093] In the first implementation of the fifth aspect, the first requirement includes a first identifier and a first AI resource requirement, the second requirement includes a second identifier and a second AI resource requirement, the first computing card satisfies the first AI resource requirement, the second computing card satisfies the second AI resource requirement, the task execution device further includes an acquisition unit, the acquisition unit is used to acquire a service address table according to the AI ​​resource requirements of the first task, the service address table includes the correspondence between the first identifier and the first service address, and the correspondence between the second identifier and the second service address; the first sending unit is used to send a resource response to the task agent, the resource response carrying the service address table.

[0094] In an optional technical solution of the first implementation, the resource manager includes a resource proxy and a resource controller, the resource controller being used to manage computing cards in the computing cluster; the task execution device further includes a second receiving unit and a second sending unit. The first receiving unit is used for the resource proxy to receive the resource request sent by the task proxy; the first sending unit is also used for the resource proxy to send the resource request to the resource controller; the second receiving unit is used for the resource controller to receive the resource request sent by the resource proxy; the acquisition unit is used for the resource controller to acquire the service address table based on the AI ​​resource requirements of the first task carried in the resource request; the second sending unit is used for the resource controller to send the resource response to the resource proxy; the first sending unit is used for the resource proxy to send the resource response to the task proxy. In this optional technical solution, the first receiving unit and the first sending unit are both located in the resource proxy, and the second receiving unit, the second sending unit, and the acquisition unit are all located in the resource controller.

[0095] In an optional technical solution of the first implementation, the task agent includes a first agent module and a second agent module, the first receiving unit is used to receive a resource request sent by the first agent module, and the first sending unit is used to send a resource response to the first agent module.

[0096] In the second implementation of the fifth aspect, the first requirement includes a first identifier and a first AI resource requirement, the second requirement includes a second identifier and a second AI resource requirement, the first computing card satisfies the first AI resource requirement, the second computing card satisfies the second AI resource requirement, and the task execution device further includes an acquisition unit, a storage unit, and a determination unit. The acquisition unit is used to acquire a service address table according to the AI ​​resource requirements of the first task, the service address table including the correspondence between the first identifier and the first service address, and the correspondence between the second identifier and the second service address; the storage unit is used to store the service address table; the first receiving unit is further used to receive a first query request and a second query request sent by the task agent, the first query request carrying the first identifier, and the second query request carrying the second identifier; the determination unit is used to determine the first service address according to the service address table and the first identifier carried in the first query request, and to determine the second service address according to the service address table and the second identifier carried in the second query request; the first sending unit is used to send a first query response to the task agent, the first query response carrying the first service address, and to send a second query response to the task agent, the second query response carrying the second service address.

[0097] In the alternative technical solution of the second implementation, the resource manager includes a resource agent and a resource controller. The resource controller is used to manage the computing cards in the computing cluster, and the task execution device further includes a second receiving unit and a second sending unit. The first receiving unit is used for the resource proxy to receive the resource request sent by the task proxy; the first sending unit is also used for the resource proxy to send the resource request to the resource controller; the second receiving unit is used for the resource controller to receive the resource request sent by the resource proxy; the obtaining unit is used for the resource controller to obtain the service address table according to the AI ​​resource requirements of the first task carried in the resource request; the second sending unit is used for the resource controller to send the service address table to the resource proxy; the storage unit is used for the resource proxy to store the service address table; the first receiving unit is used for the resource proxy to receive the first query request and the second query request sent by the task proxy; the determining unit is used for the resource proxy to obtain the first service address according to the service address table and the first identifier carried in the first query request, and the resource proxy to obtain the second service address according to the service address table and the second identifier carried in the second query request; the first sending unit is used for the resource proxy to send the first query response to the task proxy, and the resource proxy to send the second query response to the task proxy. In this optional technical solution, the first receiving unit, the first sending unit, the storage unit, and the determining unit are all located in the resource proxy, while the second receiving unit, the second sending unit, and the obtaining unit are all located in the resource controller.

[0098] In an optional technical solution of the second implementation, the task agent includes a first agent module and a second agent module. The first receiving unit is used to receive a resource request sent by the first agent module, a first query request sent by the first agent module, and a second query request sent by the second agent module. The first sending unit is used to send a resource response to the first agent module, a first query response to the first agent module, and a second query response to the second agent module.

[0099] For ease of distinction, the resource request described above in the fifth aspect is referred to as the first resource request. Optionally, the first receiving unit is further configured to receive a second resource request sent by the task agent, the second resource request carrying the AI ​​resource requirements of the first task; the first sending unit is further configured to send a third service address to the task agent, the third service address being the service address of a third computing card in the computing cluster, the third computing card satisfying the AI ​​resource requirements of the first task, the node where the third computing card is located being a different node from the node running the task agent, and the third service address being used by the task agent to invoke the third computing card to execute the AI ​​operation corresponding to the first task.

[0100] Sixthly, a task execution apparatus is provided for a first service process. The task execution apparatus includes at least one functional unit for performing the method provided by the third aspect or any optional implementation thereof. The at least one functional unit can be implemented based on software or a combination of software and hardware, and can be arbitrarily combined or divided based on implementation.

[0101] Specifically, the task execution device includes a processing unit and a receiving unit. The processing unit is configured to establish a first connection with a task agent based on a first service address, where the first service address is the service address of a first computing card in the computing cluster. The first service process manages the first computing card, which meets the AI ​​resource requirements of the first task. The node where the first computing card resides is a different node from the node running the task agent. The receiving unit is configured to receive a call request for the first task sent by the task agent through the first connection. The processing unit is further configured to invoke the first computing card to process the call request for the first task to execute the AI ​​operation corresponding to the first task.

[0102] Optionally, the processing unit is configured to: receive a connection establishment request sent by the task agent based on the first service address using the receiving unit; and establish the first connection with the task agent in response to the connection establishment request.

[0103] Optionally, the processing unit is configured to: cache the call request of the first task in a first request queue, the first request queue corresponding to the first task; and call the first computing card to process the call request in the first request queue.

[0104] Optionally, the connection establishment request carries the task identifier of the first task, and the processing unit is further configured to create the first request queue in response to the connection establishment request.

[0105] Optionally, the connection establishment request also carries the priority of the first task, and the processing unit is further configured to cache the task identifier of the first task in a first task queue in response to the priority of the first task satisfying the first priority. The first task queue corresponds to the first priority and is used to cache the task identifier of tasks whose priority satisfies the first priority.

[0106] Optionally, the processing unit is configured to: schedule task identifiers in the first task queue in a time-slice round-robin manner; when the minimum interference level corresponding to the first task is not less than a threshold and the task identifier of the first task in the first task queue is scheduled, call the first computing card to process the call request in the first request queue within the first time slice corresponding to the first task, so that the first task exclusively occupies the first computing card, the minimum interference level corresponding to the first task is the interference level of the second task on the first task, the second task is the unfinished task with the least interference level on the first task among the tasks whose task identifiers are cached in the second task queue, the second task queue corresponds to a second priority, the first priority is higher than the second priority, and the second task queue is used to cache task identifiers of tasks whose priority meets the second priority; when the minimum interference level corresponding to the first task is less than the threshold and the task identifier of the first task in the first task queue is scheduled, call the first computing card to process the call request in the first request queue and the call request in the second request queue within the first time slice, so that the first task and the second task share the first computing card in a space-division multiplexing manner, and the second request queue corresponds to the second task.

[0107] Optionally, the processing unit is further configured to, if the interference level of the second task on the first task is less than the threshold, if the second task is completed within the first time slice, call the first computing card to process the call requests in the first request queue and the call requests in the third request queue during the remaining time period of the first time slice, so that the first task and the third task share the first computing card in a space-division multiplexing manner, wherein the third request queue corresponds to the third task, and the third task is the unfinished task with the least interference level on the first task, excluding the second task, among the tasks whose task identifiers are cached in the second task queue.

[0108] Optionally, the processing unit is further configured to, if the first task is not completed at the end of the first time slice, adjust the task identifier of the first task from the head of the first task queue to the tail of the first task queue.

[0109] Optionally, the processing unit is further configured to delete the task identifier of the first task from the first task queue if the first task is completed within the first time slice.

[0110] Optionally, the processing unit is further configured to determine the minimum interference level corresponding to the first task within a second time slice corresponding to the first task, provided that the task identifier of the first task is scheduled to the first task queue, wherein the second time slice is located before the first time slice.

[0111] Optionally, the second task queue caches task identifiers for at least one task. The processing unit is configured to: within the second time slice: first, invoke the first computing card to process m call requests in the first request queue, so that the first task exclusively occupies the first computing card, where m is a positive integer; then, sequentially schedule at least one task group, wherein the at least one task group corresponds one-to-one with the at least one task, and each task group includes the first task and the task corresponding to each task group; when any task group in the at least one task group is scheduled to: invoke the first computing card to process the m call requests in the first request queue and the call requests in the request queue of the task corresponding to the task of the task group, so that the first task and the task corresponding to the task group share the first computing card in a space-division multiplexing manner; and, based on the period during which the first task exclusively occupies the first computing card, the first computing card processes the m call requests in the first request queue. The time taken to process m call requests in the first request queue is used to determine the exclusive throughput of the first task, which is the throughput of the first task when it exclusively uses the first computing card. Based on the time taken for the first task and any task group to share the first computing card, the shared throughput of the first task with any task group is determined, which is the throughput of the first task when it shares the first computing card with any task group. Based on the exclusive throughput and at least one shared throughput of the first task, at least one level of interference is determined for the first task, where each shared throughput corresponds to one of the at least one task group, and each interference level corresponds to one of the at least one shared throughput. The minimum value among the at least one level of interference is determined as the minimum interference level for the first task.

[0112] Optionally, the processing unit is further configured to, when there are no unfinished task identifiers in the first task queue, invoke the first computing card to process the call requests in the request queues of at least two tasks corresponding to at least two task identifiers in the second task queue, so that the at least two tasks share the first computing card in a space-division multiplexing manner, wherein the at least two tasks are unfinished tasks.

[0113] Optionally, the first service process includes at least one thread, and the at least one thread corresponds one-to-one with at least one task, the at least one task including the first task; the processing unit is used to call the first computing card to process the call request of the corresponding task using each of the at least one thread.

[0114] Optionally, the processing unit is further configured to disconnect the first connection in response to the first computing card meeting a disconnection condition. For example, the disconnection condition may include a malfunction of the first computing card or the load on the first computing card exceeding a preset load.

[0115] A seventh aspect provides a task execution apparatus, including a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program stored in the memory so that the task execution apparatus performs the method provided by the first aspect or any alternative implementation thereof.

[0116] Eighthly, a task execution apparatus is provided, including a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program stored in the memory so that the task execution apparatus performs the method provided by the second aspect or any alternative implementation thereof.

[0117] A ninth aspect provides a task execution apparatus, including a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program stored in the memory so that the task execution apparatus performs the method provided by the third aspect or any alternative implementation thereof.

[0118] In a tenth aspect, a task execution system is provided, the task execution system including a task agent and a resource manager, the task agent including any optional implementation of the fourth aspect above or the task execution apparatus provided in the seventh aspect above, and the resource manager including any optional implementation of the fifth aspect above or the task execution apparatus provided in the eighth aspect above.

[0119] Optionally, the task execution system further includes a first service process, which includes the task execution apparatus provided as described in the sixth aspect, any optional implementation of the sixth aspect, or the ninth aspect.

[0120] Eleventhly, a computer-readable storage medium is provided, which stores a computer program that, when executed, implements the method provided by the first aspect or any optional implementation thereof, or implements the method provided by the second aspect or any optional implementation thereof, or implements the method provided by the third aspect or any optional implementation thereof.

[0121] In a twelfth aspect, a computer program product is provided, comprising a program or code that, when executed, implements the method provided by the first aspect or any optional implementation thereof, or implements the method provided by the second aspect or any optional implementation thereof, or implements the method provided by the third aspect or any optional implementation thereof.

[0122] The technical effects of aspects four through twelfth and their optional implementations, which are not shown above, can be referred to the technical effects of aspects one through three and their optional implementations, and therefore are not elaborated upon. Attached Figure Description

[0123] Figure 1 This is a diagram illustrating how the CPU calls the GPU to execute model training tasks. Figure 2 This is a schematic diagram of a computing cluster provided in an embodiment of this application; Figure 3 This is a schematic diagram of another computing cluster provided in an embodiment of this application; Figure 4 This is a schematic diagram of another computing cluster provided in an embodiment of this application; Figure 5 This is a schematic diagram of yet another computing cluster provided in an embodiment of this application; Figure 6 This is a flowchart of a task execution method provided in an embodiment of this application; Figure 7 This is a flowchart of another task execution method provided in an embodiment of this application; Figure 8 This is a schematic diagram of a task execution method provided in an embodiment of this application; Figure 9 This is a schematic diagram of another task execution method provided in an embodiment of this application; Figure 10This is a schematic diagram of another task execution method provided in an embodiment of this application; Figure 11 This is a schematic diagram of yet another task execution method provided in an embodiment of this application; Figure 12 This is a flowchart of another task execution method provided in the embodiments of this application; Figure 13 This is a schematic diagram of yet another task execution method provided in an embodiment of this application; Figure 14 This is a schematic diagram of a task execution device provided in an embodiment of this application; Figure 15 This is a schematic diagram of another task execution device provided in an embodiment of this application; Figure 16 This is a schematic diagram of another task execution device provided in the embodiments of this application. Detailed Implementation

[0124] The embodiments of this application will now be described in further detail with reference to the accompanying drawings.

[0125] A computing cluster typically includes a scheduler and multiple compute nodes. The scheduler distributes tasks from tenants to these compute nodes for execution. Specifically, each compute node includes a central processing unit (CPU) and compute cards. After the scheduler assigns a task to any compute node, the CPU on that node invokes the compute card on that node to execute the task (specifically, to perform the corresponding operation, also known as the task-related operation). A compute card is a computing chip or component, other than a CPU, that possesses dedicated computing functions such as high-performance computing (HPC) and artificial intelligence (AI) computing. For example, a compute card could be a graphics processing unit (GPU). Tasks can be model training tasks (also known as deep learning tasks), model inference tasks, AI computing tasks (such as high-performance computing tasks), etc. In some documents, tasks are also referred to as jobs, and the scheduler is called a task scheduler, cluster scheduler, job scheduler, or similar terms.

[0126] However, in current technology, a CPU can only execute tasks using the computing cards within its own computing node, resulting in low utilization of computing card resources. For example, because a CPU can only execute tasks using the computing cards within its own computing node, when the scheduler needs to assign a task to a computing node, that node must have sufficient CPU and computing card resources. If the CPU resources of a computing node are exhausted, even if the node has idle computing card resources, it cannot accept new tasks, resulting in idle computing card resources and low utilization of those resources. Here, CPU resources refer to the resources of the CPU itself, and computing card resources refer to the resources of the computing card; both CPU resources and computing card resources include computing resources, memory resources, etc.

[0127] Take model training as an example. The execution process of a model training task typically involves multiple training rounds, each called an epoch. Within each epoch, the computing cluster iterates through the model to be trained on the training dataset. Specifically, a model training task requires at least one computing node in the computing cluster (larger model training tasks may require multiple computing nodes). During each iteration, the CPU in the computing node calls the computing card in that node to execute the operations corresponding to the model training task to train the model. The execution process of a model training task is computationally intensive and places extremely high demands on hardware execution capabilities. Using computing cards with strong computing capabilities, such as GPUs, to execute the operations corresponding to the model training task can accelerate the execution process.

[0128] In this process, the CPU sends a request to the computing card to invoke the operations corresponding to the model training task. Taking a GPU as an example... Figure 1 This is a diagram illustrating the operations performed by the CPU when calling the GPU to execute a model training task. For example... Figure 1As shown, a CPU can run training scripts, deep learning frameworks, GPU runtimes, and GPU drivers, with the training scripts running on top of the deep learning framework. For example, the CPU runs a job process corresponding to the model training task, which includes the deep learning framework, the GPU runtime, the GPU driver, and the training script. The training script and deep learning framework are application-level functional modules, while the GPU runtime and GPU driver are operating system (OS)-level functional modules. The CPU triggers the deep learning framework to send a GPU application program interface (API) call request to the GPU runtime by executing the training script. Upon receiving the GPU API call request, the GPU runtime converts it into a system call request and sends it to the GPU driver. Upon receiving the system call request, the GPU driver converts it into a peripheral component interconnect express (PCIe) call request and sends it to the GPU to invoke the GPU to perform the operations corresponding to the model training task.

[0129] As model size and training tasks increase, the computational demands for training rise, leading to a corresponding expansion of computing clusters. For example, one data center plans to increase its GPU count from 100,000 to 200,000, while other projects aim to build computing clusters with hundreds of thousands of GPUs. However, with this expansion, the resource utilization of individual GPUs becomes increasingly critical. For instance, reports indicate that one company's computing cluster, comprising over 6,000 GPUs, had over 80% of its GPUs with resource utilization below 35%; another company's cluster, with 100 GPUs, had a staggering 90% of its GPUs with resource utilization below 27%.

[0130] Taking GPUs as an example, to improve GPU resource utilization, the industry has developed various GPU sharing technologies. GPU sharing technology aims to reuse the resources of a single GPU for multiple tasks, enabling cloud service providers to improve the overall computing efficiency of their clusters without increasing hardware investment. Furthermore, GPU sharing technology can provide individual users and small and medium-sized organizations with a low-cost way to use GPUs, thereby accelerating the popularization of AI technology. Currently, the industry has developed GPU sharing technologies including application-level GPU sharing and operating system-level (OS-level) GPU sharing. These two GPU sharing technologies are introduced below.

[0131] Application-level GPU sharing technology enables multiple tasks to share the same GPU by optimizing at the application level. In this GPU sharing technology, AI frameworks (e.g., [examples of optimizations]) are modified, enhanced, or restructured. Figure 1 Deep learning frameworks in China), and job scripts (e.g.) Figure 1 The system obtains the operational semantics (such as execution path, dependencies, and resource requirements) of tasks from training scripts and other means, and implements reasonable optimization strategies based on the operational semantics of tasks during resource scheduling, enabling multiple tasks to share the same GPU. For example, Orion, presented at the European Conference on Computer Systems (EuroSys'24) in 2024, is a typical example of this type of GPU sharing technology. Orion uses performance profiling tools to analyze tasks and obtain the resource requirements of each operator (also called a subtask) of a task across a series of dimensions; during resource scheduling, it runs operators with complementary resource requirements from different tasks in parallel based on the resource requirements of the operators in different tasks, allowing different tasks to share the same GPU in a space-division multiplexing manner.

[0132] While application-level GPU sharing technology can improve GPU resource utilization, it requires modifications at the application level, thus lacking transparency to the application layer. This leads to poor maintainability and high usage costs. Specifically, application-level GPU sharing technology is deeply embedded in the application layer and is highly coupled with specific AI frameworks. Once an AI framework releases a new version or changes its underlying implementation, the GPU sharing technology needs to be adapted and updated synchronously. Given the rapid iteration of the current deep learning ecosystem and frequent updates to AI frameworks, the cost of continuous adaptation is high, making it difficult to maintain stability in long-term production environments, resulting in poor maintainability. Furthermore, application-level GPU sharing technology requires customized modifications to AI frameworks or requires users to modify job scripts and introduce specialized runtime components, which is significantly intrusive to existing workflows. In multi-tenant or large-scale shared environments, it is almost impossible for all users to develop and deploy according to these requirements, hindering the widespread adoption of this GPU sharing technology. Furthermore, application-level GPU sharing technology relies on task preprocessing. For example, Orion requires a task to have its own dedicated GPU to analyze the task. If this GPU sharing technology is deployed in a multi-task scenario, the preprocessing process will introduce a long queuing time, leading to increased costs.

[0133] OS-level GPU sharing technology is a technique that optimizes the OS layer to allow multiple tasks to share the same GPU. OS-level GPU sharing is also known as GPU virtualization. In this technology, calls from the CPU to the GPU are intercepted at the OS layer, for example, by intercepting calls from AI frameworks (e.g.,...). Figure 1This involves intercepting GPU API call requests sent by deep learning frameworks to the GPU runtime, or intercepting system call requests sent by the GPU runtime to the GPU driver, and scheduling the intercepted call requests according to a certain strategy to enable multiple tasks to share the same GPU. For example, transparent GPU sharing (TGS), presented at the Symposium on Network System Design and Implementation (NSDI'23) in 2023, is a typical example of this type of GPU sharing technology. Specifically, the call requests sent from the CPU to the GPU include call requests for productive tasks and call requests for opportunistic tasks, with productive tasks having higher priority than opportunistic tasks. TGS intercepts the call requests sent by the CPU to each task at the OS level and uses the arrival rate of call requests for productive tasks as a control signal to control the sending rate of call requests for opportunistic tasks. This allows opportunistic tasks to utilize idle GPU resources as much as possible without affecting productive tasks, thus enabling productive and opportunistic tasks to share the same GPU.

[0134] OS-level GPU sharing technology requires no modification to job scripts, AI frameworks, or task operation semantics, and is completely transparent to the application layer. It can improve GPU resource utilization without altering existing workflows, exhibiting good versatility and deployment universality. This transparent and non-intrusive nature makes it a more readily adopted sharing solution in multi-tenant computing clusters. However, despite its ability to improve GPU resource utilization, the overall GPU resource utilization remains relatively low. Specifically, in this technology, the CPU can only utilize the GPU within its own compute node to execute tasks. When a task needs to be assigned to a compute node, that node must have sufficient CPU and GPU resources. If the compute node's CPU resources are exhausted, even with available GPU resources, it cannot accept new tasks, resulting in idle GPU resources (a problem also known as GPU external fragmentation) and low GPU resource utilization. On the other hand, in this GPU sharing technology, each task needs to exclusively occupy a GPU context, and multiple GPU contexts can only share the same GPU in a time-slice round-robin manner (i.e., multiple tasks can only share the same GPU in a time-slice round-robin manner, or multiple tasks can only share the same GPU in a time-division multiplexing manner). For tasks with low resource requirements, the GPU will have a large amount of idle resources within the time slice corresponding to the task (this problem is also known as the GPU internal fragmentation problem), resulting in low GPU resource utilization. Although some GPU virtualization technologies support multiple tasks sharing the same GPU in a space-division multiplexing manner, due to their static resource allocation method, the GPU resource utilization is limited, and the GPU internal fragmentation problem still exists. Here, a GPU context refers to an abstract set of independent running states, resource allocation information, and execution environment maintained by the GPU for the task's job process during task execution. In current OS-level GPU sharing technologies, one GPU context corresponds to one job process. For single-process tasks, one GPU context corresponds to one task.

[0135] This application provides a technical solution that is completely transparent to the application layer and has good versatility and applicability. This application allows a CPU to call a computing card external to the node where the CPU resides (e.g., a compute node) to execute tasks, thereby improving the resource utilization of the computing card. Furthermore, this application allows a CPU to call any computing card external to the node where the CPU resides to execute tasks, which not only improves the resource utilization of the computing card but also enhances the flexibility of computing card calls, enabling flexible mapping between tasks and computing cards. For example, if the computing card is a GPU, and a node has idle GPU resources but its CPU resources are exhausted, CPUs in other nodes can call the GPU in that node to execute tasks, solving the problem of external GPU fragmentation. Moreover, in this application, multiple tasks can share the same GPU using a spatial multiplexing method, solving the problem of internal GPU fragmentation and improving GPU resource utilization.

[0136] The technical solutions of the embodiments of this application are described below. First, the application scenarios of the embodiments of this application are introduced.

[0137] This application provides a task execution system. Please refer to the following examples. Figure 2 The diagram illustrates a task execution system provided in an embodiment of this application. The task execution system includes a task agent 10, a resource manager 20, and at least one computing card. Figure 2 Taking n computing cards as an example (n is a positive integer), the resource manager 20 is communicatively connected to the task agent 10 and computing cards 1~n respectively, and the task agent 10 is communicatively connected to computing cards 1~n. Figure 2 For simplicity, the connection lines between the task agent 10 and computing cards 1-n are not shown. The resource manager 20 manages computing cards 1-n and allocates computing cards within them for the first task that the task execution system needs to complete. The task agent 10, based on the service address of the computing card allocated by the resource manager 20 for the first task, calls the computing card to execute the AI ​​operation corresponding to the first task to complete the first task. The computing cards 1-n are located in the computing cluster, specifically within the nodes included in the computing cluster.

[0138] The first task refers to all possible tasks that the task execution system can perform. For example, the first task could be a model training task (also known as a deep learning task), a model inference task, or an AI computing task. A computing card is a computing chip or component other than a CPU that has dedicated computing functions such as HPC and AI computing. For example, a computing card could be a GPU, a neural processing unit (NPU), or a tensor processing unit (TPU). Computing cards are also called extended processing units (XPUs), accelerators, AI accelerators, or AI computing cards. A computing card can include computing resources (also known as AI computing resources) and memory resources (also known as AI memory resources). The computing resources of a computing card can be computing cores or integrated circuits. Integrated circuits can be application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or any combination thereof. PLDs can be complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), generic array logic (GALs), or any combination thereof. The memory resources of a computing card can be high bandwidth memory (HBM) resources or other memory resources.

[0139] In an optional embodiment, the task execution system is a computing cluster comprising multiple nodes. The task agent 10 runs on at least one node, the resource manager 20 runs on at least one node, and computing cards 1 to n are located on at least one node. The node running the task agent 10 and the node running the resource manager 20 may or may not contain computing cards. The node running the resource manager 20 may or may not contain computing cards. Nodes can be servers. These multiple nodes can be connected via network devices (e.g., switches), local area networks, the Internet, private networks, etc., enabling communication between components within these nodes (e.g., task agent 10, resource manager 20, computing cards, etc.). The computing cluster can be an HPC cluster, an AI computing cluster, or a data center computing cluster, or it can be a cloud-native computing cluster; this embodiment does not limit this.

[0140] In this embodiment, the resource manager 20 allocates at least one computing card among computing cards 1 to n for the first task. The node where all or some of these computing cards reside is a different node from the node running the task agent 10. This allows the task agent 10 to call a computing card outside the node where it resides to execute the AI ​​operation corresponding to the first task, thus improving the resource utilization of the computing cards. For example, if the task agent 10 runs on the CPU and is a functional module within the CPU, this embodiment allows the CPU to call a computing card outside the node where it resides to execute the AI ​​operation corresponding to the first task, further improving the resource utilization of the computing cards.

[0141] As an example, after the task execution system accepts the first task, the task agent 10 obtains the AI ​​resource requirements of the first task and sends a resource request (for example, referred to as resource request X) to the resource manager 20. Resource request X is the resource request of the first task, carrying the AI ​​resource requirements of the first task. After receiving resource request X, resource manager 20 responds to resource request X by allocating a first computing card among computing cards 1 to n that meets the AI ​​resource requirements of the first task. The node where the first computing card is located is a different node from the node running task agent 10. Then, resource manager 20 sends the service address of the first computing card (for example, referred to as the first service address) to task agent 10. After receiving the first service address, task agent 10 calls the first computing card to execute the AI ​​operation corresponding to the first task based on the first service address. In an optional embodiment, the first task is relatively large in scale and has high AI resource requirements. In response to resource request X, resource manager 20 allocates a second computing card from computing cards 1-n to meet the AI ​​resource requirements of the first task. Then, resource manager 20 sends the service address of the second computing card (for example, referred to as the second service address) to task agent 10. After receiving the second service address, task agent 10 invokes the second computing card to execute the AI ​​operation corresponding to the first task based on the second service address. The first and second computing cards can be any two computing cards from computing cards 1-n that meet the AI ​​resource requirements of the first task. The node where the second computing card is located is a different node from the node running task agent 10, or the node where the second computing card is located is the same node as the node running task agent 10. The first and second computing cards can be located on the same node or on different nodes. For example, the first calculation card is calculation card 1, and the second calculation card is calculation card 2; or, the first calculation card is calculation card 1, and the second calculation card is calculation card n; or, the first calculation card is calculation card 2, and the second calculation card is calculation card 1.

[0142] In an optional embodiment, please refer to Figure 3 This illustrates a schematic diagram of another task execution system provided in an embodiment of this application. Figure 2 Based on this, the task execution system also includes service processes 1 to n, which are connected one-to-one with computing cards 1 to n. Resource manager 20 communicates with service processes 1 to n, thereby communicating with computing cards 1 to n. Task agent 10 communicates with service processes 1 to n. Figure 3For simplicity, the connection lines between the task agent 10 and service processes 1-n are not shown, thus enabling communication with computing cards 1-n. That is, the resource manager 20 communicates with computing cards 1-n through service processes 1-n, and the task agent 10 also communicates with computing cards 1-n through service processes 1-n. Each service process 1-n manages its corresponding computing card and invokes the corresponding computing card to perform AI operations; the resource manager 20 manages computing cards 1-n through service processes 1-n; and the task agent 10, based on the service address of the computing card allocated by the resource manager 20 for the first task, invokes the corresponding service process of that computing card to perform AI operations. In a specific embodiment, each service process (1-n) registers its corresponding computing card's information with the resource manager 20 during the system initialization phase (e.g., the initialization phase of the computing card corresponding to the service process, or the initialization phase of the node where the computing card is located). This gives the resource manager 20 a database containing information about computing cards 1-n. Subsequently, the service processes can periodically update the status information of their corresponding computing cards to the resource manager 20, causing the resource manager 20 to periodically update the database. For example, a service process can periodically send heartbeat packets to the resource manager 20 to update the status information of its corresponding computing card. The computing card's information includes its service address and, potentially, its status information. The service address is the communication address of the service process corresponding to the computing card; for example, the service address is an Internet Protocol (IP) address, generated by the service process corresponding to the computing card. The status information includes, for example, the computing card's health status, resource utilization, and load. In a specific embodiment, the task agent 10 establishes a communication connection with the service process corresponding to the computing card based on the service address of the computing card, and sends a call request to the service process corresponding to the computing card through the communication connection, so that the service process corresponding to the computing card calls the computing card to perform AI operations according to the call request. Each service process 1 to n and its corresponding computing card can reside in the same node; this embodiment does not limit this. When the computing card is a GPU, the service process corresponding to the computing card is also called a GPU service process.

[0143] For ease of description, the service process corresponding to the first computing card is referred to as the first service process, and the service process corresponding to the second computing card is referred to as the second service process. The task agent 10 is used to: invoke the first computing card through the first service process based on the first service address to execute the AI ​​operation corresponding to the first task; and invoke the second computing card through the second service process based on the second service address to execute the AI ​​operation corresponding to the first task. In a specific embodiment, the task agent 10 establishes a first connection with the first service process based on the first service address; the task agent 10 sends a call request for the first task to the first service process through the first connection; the first service process receives the call request for the first task through the first connection; the first service process invokes the first computing card to process the call request for the first task, thereby invoking the first computing card to execute the AI ​​operation corresponding to the first task. Similarly, the task agent 10 establishes a second connection with the second service process based on the second service address; the task agent 10 sends a call request for the first task to the second service process through the second connection; the second service process receives the call request for the first task through the second connection; and the second service process invokes the second computing card to process the call request for the first task, thereby invoking the second computing card to execute the AI ​​operation corresponding to the first task. The first service process and the first computing card can be located in the same node, and the second service process and the second computing card can be located in the same node.

[0144] In an optional embodiment, please refer to Figure 4 This illustrates a schematic diagram of another task execution system provided in an embodiment of this application. Figure 4As shown, the resource manager 20 includes a resource proxy 201 and a resource controller 202. The resource proxy 201 is communicatively connected to the task proxy 10 and the resource controller 202, respectively. The resource controller 202 is communicatively connected to computing cards 1-n. The resource controller 202 is the central scheduler of AI resources for this task execution system and is used to manage computing cards 1-n. In a specific embodiment, the resource controller 202 is communicatively connected to service processes 1-n, thereby communicating with computing cards 1-n through service processes 1-n. The resource controller 202 is used to manage computing cards 1-n through service processes 1-n. Each service process in service processes 1-n registers the relevant information of the computing card corresponding to the service process with the resource controller 202 during the system initialization phase, and then periodically updates the status information of the computing card corresponding to the service process to the resource controller 202. The resource proxy 201 is used to interface with the task proxy 10 and to request AI resources for the first task from the resource controller 202 on behalf of the task proxy 10. Resource agent 201 runs on at least one node, and resource controller 202 runs on at least one node. The node running resource agent 201 and the node running resource controller 202 may be the same or different. The node running resource agent 201 and the node running task agent 10 may be the same or different. The node running resource controller 202 and the node running task agent 10 may be the same or different. The node running resource agent 201 may or may not include a computing card. The node running resource controller 202 may or may not include a computing card. In optional embodiments, both resource agent 201 and resource controller 202 include processes. For example, resource controller 202 includes multiple processes, which are distributed across multiple nodes; this embodiment does not limit this.

[0145] In optional embodiments, please continue to refer to Figure 4 Task proxy 10 includes a first proxy module and a second proxy module. Both the first and second proxy modules can be API proxy modules. They can reside on the same node or on different nodes. Both the first and second proxy modules are communicatively connected to resource proxy 201, and the first proxy module is communicatively connected to computing cards 1-n (specifically, the first proxy module communicates with computing cards 1-n through service processes 1-n). Figure 4 For simplicity, the connection lines between the first agent module and service processes 1~n are not shown. The second agent module is communicatively connected to computing cards 1~n (specifically, the second agent module communicates with computing cards 1~n through service processes 1~n). Figure 4For simplicity, the connection lines between the first agent module and service processes 1~n are not shown. The resource proxy 201 is used to interface with the first agent module and / or the second agent module. In a specific embodiment, the first agent module and the second agent module may include a master agent module and a slave agent module. The master agent module is used to request AI resources for the first task, and the resource proxy 201 is used to proxy the master agent module in requesting AI resources from the resource controller 202 for the first task. For example... Figure 4 As shown, the task execution system also includes a first job process and a second job process. Both the first and second job processes are job processes for the first task, and each includes a master job process and a slave job process. The first job process includes a first proxy module, and the second job process includes a second proxy module. The proxy module located in the master job process is the master proxy module, and the proxy module located in the slave job process is the slave proxy module. For example, if the first job process is the master job process and the second job process is the slave job process, then the first proxy module is the master proxy module, and the second proxy module is the slave proxy module. When the first job process (i.e., the master job process) starts, the first proxy module (i.e., the master proxy module) obtains the AI ​​resource requirements of the first task and sends a resource request X to the resource proxy 201 to trigger the resource application process. By executing the resource application process, the first proxy module can determine the first service address, and the second proxy module can determine the second service address. Furthermore, the first agent module can invoke the first computing card to execute the AI ​​operation corresponding to the first task based on the first service address (specifically, the first agent module invokes the first computing card to execute the AI ​​operation corresponding to the first task through the first service process based on the first service address), and the second agent module can invoke the second computing card to execute the AI ​​operation corresponding to the first task based on the second service address (specifically, the second agent module invokes the second computing card to execute the AI ​​operation corresponding to the first task through the second service process based on the second service address). For example... Figure 4As shown, the first job process includes a first job script and a first AI framework, and the second job process includes a second job script and a second AI framework. The first proxy module can be a low-level functional module embedded in the first job process as a dynamic library, and the second proxy module can be a low-level functional module embedded in the second job process as a dynamic library. Both the first job script and the second job script are job scripts for the first task, and the first job script and the second job script can be the same. The first AI framework and the second AI framework can also be the same. After the first job process starts, it can trigger the first AI framework to send a call request for the first task by executing the first job script. The first proxy module intercepts the call request for the first task sent by the first AI framework and sends (e.g., forwards) the call request for the first task to the first service process to call the first computing card to perform the AI ​​operation of the first task. After the second job process starts, it can trigger the second AI framework to send a call request for the first task by executing the second job script. The second proxy module intercepts the call request for the second task sent by the second AI framework and sends (e.g., forwards) the call request for the first task to the second service process to call the second computing card to perform the AI ​​operation of the first task. In an optional embodiment, the first proxy module sends an API call request to the first service process for the first task. The first service process includes a computing card runtime and a computing card driver (e.g., if the first computing card is a GPU, then the computing card runtime is a GPU runtime and the computing card driver is a GPU driver). The computing card runtime converts the API call request into a system call request and sends the system call request to the computing card driver. The computing card driver converts the system call request into a PCIe call request and sends the PCIe call request to the first computing card to invoke the first computing card to perform the AI ​​operation of the first task. Similarly, the second proxy module can also send an API call request to the second service process for the first task. The second service process can also include a computing card runtime and a computing card driver. The computing card runtime converts the API call request into a system call request and sends the system call request to the computing card driver. The computing card driver converts the system call request into a PCIe call request and sends the PCIe call request to the second computing card to invoke the second computing card to perform the AI ​​operation of the first task.

[0146] Based on the preceding text, it is easy to understand that at least two computing cards from computing cards 1 to n are located in the same node, or any two computing cards from computing cards 1 to n are located in different nodes. Each service process from service processes 1 to n and its corresponding computing card can be located in the same node. The first job process, the second job process, the resource agent 201, the resource controller 202, and all or part of the service processes 1 to n can be located in the same node or in different nodes. As an example, if computing cards 1 to n are located in n nodes in a one-to-one correspondence, each service process from service processes 1 to n and its corresponding computing card are located in the same node, the resource agent 201 and the resource controller 202 are located in different nodes, and the first job process, the second job process, and the resource agent 201 are located in the same node, then the task execution system provided in this application embodiment can also be as follows: Figure 5 As shown. See also Figure 5 The task execution system includes node A, node B, and nodes 1 through n. Node B is communicatively connected to node A and nodes 1 through n, and node A is communicatively connected to nodes 1 through n. Figure 5 For simplicity, the connection lines between node A and nodes 1-n are not shown. For example, each of nodes A, B, and 1-n includes a network interface card (NIC). The NIC in node B is communicatively connected to the NICs in node A and nodes 1-n via network devices, local area networks (LANs), the Internet, private networks, etc. Similarly, the NICs in node A are communicatively connected to the NICs in nodes 1-n via network devices, LANs, the Internet, private networks, etc. The first job process, the second job process, and the resource proxy 201 are all located in node A, the resource controller 202 is located in node B, computing cards 1-n are located one-to-one in nodes 1-n, and service processes 1-n are located one-to-one in nodes 1-n. In a specific embodiment, each of nodes A, B, and 1-n includes a CPU. The first job process, the second job process, and the resource proxy 201 run on the CPU in node A, the resource controller 202 runs on the CPU in node B, and each service process 1-n runs on the CPU in its corresponding node. Figure 5 As shown, each of nodes A, B, and nodes 1 to n also includes host memory and a storage disk for storing information.

[0147] In optional embodiments, such as Figure 4 and Figure 5 As shown, the task execution system provided in this application embodiment includes a tenant side and a computing side, with the tenant side also referred to as the business side. The task agent 10 (including a first agent module and a second agent module), the first job process, the second job process, and the resource agent 201 can all be located on the tenant side, while the resource controller 202, computing cards 1~n, and service processes 1~n can all be located on the computing side. Figure 5 As shown, nodes A and B both include CPUs but do not include computing cards, and nodes 1 to n all include computing cards. Nodes that include CPUs but do not include computing cards are also called general-purpose computing nodes, while nodes that include computing cards are also called dedicated computing nodes, AI computing nodes, or intelligent computing nodes. In an optional embodiment, the tenant side consists of general-purpose computing nodes accessible to the tenant. These general-purpose computing nodes may not be equipped with computing cards. The computing side includes intelligent computing nodes equipped with computing cards, and may also include general-purpose computing nodes. The tenant side can be responsible for interfacing with the tenant and calling the computing side to execute the AI ​​operations corresponding to the tenant's tasks; the computing side can be responsible for processing requests from the tenant side and executing the AI ​​operations corresponding to the tasks under the invocation of the tenant side. It should be noted that the description in this paragraph is only an example and is not intended to limit the technical solutions of the embodiments of this application. In some embodiments, at least one of the task agent 10, the first job process, the second job process, and the resource agent 201 can be located on the computing side. For example, at least one of the task agent 10, the first job process, the second job process, the resource agent 201, and the resource controller 202 runs in an intelligent computing node that includes a computing card. In other embodiments, the task execution system may not distinguish between the tenant side and the computing side, and this application embodiment does not limit this.

[0148] In an optional embodiment, resource proxy 201 corresponds to the first tenant and serves the first tenant. Task proxy 10 corresponds to the first task, which is a task of the first tenant. Each task of the first tenant corresponds to a task proxy. Resource proxy 201 is used to request AI resources for the task from resource controller 202 on behalf of the task proxy corresponding to each task of the first tenant. The task proxy corresponding to each task of the first tenant is used to call the computing card allocated by resource controller 202 for the task to execute the AI ​​operation corresponding to the task. In an optional embodiment, the task execution system serves multiple tenants. The task execution system includes multiple resource proxies that correspond one-to-one with the multiple tenants. Each resource proxy is used to request AI resources for the task from resource controller 202 on behalf of the task proxy corresponding to the task of the corresponding tenant. The multiple tenants include the first tenant, and the multiple resource proxies include resource proxy 201. It should be noted that the description in this paragraph is only for example and is not intended to limit the technical solution of the embodiments of this application. In some embodiments, multiple tenants may correspond to one resource proxy, and multiple tasks may correspond to one task proxy. For example, a resource broker is located on the compute side, and multiple tenants can correspond to one resource broker.

[0149] It should be noted that, Figure 4 and Figure 5The following examples illustrate how Task Agent 10 includes a first agent module and a second agent module, and Resource Manager 20 includes a resource agent 201 and a resource controller 202. This does not mean that if Task Agent 10 includes a first agent module and a second agent module (e.g., Task Agent 10 includes multiple agent modules), Resource Manager 20 necessarily includes resource agent 201 and resource controller 202. Nor does it mean that if Resource Manager 20 includes resource agent 201 and resource controller 202, Task Agent 10 necessarily includes a first agent module and a second agent module (e.g., Task Agent 10 includes multiple agent modules). The inclusion of a first agent module and a second agent module in Task Agent 10 has no direct or indirect binding relationship with Resource Manager 20 including resource agent 201 and resource controller 202. Even if Task Agent 10 includes only one agent module, Resource Manager 20 may still include resource agent 201 and resource controller 202. In the case where the resource manager 20 is treated as a whole and not further divided into resource agent 201 and resource controller 202 (for example, where the resource manager 20 only includes resource controller 202), the task agent 10 may also include multiple agent modules.

[0150] It should be noted that, Figures 2 to 5 The task execution system illustrated is merely an example of an application scenario for this application and is not intended to limit the application scenario of the embodiments of this application. The application scenario of the embodiments of this application can be flexibly adjusted according to actual conditions. For example, the task execution system may also include a task scheduler, which is used to schedule tasks from tenants; after the task scheduler schedules a task to a general-purpose computing node, the CPU in the general-purpose computing node allocates at least one job process for the task. For example, the task scheduler schedules a first task to node A, the CPU in node A allocates a first job process (i.e., a master job process) to the first task, and the first job process, as the master job process, allocates a second job process (i.e., a slave job process) to the first task. The embodiments of this application do not limit this.

[0151] It should also be noted that the task broker, resource broker, resource controller, service process, etc., provided in the embodiments of this application can be software components. These software components can be released and upgraded through system services, container images, or middleware, etc., requiring only the installation or flashing of the relevant software on nodes (e.g., servers) or virtual machines. The embodiments of this application represent a "soft modification" of the server; the technical solution of the embodiments of this application can be implemented by deploying the above-mentioned software components at the operating system layer and user space, without requiring changes to the server hardware or the addition of dedicated hardware. The software components provided in the embodiments of this application can be deployed as independent software products in user-built computing clusters, or provided externally as cloud services. For example, they can be provided to tenants in the form of Infrastructure as a Service (IaaS) or Platform as a Service (PaaS). The software components provided in the embodiments of this application can also be combined with existing container orchestration platforms (such as Kubernetes), for example, the software components can be integrated into the container orchestration platform as plugins or system components. Although this example uses software components such as the task agent, resource agent, resource controller, and service process as illustrations, it is understood that these components can also be implemented using hardware or a combination of software and hardware. For example, they can be implemented using ASICs or PLDs, where the PLD can be a CPLD, FPGA, GAL, or any combination thereof. This application does not limit the implementation method of the task agent, resource agent, resource controller, and service process.

[0152] The above describes the application scenarios of the embodiments of this application. The following describes the method embodiments of this application in conjunction with the above application scenarios.

[0153] Please refer to Figure 6 The diagram illustrates a flowchart of a task execution method provided in an embodiment of this application. This task execution method is performed in conjunction with a task agent and a resource manager. For example, the task agent is... Figures 2 to 5 The task agent 10 in the task execution system shown is the resource manager. Figures 2 to 5 The task execution system shown is Resource Manager 20. See also... Figure 6 The method includes the following steps S601 to S606.

[0154] S601. The task agent sends a resource request X to the resource manager, which carries the AI ​​resource requirements of the first task. The resource manager manages the computing cards in the computing cluster, which includes at least one computing card.

[0155] The AI ​​resource requirements for the first task may include computing resource requirements, memory resource requirements, etc.

[0156] The task agent obtains the AI ​​resource requirements of the first task, then generates a resource request X based on these requirements and sends it to the resource manager. For example, if the first task is the first tenant's task, and the first tenant declares the AI ​​resource requirements of the first task in the task's configuration file, the task agent retrieves these requirements from the configuration file. Alternatively, the first tenant may declare the AI ​​resource requirements of the first task through environment variables, and the task agent retrieves these requirements by querying the environment variables.

[0157] In an optional embodiment, the resource manager includes a resource broker and a resource controller. The resource broker is used to interface with the task broker, and the resource controller is used to manage the computing cards in the computing cluster. The task broker sends resource requests X to the resource broker.

[0158] In an optional embodiment, when the task execution system starts the first task, the task agent executes step S601.

[0159] In an optional embodiment, the task agent includes a first agent module, which is the main agent module for the first task. Step S601 is executed by the first agent module. For example, the first agent module is located in a first job process, which is the main job process for the first task. When the first job starts (i.e., when the main job process starts), the first agent module executes step S601. The main job process can be understood as the core control process of the first task, and its lifecycle is consistent with the state of the first task. For example, when the main job process starts, the first task also starts accordingly; when the main job process terminates, the first task also stops accordingly.

[0160] S602. The resource manager receives resource request X.

[0161] In an optional embodiment, the task agent includes a first agent module, and step S601 is performed by the first agent module, wherein the resource agent receives the resource request X sent by the first agent module.

[0162] In an optional embodiment, the resource manager includes a resource proxy and a resource controller, and step S602 is performed by the resource proxy. After receiving resource request X, the resource proxy can send resource request X to the resource controller so that the resource controller allocates a computing card for the first task.

[0163] S603. In response to resource request X, the resource manager allocates a first computing card to the first task in the computing cluster. The first computing card meets the AI ​​resource requirements of the first task. The node where the first computing card is located is a different node from the node running the task agent.

[0164] In an optional embodiment, the first computing card meets the AI ​​resource requirements of the first task, and the first computing card is a computing card with a low load in the computing cluster. The resource manager obtains the AI ​​resource requirements of the first task from the resource request X, and then allocates the first computing card to the first task in the computing cluster based on the AI ​​resource requirements of the first task and the status of each computing card in the computing cluster.

[0165] In an optional embodiment, the AI ​​resource requirements of the first task include a first requirement. The resource manager, based on the first requirement and the availability of each computing card in the computing cluster, allocates a first computing card to the first task within the computing cluster. The first computing card satisfies the first requirement. In a specific embodiment, the first requirement includes a first identifier and a first AI resource requirement. The resource manager, based on the first AI resource requirement and the availability of each computing card in the computing cluster, allocates a first computing card to the first task within the computing cluster. The first computing card satisfies the first AI resource requirement.

[0166] In an optional embodiment, the resource manager includes a resource agent and a resource controller, and step S603 is performed by the resource controller.

[0167] S604. The resource manager sends the first service address to the task agent. The first service address is the service address of the first computing card.

[0168] The resource manager records the service addresses of each computing card managed by the computing manager. After the resource manager allocates the first computing card to the first task, it sends the first service address to the task agent. For example, during the system initialization phase, such as the initialization phase of the first computing card or the initialization phase of the node where the first computing card resides, the first service process used to manage the first computing card registers the relevant information of the first computing card with the resource manager. This relevant information includes the first service address (i.e., the service address of the first computing card). After receiving the relevant information of the first computing card, the resource manager records the first service address.

[0169] In an optional embodiment, the task agent includes a first agent module, to which the resource manager sends a first service address.

[0170] In an optional embodiment, the resource manager includes a resource proxy and a resource controller, and step S604 is executed by the resource proxy. In a specific embodiment, step S603 is executed by the resource controller. After the resource controller allocates the first computing card to the first task, it sends a first service address to the resource proxy; after receiving the first service address, the resource proxy executes step S604.

[0171] S605. The task agent receives the first service address.

[0172] In an optional embodiment, the resource manager includes a resource proxy and a resource controller. The resource proxy sends a first service address to the task proxy, and the task proxy receives the first service address sent by the resource proxy.

[0173] In an optional embodiment, the task agent includes a first agent module, and step S605 is executed by the first agent module.

[0174] S606. The task agent calls the first computing card based on the first service address to execute the AI ​​operation corresponding to the first task.

[0175] In a specific embodiment, the task agent establishes a first connection based on a first service address, and the first connection corresponds to a first computing card; the task agent invokes the first computing card to execute the AI ​​operation corresponding to the first task based on the first connection. For example, the task agent sends an invocation request for the first task through the first connection to invoke the first computing card to execute the AI ​​operation corresponding to the first task.

[0176] In a specific embodiment, the first service address is the communication address (e.g., IP address) of the first service process. The first service process manages the first computing card, and the first service process and the first computing card reside on the same node. The task agent, based on the first service address, calls the first computing card through the first service process to execute the AI ​​operation corresponding to the first task. For example, the task agent establishes a first connection with the first service process based on the first service address, and sends a call request for the first task to the first service process through the first connection, thereby triggering the first service process to call the first computing card to process the call request for the first task and execute the AI ​​operation corresponding to the first task.

[0177] In an optional embodiment, the task agent includes a first agent module, and step S606 is executed by the first agent module.

[0178] In summary, the technical solution provided in this application embodiment allocates a first computing card that meets the AI ​​resource requirements of the first task within the computing cluster it manages. Furthermore, the node where the first computing card resides is a different node from the node running the task agent. Therefore, the task agent can invoke the first computing card outside the node where the task agent resides to execute the AI ​​operation corresponding to the first task, which helps improve the resource utilization of the first computing card. Moreover, in this application embodiment, the resource manager can allocate any computing card within the computing cluster that meets the AI ​​resource requirements of the first task. This allows the task agent to invoke any computing card outside the node where the task agent resides to execute the AI ​​operation corresponding to the first task. In other words, the resource manager can flexibly allocate computing cards to the first task, and the task agent can flexibly invoke computing cards outside the node where the task agent resides to execute the AI ​​operation corresponding to the first task. This not only improves the resource utilization of the computing cards but also enhances the flexibility of computing card invocation, achieving a flexible mapping between the first task and the computing card.

[0179] Since the task agent can invoke computing cards outside the node where it resides to execute AI operations corresponding to a task, this application embodiment can solve the problem of external fragmentation of computing cards (e.g., GPUs). For example, if the task agent runs on a CPU, this application embodiment can enable the CPU to invoke computing cards outside the node where the CPU resides to execute AI operations corresponding to a task. If a node has idle computing card resources but its CPU resources are exhausted, CPUs in other nodes can invoke computing cards in that node to execute tasks. This expands the service scope of the computing card from a single node to the entire cluster, freeing the computing card from the availability of the local CPU. This allows a task to correspond to CPUs and computing cards across nodes, thus solving the problem of external fragmentation of computing cards. For example, experiments show that the technical solution provided by this application embodiment can reduce external fragmentation of computing cards by 74.1% and reduce the probability of task execution failure by 38.2%.

[0180] In an optional embodiment, please refer to Figure 7 The diagram illustrates a flowchart of another task execution method provided in an embodiment of this application. This task execution method is still performed by a task agent and a resource manager. See also... Figure 7 ,exist Figure 6 Based on the illustrated embodiment, the method further includes the following steps S607 to S610.

[0181] S607. In response to resource request X, the resource manager allocates a second computing card to the first task in the computing cluster. The second computing card meets the AI ​​resource requirements of the first task.

[0182] In optional embodiments, the second computing card meets the AI ​​resource requirements of the first task, and the second computing card is a low-load computing card in the computing cluster. The first and second computing cards are located on the same node; or, the first and second computing cards are located on different nodes. The node where the second computing card is located is a different node from the node running the task agent, or, the node where the second computing card is located is the same node as the node running the task agent. For example, the first and second computing cards are located on the same node, but the node where the second computing card is located is a different node from the node running the task agent. The resource manager obtains the AI ​​resource requirements of the first task from the resource request X, and then allocates the second computing card to the first task in the computing cluster based on the AI ​​resource requirements of the first task and the status of each computing card in the computing cluster.

[0183] In an optional embodiment, the AI ​​resource requirements of the first task include a second requirement. The resource manager, based on the second requirement and the availability of various computing cards in the computing cluster, allocates a second computing card to the first task within the computing cluster. The second computing card satisfies the second requirement. In a specific embodiment, the second requirement includes a second identifier and a second AI resource requirement. The resource manager, based on the second AI resource requirement and the availability of various computing cards in the computing cluster, allocates a second computing card to the first task within the computing cluster. The second computing card satisfies the second AI resource requirement.

[0184] In an optional embodiment, the resource manager includes a resource agent and a resource controller, and step S607 is performed by the resource controller.

[0185] S608. The resource manager sends a second service address to the task agent. The second service address is the service address of the second computing card.

[0186] The resource manager records the service addresses of each computing card managed by the computing manager. After allocating a second computing card for the first task, the resource manager sends the second service address to the task agent. For example, during the system initialization phase, such as the initialization phase of the second computing card, or the initialization phase of the node where the second computing card resides, the second service process used to manage the second computing card registers the relevant information of the second computing card with the resource manager. This relevant information includes the second service address (i.e., the service address of the second computing card). After receiving the relevant information of the second computing card, the resource manager records the second service address.

[0187] In an optional embodiment, the resource manager includes a resource proxy and a resource controller, and step S608 is executed by the resource proxy. In a specific embodiment, step S607 is executed by the resource controller. After the resource controller allocates the second computing card for the first task, it sends a second service address to the resource proxy; after receiving the second service address, the resource proxy executes step S608.

[0188] S609. The task agent receives the second service address.

[0189] In an optional embodiment, the resource manager includes a resource proxy and a resource controller. The resource proxy sends a second service address to the task proxy, and the task proxy receives the second service address sent by the resource proxy.

[0190] S610. The task agent calls the second computing card based on the second service address to execute the AI ​​operation corresponding to the first task.

[0191] In a specific embodiment, the task agent establishes a second connection based on a second service address, and the second connection corresponds to a second computing card. The task agent then invokes the second computing card to execute the AI ​​operation corresponding to the first task based on the second connection. For example, the task agent sends an invocation request for the first task through the second connection to invoke the second computing card to execute the AI ​​operation corresponding to the first task.

[0192] In a specific embodiment, the second service address is the communication address (e.g., IP address) of the second service process. The second service process manages the second computing card, and the second service process and the second computing card reside on the same node. The task agent, based on the second service address, calls the second computing card through the second service process to execute the AI ​​operation corresponding to the first task. For example, the task agent establishes a second connection with the second service process based on the second service address, and sends a call request for the first task to the second service process through the second connection, thereby triggering the second service process to call the second computing card to process the call request for the first task and execute the AI ​​operation corresponding to the first task.

[0193] In an optional embodiment, the task agent further includes a second agent module, and step S610 is performed by the second agent module.

[0194] It should be noted that, Figure 7 The implementation process of steps S601 to S606 in the illustrated embodiment can be referred to Figure 6The illustrated embodiments are not described in detail here. S601 to S610 are merely step numbers and are not intended to limit the execution order of the steps. The execution order of the steps in the task execution method provided in this application can be flexibly adjusted according to actual circumstances. For example, steps S603 and S607 can be executed synchronously. In possible implementations, steps S604 and S608 can also be executed synchronously, as can steps S605 and S609.

[0195] In an optional embodiment, after the resource manager allocates the first computing card and the second computing card to the first task, the resource manager sends a resource response to the task agent. In one possible implementation, the resource manager sends the first service address and the second service address in the resource response to the task agent. Specifically, the resource manager sends the first service address and the second service address to the task agent by sending a resource response carrying both the first and second service addresses. The task agent receives the first service address and the second service address by receiving the resource response sent by the resource manager. In this implementation, steps S604 and S608 are executed synchronously, and steps S605 and S609 are executed synchronously. For example, the content sent in steps S604 and S608 can be implemented by sending it in a single step. In another possible implementation, after the resource manager allocates the first computing card and the second computing card to the first task, the resource manager stores the first service address and the second service address. The resource response sent by the resource manager to the task agent only indicates that the resource allocation was successful and does not carry the first service address and the second service address. The task agent obtains the first service address and the second service address by sending a query request to the resource manager. The resource manager sends the first service address and the second service address in the query response to the task agent. In this implementation, steps S604 and S608 can be executed asynchronously, and steps S605 and S609 can be executed synchronously.

[0196] Based on the different implementation methods of the resource manager sending the first service address and the second service address to the task agent, the embodiments of this application may include the following two cases. In an optional embodiment, the AI ​​resource requirements of the first task include a first requirement and a second requirement. The first requirement includes a first identifier and a first AI resource requirement, and the second requirement includes a second identifier and a second AI resource requirement. The first computing card satisfies the first AI resource requirement, and the second computing card satisfies the second AI resource requirement. After the resource manager allocates the first computing card and the second computing card to the first task, in the following two cases, the resource manager obtains (e.g., generates) a service address table (e.g., refers to this service address table as service address table F). Service address table F includes the correspondence between the first identifier and the first service address, and the correspondence between the second identifier and the second service address.

[0197] In the first scenario, the resource manager sends the first and second service addresses in the resource response to the task agent.

[0198] After the resource manager obtains the service address table F, it sends a resource response (e.g., referred to as resource response X) carrying the service address table F to the task agent. This enables the resource manager to send the first and second service addresses to the task agent. Resource response X corresponds to resource request X. The task agent receives resource response X, which corresponds to the resource manager sending resource response X. In this first case, steps S604 and S608 specifically involve the resource manager sending resource response X to the task agent, and steps S605 and S609 specifically involve the task agent receiving resource response X. In an optional embodiment, after receiving a resource response X, the task agent obtains and stores the service address table F carried by the resource response X. Then, the task agent determines a first service address based on a first identifier (e.g., the first identifier included in the first requirement) and the stored service address table F, and invokes the first computing card to execute the AI ​​operation corresponding to the first task based on the first service address. Additionally, the task agent determines a second service address based on a second identifier (e.g., the second identifier included in the second requirement) and the stored service address table F, and invokes the second computing card to execute the AI ​​operation corresponding to the first task based on the second service address. For example, the task agent invokes the first computing card to execute the AI ​​operation corresponding to the first task through a first service process based on the first service address, and the task agent invokes the second computing card to execute the AI ​​operation corresponding to the first task through a second service process based on the second service address.

[0199] by Figure 3 The following example illustrates the task execution system. For instance, if the first computing card is computing card 1 and the second computing card is computing card 2, then the first service process is service process 1, and the second service process is service process 2. As another example, if the first computing card is computing card 1 and the second computing card is computing card n, then the first service process is service process 1, and the second service process is service process n. And yet another example, if the first computing card is computing card 2 and the second computing card is computing card 1, then the first service process is service process 2, and the second service process is service process 1. This example uses computing card 1 as the first computing card and computing card n as the second computing card. Please refer to [the example provided]. Figure 8 The diagram illustrates a task processing method provided in an embodiment of this application. Figure 8 This task processing method is applied to Figure 3 The task execution system shown is used as an example for illustration. Figure 8As shown, after the task execution system receives the first task, the task agent 10 sends a resource request X to the resource manager 20. Resource request X carries the AI ​​resource requirements of the first task, which include a first requirement and a second requirement. The first requirement includes a first identifier and a first AI resource requirement, and the second requirement includes a second identifier and a second AI resource requirement. Upon receiving resource request X, resource manager 20 responds by allocating a first computing card (i.e., computing card 1) and a second computing card (i.e., computing card n) from computing cards 1 to n for the first task. The first computing card satisfies the first AI resource requirement, and the second computing card satisfies the second AI resource requirement. Resource manager 20 then obtains the service address table F. After obtaining the service address table F, resource manager 20 sends a resource response X carrying the service address table F to task agent 10. After receiving a resource response X, task agent 10 obtains and stores the service address table F carried by resource response X. Furthermore, task agent 10 determines a service address based on the stored service address table F (for example, task agent 10 determines a first service address based on a first identifier and the stored service address table F, and task agent 10 determines a second service address based on a second identifier and the stored service address table F). After determining the first service address, task agent 10 invokes a first computing card to perform AI operations based on the first service address through a first service process. After determining the second service address, task agent 10 invokes a second computing card to perform AI operations based on the second service address through a second service process.

[0200] In an optional embodiment, the task agent includes a first agent module and a second agent module, where the first agent module is the master agent module and the second agent module is the slave agent module. In this first scenario: the first agent module and the second agent module run on the same node. In step S601, the first agent module sends a resource request X to the resource manager. In the specific implementations of steps S604 and S608, the resource manager sends a resource response X to the first agent module. In the specific implementations of steps S605 and S609, the first agent module receives the resource response X sent by the resource manager. After receiving the resource response X, the first agent module obtains the service address table F carried by the resource response X and stores the service address table F in the shared memory region S, which is a shared memory region between the first agent module and the second agent module. Then, the first agent module determines a first service address based on a first identifier and the service address table F stored in the shared memory region S, and calls the first computing card to execute the AI ​​operation corresponding to the first task based on the first service address. Similarly, the second agent module determines a second service address based on a second identifier and the service address table F stored in the shared memory region S, and calls the second computing card to execute the AI ​​operation corresponding to the first task based on the second service address. In a specific embodiment, the task execution system includes a first job process and a second job process. The first job process and the second job process run on the same node. Both the first job process and the second job process are job processes of the first task. The first job process is the master job process and the second job process is the slave job process. The first job process includes a first agent module and the second job process includes a second agent module.

[0201] In an optional embodiment, the resource manager includes a resource proxy and a resource controller. The resource controller manages the computing cards in the computing cluster, and the resource proxy interfaces with the task proxy. In step S602, the resource proxy receives a resource request X sent by the task proxy. After receiving the resource request X, the resource proxy sends the resource request X to the resource controller. In response to the resource request X, the resource controller allocates a first computing card and a second computing card for the first task and obtains a service address table F. The resource controller then sends a resource response X carrying the service address table F to the resource proxy. In the first case, after receiving the resource response X, the resource proxy sends the resource response X to the task proxy.

[0202] This example illustrates a task agent comprising a first agent module and a second agent module, and a resource manager comprising a resource agent and a resource controller. For further details, please refer to [reference needed]. Figure 9 The diagram illustrates another task processing method provided in the embodiments of this application. Figure 9 This task processing method is applied to Figure 4 The computing cluster shown is used as an example for illustration, and Figure 9 Let's take computing card 1 as the first computing card, computing card n as the second computing card, and the first agent module as the main agent module, and the second agent module as the secondary agent module, as an example. For... Figure 9 In the illustrated embodiment, the first job process and the second job process run on the same node (e.g., the first job process and the second job process run on the same CPU). Figure 9 For simplicity, the job scripts and AI frameworks included in the first and second job processes are not shown. For example... Figure 9 As shown, after the computing cluster undertakes the first task, for example, when the first job process (i.e., the main job process) starts, the first agent module obtains the AI ​​resource requirements of the first task and sends a resource request X to the resource agent 201. The resource request X carries the AI ​​resource requirements of the first task, which include a first requirement and a second requirement. The first requirement includes a first identifier and a first AI resource requirement, and the second requirement includes a second identifier and a second AI resource requirement. After receiving the resource request X, the resource agent 201 sends the resource request X to the resource controller 202. After receiving the resource request X, the resource controller 202 responds to the resource request X by allocating a first computing card (i.e., computing card 1) and a second computing card (i.e., computing card n) for the first task from computing cards 1 to n. The first computing card satisfies the first AI resource requirement, and the second computing card satisfies the second AI resource requirement. Furthermore, the resource controller 202 obtains the service address table F. Afterward, the resource controller 202 sends a resource response X carrying the service address table F to the resource agent 201. After receiving the resource response X, resource proxy 201 sends the resource response X to the first proxy module. Upon receiving the resource response X, the first proxy module retrieves the service address table F carried in the resource response X and stores the service address table F in the shared memory region S. Then, the first proxy module determines the first service address based on the service address table F stored in the shared memory region S (for example, the first proxy module determines the first service address based on the first identifier and the service address table F stored in the shared memory region S), and calls the first computing card to perform AI operations based on the first service address through the first service process (i.e., service process 1). Furthermore, after the first proxy module stores the service address table F in the shared memory region S, when the second job process starts, the second proxy module determines the second service address based on the service address table F stored in the shared memory region S (for example, the second proxy module determines the second service address based on the second identifier and the service address table F stored in the shared memory region S), and calls the second computing card to perform AI operations based on the second service address through the second service process (i.e., service process 2).

[0203] In the second scenario, the resource manager sends the first and second service addresses in the query response to the task agent.

[0204] After the resource manager obtains the service address table F, it stores the service address table F and sends a resource response (e.g., referred to as resource response X') to the task agent. Resource response X' corresponds to resource request X and indicates successful resource allocation, but does not carry the service address table F. Upon receiving resource response X', the task agent obtains a first service address and a second service address by sending query requests to the resource manager. Then, based on the first service address, it calls the first computing card to execute the AI ​​operation corresponding to the first task, and based on the second service address, it calls the second computing card to execute the AI ​​operation corresponding to the first task. For example, after receiving resource response X', the task agent sends a first query request and a second query request to the resource manager. The first query request carries a first identifier, and the second query request carries a second identifier. Upon receiving the first query request, the resource manager determines the first service address based on the service address table F and the first identifier carried in the first query request, and sends a first query response carrying the first service address to the task agent, thus enabling the resource manager to send the first service address to the task agent. After receiving the second query request, the resource manager determines the second service address based on the service address table F and the second identifier carried in the second query request, and sends a second query response carrying the second service address to the task agent, thereby enabling the resource manager to send the second service address to the task agent. Corresponding to the resource manager's first query response to the task agent, the task agent receives the first query response. Corresponding to the resource manager's second query response to the task agent, the task agent receives the second query response. In this second case, step S604 specifically involves the resource manager sending the first query response to the task agent, step S605 specifically involves the task agent receiving the first query response, step S608 specifically involves the resource manager sending the second query response to the task agent, and step S609 specifically involves the task agent receiving the second query response.

[0205] by Figure 3 The task execution system shown is used as an example for illustration, and we will still use computing card 1 as the first computing card and computing card n as the second computing card as an example. Please refer to [the example provided]. Figure 10 The diagram illustrates another task processing method provided in the embodiments of this application. Figure 10 This task processing method is applied to Figure 3 The task execution system shown is used as an example for illustration. Figure 10As shown, after the task execution system receives the first task, the task agent 10 sends a resource request X to the resource manager 20. Resource request X carries the AI ​​resource requirements of the first task, which include a first requirement and a second requirement. The first requirement includes a first identifier and a first AI resource requirement, and the second requirement includes a second identifier and a second AI resource requirement. After receiving resource request X, resource manager 20 responds by allocating a first computing card (i.e., computing card 1) and a second computing card (i.e., computing card n) for the first task from computing cards 1 to n. The first computing card satisfies the first AI resource requirement, and the second computing card satisfies the second AI resource requirement. Resource manager 20 then obtains the service address table F. After obtaining the service address table F, resource manager 20 stores the service address table F. Then, resource manager 20 sends a resource response X' to the task agent 10. After receiving resource response X', task agent 10 sends a first query request to resource manager 20. Upon receiving the first query request, resource manager 20 determines a first service address based on the stored service address table F and the first identifier carried in the first query request, and sends a first query response carrying the first service address to task agent 10. Upon receiving the first query response, task agent 10, based on the first service address, calls the first computing card through the first service process (i.e., service process 1) to execute the AI ​​operation corresponding to the first task. After receiving resource response X', task agent 10 also sends a second query request to resource manager 20. Upon receiving the second query request, resource manager 20 determines a second service address based on the stored service address table F and the second identifier carried in the second query request, and sends a second query response carrying the second service address to task agent 10. Upon receiving the second query response, task agent 10, based on the second service address, calls the second computing card through the second service process (i.e., service process n) to execute the AI ​​operation corresponding to the second task.

[0206] In an optional embodiment, the task proxy includes a first proxy module and a second proxy module, where the first proxy module is the master proxy module and the second proxy module is the slave proxy module. In this second scenario: the first and second proxy modules can run on the same node or on different nodes. In step S601, the first proxy module sends a resource request X to the resource manager. In a specific implementation of step S604, the resource manager sends a first query response to the first proxy module, and in a specific implementation of step S605, the first proxy module receives the first query response. In a specific implementation of step S608, the resource manager sends a second query response to the second proxy module, and in a specific implementation of step S605, the second proxy module receives the second query response. After receiving the first query response, the first proxy module calls the first computing card based on the first service address to execute the AI ​​operation corresponding to the first task. After receiving the second query response, the second proxy module calls the second computing card based on the second service address to execute the AI ​​operation corresponding to the first task. For example, the task execution system includes a first job process and a second job process. Both the first job process and the second job process are job processes of the first task, and the first job process is the master job process and the second job process is the slave job process. The first job process includes a first agent module and the second job process includes a second agent module.

[0207] In an optional embodiment, the resource manager includes a resource proxy and a resource controller. The resource controller manages the computing cards in the computing cluster, and the resource proxy interfaces with the task proxy. In step S602, the resource proxy receives a resource request X sent by the task proxy. After receiving the resource request X, the resource proxy sends the resource request X to the resource controller. In response to the resource request X, the resource controller allocates a first computing card and a second computing card for the first task and obtains a service address table F. The resource controller then sends the service address table F to the resource proxy (e.g., the resource controller sends a resource response X carrying the service address table F to the resource proxy). After receiving the service address table F, the resource proxy stores the service address table F and sends a resource response X' to the task proxy. After receiving the resource response X', the task proxy sends a first query request and a second query request to the resource proxy. After receiving the first query request, the resource proxy determines a first service address based on the service address table F and a first identifier carried in the first query request, and sends a first query response carrying the first service address to the task proxy. After receiving the second query request, the resource agent determines the second service address based on the service address table F and the second identifier carried in the second query request, and sends a second query response carrying the second service address to the task agent.

[0208] This example illustrates a task agent comprising a first agent module and a second agent module, and a resource manager comprising a resource agent and a resource controller. For further details, please refer to [reference needed]. Figure 11 The diagram illustrates another task processing method provided in the embodiments of this application. Figure 11 This task processing method is applied to Figure 4 The task execution system shown is used as an example for illustration, and Figure 11 Let's take computing card 1 as the first computing card, computing card n as the second computing card, and the first agent module as the main agent module, and the second agent module as the secondary agent module, as an example. For... Figure 11 In the illustrated embodiment, the first job process and the second job process can run on the same node or on different nodes. Figure 11 For simplicity, the job scripts and AI frameworks included in the first and second job processes are not shown. For example... Figure 11As shown, after the task execution system receives the first task, for example, when the first job process (i.e., the main job process) starts, the first agent module obtains the AI ​​resource requirements of the first task and sends a resource request X to the resource agent 201. The resource request X carries the AI ​​resource requirements of the first task, which include a first requirement and a second requirement. The first requirement includes a first identifier and a first AI resource requirement, and the second requirement includes a second identifier and a second AI resource requirement. After receiving the resource request X, the resource agent 201 sends the resource request X to the resource controller 202. After receiving the resource request X, the resource controller 202 responds to the resource request X by allocating a first computing card (i.e., computing card 1) and a second computing card (i.e., computing card n) for the first task from computing cards 1 to n. The first computing card satisfies the first AI resource requirement, and the second computing card satisfies the second AI resource requirement. Furthermore, the resource controller 202 obtains the service address table F. Afterward, the resource controller 202 sends a resource response X to the resource agent 201, which carries the service address table F. After receiving resource response X, resource proxy 201 stores the service address table F. Then, resource proxy 201 sends resource response X' to the first proxy module. After receiving resource response X', the first proxy module sends a first query request to resource proxy 201. After receiving the first query request, resource proxy 201 determines the first service address based on the stored service address table F and the first identifier carried in the first query request, and sends a first query response carrying the first service address to the first proxy module. After receiving the first query response, the first proxy module calls the first computing card to execute the AI ​​operation corresponding to the first task based on the first service address through the first service process (i.e., service process 1). After the first agent module receives the resource response X', when the second job process starts, the second agent module sends a second query request to the resource agent 201. After receiving the second query request, the resource agent 201 determines the second service address based on the stored service address table F and the second identifier carried in the second query request, and sends a second query response carrying the second service address to the second agent module. After receiving the second query response, the second agent module calls the second computing card to execute the AI ​​operation corresponding to the second task based on the second service address through the second service process (i.e., service process n).

[0209] As mentioned earlier, the service address table F includes the correspondence between a first identifier and a first service address, and the correspondence between a second identifier and a second service address. For example, the service address table F is shown in Table 1 below.

[0210] Table 1 (Service Address Table F)

[0211] Since the first requirement of the first task includes a first identifier, and the second requirement of the first task includes a second identifier, both the first and second identifiers can be identifiers related to the first task. The first service address is the service address of the first computing card assigned to the first task by the resource manager, and the second service address is the service address of the second computing card assigned to the first task by the resource manager. The service address table includes the correspondence between the first identifier and the first service address, as well as the correspondence between the second identifier and the second service address. Therefore, the mapping between the first task and the computing card assigned to the first task can be achieved through this service address table. Furthermore, since the resource manager can flexibly assign computing cards to the first task, this service address table can flexibly record the service addresses of the computing cards assigned to the first task, achieving flexible mapping between the first task and the computing cards.

[0212] In an optional embodiment, the service address table F further includes relevant information about the first computing card and relevant information about the second computing card. For example, the service address table F includes the correspondence between the first identifier, the first service address, and the relevant information of the first computing card, and the service address table F includes the correspondence between the second identifier, the second service address, and the relevant information of the second computing card. For example, the relevant information of the first computing card includes the identifier of the first computing card, the model of the first computing card, and the identifier of the node where the first computing card is located; the relevant information of the second computing card includes the identifier of the second computing card, the model of the second computing card, and the identifier of the node where the second computing card is located. Then, the service address table F can be as shown in Table 2 below.

[0213] Table 2 (Service Address Table F)

[0214] In this embodiment, the service address table F includes relevant information about the first computing card and the second computing card, which facilitates the resource agent in determining the relevant information of the first and second computing cards. In the first scenario described above, it also facilitates the task agent in determining the relevant information of the first and second computing cards.

[0215] It should be noted that, Figures 8 to 11 This is merely an example. Figures 8 to 11 The step numbers in this application are for ease of description only and are not intended to limit the execution order of the steps. The execution order of the steps in this application embodiment can be flexibly adjusted according to the actual situation.

[0216] As discussed above, the task agent can invoke the first computing card to execute the AI ​​operation corresponding to the first task based on the first service address and the first service process. Similarly, the task agent can invoke the second computing card to execute the AI ​​operation corresponding to the first task based on the second service address and the second service process. The following explanation uses the example of the task agent invoking the first computing card to execute the AI ​​operation corresponding to the first task based on the first service address and the first service process. The implementation process of the task agent invoking the second computing card to execute the AI ​​operation corresponding to the first task based on the second service address and the second service process can be referred to in the following embodiment, simply by replacing the first computing card with the second computing card, the first service process with the second service process, the first agent module with the second agent module, and the first connection with the second connection.

[0217] Please refer to Figure 12 The diagram illustrates a flowchart of another task processing method provided in an embodiment of this application. This task processing method is executed in cooperation between a task agent and a first service process. The first service process manages a first computing card, and the first service process and the first computing card reside on the same node. For example, the task agent is... Figures 2 to 5 The task agent 10 in the task execution system shown has a first computing card. Figures 2 to 5 The task execution system shown has a computing card, and the first service process is... Figures 2 to 5 The service process corresponding to the first computing card in the task execution system shown. See also... Figure 12 The method includes the following steps S1201 to S1204.

[0218] S1201. The task agent establishes a first connection with the first service process based on the first service address.

[0219] In an optional embodiment, the task agent sends a connection establishment request to the first service process based on the first service address. The destination address of the connection establishment request is the first service address. The connection establishment request is used by the task agent to request the establishment of a first connection with the first service process. The first service process receives the connection establishment request and, in response to the connection establishment request, establishes a first connection with the task agent.

[0220] In an optional embodiment, the connection establishment request carries the task identifier of the first task. In response to the connection establishment request, the first service process also creates a request queue corresponding to the first task. This request queue is used to cache the invocation requests of the first task. For ease of description, this embodiment refers to the request queue corresponding to the first task as the first request queue.

[0221] In an optional embodiment, the first service process maintains a first task queue, which corresponds to a first priority. The first task queue is used to cache task identifiers of tasks whose priorities meet the first priority. The connection establishment request also carries the priority of the first task. After receiving the connection establishment request, the first service process, in response to the first task's priority meeting the first priority, caches the task identifier of the first task in the first task queue. In an optional embodiment, the first service process also maintains a second task queue, which corresponds to a second priority. The second task queue is used to cache task identifiers of tasks whose priorities meet the second priority, where the first priority is higher than the second priority.

[0222] In this embodiment, a priority satisfying a first priority can mean that the priority is the first priority or that the priority belongs to the first priority; a priority satisfying a second priority can mean that the priority is the second priority or that the priority belongs to the second priority. In specific embodiments, both the first and second priorities are coarse-grained priorities, with the first priority being high and the second priority being low. The first and second priorities are distinguished by their relative importance. A task's priority can be either coarse-grained or fine-grained. When any task has a coarse-grained priority, its priority can be either the first priority (i.e., high priority) or the second priority (i.e., low-high priority). Specifically, a task satisfying a first priority means that its priority is the first priority, and a task satisfying a second priority means that its priority is the second priority. When any task has a fine-grained priority, specifically, a task satisfying a first priority means that its priority belongs to the first priority (i.e., high priority), and a task satisfying a second priority means that its priority belongs to the second priority (i.e., low-high priority). For example, the task priority is a fine-grained priority, and the task priority is divided into 8 priority levels, namely priority 1 to 8, with priorities decreasing sequentially from 1 to 8. Priorities 1 to 4 correspond to the first priority (i.e., high priority), and priorities 5 to 8 correspond to the second priority (i.e., low priority). Then, for any task: if the task's priority is any one of priority 1 to 4, the task's priority belongs to the first priority (i.e., high priority), and thus the task's priority satisfies the first priority; if the task's priority is any one of priority 5 to 8, the task's priority belongs to the second priority (i.e., high priority), and thus the task's priority satisfies the second priority.

[0223] S1202. The task agent sends a request to invoke the first task to the first service process through the first connection.

[0224] The task agent sends a request to the first service process to invoke the first computing card to perform the AI ​​operation corresponding to the first task.

[0225] S1203. The first service process receives the call request for the first task through the first connection.

[0226] S1204. The first service process calls the first computing card to process the call request of the first task in order to execute the AI ​​operation corresponding to the first task.

[0227] In an optional embodiment, after receiving any call request for the first task, the first service process caches the call request in the first request queue. The first service process then calls the first computing card to process the call requests in the first request queue to execute the AI ​​operation corresponding to the first task. Since the call requests cached in the first request queue are call requests for the first task, the first service process calling the first computing card to process the call requests in the first request queue is equivalent to the first service process calling the first computing card to process the call requests for the first task.

[0228] In an optional embodiment, the first service process maintains a first task queue and a second task queue. The first task queue is used to cache task identifiers of tasks with a priority satisfying a first priority, and the second task queue is used to cache task identifiers of tasks with a priority satisfying a second priority. The task identifiers of the first tasks are cached in the first task queue. Furthermore, the first service process maintains request queues corresponding to the tasks indicated by each task identifier in the first task queue, and request queues corresponding to the tasks indicated by each task identifier in the second task queue. The request queue corresponding to any task is used to cache the call request for that task. It is easy to understand that the request queues maintained by the first service process include the first request queue. The first service process can schedule the task identifiers in the first task queue in a time-slice round-robin manner (also known as time-division multiplexing). When the first service process schedules any task identifier in the first task queue, the first service process calls the first computing card to process the call request in the request queue corresponding to the task indicated by that task identifier. In an optional embodiment, the first service process invokes the first computing card to process call requests in the first request queue, including: when the minimum interference level corresponding to the first task is not less than a threshold Y and the first service process is scheduled to the task identifier of the first task in the first task queue, the first service process invokes the first computing card to process (specifically, only processes) the call requests in the first request queue within the first time slice corresponding to the first task, so that the first task exclusively occupies the first computing card (specifically, the call requests of the first task exclusively occupy the first computing card); when the minimum interference level corresponding to the first task is less than the threshold Y and the first service process is scheduled to the task identifier of the first task in the first task queue, the first service process invokes the first computing card to process (specifically, only processes) the call requests in the first request queue and the call requests in the second request queue within the first time slice corresponding to the first task, so that the first task and the second task share the first computing card in a space-division multiplexing manner (specifically, the call requests of the first task and the call requests of the second task share the first computing card in a space-division multiplexing manner). The second request queue corresponds to the second task, and the second request queue is used to cache the call requests of the second task; the call requests in the second request queue are the call requests of the second task. The minimum level of interference corresponding to the first task is the level of interference of the second task on the first task. The second task is the unfinished task with the lowest level of interference to the first task among the tasks whose task identifiers are cached in the second task queue.

[0229] Since the minimum interference level corresponding to the first task is the interference level of the second task on the first task, and the second task is the incomplete task with the lowest interference level on the first task among the tasks whose task identifiers are cached in the second task queue, it means that if the minimum interference level corresponding to the first task is not less than the threshold Y, it indicates that the interference level of the tasks indicated by each task identifier in the second task queue on the first task is not less than the threshold Y, and the interference level of the tasks indicated by each task identifier in the second task queue on the first task is relatively large. In this case, if the tasks indicated by the task identifiers in the first task and the second task queue share the first computing card in a space-division multiplexing manner, it is likely to have a significant impact on the first task, such as severely affecting the execution efficiency of the first task. Therefore, if the minimum interference level corresponding to the first task is not less than the threshold Y, the first service process ensures that the first task exclusively occupies the first computing card within the first time slice corresponding to the first task, thereby guaranteeing the execution efficiency of the first task. However, if the minimum interference level corresponding to the first task is less than the threshold Y, it means that at least the interference level of the second task on the first task is relatively small. In this case, if the first task exclusively occupies the first computing card, it will cause a waste of resources on the first computing card and trigger internal fragmentation problems. Therefore, when the minimum interference level corresponding to the first task is less than the threshold Y, the first service process enables the first task and the second task to share the first computing card in a space-division multiplexing manner within the first time slice corresponding to the first task. This can ensure the execution efficiency of the first task on the one hand, and the resource utilization of the first computing card on the other hand, thus solving the internal fragmentation problem of the first computing card.

[0230] In an optional embodiment, if the minimum interference level corresponding to the first task (i.e., the interference level of the second task on the first task) is less than a threshold Y, the first service process calls the first computing card within the first time slice corresponding to the first task to process the call requests in the first request queue and the second request queue, so that the first task and the second task share the first computing card in a space-division multiplexing manner. If the second task is completed within the first time slice, that is, if the first time slice has not ended, the first task has not been completed, and the second task has been completed, the first service process calls the first computing card within the remaining time period of the first time slice to process (specifically, it can only process) the call requests in the first request queue and the third request queue, so that the first task and the third task share the first computing card in a space-division multiplexing manner (specifically, the call requests of the first task and the call requests of the third task share the first computing card in a space-division multiplexing manner). The third request queue corresponds to the third task, and the third request queue is used to cache the call requests of the third task. The call requests in the third request queue are the call requests of the third task. The third task is the unfinished task with the least interference level on the first task, excluding the second task, among the tasks whose task identifiers are cached in the second task queue.

[0231] The first service process schedules task identifiers in the first task queue in a time-slice round-robin manner. If the first service process schedules any task identifier in the first task queue, it indicates that the task identifier is at the head of the first task queue. If the first service process schedules the task identifier of the first task in the first task queue, it indicates that the task identifier of the first task is at the head of the first task queue. In an optional embodiment, if the first task is not completed by the end of the first time slice, the first service process moves the task identifier of the first task from the head to the tail of the first task queue. If the first task is completed within the first time slice, the first service process deletes the task identifier of the first task from the first task queue, specifically from the head of the first task queue. In both cases, the task identifier at the head of the first task queue changes. The first service process continues to schedule the task identifier at the head of the first task queue (specifically, the changed task identifier) ​​and calls the first computing card to process the call request in the request queue of the task indicated by the task identifier. The specific implementation process can be referred to the implementation process of the first service process calling the first computing card to process the call request in the first request queue, which will not be elaborated here in this embodiment.

[0232] In an optional embodiment, when the first service process schedules a task identifier for a first task in the first task queue, the first service process determines the minimum interference level corresponding to the first task within the second time slice corresponding to the first task. Then, the first service process determines the first task's allocation method for the first computing card based on the minimum interference level. The second time slice is located before the first time slice. The first task's allocation method for the first computing card includes the first task exclusively occupying the first computing card, or the first task and the task indicated by the task identifier in the second task queue sharing the first computing card in a space-division multiplexing manner. If the minimum interference level corresponding to the first task is not less than a threshold Y, the first service process determines that the first task exclusively occupies the first computing card. If the minimum interference level corresponding to the first task is less than the threshold Y, the first service process determines that the first task and the task indicated by the task identifier in the second task queue sharing the first computing card in a space-division multiplexing manner.

[0233] The following describes the implementation process of the first service process determining the minimum level of interference corresponding to the first task within the second time slice.

[0234] In a specific embodiment, the second task queue caches task identifiers for at least one task. During the second time slice: First, the first service process calls the first computing card to process (specifically, only processes) m call requests in the first request queue, so that the first task exclusively occupies the first computing card (specifically, the m call requests of the first task exclusively occupy the first computing card), where m is a positive integer; then, the first service process sequentially schedules at least one task group, which corresponds one-to-one with the at least one task (i.e., the task indicated by at least one task identifier in the second task queue). Each task group in the at least one task group includes the first task and the task corresponding to each task group. When the first service process schedules to any task group in the at least one task group: the first service process calls the first computing card to process (specifically, only processes) the m call requests in the first request queue and the call requests in the request queue of the task corresponding to the task of the task group, so that the first task and the task corresponding to the task group share the first computing card in a space-division multiplexing manner (specifically, the m call requests of the first task and the call requests of the task corresponding to the task group share the first computing card in a space-division multiplexing manner). The first service process determines the exclusive throughput of the first task based on the time taken for the first computing card to process m call requests in the first request queue during the period when the first task exclusively occupies the first computing card within the second time slice. This exclusive throughput is the throughput of the first task when it exclusively occupies the first computing card. The first service process also determines the shared throughput of the first task with any given task group during the period when the first computing card is shared with the first task within the second time slice. This shared throughput is the throughput of the first task when it shares the first computing card with any given task group. The first computing card can obtain at least one shared throughput of the first task, and this at least one shared throughput corresponds one-to-one with the at least one task group. Based on the exclusive throughput of the first task and the at least one shared throughput of the first task, the first service process determines at least one level of interference corresponding to the first task, and this at least one level of interference corresponds one-to-one with the at least one shared throughput. The first service process determines the minimum value among these at least one level of interference as the minimum level of interference corresponding to the first task. The task corresponding to the task group corresponding to the minimum value among these at least one level of interference is the second task.

[0235] In an optional embodiment, the first service process determines the average time taken for the first computing card to process each of the m call requests in the first request queue during the period when the first task exclusively occupies the first computing card within the second time slice. This average time is the exclusive throughput of the first task. For example, the first service process can obtain the exclusive throughput of the first task by dividing the time taken for the first computing card to process the m call requests in the first request queue during the period when the first task exclusively occupies the first computing card within the second time slice by m.

[0236] In an optional embodiment, the first service process determines the average time taken for the first computing card to process each of the m call requests in the first request queue during the period when the first task and any task group share the first computing card within the second time slice. This average time is the shared throughput of the first task with the corresponding task group. For example, the first service process divides the time taken for the first computing card to process the m call requests in the first request queue during the period when the first task and any task group share the first computing card within the second time slice by m to obtain the shared throughput of the first task with the corresponding task group.

[0237] In an optional embodiment, during the process of the first service process determining at least one level of interference corresponding to the first task based on the exclusive throughput of the first task and at least one shared throughput of the first task, the first service process determines the difference between the exclusive throughput and each of the at least one shared throughput, and the first service process determines the difference as a level of interference corresponding to the first task.

[0238] In an optional embodiment, the tasks indicated by the task identifiers in the first and second task queues are both model training tasks. The "m call requests" mentioned above can be adaptively adjusted to m iterations of training. For example, within the second time slice: First, the first service process calls the first computing card to process (specifically, only processes) the call requests in the first request queue, so that the first task exclusively occupies the first computing card and the first computing card completes m iterations of training for the first task; then, the first service process sequentially schedules the above-mentioned at least one task group; when the first service process schedules to any one of the at least one task group: the first service process calls the first computing card to process (specifically, only processes) the call requests in the first request queue and the call requests in the request queue of the task corresponding to that task group, so that the first task and the task corresponding to that task group share the first computing card in a space-division multiplexing manner and the first computing card completes m iterations of training for the first task. The first service process determines the exclusive throughput of the first task based on the time taken for the first computing card to complete m iterations of training for the first task during the period when the first task exclusively occupies the first computing card within the second time slice. The first service process determines the shared throughput of the first task and its corresponding task group based on the time taken for the first computing card to complete m iterations of training for the first task during the shared first computing card period of the first task and any task group within the second time slice. The first computing card can obtain at least one shared throughput of the first task. Based on the exclusive throughput of the first task and the at least one shared throughput of the first task, the first service process determines at least one level of interference corresponding to the first task, and the minimum value among the at least one level of interference is determined as the minimum level of interference corresponding to the first task.

[0239] In an optional embodiment, the first service process determines the average time taken for the first computing card to complete m iterations of training during the period when the first task exclusively occupies the first computing card within the second time slice. This average time is the exclusive throughput of the first task. For example, the first service process can divide the time taken for the first computing card to complete m iterations of training during the period when the first task exclusively occupies the first computing card within the second time slice by m to obtain the exclusive throughput of the first task.

[0240] In an optional embodiment, the first service process determines the average time taken for the first computing card to complete m iterations of training for the first task during the period when the first task and any task group share the first computing card within the second time slice. This average time is the shared throughput of the first task with the corresponding task group. For example, the first service process divides the time taken for the first computing card to complete m iterations of training for the first task during the period when the first task and any task group share the first computing card within the second time slice by m to obtain the shared throughput of the first task with the corresponding task group.

[0241] It should be noted that the number of second time slices can be one or more. When there are multiple second time slices, the first service process can determine a minimum interference level corresponding to the first task based on each second time slice. The first service process can obtain multiple minimum interference levels corresponding to the first task. The first service process can determine the statistical value (e.g., maximum, minimum, or average value) of these multiple minimum interference levels as the final minimum interference level corresponding to the first task. The first service process determines the occupation method of the first task on the first computing card based on this final minimum interference level. For example, the first service process determines the minimum value among the multiple minimum interference levels as the final minimum interference level corresponding to the first task. The implementation method of the first service process determining the minimum interference level corresponding to the first task based on each second time slice can be referred to the description of the above embodiments. In optional embodiments, the number of first time slices can also be one or more. When there are multiple first time slices, each second time slice is located before the multiple first time slices. For example, the second time slice is the time slice at the initial stage of the first task's startup, and the first time slice is the time slice during the execution of the first task excluding the second time slice.

[0242] In an optional embodiment, the first service process schedules task identifiers in the first task queue in a time-slice round-robin manner. If there are no incomplete task identifiers in the first task queue (e.g., the first task queue is empty and the second task queue contains at least two incomplete task identifiers), the first service process invokes the first computing card to process the call requests in the request queues of the at least two tasks corresponding to the at least two task identifiers in the second task queue. This allows the at least two tasks to share the first computing card in a space-division multiplexing manner. These at least two tasks are incomplete tasks. For example, the at least two tasks are all incomplete tasks whose task identifiers are cached in the second task queue. Sharing the first computing card in a space-division multiplexing manner can improve the resource utilization of the first computing card.

[0243] In an optional embodiment, the first service process includes at least one thread, each of which corresponds one-to-one with at least one task. Each thread is used to invoke the first computing card to process the invocation request of the corresponding task. Specifically, the first service process invoking the first computing card to process the invocation request of any task can be achieved by the thread corresponding to the task invoking the first computing card to process the invocation request of that task. Threads are also called workers. The at least one task includes a first task. For example, the thread corresponding to the first task is referred to as the first thread. Specifically, the first service process invoking the first computing card to process the invocation request of the first task can be achieved by the first thread invoking the first computing card to process the invocation request of the first task. For example, the first service process invoking the first computing card to process invocation requests in the first request queue.

[0244] In an optional embodiment, the first service process further includes a scheduling module. This scheduling module is used to schedule task identifiers in the first task queue in a time-slice round-robin manner, and, depending on the minimum interference level corresponding to the task indicated by the task identifier in the first task queue, schedule task identifiers in the second task queue as appropriate. If the scheduling module does not schedule a task identifier (which may be a task identifier in the first task queue or a task identifier in the second task queue), the thread corresponding to the task indicated by that task identifier is in a sleep state. If the scheduling module schedules any task identifier, the scheduling module activates (or wakes up) the thread corresponding to the task indicated by that task identifier, causing the thread to call the first computing card to process the call request in the request queue corresponding to that task; if the scheduling module ends the scheduling of that task identifier, the scheduling module triggers the thread to enter a sleep state.

[0245] When the first service process includes a scheduling module, the implementation processes of determining the exclusive throughput of the first task, determining the shared throughput of the first task, determining the interference level corresponding to the first task, and determining the minimum interference level corresponding to the first task based on at least one interference level, all involved in the above embodiments, can be executed by the scheduling module. In other embodiments, the first service process may include a processing module, which determines the exclusive throughput of the first task, the shared throughput of the first task, the interference level corresponding to the first task, and determines the minimum interference level corresponding to the first task based on at least one interference level. This application embodiment does not limit this aspect.

[0246] In an optional embodiment, any task call request received by the first service process carries AI operation parameters. These AI operation parameters are used by the first computing card to execute the AI ​​operation corresponding to the task. Call requests in the task queue corresponding to any task also carry AI operation parameters. During the process of the first service process calling the first computing card to process any call request for any task, the first computing card executes the AI ​​operation corresponding to the task according to the AI ​​operation parameters carried in the call request, for example, if the task is the first task. The AI ​​operation may include AI computation operations, AI memory operations, etc. AI memory is the memory of the computing card; for example, if the computing card is a GPU, AI memory is video memory. AI memory operations may include AI memory allocation operations, AI memory access operations, etc. For example, a call request to request the first computing card to allocate AI memory is a cudaMalloc call request. This call request carries the size of the AI ​​memory to be allocated. During the process of the first service process calling the first computing card to process this call request, the first computing card allocates AI memory according to the size of the AI ​​memory to be allocated carried in the call request.

[0247] In an optional embodiment, any task call request received by the first service process is an API call request. For any call request of that task (i.e., an API call request): the first service process first converts the call request into a system call request, then converts the system call request into a PCIe call request, and then sends the PCIe call request to the first computing card to invoke the first computing card to execute the AI ​​operation corresponding to the task. In a specific embodiment, the first service process includes a computing card runtime and a computing card driver (e.g., if the computing card is a GPU, then the computing card runtime is a GPU runtime, and the computing card driver is a GPU driver). The computing card runtime converts the API call request into a system call request and sends the system call request to the computing card driver. The computing card driver converts the system call request into a PCIe call request and sends the PCIe call request to the first computing card. For any task, if the first service process includes a thread corresponding to the task, both the computing card runtime and the computing card driver can reside in that thread. For example, the first service process includes at least one thread, and each of the at least one thread includes a computing card runtime and a computing card driver. For example, any task in this paragraph is a first task. This example illustrates the scenario where the first service process receives an API call request; that is, the task agent sends an API call request to the first service process. In other embodiments, when the computing card is running or the computing card driver is located in a job process, the call request sent by the task agent to the first service process can be a system call request or a PCIe call request.

[0248] In an optional embodiment, after the first service process invokes the first computing card to execute the AI ​​operation corresponding to the first task, the first service process sends the invocation result to the task agent through the first connection. After receiving the invocation result, the task agent can pass the invocation result to the upper layer. For example, a certain invocation request of the first task is used to request the first computing card to allocate AI memory. After the first service process invokes the first computing card to allocate AI memory, the first service process sends the address of the allocated AI memory to the task agent through the first connection.

[0249] In an optional embodiment, the task agent includes a first agent module. Figure 12 In the illustrated embodiment, the operations performed by the task agent are specifically executed by the first agent module. For example, the first agent module is located in the first job process. After receiving the call result, the first agent module passes the call result to the upper layer of the first job process (e.g., an AI framework).

[0250] In summary, the technical solution provided in this application allows the task agent to invoke the first computing card outside the node where the task agent resides to execute the AI ​​operation corresponding to the first task through the first service process used to manage the first computing card, which helps improve the resource utilization of the first computing card. Since the task agent can invoke the computing card outside the node where the task agent resides to execute the AI ​​operation corresponding to the task, this application can solve the problem of external fragmentation of computing cards (e.g., GPUs).

[0251] To facilitate understanding of the implementation process of step S1201, the specific implementation process of step S1201 will be described below with reference to the accompanying drawings.

[0252] Please refer to Figure 13 This illustrates a schematic diagram of yet another task processing method provided in an embodiment of this application. For example... Figure 13 As shown, the first service process maintains a first task queue, a second task queue, and k request queues, where k is a positive integer. These k request queues are request queues 1 through k, each corresponding to task 1 through task k. Each request queue in request queues 1 through k is used to cache the call requests for the corresponding task. For example, if tasks 1 through 2 satisfy the first priority and tasks 3 through k satisfy the second priority, then the task identifiers for tasks 1 through 2 are cached in the first task queue, and the task identifiers for tasks 3 through k are cached in the second task queue. The first service process schedules the task identifiers in the first task queue using a time-slice round-robin method. When the first service process schedules the task identifier for task 1 in the first task queue, it calls the first computing card to process the call request in request queue 1. When the first service process schedules the task identifier for task 2 in the first task queue, it calls the first computing card to process the call request in request queue 2.

[0253] When the first service process is scheduled to the task identifier of task 1 in the first task queue, the first service process determines the minimum interference level corresponding to task 1 within the time slice T11 corresponding to task 1. For example, time slice T11 is the first time slice corresponding to task 1. In a specific embodiment, task 1 and each of tasks 3 to k form a task group, with a total of k-2 task groups; for example, these k-2 task groups are called task groups 13 to 1k, where task group 13 includes task 1 and task 3, task group 14 includes task 1 and task 4, and so on, with task group 1k including task 1 and task k. Within time slice T11: First, the first service process calls the first computing card to process only m call requests in request queue 1, allowing task 1 to exclusively use the first computing card; then, the first service process schedules task groups 13 to 1k in sequence. When the first service process is scheduled to task group 13: The first service process calls the first computing card to process only m call requests in request queue 1 and call requests in request queue 3, allowing task 1 and task 3 to share the first computing card in a space-division multiplexing manner. When the first service process is scheduled to task group 14: the first service process calls the first computing card to process only m call requests in request queue 1 and call requests in request queue 4, allowing task 1 and task 4 to share the first computing card in a space-division multiplexing manner. Similarly, when the first service process is scheduled to task group 1k: the first service process calls the first computing card to process only m call requests in request queue 1 and call requests in request queue k, allowing task 1 and task k to share the first computing card in a space-division multiplexing manner. Based on the time taken for the first computing card to process m call requests in request queue 1 during the time slice T11 when task 1 exclusively uses the first computing card, the first service process determines the exclusive throughput of task 1. This exclusive throughput is the throughput of task 1 when it exclusively uses the first computing card. Based on the time taken for the first computing card to process m call requests in request queue 1 during the time slice T11 when task 1 and task 3 share the first computing card, the first service process determines a shared throughput of task 1 (for example, referred to as shared throughput 13). Shared throughput 13 is the throughput of task 1 when task 1 and task 3 share the first computing card. The first service process determines a shared throughput of task 1 (for example, calling this shared throughput 14) based on the time taken for the first computing card to process m call requests in request queue 1 during the time slice T11 when task 1 and task 4 share the first computing card. Shared throughput 14 is the throughput of task 1 when task 1 and task 4 share the first computing card.Similarly, the first service process determines a shared throughput for task 1 (e.g., shared throughput 1k) based on the time taken for the first computing card to process m call requests in request queue 1 during the time slice T11 when task 1 and task k share the first computing card. Shared throughput 1k is the throughput of task 1 when task 1 and task k share the first computing card. The first service process can obtain k-2 shared throughputs for task 1, which are shared throughputs 13 to 1k. Based on the exclusive throughput and shared throughput 13 of task 1, the first service process determines a level of interference for task 1 (e.g., interference level 13). Based on the exclusive throughput and shared throughput 14 of task 1, the first service process determines a level of interference for task 1 (e.g., interference level 14). And so on, based on the exclusive throughput and shared throughput 1k of task 1, the first service process determines a level of interference for task 1 (e.g., interference level 1k). The first service process can obtain k-2 interference levels corresponding to task 1, which are interference levels 13 to 1k. The first service process determines the minimum interference level among interference levels 13 to 1k as the minimum interference level corresponding to task 1. For example, if the minimum interference level among interference levels 13 to 1k is interference level 13, then interference level 13 is the interference level of task 3 on task 1.

[0254] After determining the minimum interference level for Task 1, the first service process determines the allocation method of the first computing card for Task 1 based on this minimum interference level. If the minimum interference level for Task 1 is not less than the threshold Y1, the first service process determines that Task 1 exclusively occupies the first computing card. If the minimum interference level for Task 1 is less than the threshold Y1, the first service process determines that Task 1 and the unfinished task with the minimum interference level for Task 1 among Tasks 3~k will share the first computing card using a space-division multiplexing method.

[0255] If the minimum interference level corresponding to task 1 is not less than the threshold Y1 and the first service process is scheduled to the task identifier of task 1 in the first task queue, the first service process calls the first computing card to process the call request in the request queue 1 within the time slice T12 corresponding to task 1, so that task 1 exclusively occupies the first computing card until the end of time slice T12, or until task 1 is completed within time slice T12.

[0256] Taking the minimum interference level corresponding to Task 1 as interference level 13, with interference levels increasing sequentially from 13 to 1k, as an example: When the minimum interference level corresponding to Task 1 is less than the threshold Y1 and the first service process is scheduled to the task identifier of Task 1 in the first task queue, the first service process calls the first computing card within the time slice T12 corresponding to Task 1 to process the call requests in request queue 1 and request queue 3, allowing Task 1 and Task 3 to share the first computing card in a space-division multiplexing manner. If Task 3 completes within time slice T12 (i.e., time slice T12 has not ended, Task 1 has not completed, and Task 3 has completed), the first service process calls the first computing card within the remaining time period of time slice T12 to process the call requests in request queue 1 and request queue 4, allowing Task 1 and Task 4 to share the first computing card in a space-division multiplexing manner. If task 4 also completes within time slice T12 (i.e., time slice T12 has not ended, task 1 has not completed, and task 4 has completed), the first service process will use the first computing card to process the call requests in request queue 1 and request queue 5 during the remaining time period of time slice T12, allowing task 1 and task 5 to share the first computing card in a space-division multiplexing manner. This continues until time slice T12 ends, or until task 1 completes within time slice T12, or until tasks 3 through k are all completed within time slice T12. If tasks 3 through k are all completed within time slice T12, the first service process will use the first computing card to process the call requests in request queue 1 during the remaining time period of time slice T12, allowing task 1 to exclusively use the first computing card until time slice T12 ends, or until task 1 completes within time slice T12.

[0257] If Task 1 is not completed by the end of time slice T12, the first service process will move the task identifier of Task 1 from the head to the tail of the first task queue at the end of time slice T12, and then the first service process will continue to schedule the next task identifier in the first task queue. If Task 1 is completed within time slice T12 (including the end of time slice T12), the first service process will remove the task identifier of Task 1 from the first task queue after Task 1 is completed, and will continue to schedule the next task identifier in the first task queue.

[0258] Time slice T12 is located after time slice T11. Time slice T12 can be any time slice after time slice T11 corresponding to task 1. The implementation process of the first service process calling the first computing card to process the call request in request queue 1 within any time slice after time slice T11 corresponding to task 1 can be referred to as the implementation process of the first service process calling the first computing card to process the call request in request queue 1 within time slice T12. The threshold Y1 can be the same as or different from the threshold Y mentioned above.

[0259] When the first service process is scheduled to the task identifier of task 2 in the first task queue, the first service process determines the minimum interference level corresponding to task 2 within the time slice T21 corresponding to task 2. For a detailed implementation, refer to the implementation process of the first service process determining the minimum interference level corresponding to task 1 within time slice T11. After determining the minimum interference level corresponding to task 2, the first service process determines the usage method of task 2 on the first computing card based on the minimum interference level, and then calls the first computing card to process the call requests in request queue 2 according to the usage method of task 2. For a detailed implementation, refer to the implementation process of the first service process calling the first computing card to process the call requests in request queue 1.

[0260] The first service process schedules the task identifiers in the first task queue using a time-slice round-robin method. If there are no unfinished task identifiers in the first task queue, the first service process calls the first computing card to process the call requests in the request queues 3~k, so that tasks 3~k share the first computing card in a space-division multiplexing manner.

[0261] like Figure 13As shown, the first service process includes a scheduling module and threads 1-k, each corresponding to a task. Each thread in threads 1-k is used to invoke the first computing card to process the corresponding task's request. The scheduling module schedules task identifiers in the first task queue in a time-slice round-robin fashion, and, based on the minimum interference level indicated by the task identifiers in the first task queue, schedules task identifiers in the second task queue as needed. When the scheduling module schedules the task identifier for task 1 in the first task queue, it activates thread 1, enabling thread 1 to invoke the first computing card to process the request in request queue 1 within the time slice corresponding to task 1. After any time slice for task 1 ends, the scheduling module triggers thread 1 to enter a sleep state. When the scheduling module schedules the task identifier for task 2 in the first task queue, it activates thread 2, enabling thread 2 to invoke the first computing card to process the request in request queue 2 within the time slice corresponding to task 2. After any time slice for task 2 ends, the scheduling module triggers thread 2 to enter a sleep state. When the scheduling module schedules a task identifier for task 3 in the second task queue, it activates thread 3, causing thread 3 to call the first computing card to process the call request in request queue 3. When the scheduling module terminates the scheduling of task identifier 3, it triggers thread 3 to enter a sleep state. Similarly, when the scheduling module schedules a task identifier for task 4 in the second task queue, it activates thread 4, causing thread 4 to call the first computing card to process the call request in request queue 4. When the scheduling module terminates the scheduling of task identifier 4, it triggers thread 4 to enter a sleep state. This process continues in the same manner.

[0262] In the technical solutions provided in this application, one service process can correspond to one computing card context. For example, if the computing card is a GPU, the computing card context is a GPU context. Based on Figure 12 and Figure 13As illustrated in the embodiments, multiple tasks can share a single computing card context. The service process can detect the performance (specifically, throughput) of tasks during execution and schedule tasks based on this performance. This allows tasks meeting the first priority and tasks meeting the second priority to share the same computing card using space-division multiplexing, thereby solving the fragmentation problem within the computing card and improving the throughput of tasks meeting the first priority. For example, the first priority is referred to as high priority, the second priority as low priority, tasks meeting the first priority as high-priority tasks, and tasks meeting the second priority as low-priority tasks. Experiments show that the technical solution provided in this embodiment can increase the throughput of high-priority tasks by 67.0% and effectively recover 17.4% of the fragmentation within the computing card. Furthermore, this embodiment can combine time-division multiplexing and space-division multiplexing, breaking through the limitations of traditional single multiplexing methods and effectively solving the fragmentation problem within the computing card.

[0263] In the technical solution provided in this application embodiment, the process of the service process detecting task performance is transparent to the task's job process. This process is independent of specific hardware or frameworks, exhibiting high versatility and maintainability. Furthermore, this application embodiment can detect the performance degradation caused by co-located tasks in real time without blocking high-priority tasks, providing a reliable basis for spatial multiplexing scheduling. In this application embodiment, the service process detects task performance and performs task scheduling to achieve compute card reuse. The service process and the AI ​​framework reside on different nodes; therefore, the process of the service process detecting task performance and performing task scheduling are independent of the AI ​​framework. In other embodiments, compute card reuse can be achieved by embedding scheduling logic into the AI ​​framework. For example, scheduling logic can be embedded in the AI ​​framework to determine which tasks can be executed concurrently on the same compute card based on the task's operator graph or static profiling results. Alternatively, the performance interference estimation module may be hidden in the inference service gateway or task manager, using different metrics to drive scheduling decisions to achieve compute card reuse.

[0264] Since one service process manages one computing card, and one service process corresponds to one computing card context, the embodiments of this application can achieve management of computing cards in a computing cluster at the granularity of computing card context. Furthermore, combined with... Figure 6 , Figure 7 and Figure 12 As can be seen from the embodiments shown, the embodiments of this application can establish a service address table for the task during the task startup phase through the collaboration of the task agent, resource agent and resource controller, so that the task agent can call the computing card to execute the AI ​​operation corresponding to the task according to the service address recorded in the service address table.

[0265] The embodiments of this application can realize the unified organization and scheduling of computing card contexts across the cluster (e.g., implemented by a resource controller), be compatible with existing cluster scheduling strategies, have good scalability, support real-time scheduling of computing cards at the level of tens of thousands of cards in large-scale environments, and have a computing card response latency of only 47.8ms; under high concurrency requests, the performance is predictable, and the response latency increases linearly.

[0266] The technical solution provided in this application decouples the computing card context from the node where the job process resides and maps it through a service address table. The overhead introduced by decoupling and mapping is extremely low, only 0.7%-3.0%. Furthermore, the technical solution provided in this application has strong portability. For example, the technical solution provided in this application is applicable to heterogeneous scenarios with multiple types of computing cards, i.e., scenarios where the computing cluster includes multiple types of computing cards, such as GPUs, NPUs, and TPUs. As another example, the technical solution provided in this application is applicable to edge computing scenarios. When an edge node only has a CPU, the CPU in that edge node can call the computing card in the edge data center to perform AI operations, thereby realizing on-demand remote computing power in environments with distributed computing power and complex network topologies to accelerate AI tasks. As yet another example, the technical solution provided in this application is applicable to multi-cloud or cross-regional scenarios. The CPU can call the computing card across clouds and regions to perform AI operations, reserving an implementation path for future computing power interconnection and computing power network forms.

[0267] In the technical solution provided in this application embodiment, the application layer on the tenant side (e.g., the application layer of the job process) is unaware of the remote computing card. For the CPU on the tenant side, there is no difference between the AI ​​operation performed by the remote computing card and the AI ​​operation performed by the local computing card. Therefore, for the CPU on the tenant side, the computing card located at the remote end is also called a virtual computing card.

[0268] The technical solutions provided in this application are completely transparent to the application layer and have good versatility and universality of deployment.

[0269] The above embodiments illustrate the example of the task agent requesting AI resources for the first task when the task execution system starts the first task (or before starting it). In optional embodiments, the task agent can also request AI resources for the first task during its execution. For example, if the task agent determines that the AI ​​resource requirement of the first task increases during its execution, the task agent can request AI resources for the first task. As another example, if the computing card already allocated to the first task by the resource manager malfunctions or its load exceeds a preset limit during the execution of the first task, the task agent can re-request AI resources for the first task to replace the current computing card.

[0270] As an example, as mentioned earlier, after the first service process establishes a first connection with the task agent, the first service process receives the invocation request of the first task through the first connection and invokes the first computing card to process the invocation request to execute the AI ​​operation corresponding to the first task. In an optional embodiment, during the process of the first service process invoking the first computing card to process the invocation request, the first service process monitors the status of the first computing card. If the first service process determines that the status of the first computing card is abnormal, such as a malfunction or the load exceeding a preset load, the first service process disconnects the first connection. For example, if the first service process determines that the status of the first computing card is abnormal, it determines that the first computing card meets the disconnection conditions, including a malfunction or the load exceeding a preset load. The first service process disconnects the first connection in response to the first computing card meeting the disconnection conditions. After the first service process disconnects the first connection, the task agent (specifically, a first agent module) can detect the disconnection. After the task agent determines that the first connection is disconnected, it re-applies for AI resources for the first task. For example, in response to the first connection being disconnected, the task agent sends a second resource request to the resource manager, carrying the AI ​​resource requirements of the first task. After receiving the second resource request, the resource manager allocates a third computing card in the computing cluster to meet the AI ​​resource requirements of the first task and sends the service address of the third computing card (e.g., referred to as the third service address) to the task agent. After receiving the third service address sent by the resource manager, the task agent calls the third computing card to execute the AI ​​operation corresponding to the first task based on the third service address. The node where the third computing card is located is a different node from the node running the task agent. In this case, the first computing card used to execute the AI ​​operation corresponding to the first task is replaced by the third computing card. For example, the task agent calls the third computing card to execute the AI ​​operation corresponding to the first task through a third service process based on the third service address. The third service process is used to manage the third computing card, and the third service process and the third computing card are located on the same node. In an optional embodiment, after the first computing card is replaced by the third computing card, the third service process obtains the computing card context corresponding to the first service process and continues to execute the AI ​​operation corresponding to the first task based on the computing card context corresponding to the first service process. For example, the computing card context corresponding to the first service process is stored in a shared storage space. This shared storage space is shared by multiple service processes, including the first service process and the third service process. The third service process obtains the computing card context corresponding to the first service process from this shared storage space. The implementation process of the task agent re-applying for AI resources for the first task and the implementation process of the task agent calling the third computing card based on the third service address to execute the AI ​​operation corresponding to the first task can both be referred to the description in the previous embodiments, and will not be repeated here.Furthermore, the operations described in this paragraph that are performed by the task agent can be specifically performed by the first agent module included in the task agent.

[0271] This explanation uses the first connection disconnection as an example. Even if the second connection is disconnected, the task agent can still detect the second connection disconnection and re-apply for AI resources for the first task. For instance, the second connection might be located between the second service process and the second agent module included in the task agent, where the first agent module is the master agent module and the second agent module is the slave agent module. Specifically, in the case of the second connection disconnection, the second agent module is the first to detect it. The second agent module can then transmit this event to the first agent module through various possible means, allowing the first agent module to re-apply for AI resources for the first task.

[0272] As can be seen from the description here, the embodiments of this application not only support the allocation of computing cards to the first task when the task execution system starts the first task (or before starting it), but also allow the allocation or replacement of computing cards to the first task dynamically based on the actual situation during the operation of the first task. This not only improves the resource utilization of the computing cards and ensures the sequential execution of the first task, but also supports the dynamic and flexible allocation of AI resources to the task.

[0273] The above is a description of the method embodiments of this application. The following describes the apparatus embodiments of this application, which are used to execute the method of this application. For details not disclosed in the apparatus embodiments, please refer to the method embodiments.

[0274] Please refer to Figure 14 This illustration shows a schematic diagram of a task execution device 1400 provided in an embodiment of this application. The task execution device 1400 is applied to a task agent, which may be the task agent 10 in the above embodiments. The task execution device 1400 includes at least one functional unit for performing operations executed by the task agent in the above method embodiments. The at least one functional unit may be implemented based on software or a combination of software and hardware, and the at least one functional unit may be arbitrarily combined or divided based on the implementation.

[0275] In specific embodiments, such as Figure 14As shown, the task execution device 1400 includes a first sending unit 1401, a first receiving unit 1402, and a first calling unit 1403. The first sending unit 1401 sends a resource request to a resource manager, which carries the AI ​​resource requirements of a first task. The resource manager manages computing cards in a computing cluster, which includes at least one computing card. The first receiving unit 1402 receives a first service address sent by the resource manager. The first service address is the service address of a first computing card in the computing cluster, and the first computing card meets the AI ​​resource requirements of the first task. The first calling unit 1403, based on the first service address, calls the first computing card to execute the AI ​​operation corresponding to the first task. The node where the first computing card is located is a different node from the node running the task agent. The functionality of the first sending unit 1401 can be referred to the relevant description in step S601 above. The functionality of the first receiving unit 1402 can be referred to the relevant description in step S605 above. The functionality of the first calling unit 1403 can be referred to the relevant description in step S606 above.

[0276] In an optional embodiment, the AI ​​resource requirements of the first task include a first requirement and a second requirement, and the first computing card meets the first requirement, such as... Figure 14 As shown, the task execution device 1400 further includes a second receiving unit 1404 and a second calling unit 1405. The second receiving unit 1404 is used to receive a second service address sent by the resource manager. The second service address is the service address of a second computing card in the computing cluster, and the second computing card meets the second requirement. The second calling unit 1405 is used to call the second computing card to execute the AI ​​operation corresponding to the first task based on the second service address. The functional implementation of the second receiving unit 1404 can also be referred to the relevant description in step S609 above. The functional implementation of the second calling unit 1405 can also be referred to the relevant description in step S610 above.

[0277] In an optional embodiment, the first computing card and the second computing card are located on the same node; or, the first computing card and the second computing card are located on different nodes.

[0278] In an optional embodiment, the node where the second computing card is located is a different node from the node running the task agent; or, the node where the second computing card is located is the same node as the node running the task agent.

[0279] In optional embodiments, this application includes the following two implementation methods.

[0280] In the first implementation, the first requirement includes a first identifier and a first AI resource requirement, and the second requirement includes a second identifier and a second AI resource requirement. The first computing card satisfies the first AI resource requirement, and the second computing card satisfies the second AI resource requirement. The first receiving unit 1402 and the second receiving unit 1404 are the same receiving unit. The first receiving unit 1402 is used to: receive a resource response sent by the resource manager, the resource response carrying a service address table, the service address table including the correspondence between the first identifier and the first service address, and the correspondence between the second identifier and the second service address.

[0281] In an optional implementation of the first approach, the task agent includes a first agent module and a second agent module, and the task execution device 1400 further includes a storage unit 1406. A first sending unit 1401 is used by the first agent module to send the resource request to the resource manager. A first receiving unit 1402 is used by the first agent module to receive the resource response sent by the resource manager. The storage unit 1406 is used by the first agent module to store the service address table in a shared memory area, which is a shared memory area between the first and second agent modules. In this optional implementation, the first sending unit 1401, the first receiving unit 1402, and the storage unit 1406 are all located within the first agent module.

[0282] In the alternative implementation schemes of the first implementation method, such as Figure 14 As shown, the task execution device 1400 further includes a first determining unit 1407 and a second determining unit 1408. The first determining unit 1407 is used by the first proxy module to determine a first service address based on a first identifier and a service address table stored in the shared memory region. The second determining unit 1408 is used by the second proxy module to determine a second service address based on a second identifier and a service address table stored in the shared memory region. The first calling unit 1403 is used by the first proxy module to call the first computing card to execute the AI ​​operation corresponding to the first task based on the first service address. The second calling unit 1405 is used by the second proxy module to call the second computing card to execute the AI ​​operation corresponding to the first task based on the second service address. In this optional implementation, the first determining unit 1407 and the first calling unit 1403 are both located in the first proxy module, and the second determining unit 1408 and the second calling unit 1405 are both located in the second proxy module.

[0283] In an optional implementation of the first approach, the resource manager receives a resource request from the task agent, obtains a service address table based on the AI ​​resource requirements of the first task carried in the resource request, and sends the resource response to the task agent.

[0284] In an optional implementation of the first approach, the resource manager includes a resource proxy and a resource controller. The resource controller manages the computing cards in the computing cluster. The resource proxy receives resource requests sent by the task proxy and sends the resource request to the resource controller. The resource controller obtains a service address table based on the AI ​​resource requirements of the first task carried in the resource request and sends the resource response to the resource proxy. The resource proxy sends the resource response to the task proxy.

[0285] In the second implementation, the first requirement includes a first identifier and a first AI resource requirement, and the second requirement includes a second identifier and a second AI resource requirement. The first computing card satisfies the first AI resource requirement, and the second computing card satisfies the second AI resource requirement, such as... Figure 14 As shown, the task execution device 1400 further includes a second sending unit 1409. The first sending unit 1401 is further configured to send a first query request to the resource manager, the first query request carrying a first identifier. The second sending unit 1409 is configured to send a second query request to the resource manager, the second query request carrying a second identifier. The first receiving unit 1402 is configured to receive a first query response sent by the resource manager, the first query response carrying a first service address; the second receiving unit 1404 is configured to receive a second query response sent by the resource manager, the second query response carrying a second service address.

[0286] In an optional implementation of the second approach, the task proxy includes a first proxy module and a second proxy module; a first sending unit 1401, used by the first proxy module to send a resource request to the resource manager. The first sending unit 1401 is also used by the first proxy module to send a first query request to the resource manager. A first receiving unit 1402 is used by the first proxy module to receive the first query response sent by the resource manager. A first calling unit 1403 is used by the first proxy module to call the first computing card based on the first service address to execute the AI ​​operation corresponding to the first task. A second sending unit 1409 is used by the second proxy module to send a second query request to the resource manager. A second receiving unit 1404 is used by the second proxy module to receive the second query response sent by the resource manager. A second calling unit 1405 is used by the second proxy module to call the second computing card based on the second service address to execute the AI ​​operation corresponding to the first task. In this optional implementation, the first sending unit 1401, the first receiving unit 1402, and the first calling unit 1403 are all located in the first proxy module, and the second sending unit 1409, the second receiving unit 1404, and the second calling unit 1405 are all located in the second proxy module.

[0287] In an optional implementation of the second approach, the resource manager is configured to: receive a resource request sent by the task agent; obtain a service address table based on the AI ​​resource requirements of the first task carried in the resource request; store the service address table; and send a resource response to the task agent. The resource response indicates successful resource allocation and does not carry the service address table. The service address table includes a mapping between a first identifier and a first service address, and a mapping between a second identifier and a second service address. Furthermore, the resource manager is configured to: receive a first query request sent by the task agent (specifically, the first agent module); determine a first service address based on the service address table and the first identifier carried in the first query request; and send a first query response to the task agent (specifically, the first agent module). Additionally, the resource manager is configured to: receive a second query request sent by the task agent (specifically, the second agent module); determine a second service address based on the service address table and the second identifier carried in the second query request; and send a second query response to the task agent (specifically, the second agent module).

[0288] In an optional implementation of the second approach, the resource manager includes a resource proxy and a resource controller. The resource controller manages the computing cards in the computing cluster. The resource proxy receives resource requests from the task proxy and sends the resource requests to the resource controller. The resource controller obtains a service address table based on the AI ​​resource requirements of the first task carried in the resource request and sends the service address table to the resource proxy. The resource proxy stores the service address table and sends the resource response to the task proxy. Furthermore, the resource proxy is also used to: receive a first query request from the task proxy (specifically, the first proxy module), determine a first service address based on the service address table and a first identifier carried in the first query request, and send a first query response to the task proxy (specifically, the first proxy module); receive a second query request from the task proxy (specifically, the second proxy module), determine a second service address based on the service address table and a second identifier carried in the second query request, and send a second query response to the task proxy (specifically, the second proxy module).

[0289] In an optional embodiment, the first calling unit 1403 is configured to: establish a first connection based on a first service address; and call the first computing card to execute the AI ​​operation corresponding to the first task based on the first connection.

[0290] In an optional embodiment, the first calling unit 1403 is configured to: establish a first connection with a first service process based on a first service address, wherein the first service process is used to manage a first computing card; and use the first sending unit 1401 to send a call request for a first task to the first service process through the first connection, thereby triggering the first service process to call the first computing card to process the call request for the first task in order to execute the AI ​​operation corresponding to the first task.

[0291] In an optional embodiment, the first calling unit 1403 is configured to: send a connection establishment request to the first service process based on the first service address using the first sending unit 1401, the connection establishment request being used to request the establishment of a first connection with the first service process.

[0292] In an optional embodiment, the second calling unit 1405 is used to: establish a second connection based on the second service address; and call the second computing card to execute the AI ​​operation corresponding to the first task based on the second connection.

[0293] In an optional embodiment, the second calling unit 1405 is configured to: establish a second connection with a second service process based on a second service address, wherein the second service process is used to manage the second computing card; and use the second sending unit 1409 to send a call request for the first task to the second service process through the second connection, thereby triggering the second service process to call the second computing card to process the call request for the first task and execute the AI ​​operation corresponding to the first task.

[0294] In an optional embodiment, the second calling unit 1405 is configured to: use the second sending unit 1409 to send a connection establishment request to the second service process based on the second service address, the connection establishment request being used to request the establishment of a second connection with the second service process.

[0295] To make it easier to distinguish, Figure 14 The resource request described above in the illustrated embodiment is referred to as the first resource request. In an optional embodiment, the first sending unit 1401 is further configured to send a second resource request to the resource manager in response to the first connection being disconnected. The second resource request carries the AI ​​resource requirements of the first task. The first receiving unit 1402 is further configured to receive a third service address sent by the resource manager. The third service address is the service address of a third computing card in the computing cluster. The third computing card meets the AI ​​resource requirements of the first task. The first calling unit 1403 is further configured to call the third computing card to execute the AI ​​operation corresponding to the first task based on the third service address. The node where the third computing card is located is a different node from the node running the task agent.

[0296] Please refer to Figure 15 This illustration shows a schematic diagram of a task execution device 1500 provided in an embodiment of this application. The task execution device 1500 is applied to a resource manager, which may be the resource manager 20 in the above embodiments. The task execution device 1500 includes at least one functional unit for performing operations executed by the resource manager in the above method embodiments. The at least one functional unit may be implemented based on software or a combination of software and hardware, and the at least one functional unit may be arbitrarily combined or divided based on the implementation.

[0297] In specific embodiments, such as Figure 15 As shown, the task execution device 1500 includes a first receiving unit 1501 and a first sending unit 1502. The first receiving unit 1501 receives a resource request sent by a task agent, which carries the AI ​​resource requirements of a first task. The resource agent manages computing cards in a computing cluster, which includes at least one computing card. The first sending unit 1502 sends a first service address to the task agent. The first service address is the service address of a first computing card in the computing cluster. The first computing card meets the AI ​​resource requirements of the first task. The node where the first computing card is located is a different node from the node running the task agent. The task agent uses the first service address to invoke the first computing card to execute the AI ​​operation corresponding to the first task. The functional implementation of the first receiving unit 1501 can be referred to the relevant description in step S602 above. The functional implementation of the first sending unit 1502 can be referred to the relevant description in step S604 above.

[0298] In an optional embodiment, the AI ​​resource requirements of the first task include a first requirement and a second requirement. The first computing card meets the first requirement. The first sending unit 1502 is further configured to send a second service address to the task agent. The second service address is the service address of the second computing card in the computing cluster. The second computing card meets the second requirement. The second service address is used by the task agent to invoke the second computing card to execute the AI ​​operation corresponding to the first task. The functional implementation of the first sending unit 1502 can also refer to the relevant description in step S608 above.

[0299] In an optional embodiment, the first computing card and the second computing card are located on the same node; or, the first computing card and the second computing card are located on different nodes.

[0300] In an optional embodiment, the node where the second computing card is located is a different node from the node running the task agent; or, the node where the second computing card is located is the same node as the node running the task agent.

[0301] In optional embodiments, this application includes the following two implementation methods.

[0302] In the first implementation, the first requirement includes a first identifier and a first AI resource requirement, and the second requirement includes a second identifier and a second AI resource requirement. The first computing card satisfies the first AI resource requirement, and the second computing card satisfies the second AI resource requirement, such as... Figure 15As shown, the task execution device 1500 further includes an acquisition unit 1503. The acquisition unit 1503 is used to acquire a service address table according to the AI ​​resource requirements of the first task. The service address table includes a correspondence between a first identifier and a first service address, and a correspondence between a second identifier and a second service address. The first sending unit 1502 is used to send a resource response to the task agent. The resource response carries the service address table.

[0303] In an optional implementation of the first approach, the resource manager includes a resource broker and a resource controller, with the resource controller managing the computing cards in the computing cluster; for example... Figure 15 As shown, the task execution device 1500 further includes a second receiving unit 1504 and a second sending unit 1505. The first receiving unit 1501 is used for the resource proxy to receive a resource request sent by the task proxy. The first sending unit 1502 is also used for the resource proxy to send the resource request to the resource controller. The second receiving unit 1504 is used for the resource proxy to receive the resource request sent by the task proxy. The obtaining unit 1503 is used for the resource controller to obtain a service address table based on the AI ​​resource requirements of the first task carried in the resource request. The second sending unit 1505 is used for the resource controller to send the resource response to the resource proxy. The first sending unit 1502 is used for the resource proxy to send the resource response to the task proxy. In this optional implementation, the first receiving unit 1501 and the first sending unit 1502 are both located in the resource proxy, while the second receiving unit 1504, the second sending unit 1505, and the obtaining unit 1503 are all located in the resource controller.

[0304] In an optional implementation of the first implementation method, the task agent includes a first agent module and a second agent module, a first receiving unit 1501 for receiving a resource request sent by the first agent module, and a first sending unit 1502 for sending the resource response to the first agent module.

[0305] In the second implementation, the first requirement includes a first identifier and a first AI resource requirement, and the second requirement includes a second identifier and a second AI resource requirement. The first computing card satisfies the first AI resource requirement, and the second computing card satisfies the second AI resource requirement, such as... Figure 15As shown, the task execution device 1500 further includes an acquisition unit 1503, a storage unit 1506, and a determination unit 1507. The acquisition unit 1503 is used to acquire a service address table based on the AI ​​resource requirements of the first task. The service address table includes a correspondence between a first identifier and a first service address, and a correspondence between a second identifier and a second service address. The storage unit 1506 is used to store the service address table. The first sending unit 1502 is further used to send a resource response to the task agent, the resource response indicating successful resource allocation and not carrying the service address table. The first receiving unit 1501 is further used to receive a first query request and a second query request sent by the task agent, the first query request carrying a first identifier and the second query request carrying a second identifier. The determination unit 1507 is used to: determine a first service address based on the service address table and the first identifier carried in the first query request; and determine a second service address based on the service address table and the second identifier carried in the second query request. The first sending unit 1502 is further configured to: send a first query response to the task agent, the first query response carrying a first service address, and send a second query response to the task agent, the second query response carrying a second service address.

[0306] In an optional implementation of the second approach, the resource manager includes a resource broker and a resource controller, with the resource controller managing the computing cards in the computing cluster; for example... Figure 15As shown, the task execution device 1500 further includes a second receiving unit 1504 and a second sending unit 1505. The first receiving unit 1501 is used for the resource proxy to receive a resource request sent by the task proxy. The first sending unit 1502 is also used for the resource proxy to send the resource request to the resource controller. The second receiving unit 1504 is used for the resource controller to receive the resource request sent by the resource proxy. The acquisition unit 1503 is used for the resource controller to acquire a service address table based on the AI ​​resource requirements of the first task carried in the resource request. The second sending unit 1505 is used for the resource controller to send the service address table to the resource proxy. The storage unit 1506 is used for the resource proxy to store the service address table. The first receiving unit 1501 is also used for the resource proxy to receive a first query request and a second query request sent by the task proxy; the determination unit 1507 is used for the resource proxy to acquire a first service address based on the service address table and a first identifier carried in the first query request, and for the resource proxy to acquire a second service address based on the service address table and a second identifier carried in the second query request. The first sending unit 1502 is further configured to send a first query response from the resource broker to the task broker, and to send a second query response from the resource broker to the task broker. In this optional implementation, the first receiving unit 1501, the first sending unit 1502, the storage unit 1506, and the determining unit 1507 are all located in the resource broker, while the second receiving unit 1504, the second sending unit 1505, and the obtaining unit 1503 are all located in the resource controller.

[0307] In an optional implementation of the second implementation method, the task proxy includes a first proxy module and a second proxy module. The first receiving unit 1501 is used to receive resource requests sent by the first proxy module, receive first query requests sent by the first proxy module, and receive second query requests sent by the second proxy module. The first sending unit 1502 is used to send the resource response to the first proxy module, send the first query response to the first proxy module, and send the second query response to the second proxy module.

[0308] To make it easier to distinguish, Figure 15 The resource request described above in the illustrated embodiment is referred to as the first resource request. In an optional embodiment, the first receiving unit 1501 is further configured to receive a second resource request sent by the task agent, the second resource request carrying the AI ​​resource requirements of the first task; the first sending unit 1502 is further configured to send a third service address to the task agent, the third service address being the service address of a third computing card in the computing cluster, the third computing card meeting the AI ​​resource requirements of the first task, the node where the third computing card is located being a different node from the node running the task agent, and the third service address being used by the task agent to call the third computing card to execute the AI ​​operation corresponding to the first task.

[0309] Please refer to Figure 16 The diagram illustrates a task execution device 1600 provided in an embodiment of this application. The task execution device 1600 is applied to a first service process. The task execution device 1600 includes at least one functional unit for performing operations executed by the resource manager in the above method embodiments. This at least one functional unit can be implemented based on software or a combination of software and hardware, and can be arbitrarily combined or divided based on the implementation.

[0310] In specific embodiments, such as Figure 16 As shown, the task execution device 1600 includes a processing unit 1601 and a receiving unit 1602. The processing unit 1601 is used to establish a first connection with the task agent based on a first service address. The first service address is the service address of a first computing card in the computing cluster. A first service process manages the first computing card, which meets the AI ​​resource requirements of the first task. The node where the first computing card is located is a different node from the node running the task agent. The receiving unit 1602 is used to receive a call request for the first task sent by the task agent through the first connection. The processing unit 1601 is also used to invoke the first computing card to process the call request for the first task to execute the AI ​​operation corresponding to the first task. The functional implementation of the processing unit 1601 can be referred to the relevant descriptions in steps S1201 and S1204 above. The functional implementation of the receiving unit 1602 can be referred to the relevant descriptions in step S1203 above.

[0311] In an optional embodiment, the processing unit 1601 is configured to: receive a connection establishment request sent by the task agent based on a first service address using the receiving unit 1602; and establish a first connection with the task agent in response to the connection establishment request.

[0312] In an optional embodiment, the processing unit 1601 is configured to: cache the call request of the first task in a first request queue, the first request queue corresponding to the first task; and call the first computing card to process the call request in the first request queue.

[0313] In an optional embodiment, the connection establishment request carries the task identifier of the first task, and the processing unit 1601 is further configured to create a first request queue in response to the connection establishment request.

[0314] In an optional embodiment, the connection establishment request also carries the priority of the first task. The processing unit 1601 is further configured to cache the task identifier of the first task in the first task queue in response to the priority of the first task satisfying the first priority. The first task queue corresponds to the first priority and is used to cache the task identifier of the task whose priority satisfies the first priority.

[0315] In an optional embodiment, the processing unit 1601 is configured to: schedule task identifiers in the first task queue in a time-slice round-robin manner; when the minimum interference level corresponding to the first task is not less than a threshold and the task identifier of the first task is scheduled to the first task queue, call the first computing card to process the call request in the first request queue within the first time slice corresponding to the first task, so that the first task exclusively occupies the first computing card; the minimum interference level corresponding to the first task is the interference level of the second task on the first task; the second task is the unfinished task with the least interference level on the first task among the tasks whose task identifiers are cached in the second task queue; the second task queue corresponds to a second priority, and the first priority is higher than the second priority; the second task queue is used to cache task identifiers of tasks whose priority meets the second priority; when the minimum interference level corresponding to the first task is less than the threshold and the task identifier of the first task is scheduled to the first task queue, call the first computing card to process the call request in the first request queue and the call request in the second request queue within the first time slice, so that the first task and the second task share the first computing card in a space-division multiplexing manner; the second request queue corresponds to the second task.

[0316] In an optional embodiment, the processing unit 1601 is further configured to, if the interference level of the second task on the first task is less than the threshold, call the first computing card to process the call requests in the first request queue and the call requests in the third request queue during the remaining time period of the first time slice if the second task is completed within the first time slice, so that the first task and the third task share the first computing card in a space-division multiplexing manner, the third request queue corresponds to the third task, and the third task is the unfinished task with the least interference level on the first task, excluding the second task, among the tasks whose task identifier is cached in the second task queue.

[0317] In an optional embodiment, the processing unit 1601 is further configured to adjust the task identifier of the first task from the head of the first task queue to the tail of the first task queue if the first task is not completed at the end of the first time slice.

[0318] In an optional embodiment, the processing unit 1601 is further configured to delete the task identifier of the first task from the first task queue if the first task is completed within the first time slice.

[0319] In an optional embodiment, the processing unit 1601 is further configured to determine the minimum interference level corresponding to the first task within a second time slice corresponding to the first task, provided that the task identifier of the first task is scheduled to the first task queue, and the second time slice is located before the first time slice.

[0320] In an optional embodiment, the second task queue caches task identifiers for at least one task. The processing unit 1601 is configured to: within a second time slice: first, invoke the first computing card to process m call requests in the first request queue, so that the first task exclusively occupies the first computing card, where m is a positive integer; then, sequentially schedule at least one task group, which corresponds one-to-one with the at least one task, each task group including the first task and the task corresponding to each task group; when any task group in the at least one task group is scheduled: invoke the first computing card to process the m call requests in the first request queue and the call requests in the request queue of the task corresponding to the task group, so that the first task and the task corresponding to the task group share the first computing card in a space-division multiplexing manner; based on the period during which the first task exclusively occupies the first computing card, the first computing card processes the m call requests in the first request queue. The exclusive throughput of the first task is determined by calculating the time taken for m call requests. This exclusive throughput is the throughput of the first task when it exclusively uses the first computing card. The shared throughput of the first task with the corresponding task group is determined by calculating the time taken for the first computing card to process m call requests in the first request queue during the period when the first task and the corresponding task group share the first computing card. This shared throughput is the throughput of the first task when the first task and the corresponding task group share the first computing card. At least one level of interference is determined based on the exclusive throughput and at least one shared throughput of the first task. This at least one shared throughput corresponds one-to-one with the at least one task group, and this at least one level of interference corresponds one-to-one with the at least one shared throughput. The minimum value among these at least one level of interference is determined as the minimum level of interference for the first task.

[0321] In an optional embodiment, the processing unit 1601 is further configured to, when there are no unfinished task identifiers in the first task queue, call the first computing card to process the call requests in the request queues of at least two tasks corresponding to at least two task identifiers in the second task queue, so that the at least two tasks share the first computing card in a space-division multiplexing manner, and the at least two tasks are unfinished tasks.

[0322] In an optional embodiment, the first service process includes at least one thread, which corresponds one-to-one with at least one task, which includes a first task; the processing unit 1601 is used to call the first computing card to process the call request of the corresponding task using each of the at least one thread.

[0323] In an optional embodiment, the processing unit 1601 is further configured to disconnect the first connection in response to the first computing card meeting a disconnection condition. For example, the disconnection condition may include a malfunction of the first computing card, or the load on the first computing card exceeding a preset load.

[0324] It should be understood that the task execution device provided in the embodiments of this application can also be implemented using an ASIC or a PLD. The PLD can be a CPLD, FPGA, GAL, or any combination thereof. The task execution device provided in the above method embodiments can also be implemented by software. When the task execution device provided in the above method embodiments is implemented by software, each module in the task execution device can also be a software module.

[0325] Based on the same inventive concept, embodiments of this application provide a task execution device, which includes a memory and a processor. The memory stores computer programs. The processor executes the computer programs stored in the memory, causing the task execution device to perform actions such as... Figure 6 , Figure 7 and Figure 12 The operations shown in the method embodiment are performed by a task agent. For example, the task execution device is a node running a task agent. Figure 5 Node A in the task execution system shown.

[0326] Based on the same inventive concept, embodiments of this application provide a task execution device, which includes a memory and a processor. The memory stores computer programs. The processor executes the computer programs stored in the memory, causing the task execution device to perform actions such as... Figure 6 , Figure 7 and Figure 12 The operations shown in the method embodiment are performed by a resource manager. For example, the task execution device is a node running a resource manager. Figure 5 The task execution system shown is either node A or node B.

[0327] Based on the same inventive concept, embodiments of this application provide a task execution device, which includes a memory and a processor. The memory stores computer programs. The processor executes the computer programs stored in the memory, causing the task execution device to perform actions such as... Figure 6 , Figure 7 and Figure 12 The operations shown in the method embodiment are performed by the first service process. For example, the task execution device is a node running the first service process. Figure 5 The node shown is one of nodes 1 to n in the task execution system.

[0328] Based on the same inventive concept, embodiments of this application provide a task execution system, which includes a task agent and a resource manager. For example, the task agent includes, Figure 14The illustrated embodiment provides a task execution device 1400, the resource manager including, as shown in the example Figure 15 The illustrated embodiment provides a task execution device 1500.

[0329] In an optional embodiment, the task execution system further includes a first service process, the first service process including, for example: Figure 16 The illustrated embodiment provides a task execution device 1600.

[0330] As an example, the task execution system is as follows: Figures 2 to 5 Any of the task execution systems shown.

[0331] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed, implements... Figure 6 , Figure 7 and Figure 12 The method embodiment shown. For example, when the computer program is executed by the task agent, it implements as follows: Figure 6 , Figure 7 and Figure 12 The operations performed by the task agent in the illustrated method embodiment. When this computer program is executed by the resource manager, it implements the following... Figure 6 , Figure 7 and Figure 12 The operation performed by the resource manager in the illustrated method embodiment. When this computer program is executed by the service process, it implements the following... Figure 6 , Figure 7 and Figure 12 The operations performed by the service process in the illustrated method embodiment.

[0332] Based on the same inventive concept, embodiments of this application provide a computer program product, which includes a program or code, and which, when executed, implements... Figure 6 , Figure 7 and Figure 12 The method embodiment shown. For example, when the program or code is executed by the task agent, it implements as follows: Figure 6 , Figure 7 and Figure 12 The operations performed by the task agent in the illustrated method embodiment. When this program or code is executed by the resource manager, it implements the following... Figure 6 , Figure 7 and Figure 12 The operation is performed by the resource manager in the method embodiment shown. When the program or code is executed by the service process, it implements the following... Figure 6 , Figure 7 and Figure 12 The operations performed by the service process in the illustrated method embodiment.

[0333] It should be noted that the above embodiments of task execution device, task execution system, computer-readable storage medium, computer program product, etc., have the same or similar technical effects as the previous embodiments of task execution method, and therefore will not be described in detail.

[0334] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented entirely or partially as a computer program product, which includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this application are generated entirely or partially. The computer can be a general-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium, or a semiconductor medium (e.g., solid-state drive), etc.

[0335] It should be understood that the term "at least one" in this application refers to one or more, and "multiple" refers to two or more. In this application, unless otherwise stated, the symbol " / " generally means "or," for example, A / B can mean A or B. The term "and / or" in this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, for clarity, this application uses terms such as "first," "second," and "third" to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first," "second," and "third" do not limit the quantity or order of execution.

[0336] The different types of embodiments, such as the method embodiments and device embodiments provided in this application, can be referenced to each other. The order of operations in the method embodiments can be adjusted appropriately, and the operations can be added or removed in response to the situation. Any variations that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application, and therefore will not be described in detail.

[0337] In the corresponding embodiments provided in this application, it should be understood that the disclosed devices, etc., can be implemented through other configurations. For example, the device embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed between devices or modules may be through some interfaces, or indirect coupling or communication connection between devices or modules, which may be electrical or other forms. Modules described as separate components may or may not be physically separate, and components described as modules may or may not be physical modules; they may be located in one place or distributed across multiple network nodes. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0338] The above description is merely an exemplary embodiment of this application, but the scope of protection of this application is not limited thereto. Any equivalent modifications or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A task execution method characterized by, The method includes: The task agent sends a resource request to the resource manager, the resource request carrying the AI ​​resource requirements of the first task, the resource manager being used to manage computing cards in the computing cluster, the computing cluster including at least one computing card; The task agent receives a first service address sent by the resource manager. The first service address is the service address of the first computing card in the computing cluster. The first computing card meets the AI ​​resource requirements of the first task. The task agent invokes the first computing card to execute the AI ​​operation corresponding to the first task based on the first service address, wherein the node where the first computing card is located is a different node from the node running the task agent.

2. The method according to claim 1, characterized in that, The AI ​​resource requirements of the first task include a first requirement and a second requirement. The first computing card meets the first requirement. The method further includes: The task agent receives a second service address sent by the resource manager. The second service address is the service address of the second computing card in the computing cluster, and the second computing card meets the second requirement. The task agent invokes the second computing card to execute the AI ​​operation corresponding to the first task based on the second service address.

3. The method according to claim 2, characterized in that, The first computing card and the second computing card are located on the same node; or, The first computing card and the second computing card are located on different nodes.

4. The method according to claim 2 or 3, characterized in that, The first requirement includes a first identifier and a first AI resource requirement; the second requirement includes a second identifier and a second AI resource requirement; the first computing card satisfies the first AI resource requirement; the second computing card satisfies the second AI resource requirement; the task agent receives the first service address and the second service address sent by the resource manager, including: The task agent receives a resource response sent by the resource manager. The resource response carries a service address table, which includes the correspondence between the first identifier and the first service address, and the correspondence between the second identifier and the second service address.

5. The method according to claim 4, characterized in that, The task agent includes a first agent module and a second agent module; The task agent sends the resource request to the resource manager, including: the first agent module sending the resource request to the resource manager; The task agent receiving the resource response sent by the resource manager includes: the first agent module receiving the resource response sent by the resource manager; The method further includes: the first proxy module storing the service address table in a shared memory region, wherein the shared memory region is a memory region shared by the first proxy module and the second proxy module.

6. The method according to claim 5, characterized in that, The method further includes: The first proxy module determines the first service address based on the first identifier and the service address table stored in the shared memory region; The second proxy module determines the second service address based on the second identifier and the service address table stored in the shared memory region; The task agent invokes the first computing card to execute the AI ​​operation corresponding to the first task based on the first service address, including: the first agent module invokes the first computing card to execute the AI ​​operation corresponding to the first task based on the first service address; The task agent invokes the second computing card to execute the AI ​​operation corresponding to the first task based on the second service address, including: the second agent module invokes the second computing card to execute the AI ​​operation corresponding to the first task based on the second service address.

7. The method according to claim 2 or 3, characterized in that, The first requirement includes a first identifier and a first AI resource requirement, the second requirement includes a second identifier and a second AI resource requirement, the first computing card satisfies the first AI resource requirement, the second computing card satisfies the second AI resource requirement, and the method further includes: The task agent sends a first query request and a second query request to the resource manager. The first query request carries the first identifier, and the second query request carries the second identifier. The task agent receiving the first service address sent by the resource manager includes: the task agent receiving a first query response sent by the resource manager, wherein the first query response carries the first service address; The task agent receiving the second service address sent by the resource manager includes: the task agent receiving a second query response sent by the resource manager, the second query response carrying the second service address.

8. The method according to claim 7, characterized in that, The task agent includes a first agent module and a second agent module; The task agent sends the first query request to the resource manager, including: the first agent module sending the first query request to the resource manager; The task agent receiving the first query response sent by the resource manager includes: the first agent module receiving the first query response sent by the resource manager; The task agent invokes the first computing card to execute the AI ​​operation corresponding to the first task based on the first service address, including: the first agent module invokes the first computing card to execute the AI ​​operation corresponding to the first task based on the first service address; The task agent sends the second query request to the resource manager, including: the second agent module sending the second query request to the resource manager; The task agent receiving the second query response sent by the resource manager includes: the second agent module receiving the second query response sent by the resource manager; The task agent invokes the second computing card to execute the AI ​​operation corresponding to the first task based on the second service address, including: the second agent module invokes the second computing card to execute the AI ​​operation corresponding to the first task based on the second service address.

9. The method according to any one of claims 1 to 8, characterized in that, The task agent, based on the first service address, invokes the first computing card to execute the AI ​​operation corresponding to the first task, including: The task agent establishes a first connection based on the first service address; The task agent invokes the first computing card based on the first connection to execute the AI ​​operation corresponding to the first task.

10. The method according to claim 9, characterized in that, The task agent establishes the first connection based on the first service address, including: the task agent establishes the first connection with a first service process based on the first service address, and the first service process is used to manage the first computing card; The task agent invokes the first computing card to execute the AI ​​operation corresponding to the first task based on the first connection, including: the task agent sends the invocation request of the first task to the first service process through the first connection to trigger the first service process to invoke the first computing card to process the invocation request of the first task to execute the AI ​​operation corresponding to the first task.

11. A task execution method, characterized in that, The method includes: The resource manager receives a resource request sent by the task agent. The resource request carries the AI ​​resource requirements of the first task. The resource manager is used to manage the computing cards in the computing cluster, which includes at least one computing card. The resource manager sends a first service address to the task agent. The first service address is the service address of the first computing card in the computing cluster. The first computing card meets the AI ​​resource requirements of the first task. The node where the first computing card is located is a different node from the node running the task agent. The first service address is used by the task agent to call the first computing card to execute the AI ​​operation corresponding to the first task.

12. The method according to claim 11, characterized in that, The AI ​​resource requirements of the first task include a first requirement and a second requirement. The first computing card meets the first requirement. The method further includes: The resource manager sends a second service address to the task agent. The second service address is the service address of the second computing card in the computing cluster. The second computing card meets the second requirement. The task agent uses the second service address to call the second computing card to execute the AI ​​operation corresponding to the first task.

13. The method according to claim 12, characterized in that, The first computing card and the second computing card are located on the same node; or, The first computing card and the second computing card are located on different nodes.

14. The method according to claim 12 or 13, characterized in that, The first requirement includes a first identifier and a first AI resource requirement, the second requirement includes a second identifier and a second AI resource requirement, the first computing card satisfies the first AI resource requirement, the second computing card satisfies the second AI resource requirement, and the method further includes: The resource manager obtains a service address table based on the AI ​​resource requirements of the first task. The service address table includes the correspondence between the first identifier and the first service address, and the correspondence between the second identifier and the second service address. Sending the first service address and the second service address to the task agent by the resource manager includes: the resource manager sending a resource response to the task agent, the resource response carrying the service address table.

15. The method according to claim 14, characterized in that, The resource manager includes a resource broker and a resource controller, the resource controller being used to manage the computing cards in the computing cluster; The resource manager receiving the resource request sent by the task agent includes: the resource agent receiving the resource request sent by the task agent; The method further includes: the resource agent sending the resource request to the resource controller; The resource manager obtains the service address table according to the AI ​​resource requirements of the first task, including: the resource controller obtains the service address table according to the AI ​​resource requirements of the first task carried in the resource request; The method further includes: the resource controller sending the resource response to the resource broker; The resource manager sending the resource response to the task agent includes: the resource agent sending the resource response to the task agent.

16. The method according to claim 12 or 13, characterized in that, The first requirement includes a first identifier and a first AI resource requirement, the second requirement includes a second identifier and a second AI resource requirement, the first computing card satisfies the first AI resource requirement, the second computing card satisfies the second AI resource requirement, and the method further includes: The resource manager obtains and stores the service address table according to the AI ​​resource requirements of the first task. The service address table includes the correspondence between the first identifier and the first service address, and the correspondence between the second identifier and the second service address. The resource manager receives a first query request and a second query request sent by the task agent, wherein the first query request carries the first identifier and the second query request carries the second identifier. The resource manager determines the first service address based on the service address table and the first identifier carried in the first query request; The resource manager determines the second service address based on the service address table and the second identifier carried in the second query request; Sending the first service address to the task agent by the resource manager includes: the resource manager sending a first query response, wherein the first query response carries the first service address; Sending the second service address to the task agent by the resource manager includes: the resource manager sending a second query response to the task agent, the second query response carrying the second service address.

17. The method according to claim 16, characterized in that, The resource manager includes a resource broker and a resource controller, the resource controller being used to manage the computing cards in the computing cluster. The resource manager receiving the resource request sent by the task agent includes: the resource agent receiving the resource request sent by the task agent; The method further includes: the resource agent sending the resource request to the resource controller; The resource manager obtains the service address table according to the AI ​​resource requirements of the first task, including: the resource controller obtains the service address table according to the AI ​​resource requirements of the first task carried in the resource request; The method further includes: the resource controller sending the service address table to the resource proxy; the resource proxy storing the service address table; The resource manager receiving the first query request and the second query request sent by the task agent includes: the resource agent receiving the first query request and the second query request sent by the task agent; The resource manager determines the first service address based on the service address table and the first identifier carried in the first query request, including: the resource agent obtains the first service address based on the service address table and the first identifier carried in the first query request; The resource manager determines the second service address based on the service address table and the second identifier carried in the second query request, including: the resource agent obtains the second service address based on the service address table and the second identifier carried in the second query request; The resource manager sending the first query response to the task agent includes: the resource agent sending the first query response to the task agent; The resource manager sending the second query response to the task agent includes: the resource agent sending the second query response to the task agent.

18. A task execution device, characterized in that, Applied to a task agent, the task execution device includes at least one functional unit for performing the method as described in any one of claims 1 to 10.

19. A task execution device, characterized in that, Applied to a resource manager, the task execution device includes at least one functional unit, the at least one functional unit being used to perform the method as described in any one of claims 11 to 17.

20. A task execution device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to execute a computer program stored in the memory to cause the task execution device to perform the method as described in any one of claims 1 to 10.

21. A task execution device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to execute a computer program stored in the memory to cause the task execution device to perform the method as described in any one of claims 11 to 17.

22. A task execution system, characterized in that, It includes a task agent and a resource manager, wherein the task agent includes the task execution device as described in claim 18 or 20, and the resource manager includes the task execution device as described in claim 19 or 21.

23. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the method as described in any one of claims 1 to 17.

24. A computer program product, characterized in that, The computer program product includes a program or code that, when executed, implements the method as described in any one of claims 1 to 17.

Images