Task execution method for use in artificial intelligence accelerator card

Abstract

The present disclosure relates to a task execution method used in an artificial intelligence acceleration card, comprising: recording execution state information of a task in a task execution process; in the case of detecting a task execution failure, determining a minimum execution range of the task that needs to be rolled back based on the execution state information of the task; rolling back the execution state associated with the task execution failure in the minimum execution range of the task that needs to be rolled back; and rescheduling a subtask associated with the task execution failure in the task for re-execution.

Description

Technical Field

[0001] This disclosure relates to the field of computer science, and more specifically to a task execution method used in an artificial intelligence accelerator card. Background Technology

[0002] Currently, the software systems used in Artificial Intelligence (AI) accelerator cards typically employ an architecture that separates the runtime system from the driver system. The runtime system provides asynchronous execution interfaces to upper-layer applications or deep learning frameworks to implement functions such as task submission, stream management, event synchronization, and memory management, while interacting with the hardware or firmware of the AI ​​accelerator card through the driver system. Summary of the Invention

[0003] A task execution method for use in an AI accelerator card according to an embodiment of the present disclosure includes: recording task execution status information during task execution; determining the minimum execution range that the task needs to be rolled back based on the task execution status information when task execution failure is detected; rolling back the execution status associated with task execution failure within the minimum execution range that the task needs to be rolled back; and rescheduling subtasks associated with task execution failure in the task for re-execution.

[0004] These and other features of the task execution method disclosed herein, as well as the operational methods and functions and economics of manufacture of the associated structural elements and combinations thereof, will become more apparent upon consideration of the following description and appended claims with reference to the accompanying drawings (all of which form a part of this specification, wherein like reference numerals identify corresponding portions in the drawings). However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and not for limiting purposes. Attached Figure Description

[0005] Figure 1 This is a schematic block diagram of a task execution system used in an AI accelerator card according to an embodiment of the present disclosure.

[0006] Figure 2 This is a schematic flowchart of a task execution method used in an AI accelerator card according to an embodiment of the present disclosure.

[0007] Figure 3 This is a schematic block diagram of an example computing device that can implement a task execution method in an AI accelerator card according to embodiments of the present disclosure. Detailed Implementation

[0008] The features and exemplary embodiments of various aspects of this application will now be described in detail. Numerous specific details are set forth in the following detailed description to provide a comprehensive understanding of this application. However, it will be apparent to those skilled in the art that this application can be implemented without requiring some of these specific details. The following description of embodiments is merely intended to provide a better understanding of this application by illustrating examples. This application is by no means limited to any specific configuration presented below, but covers any modifications, substitutions, and improvements to elements, components, and algorithms without departing from the spirit of this application. Well-known structures and techniques are not shown in the accompanying drawings and the following description in order to avoid unnecessary obfuscation of this application.

[0009] Furthermore, the various operations will be described as multiple discrete operations in a manner most conducive to understanding the illustrative embodiments; however, the order of description should not be construed as implying that these operations must depend on the order. In particular, these operations do not need to be performed in the order presented.

[0010] The phrases “in an embodiment,” “in one embodiment,” and “in some embodiments” are used repeatedly throughout this document. These phrases do not typically refer to the same embodiment; however, they may refer to the same embodiment. Unless the context otherwise specifies, the terms “comprising,” “having,” and “including” are synonyms. The phrases “A or B” and “A / B” mean “(A), (B) or (A and B).”

[0011] Currently, in the software stack of AI accelerator cards, upper-layer applications typically submit tasks to the runtime system for execution in the form of operators, kernels, tasks, or subtasks. The runtime system asynchronously schedules tasks through streams, queues, or scheduling units, and returns the execution result to the upper-layer application after the task is completed. During task execution, tasks may fail due to insufficient resources, execution timeouts, device malfunctions, or inconsistent runtime states. When the runtime system detects a task execution failure, it typically adopts one of the following processing methods: 1) directly returns an error status to the upper-layer application, allowing the upper-layer application to decide whether to resubmit the task; 2) requires the upper-layer application to resubmit the entire task or complete execution chain, regardless of the execution state of different stages within the task; 3) marks the execution context as an abnormal state, rebuilds the execution context, or restarts the process to resume task execution.

[0012] The above-mentioned handling method when the runtime system detects task execution failure has one or more of the following problems: 1) Even if some stages of the task have been successfully completed, the entire task or task chain must be re-executed; 2) Successfully completed calculation results are discarded, resulting in repeated calculations, which increases execution latency and computing power consumption; 3) In scenarios such as large model inference, pipeline parallelism, or long-running tasks, failure of any stage of the task will cause the entire task to be rolled back, which seriously affects system throughput; 4) Task-level failures may spread to stream-level or execution context-level anomalies, thereby affecting the execution of other unrelated tasks.

[0013] In view of one or more problems existing in the above-mentioned handling methods when the runtime system detects task execution failure, a task execution method and system for use in an AI accelerator card according to embodiments of the present disclosure are proposed. By introducing task execution status recording, task rollback range determination, and subtask rescheduling, only the part associated with the task execution failure can be rolled back and rescheduled when task execution failure is detected, thereby improving system execution efficiency and stability.

[0014] Figure 1 This is a schematic block diagram of a task execution system used in an AI accelerator card according to an embodiment of the present disclosure. Figure 2 This is a schematic flowchart of a task execution method used in an AI accelerator card according to an embodiment of the present disclosure. The following is in conjunction with... Figure 1 and Figure 2 This document describes in detail a task execution system and method for use in an AI accelerator card according to embodiments of the present disclosure.

[0015] like Figure 1 As shown, the task execution system 100 used in the AI ​​accelerator card includes an execution status recording module 102, a rollback range determination module 104, a partial rollback control module 106, and a subtask rescheduling module 108. The execution status recording module 102 is configured to record the execution status information of the task during task execution (i.e., the execution state of the task). Figure 2 The processing step S202 shown); the rollback range determination module 104 is configured to determine the minimum execution range (i.e., the execution range) that the task needs to be rolled back based on the task's execution status information when a task execution failure is detected. Figure 2 The process shown in S204); the partial rollback control module 106 is configured to roll back the execution state associated with task execution failure within the minimum execution range that the task needs to be rolled back (i.e., execution...). Figure 2 The processing shown in S206); the subtask rescheduling module 108 is configured to reschedule the subtasks in the task that are associated with the task execution failure for re-execution (i.e., execution). Figure 2 The process shown is S208.

[0016] In some embodiments, the execution status recording module 102, the rollback range determination module 104, the partial rollback control module 106, and the subtask rescheduling module 108 can be uniformly managed by the runtime system in the AI ​​acceleration card.

[0017] In some embodiments, the task execution status information includes information about one or more of the following: the task phase division of the task, the completion status of each task phase of the task, the resource usage status related to the task, and the dependencies between subtasks in the task.

[0018] In some embodiments, the rollback scope determination module 104 is configured to determine the minimum execution scope of a task that needs to be rolled back using a rollback boundary of any of the following granularities: a single subtask or operator of the task, a single task stage of the task, or a portion of subtasks in a subtask dependency chain within the task. In other words, the minimum execution scope of a task that needs to be rolled back uses a rollback boundary of any of the following granularities: a single subtask or operator of the task, a single task stage of the task, or a portion of subtasks in a subtask dependency chain within the task.

[0019] In some embodiments, the rollback range determination module 104 is configured to retain the execution results of the following task phases in the task when determining the minimum execution range in which the task needs to be rolled back: the task phases that have been successfully executed and whose execution results do not depend on the execution results of the task phases that failed to execute.

[0020] In some embodiments, the partial rollback control module 106 is configured to roll back the execution states associated with task execution failure within the minimum execution range that the task needs to be rolled back through the following processes: releasing or resetting resources associated with task execution failure, rolling back intermediate states corresponding to task execution failure, and cleaning up execution records corresponding to task execution failure.

[0021] In some embodiments, the partial rollback control module 106 is configured to retain, during the rollback of the execution states associated with task execution failure within the minimum execution range that the task needs to be rolled back, the execution states that are not affected by task execution failure.

[0022] In some embodiments, the subtask rescheduling module 108 is configured to reschedule subtasks associated with task execution failure within the minimum execution range that the task needs to be rolled back, either within the same stream or different streams, or to schedule subtasks associated with task execution failure within the minimum execution range that the task needs to be rolled back, after resource conditions have been met and a delay has elapsed. It should be noted that this subtask rescheduling process does not affect the execution results of completed subtasks within the minimum execution range that the task needs to be rolled back. Once the subtasks associated with task execution failure within the minimum execution range that the task needs to be rolled back have completed execution, the task continues to execute.

[0023] Figure 3 This is a schematic block diagram of an example computing device capable of implementing a task execution method in an AI accelerator card according to embodiments of the present disclosure. Figure 3 As shown, computing device 300 may include bus 302 or other communication mechanisms for transmitting information and one or more hardware processors 304 coupled to bus 302 for processing information. The one or more hardware processors 304 may include, for example, one or more general-purpose microprocessors.

[0024] like Figure 3 As shown, in some embodiments, computing device 300 may further include main memory 306 coupled to bus 302 for storing information and instructions executed by one or more processors 304, such as random access memory (RAM), cache, and / or other dynamic storage devices. Main memory 306 may also be used to store temporary variables or other intermediate information during the execution of instructions executed by one or more processors 304. These instructions, when stored in storage media accessible to one or more processors 304, can cause computing device 300 to become a dedicated machine customized to perform the operations specified in the instructions. Storage device 308 may include non-volatile and / or volatile storage media. Non-volatile storage media may include, for example, optical discs or magnetic disks. Volatile storage media may include dynamic memory. Common forms of storage media may include, for example, floppy disks, flexible disks, hard disks, solid-state drives, magnetic tape, or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with a perforated pattern, RAM, DRAM, PROM, and EPROM, FLASH-EPROM, NVRAM, any other memory chip or cartridge, or their networking versions.

[0025] like Figure 3As shown, in some embodiments, computing device 300 may further include a communication interface 310 coupled to bus 302. Communication interface 310 may provide bidirectional data communication coupling to one or more network links connected to one or more networks. As another example, communication interface 310 may be a local area network (LAN) card to provide data communication connectivity to a LAN-compatible (or WAN component communicating with a WAN) network. Wireless links may also be implemented.

[0026] The execution of certain operations can be distributed across processors rather than residing within a single machine, but rather deployed across multiple machines. In some example embodiments, the processor or processor-implemented engine may reside in a single geographic location (e.g., in a home environment, office environment, or server farm). In other example embodiments, the processor or processor-implemented engine may be distributed across multiple geographic locations.

[0027] Each of the processes, methods, and algorithms described in the preceding sections may be embodied in code modules executed by one or more computer systems or computer processors including computer hardware, and may be fully or partially automated by these code modules. The processes and algorithms may be implemented, partially or fully, in dedicated circuit systems.

[0028] When the functions disclosed herein are implemented as software functional units and sold or used as stand-alone products, they may be stored in a processor-executable, non-volatile, computer-readable storage medium. Specific technical solutions (all or part) disclosed herein, or aspects contributing to the prior art, may be embodied in the form of a software product. The software product may be stored in a storage medium and includes several instructions that cause a computing device (which may be a personal computer, server, network device, etc.) to perform all or some steps of the methods of the embodiments of this application. The storage medium may include a flash drive, portable hard disk drive, ROM, RAM, magnetic disk, optical disk, other media operable to store program code, or any combination thereof.

[0029] Specific embodiments further provide a system including a processor and a non-transitory computer-readable storage medium storing instructions executable by the processor to cause the system to perform operations corresponding to steps in any method of the embodiments disclosed above. Specific embodiments further provide a non-transitory computer-readable storage medium storing instructions executable by one or more processors to cause the one or more processors to perform operations corresponding to steps in any method of the embodiments disclosed above.

[0030] The embodiments disclosed herein can be implemented via a cloud platform, server, or server cluster (collectively referred to below as the "Service System") that interacts with a client. The client can be a terminal device or a client registered by a user at the platform, wherein the terminal device can be a mobile terminal, a personal computer (PC), or any device that can have the platform application installed.

[0031] The various features and processes described above can be used independently of each other or combined in various ways. All possible combinations and sub-combinations are intended to fall within the scope of this disclosure. Additionally, certain method or process blocks may be omitted in some embodiments. The methods and processes described herein are not limited to any particular order, and their associated blocks or states may be executed in other suitable orders. For example, described blocks or states may be executed in an order other than that specifically disclosed, or multiple blocks or states may be combined into a single block or state. Example blocks or states may be executed sequentially, in parallel, or in some other manner. Blocks or states may be added to or removed from the disclosed example embodiments. The exemplary systems and components described herein may be configured differently than described. For example, components may be added to, removed from, or rearranged compared to the disclosed example embodiments.

[0032] The various operations of the exemplary methods described herein can be performed at least in part by an algorithm. The algorithm may be included in program code or instructions stored in memory (e.g., the aforementioned non-transitory computer-readable storage medium). This algorithm may include a machine learning algorithm. In some embodiments, the machine learning algorithm may not explicitly refer to the computer as performing the function but may learn from training data to generate a predictive model of the function.

[0033] The various operations of the exemplary methods described herein can be performed, at least in part, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, these processors can constitute an engine of processor implementation that operates to perform one or more of the operations or functions described herein.

[0034] Similarly, the methods described herein may be implemented at least in part by a processor, wherein one or more specific processors are instances of hardware. For example, at least some operations of the methods may be performed by one or more processors or an engine implemented by a processor. Furthermore, one or more processors may also be operable to support the execution of relevant operations in a “cloud computing” environment or as the execution of relevant operations in a “Software as a Service” (SaaS) context. For example, at least some operations may be performed by a group of computers (as an example of a machine containing processors), wherein these operations are accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., application programming interfaces (APIs)).

[0035] The execution of certain operations can be distributed across processors rather than residing within a single machine, and can be deployed across multiple machines. In some example embodiments, the processor or processor-implemented engine may reside in a single geographic location (e.g., in a home environment, office environment, or server farm). In other example embodiments, the processor or processor-implemented engine may be distributed across multiple geographic locations.

[0036] Throughout this specification, multiple instances may be implemented as components, operations, or structures described as a single instance. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of these individual operations may be performed simultaneously, and not necessarily in the order illustrated. Structures and functions presented as separate components in the example configuration may be implemented as composite structures or components. Similarly, structures and functions presented as single components may be implemented as single components. These and other variations, modifications, additions, and improvements fall within the scope of this document.

[0037] As used herein, "or" is inclusive rather than exclusive unless explicitly indicated by the context. Therefore, in this document, "A, B, or C" means "A, B, A and B, A and C, B and C, or A, B, and C" unless explicitly indicated by the context. Furthermore, "and" is combined and separate unless explicitly indicated by the context. Therefore, in this document, "A and B" means "A and B, combined or separate" unless explicitly indicated by the context. Additionally, multiple instances of resources, operations, or structures described herein may be provided as a single instance. Furthermore, the boundaries between various resources, operations, engines, and data storage devices are somewhat arbitrary and specific operations are illustrated within the context of a particular illustrative configuration. Other functional assignments are foreseeable and fall within the scope of various embodiments of this disclosure. Generally, structures and functions presented as individual resources in example configurations may be implemented as combined structures or resources. Similarly, structures and functions presented as single resources may be implemented as single resources. These and other changes, modifications, additions, and improvements fall within the scope of the embodiments of this disclosure as expressed in the appended claims. Therefore, this specification and drawings should be considered illustrative rather than restrictive.

[0038] The terms “comprising” or “including” are used to indicate the presence of a subsequently claimed feature, but do not preclude the addition of other features. Unless otherwise specifically stated or otherwise understood in the context in which they are used, conditional language such as “may,” “can,” “may,” and “can” is generally intended to convey that certain embodiments include certain features, components, and / or steps that are not included in other embodiments. Therefore, this conditional language is generally not intended to imply that one or more embodiments require features, components, and / or steps in any way, or that one or more embodiments must include logic for determining whether such features, components, and / or steps are included in or performed in any particular embodiment, with or without user input or prompts.

[0039] Although the general outline of the subject matter has been described with reference to specific exemplary embodiments, various modifications and changes may be made to these embodiments without departing from the broad scope of embodiments of this disclosure. Where more than one embodiment is disclosed, these embodiments of the subject matter may be referred to individually or collectively herein as the term "invention," this is for convenience only and is not intended to automatically limit the scope of this application to any single disclosure or concept.

[0040] The embodiments illustrated herein have been described in sufficient detail to enable those skilled in the art to practice the teachings disclosed. Other embodiments may be used and derived therefrom, such that structural and logical substitutions and changes may be made without departing from the scope of this disclosure. Therefore, the term "implementation" is not intended to be limiting, and the scope of the various embodiments is defined only by the appended claims and their full scope.

Claims

1. A task execution method used in an artificial intelligence accelerator card, characterized in that, In the software stack of the AI ​​accelerator card, upper-layer applications submit tasks to be executed to the runtime system in the form of operators, kernels, tasks, or subtasks. The runtime system asynchronously schedules the tasks through streams, queues, or scheduling units and returns the execution result to the upper-layer application after the task is completed. The task execution method includes: Record the execution status information of the task during the execution of the task; If the task execution failure is detected, the minimum execution range that the task needs to be rolled back is determined based on the task's execution status information; Rollback the execution states associated with the task's failure within the minimum execution range that require rollback; and The subtasks associated with the task's execution failure within the task are rescheduled for re-execution, wherein... The execution status information of the task includes information about one or more of the following: the task phase division of the task, the completion status of each task phase of the task, the resource usage status related to the task, and the dependencies between subtasks in the task. The minimum execution scope that the task needs to be rolled back can be a rollback boundary of any of the following granularities: a single subtask or operator of the task, a single task stage of the task, or a portion of the subtasks in the subtask dependency chain within the task.

2. The task execution method according to claim 1, characterized in that, Rolling back the execution status associated with the task execution failure includes: releasing or resetting the resources associated with the task execution failure, rolling back the intermediate state corresponding to the task execution failure, and clearing the execution records corresponding to the task execution failure.

3. The task execution method according to claim 1, characterized in that, Reschedule subtasks associated with the failure of the task execution in the same or different streams, or schedule subtasks associated with the failure of the task execution after a delay when resource conditions are met.

4. The task execution method according to claim 1, characterized in that, When determining the minimum execution scope that the task needs to be rolled back, the execution results of the following task phases in the task are retained: the task phases that have been successfully executed and whose execution results do not depend on the execution results of the task phases that failed to execute.

5. The task execution method according to claim 1, characterized in that, During the rollback of the execution state associated with the task execution failure, the execution states that were not affected by the task execution failure are preserved within the minimum execution range that the task needs to be rolled back.

6. A non-transitory computer-readable storage medium, comprising instructions, characterized in that, When the instruction is executed by the AI ​​accelerator card, the AI ​​accelerator card performs the task execution method as described in any one of claims 1 to 5.

7. A computer program product, comprising instructions, characterized in that, When the instruction is executed by the AI ​​accelerator card, the AI ​​accelerator card performs the task execution method as described in any one of claims 1 to 5.

Images