Method for managing jobs and computing system for performing the same

Abstract

The present disclosure provides a method for managing tasks performed by at least one processor. The method comprises the steps of: distributing a plurality of tasks associated with at least one context to a plurality of processing devices; storing a task that has been processed as completed among the distributed tasks in a task pending queue; identifying a first context associated with an error in response to determining that an error has occurred in at least one context; initializing each of a plurality of command queues included in the plurality of processing devices; determining a task to be recovered based on the identified first context; and recovering the determined task to be recovered to each initialized command queue. The plurality of processing devices include a plurality of command queues that store the distributed tasks.

Description

Technology Field

[0001] The present disclosure relates to a method for managing tasks and a computing system. Specifically, the present disclosure relates to a technique for ensuring task continuity by identifying a context in which an error has occurred while running in a multi-device environment, excluding the context from service, and recovering another context using a task pending queue in which completed tasks are stored. Background Technology

[0002] To perform artificial intelligence computations, hardware specialized for AI computation is utilized. For example, AI computations are performed more quickly by using accelerators including GPUs (Graphic Processing Units) and NPUs (Neural Processing Units). Data forming the basis of AI computations is transmitted to such hardware, and the hardware can apply the received data to a machine learning model to provide AI computation results (e.g., inference results).

[0003] Artificial intelligence (AI) computations may fail due to errors occurring during the computation caused by various factors, such as errors in input data applied to machine learning models or errors in the system or chips. To prepare for such failures, hardware resources are partitioned, and systems are designed so that each task is performed independently through these partitioned resources. For example, the system can be designed by pre-separating caches, RAM (Random Access Memory), and sys pipes, allowing independent tasks to be utilized through these separated hardware resources. In such cases, AI computation tasks are performed independently of each partitioned hardware resource, and even if an error occurs in a specific task, it does not affect other tasks, thereby satisfying fault tolerance.

[0004] As such, partitioning hardware resources incurs high design costs and can also require significant hardware resources. Consequently, there is a growing need for technologies that can satisfy coupling tolerance at a lower cost.

[0005] The information described above is intended to enhance understanding of the background of the present invention and may include information that does not constitute prior art. The problem to be solved

[0006] The present disclosure provides a method for managing operations to solve the above-mentioned problems, a computer program stored on a computer-readable recording medium, a computer-readable recording medium, and a device (system). means of solving the problem

[0007] The present disclosure may be implemented in various ways, including methods, devices (systems), and / or computer programs stored on computer-readable storage media.

[0008] According to one embodiment of the present disclosure, a method for managing tasks performed by a host device may include: distributing a plurality of tasks associated with at least one context to a plurality of processing devices; storing a task that has been processed as completed among the distributed tasks in a task pending queue; identifying a first context associated with an error in response to determining that an error has occurred in at least one context; initializing each of a plurality of command queues included in the plurality of processing devices; determining a task to be recovered based on the identified first context; and recovering the determined task to be recovered to each initialized command queue. The plurality of processing devices may include a plurality of command queues that store the distributed tasks.

[0009] According to one embodiment of the present disclosure, the determining step may include determining a task not associated with a first context among tasks stored in a task pending queue as a first task to be recovered, and determining a task not associated with a first context among tasks stored in each of a plurality of command queues as a second task to be recovered.

[0010] According to one embodiment of the present disclosure, the recovery step may include a step of recovering a first recovery target task first and a step of recovering a second recovery target task sequentially having a sequence number after the first recovery target task.

[0011] According to one embodiment of the present disclosure, a task stored in a task pending queue includes node data associated with at least one processing device among a plurality of processing devices, and the step of preferentially recovering may include the step of transmitting a first task to be recovered to a processing device associated with a first task to be recovered based on the node data included in the first task to be recovered.

[0012] According to one embodiment of the present disclosure, a method for managing a task may further include, after a saving step, a step of identifying a second context that is completed among at least one context and a step of deleting a task associated with the identified second context from a task pending queue.

[0013] According to one embodiment of the present disclosure, a second context includes a first task and a second task, and the step of identifying the second context may include a step of determining whether the first task is completed, a step of determining whether the second task is completed, and a step of determining that the second context is completed in response to determining that the first task and the second task are completed.

[0014] According to one embodiment of the present disclosure, the step of identifying a second context may include identifying the number of tasks associated with the second context, counting the number of completions of tasks associated with the second context, and determining that the second context is completed in response to determining that the counted number of completions is equal to the number of tasks.

[0015] According to one embodiment of the present disclosure, a method for managing an operation may further include, prior to the step of identifying a first context, the step of receiving an error message from at least one processing device among a plurality of processing devices and the step of determining that an error has occurred in at least one context in response to receiving the error message.

[0016] According to one embodiment of the present disclosure, the step of identifying a first context may include the step of identifying an operation in which an error occurred based on an error message, and the step of determining that an error occurred in the first context in response to determining that the first context is associated with the operation in which an error occurred.

[0017] According to one embodiment of the present disclosure, a method for managing a task may further include, prior to the step of identifying a first context, the step of receiving a time-out report from at least one processing device among a plurality of processing devices and the step of determining that an error has occurred in at least one context based on the time-out report.

[0018] According to one embodiment of the present disclosure, the step of identifying a first context may include identifying the number of tasks included in the first context, counting the number of times a timeout report associated with the first context is received, and determining that an error has occurred in the first context in response to determining that the number of times received is equal to the number of tasks.

[0019] According to one embodiment of the present disclosure, the step of storing in a work pending queue may include receiving a work completion report from at least one processing device among a plurality of processing devices, identifying a work associated with the work completion report in response to receiving the work completion report, processing the work associated with the work completion report as completed, and storing the processed work in a work pending queue.

[0020] According to one embodiment of the present disclosure, the step of storing in a job pending queue may include receiving a time-out report from at least one of a plurality of processing devices, identifying a job associated with the time-out report in response to receiving the time-out report, processing the job associated with the time-out report as completed, and storing the completed job in the job pending queue.

[0021] According to one embodiment of the present disclosure, at least one context is associated with a specific user, and a method for managing a task may further include the steps of identifying the context associated with the time-out report in response to receiving a time-out report and transmitting the time-out report to the user associated with the identified context.

[0022] According to one embodiment of the present disclosure, the first context includes a third task and a fourth task, the third task includes a first command and a second command, the fourth task includes a third command and a fourth command, and at least one of the first command or the second command may be associated with at least one of the third command or the fourth command.

[0023] According to one embodiment of the present disclosure, a computer program stored on a computer-readable recording medium may be provided for executing a method of managing work on a computer.

[0024] According to one embodiment of the present disclosure, a computing system comprises a task pending queue for storing completed tasks and at least one host processor configured to manage the task pending queue. The at least one host processor may be further configured to distribute a plurality of tasks associated with at least one context to a plurality of processing devices, store completed tasks among the distributed tasks in the task pending queue, identify a first context associated with an error in response to determining that an error has occurred in at least one context, initialize each of a plurality of command queues included in the plurality of processing devices, determine a task to be recovered based on the identified first context, and recover the determined task to be recovered to each initialized command queue. A plurality of processing devices may include a plurality of command queues for storing distributed tasks.

[0025] According to one embodiment of the present disclosure, determining may include determining a task not associated with a first context among tasks stored in a task pending queue as a first task to be recovered, and determining a task not associated with a first context among tasks stored in each of a plurality of command queues as a second task to be recovered.

[0026] According to one embodiment of the present disclosure, recovery may include recovering a first recovery target task first, and recovering a second recovery target task sequentially to have a sequence number after the first recovery target task.

[0027] According to one embodiment of the present disclosure, at least one host processor may be further configured to identify a second context that has been completed among at least one context, and to remove a task associated with the identified second context from a task pending queue. Effects of the invention

[0028] According to various embodiments of the present disclosure, a host system configures a job pending queue that stores completed jobs, thereby enabling jobs with already completed dependency relationships to be restored normally even during the queue initialization and recovery process. This prevents jobs that have already been completed but whose dependency relationships have not been resolved from being deleted upon reset, thereby ensuring system stability and the continuity of the workflow.

[0029] According to various embodiments of the present disclosure, instead of adopting a method in which a host processor configures a separate queue before task distribution to distribute tasks while considering all dependency relationships, multiple tasks with dependency relationships are distributed to multiple processing devices, and then each device processor with dependency relationships can transmit and receive data to and from each other and execute commands. Through this configuration, host overhead can be effectively reduced, and dependency issues are processed in real time through dynamic interaction between devices; furthermore, bottlenecks that may occur during the task distribution process are prevented, and the processing speed of the system can be improved.

[0030] According to various embodiments of the present disclosure, a host system may provide a rapid response to a timeout detection to a user terminal before identifying whether an error has occurred. Through this configuration, the user can quickly recognize the situation and take necessary measures, the overall stability of the system may be improved, and the user experience may be enhanced.

[0031] According to various embodiments of the present disclosure, fault tolerance for errors can be satisfied without the need to partition hardware resources.

[0032] The effects of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art to which the present disclosure pertains (referred to as "person skilled in the art") from the description in the claims. Brief explanation of the drawing

[0033] Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, wherein similar reference numerals indicate similar elements, but are not limited thereto. FIG. 1 is a block diagram illustrating a processing system according to some embodiments of the present disclosure. Figure 2 is a block diagram for explaining the processing device of Figure 1 in detail. Figure 3 is a block diagram for explaining the host system of Figure 1 in detail. FIG. 4 is a block diagram illustrating a processing system according to some embodiments of the present disclosure. FIG. 5 is a diagram illustrating the relationship between a host processor and a device processor according to one embodiment of the present disclosure. FIG. 6 is a drawing for explaining examples of a command queue and a command buffer included in a processing device according to one embodiment of the present disclosure. FIG. 7 is a drawing for illustrating examples of a command buffer included in each of a pending task queue and a processing device included in a host system according to one embodiment of the present disclosure. FIG. 8 is a diagram illustrating an example in which an error exists in an artificial intelligence operation associated with a first context, according to one embodiment of the present disclosure. FIG. 9 is a diagram illustrating the process of recovering a command queue after it has been initialized according to one embodiment of the present disclosure. FIG. 10 is a drawing for illustrating examples of a command buffer included in each of a pending task queue and a processing device included in a host system according to one embodiment of the present disclosure. FIG. 11 is a drawing for illustrating an example in which a timeout is detected according to one embodiment of the present disclosure. FIG. 12 is a diagram illustrating the process of recovering a command queue after it has been initialized according to one embodiment of the present disclosure. FIG. 13 is a flowchart illustrating a work management method according to one embodiment of the present disclosure. FIG. 14 is a flowchart illustrating a work recovery method according to one embodiment of the present disclosure. Specific details for implementing the invention

[0034] Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the attached drawings. However, in the following description, specific descriptions regarding widely known functions or configurations will be omitted if there is a risk that the gist of the present disclosure may be unnecessarily obscured.

[0035] In the attached drawings, identical or corresponding components are assigned the same reference numerals. Additionally, in the description of the following embodiments, the description of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0036] The advantages and features of the disclosed embodiments and the methods for achieving them will become clear by referring to the embodiments described below in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below but may be implemented in various different forms, and the embodiments provided are merely to make the present disclosure complete and to fully inform those skilled in the art of the scope of the invention.

[0037] The terms used in this specification will be briefly explained, and the disclosed embodiments will be described in detail. The terms used in this specification have been selected to be as generally used as possible, taking into account their functions in this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this disclosure should be defined not merely by their names, but based on their meanings and the content throughout this disclosure.

[0038] In this specification, singular expressions include plural expressions unless the context clearly specifies them as singular. Additionally, plural expressions include singular expressions unless the context clearly specifies them as plural. Throughout the specification, when a part is described as including a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0039] Additionally, the terms 'module' or 'part' as used in the specification refer to software or hardware components, and the 'module' or 'part' performs certain roles. However, the meaning of 'module' or 'part' is not limited to software or hardware. The 'module' or 'part' may be configured to reside in an addressable storage medium or configured to run on one or more processors. Thus, as an example, the 'module' or 'part' may include components such as software components, object-oriented software components, class components, and task components, and at least one of processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. The components and the functions provided within the 'module' or 'part' may be combined into a smaller number of components and 'modules' or 'parts', or further separated into additional components and 'modules' or 'parts'.

[0040] According to one embodiment of the present disclosure, a 'module' or 'part' may be implemented as a processor and memory, or as a circuit (circuitry). Terms such as 'circuit' (circuitry) may refer to a circuit in hardware, but may also refer to a circuit in software. The term 'processor' should be broadly interpreted to include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, etc. In some contexts, the term 'processor' may refer to an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), etc. The term 'processor' may also refer to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors combined with a DSP core, or any other combination of such configurations. Additionally, the term 'memory' should be broadly interpreted to include any electronic component capable of storing electronic information. 'Memory' may refer to various types of processor-readable media, such as Random Access Memory (RAM), Read-Only Memory (ROM), Non-Volatile Random Access Memory (NVRAM), Programmable Read-Only Memory (PROM), Erasable-Programmable Read-Only Memory (EPROM), Electrically Erasable PROM (EEPROM), Flash Memory, Magnetic or Optical Data Storage Devices, Registers, etc. If a processor can read information from memory and / or write information to memory, the memory is said to be in an electronic communication state with the processor. Memory integrated into a processor is in an electronic communication state with the processor.

[0041] In addition, terms such as first, second, A, B, (a), (b), etc. used in the following embodiments are used merely to distinguish one component from another, and the essence, order, or sequence of the said component is not limited by such terms.

[0042] Additionally, in the following embodiments, where it is stated that one component is 'connected', 'coupled', or 'joined' to another component, it should be understood that the component may be directly connected or joined to the other component, but that another component may also be 'connected', 'coupled', or 'joined' between each component.

[0043] Additionally, the terms 'comprises' and / or 'comprising' as used in the following embodiments do not exclude the presence or addition of one or more other components, steps, actions, and / or elements to the mentioned components, steps, actions, and / or elements.

[0044] In the present disclosure, 'each of the plurality of A' may refer to each of all components included in the plurality of A, or each of some components included in the plurality of A.

[0045] Before describing the various embodiments of the present disclosure, the terms used will be explained.

[0046] In the present disclosure, a ‘machine learning model’ may include any model used to infer an answer to a given input. According to one embodiment, the machine learning model may include an artificial neural network model comprising an input layer, a plurality of hidden layers, and an output layer. Here, each layer may include a plurality of nodes. Additionally, in the present disclosure, the machine learning model may refer to an artificial neural network model, and the artificial neural network model may refer to a machine learning model.

[0047] In the present disclosure, a 'descriptor' may include at least one instruction address for executing an artificial intelligence operation. Here, the instruction address may be the address of a storage area (e.g., a buffer) where the instruction is stored. Additionally, the descriptor may be associated with at least one operation. For example, the execution of a single operation may mean that at least one instruction associated with the descriptor is executed.

[0048] Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0049] FIG. 1 is a block diagram illustrating a processing system (PS) according to some embodiments of the present disclosure. Referring to FIG. 1, a processing system (PS) according to some embodiments of the present disclosure may include a processing device (1), a host system (HS), and a host interface (HIO).

[0050] In one embodiment, the processing device (1) may be a device that performs calculations using an artificial neural network. The processing device (1) may be, for example, a device specialized in performing deep learning calculation tasks. However, the present embodiment is not limited thereto.

[0051] In one embodiment, the processing unit (1) may include one or more accelerators such as a neural processing unit (NPU), a graphics processing unit (GPU), or a central processing unit (CPU) specialized for deep learning tasks. However, the present disclosure is not limited thereto, and the processing unit (1) may be any other type of processing unit.

[0052] In one embodiment, the processing device (1) may include at least one processor. Additionally, the processing device (1) may include a memory for storing data processed by the processor. In one embodiment, a task pending queue for managing completed tasks is stored in the memory, and the processor may manage the task pending queue stored in the memory.

[0053] The host system (HS) may be a computing system that directs a processing device (1) to perform computational operations and retrieves the results of the computational operations. For example, the host system (HS) may transmit data related to artificial intelligence computation to the processing device (1) and receive the results of the artificial intelligence computation based on the transmitted data from the processing device (1). In one embodiment, the host system (HS) may be a computing system that is not specialized for deep learning computational operations compared to the processing device (1). However, the present embodiment is not limited thereto.

[0054] The host interface (HIO) can transmit data and / or control signals between the processing unit (1) and the host system (HS). The host interface (HIO) can, for example, transmit commands and / or data from the host system (HS) to the processing unit (1), thereby enabling the processing unit (1) to perform computational operations. When the processing unit (1) completes a computational operation, the result can be transmitted to the host system (HS) via an interrupt request. The host interface (HIO) may, for example, be PCIe (PCI Express), but is not limited thereto.

[0055] FIG. 2 is a block diagram for describing in detail the processing unit (1) of FIG. 1. Referring to FIG. 2, the processing unit (1) may include a neural core SoC (10), an off-chip memory (30), a non-volatile memory interface (40), and a volatile memory interface (50). In the description referring to FIG. 2, the processing unit (1) is described as, by example, a neural network processing unit.

[0056] The neural core SoC (10) may be a system-on-chip device. The neural core SoC (10) may include an accelerator as an artificial intelligence computing unit. The neural core SoC (10) may include, for example, at least one of a graphics processing unit (GPU), a field programmable gate array (FPGA), or an application-specific integrated circuit (ASIC). However, the present embodiment is not limited thereto.

[0057] The neural core SoC (10) can exchange data with other external computation units through a separate external interface. Additionally, the neural core SoC (10) can be connected to non-volatile memory (31) and volatile memory (32), respectively, through a non-volatile memory interface (40) and a volatile memory interface (50).

[0058] The off-chip memory (30) may be memory placed outside the chip of the neural core SoC (10). The off-chip memory (30) may include non-volatile memory (31) and volatile memory (32).

[0059] The non-volatile memory (31) may be a memory that retains stored information even when power is not supplied. The non-volatile memory (31) may store one or more instructions for controlling operations on a machine learning model described later. Non-volatile memory (31) is, for example, ROM (Read-Only Memory), PROM (Programmable Read-Only Memory), EAROM (Erasable Alterable ROM), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory) (e.g., NAND Flash memory, NOR Flash memory), UVEPROM (Ultra-Violet Erasable Programmable Read-Only Memory), FeRAM (Ferroelectric Random Access Memory), MRAM (Magnetoresistive Random Access Memory), PRAM (Phase-change Random Access Memory), SONOS (silicon-oxide-nitride-oxide-silicon), RRAM (Resistive Random Access Memory), NRAM (Nanotube Random Access Memory), magnetic computer memory (e.g., hard disk, floppy disk drive, magnetic tape), optical disc drive or 3D crosspoint It may include at least one of a memory (3D XPoint memory). However, the present embodiment is not limited thereto.

[0060] Volatile memory (32), unlike non-volatile memory (31), may be a memory that continuously requires power to maintain stored information. Volatile memory (32) may include, for example, at least one of DRAM (Dynamic Random Access Memory), SRAM (Static Random Access Memory), SDRAM (Synchronous Dynamic Random Access Memory), or DDR SDRAM (Double Data Rate SDRAM). However, the present embodiment is not limited thereto.

[0061] The non-volatile memory interface (40) may include, for example, at least one of PATA (Parallel Advanced Technology Attachment), SCSI (Small Computer System Interface), SAS (Serial Attached SCSI), SATA (Serial Advanced Technology Attachment), or PCIe (PCI Express). However, the present embodiment is not limited thereto.

[0062] The volatile memory interface (50) may be at least one of, for example, SDR (Single Data Rate), DDR (Double Data Rate), QDR (Quad Data Rate), ODR (Octal Data Rate), or XDR (eXtreme Data Rate). However, the present embodiment is not limited thereto.

[0063] In one embodiment, the neural core SoC (10) may include at least one processor, and the processor included in the neural core SoC (10) may receive data and / or commands from a host system through a host interface (HIO) and apply the received data to a machine learning model to perform artificial intelligence operations. In one embodiment, the neural core SoC (10) may transmit result data for the artificial intelligence operations to the host system through the host interface (HIO). For example, when the neural core SoC (10) completes an artificial intelligence operation associated with at least one task, it may transmit a task completion report to the host system through the host interface (HIO). In one embodiment, if an error occurs during the performance of the artificial intelligence operations, the neural core SoC (10) may transmit a message associated with the occurrence of the error to the host system through the host interface (HIO). In one embodiment, the neural core SoC (10) may monitor the execution time of the artificial intelligence operations and, if the execution time of the artificial intelligence operations exceeds a threshold time, transmit a timeout report to the host system through the host interface (HIO).

[0064] In one embodiment, a command queue storing buffer descriptors may be stored in off-chip memory (30). Additionally, or alternatively, a command queue storing buffer descriptors may be stored in at least one memory located inside the neural core SoC (10).

[0065] FIG. 3 is a block diagram for explaining the host system (HS) of FIG. 1 in detail. The host system (HS) may include a memory (310), a processor (320), a communication module (330), and an input / output interface (340). The host system (HS) may be configured to communicate information and / or data over a network using the communication module (330).

[0066] The memory (310) may include any non-transient computer-readable recording medium. According to one embodiment, the memory (310) may include a permanent mass storage device such as a read-only memory (ROM), a disk drive, a solid-state drive (SSD), or a flash memory. As another example, a permanent mass storage device such as a ROM, SSD, a flash memory, or a disk drive may be included in the host system (HS) as a separate permanent storage device distinct from the memory (310). Additionally, the memory (310) may store an operating system and at least one program code (e.g., code for artificial intelligence computation requests, determining recovery target tasks, queue initialization, or recovery, etc., installed and running on the host system (HS)). In FIG. 3, the memory (310) is depicted as a single memory, but this is for convenience of explanation only, and the memory (310) may include multiple memories. In one embodiment, a work pending queue in which completed work is stored may be included in at least one of memory (310) or permanent storage device.

[0067] Software components may be loaded from a computer-readable recording medium separate from memory (310). This separate computer-readable recording medium may include a recording medium that can be directly connected to this host system (HS), for example, a computer-readable recording medium such as a floppy drive, disk, tape, DVD / CD-ROM drive, or memory card. As another example, software components may be loaded into memory (310) via a communication module (330) rather than a computer-readable recording medium. For example, at least one program may be loaded into memory (310) based on a computer program (e.g., a program for artificial intelligence computation requests, determining recovery target tasks, queue initialization, or recovery, etc.) installed by files provided through the communication module (330) by developers or a file distribution system distributing installation files for applications.

[0068] The processor (320) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to a user terminal (not shown) or another external system by memory (310) or a communication module (330). For example, the processor (320) may receive work data associated with at least one context from the user terminal or another external system through the communication module (330).

[0069] The communication module (330) may provide a configuration or function for a user terminal (not shown) and a host system (HS) to communicate with each other via a network, and may provide a configuration or function for the host system (HS) to communicate with an external system (e.g., a separate cloud system). As an example, control signals, commands, data, etc. provided under the control of the processor (320) of the host system (HS) may be transmitted to the user terminal and / or the external system through the communication module (330) and the network, and through the communication module of the user terminal and / or the external system.

[0070] Additionally, the input / output interface (340) of the host system (HS) may be a means for interface with a device (not shown) for input or output that is connected to the host system (HS) or that the host system (HS) may include. For example, the input / output interface (340) may include at least one of a PCI express interface and an Ethernet interface. Although the input / output interface (340) is depicted in FIG. 3 as an element configured separately from the processor (320), it is not limited thereto, and the input / output interface (340) may be configured to be included in the processor (320). Additionally, the host system (HS) may include more components than those shown in FIG. 3.

[0071] In one embodiment, the input / output interface (340) may include a host interface (HIO) formed between a host system (HS) and a processing device. Data, commands, signals, messages, etc., may be transmitted and received through the host interface (HIO).

[0072] The processor (320) of the host system (HS) may be configured to manage, process, and / or store information and / or data received from a plurality of user terminals and / or a plurality of external systems. Additionally, the processor (320) may be configured to manage at least one queue / buffer. Additionally, the processor (320) may be further configured to manage a pending task queue, as described below.

[0073] FIG. 4 is a block diagram illustrating a processing system (PS) according to some embodiments of the present disclosure.

[0074] Referring to FIG. 4, there may be multiple processing devices (1). Each of the multiple processing devices (1) may be connected to a host system (HS) through a host interface (HIO). Although the host interface (HIO) is shown as one in FIG. 4, the host interface (HIO) may include multiple interfaces connecting each processing device (1) to the host system (HS).

[0075] A plurality of processing devices (1) can exchange data and / or signals with one another. A plurality of processing devices (1) can transmit data and / or signals to one another through separate interfaces without passing through a host system (HS). However, the present embodiment is not limited thereto.

[0076] In one embodiment, a plurality of processing devices can transmit and receive data including the results of a task execution to and from other processing devices performing tasks in a dependency relationship through a separate interface.

[0077] FIG. 5 is a diagram illustrating the relationship between a host processor (510) and device processors (522, 524, 526) according to one embodiment of the present disclosure. Here, the host processor (510) corresponds to at least one processor included in a host system (e.g., 320 in FIG. 3), and the device processors (522, 524, 526) may correspond to at least one processor included in a processing device.

[0078] Referring to FIG. 5, the host processor (510) can transmit and receive data and / or control signals with each of the device processors (522, 524, 526). For example, the host processor (510) can distribute tasks associated with artificial intelligence computation to each of the device processors (522, 524, 526) and receive the results of the execution of the distributed tasks from each of the device processors (522, 524, 526). As another example, the host processor (510) can transmit an initialization command to each of the device processors (522, 524, 526) and transmit data to each of the device processors (522, 524, 526) so that at least one task stored in the task pending queue is restored to the command queue.

[0079] Each of the device processors (522, 524, 526) can transmit and receive data to and from each other. For example, each of the device processors (522, 524, 526) can transmit and receive data including the results of the work performed with another device processor (522, 524, 526) that performs a work in a dependency relationship while performing a work related to artificial intelligence computation.

[0080] In one embodiment, a first task, a second task, and a third task associated with a first context may each be transmitted to a first device processor (522), a second device processor (524), and a third device processor (526), ​​respectively. Here, the first task may include a first command, a second command, and a third command, the second task may include a fourth command, a fifth command, and a sixth command, and the third task may include a seventh command, an eighth command, and a ninth command. Additionally, the first task, the second task, and the third task associated with the first context may each have a dependency relationship with one another. For example, the fourth command, the fifth command, and the sixth command included in the second task may be executed only after the first command, the second command, and the third command included in the first task are all completed. As another example, the 7th command included in the 3rd task can be executed only after the 1st command included in the 1st task is completed, and the 4th command included in the 2nd task can be executed only after the 7th command is completed. Examples of dependency relationships between tasks can be applied in various ways, and the present disclosure is not limited to the examples described above.

[0081] According to various embodiments of the present disclosure, instead of adopting a method in which a host processor configures a separate queue before task distribution to consider all dependency relationships and distributes tasks, multiple tasks with dependency relationships are distributed to multiple processing devices, and then each device processor with dependency relationships can transmit and receive data to and from each other and execute commands. Through this configuration, the overhead of the host can be effectively reduced. Furthermore, through such a configuration, dependency issues are processed in real time through dynamic interaction between devices, bottlenecks that may occur during the task distribution process are prevented, and the processing speed of the system can be improved.

[0082] According to various embodiments of the present disclosure, a host system configures a job pending queue that stores completed jobs, thereby enabling jobs with already completed dependency relationships to be restored normally even during the queue initialization and recovery process. This prevents jobs that have already been completed but whose dependency relationships have not been resolved from being deleted upon reset, thereby ensuring system stability and the continuity of the workflow.

[0083] Hereinafter, with reference to FIGS. 6 to 14, a method of managing tasks by a host processor (510) and device processors (522, 524, 526) will be described. The management of tasks exemplified in FIGS. 6 to 14 described below may be performed by the host processor (510) and / or device processors (522, 524, 526). For example, the management of tasks exemplified in FIGS. 6 to 14 may be associated with the operation of a driver supported by the host processor (510), and may also be associated with the operation of a driver supported by each of the device processors (522, 524, 526).

[0084] FIG. 6 is a diagram illustrating an example of a command queue and a command buffer included in a processing device (e.g., 1 of FIG. 1) according to one embodiment of the present disclosure. Referring to FIG. 6, the processing device may include a command queue and a plurality of command buffers (COMMAND BUFFER_1, COMMAND BUFFER_2, COMMAND BUFFER_3, COMMAND BUFFER_4). Each of the command queue and the plurality of command buffers may be managed by at least one device processor (e.g., 522, 524, 526 of FIG. 5) included in the processing device. Hereinafter, the operation of the processing device may be understood as the operation of the device processor, and vice versa. Each of the plurality of command buffers in the present embodiment may have a storage area that is logically separated or physically separated.

[0085] The processing device may receive work data from the host system. Here, the work data is data associated with an artificial intelligence operation and may be generated based on data and / or commands received from a user terminal. Additionally, the work data may include at least one command. For example, the host system may receive an artificial intelligence operation request from the user terminal, generate work data based on the data and / or commands included in the artificial intelligence operation request, and then transmit it to the processing device.

[0086] In response to receiving work data from a host system, the processing device may store at least one command included in the work data in at least one command buffer.

[0087] When at least one command is stored in the command buffer, a buffer descriptor may be created and stored in the command queue. The device processor may create a buffer descriptor and store the created buffer descriptor in the command queue. In one embodiment, the buffer descriptor may include the address and size of at least one command stored in the command buffer. In this embodiment, a buffer descriptor including the address of the command buffer is exemplified as being stored in the command queue; however, the present disclosure is not limited thereto, and a descriptor including the address of a storage area other than the command buffer may be stored in the command buffer. However, for convenience of explanation, the following description will assume that the buffer descriptor is stored in the command queue.

[0088] Referring to FIG. 6, a plurality of commands (CMD1, CMD2, CMD3) included in the first task data may be stored in the first command buffer (COMMAND BUFFER_1), and a first buffer descriptor (BD1) associated with the plurality of commands (CMD1, CMD2, CMD3) may be stored in the command queue. Additionally, a plurality of commands (CMD4, CMD5, CMD6) included in the second task data may be stored in the second command buffer (COMMAND BUFFER_2), and a second buffer descriptor (BD2) associated with the plurality of commands (CMD4, CMD5, CMD6) may be stored in the command queue. Additionally, a plurality of commands (CMD7, CMD8, CMD9) included in the third task data may be stored in the third command buffer (COMMAND BUFFER_3), and a third buffer descriptor (BD3) associated with the plurality of commands (CMD7, CMD8, CMD9) may be stored in the command queue. Additionally, a plurality of commands (CMD10, CMD11, CMD12) included in the fourth task data may be stored in the fourth command buffer (COMMAND BUFFER_4), and a fourth buffer descriptor (BD4) associated with the plurality of commands (CMD10, CMD11, CMD12) may be stored in the command queue. Here, the first task data to the fourth task data may be associated with different contexts, but are not limited thereto. At least two of the first task data to the fourth task data may be associated with the same context. As shown in the example of Fig. 6, buffer descriptors are stored in the command queue, and actual commands can be stored in the command buffer.

[0089] In such a system environment, when a command execution cycle arrives, the device processor may acquire the highest priority buffer descriptor stored in the command queue and execute at least one command associated with the acquired buffer descriptor. In this case, for example, the device processor may apply data included in at least one command to a machine learning model to perform an artificial intelligence operation and transmit the operation result to a host system.

[0090] According to one embodiment, a device processor may manage trace data associated with an artificial intelligence computation task. The trace data may store at least one buffer descriptor associated with a command that has not yet been executed and the order of the buffer descriptors. Additionally or alternatively, the trace data may store the address of at least one command buffer associated with the buffer descriptor. The command queue initialization and recovery described below may be performed based on the trace data.

[0091] According to one embodiment, when a buffer descriptor is stored in a command queue, the buffer descriptor stored in the command queue may also be stored in the trace data. According to some embodiments, when a buffer descriptor is stored in a command queue, at least one command buffer address associated with the buffer descriptor may be stored in the trace data. The addresses of the buffer descriptor and / or command buffer stored in the trace data may be stored in a first-in, first-out structure. That is, the addresses of the buffer descriptor and / or command buffer stored in the trace data may have sequence numbers. According to one embodiment, when a device processor completes an artificial intelligence operation associated with at least one task data, the addresses of the buffer descriptor and / or command buffer associated with the result of this operation may be deleted from the trace data. Additionally, or alternatively, the device processor may delete the addresses of the buffer descriptor and / or command buffer included in the trace data deletion command from the trace data in response to receiving a trace data deletion command from the host system. When the host system deletes a specific context and / or a specific job from the job pending queue, it may send a trace data deletion command to the processing unit so that the address of the buffer descriptor and / or command buffer associated with the specific context and / or specific job is deleted from the trace data.

[0092] FIG. 7 is a diagram illustrating examples of a job pending queue included in a host system (e.g., HS of FIG. 1) and a command queue included in each of a processing device (e.g., 1 of FIG. 1) according to one embodiment of the present disclosure. Referring to FIG. 7, the host system may include a job pending queue. Additionally, a first processing device (PROCESSING DEVICE_1) may include a first command queue (COMMAND QUEUE_1), a second processing device (PROCESSING DEVICE_2) may include a second command queue (COMMAND QUEUE_2), and a third processing device (PROCESSING DEVICE_3) may include a third command queue (COMMAND QUEUE_3). The job pending queue may be managed by at least one host processor (e.g., 510 of FIG. 5) included in the host system. In the following, the operation of the host system can be understood as the operation of the host processor, and vice versa. Additionally, each command queue can be managed by at least one device processor (e.g., 522, 524, 526 of FIG. 5) included in each processing unit. In the following, the operation of the processing unit can be understood as the operation of the device processor, and vice versa.

[0093] The host system can distribute multiple task data associated with at least one context to multiple processing devices. For example, the host system may receive an artificial intelligence operation request from a user terminal, generate task data based on the data and / or commands included in the artificial intelligence operation request, and then distribute it to each processing device. Each processing device may generate a buffer descriptor based on the received task data and store the generated buffer descriptor in a command queue. Additionally, the host system may store a buffer descriptor associated with a completed task in a task pending queue.

[0094] In one embodiment, the buffer descriptor may further include context data, task dependency data, and node data. The context data may include information about the context associated with the buffer descriptor. The task dependency data may include information indicating tasks that are in a dependency relationship with each other. For example, there may be a dependency relationship between buffer descriptors where the context data and task dependency data are the same. In the example of FIG. 7, the first buffer descriptor (BD1) and the third buffer descriptor (BD3) may be in a dependency relationship with each other. Additionally, the second buffer descriptor (BD2) and the fourth buffer descriptor (BD4) may be in a dependency relationship with each other. The node data may include information for identifying which processing device the task data has been distributed to. However, this is merely an example, and additional information may be stored in the buffer descriptor, the information described above may be stored in a different form of data, or at least one of the context data, task dependency data, or node data may not be stored.

[0095] Referring to the example in FIG. 7, the host system may distribute work data associated with a first context (CTX1) to a first processing unit and a second processing unit. In one embodiment, at least some of the commands included in the work data distributed to the first processing unit and the second processing unit, respectively, may be different. In one embodiment, at least some of the commands included in the work data distributed to the first processing unit and the second processing unit, respectively, may overlap with each other. Additionally, the host system may distribute work data associated with a second context (CTX2) to a second processing unit and a third processing unit. Additionally, the host system may transmit work data associated with a third context (CTX3) to a third processing unit. Each of the first to third processing units may generate a buffer descriptor based on the received data and store it in the first to third command queues, respectively. Each of the first to third processing units may perform a task (e.g., an artificial intelligence operation) by executing the commands associated with the buffer descriptor in the order stored in the command queues. For example, the second processing unit may execute a command associated with the second buffer descriptor (BD2), and when the execution is completed, execute a command associated with the third buffer descriptor (BD3). Additionally, the third processing unit may execute a command associated with the fourth buffer descriptor (BD4), and when the execution is completed, execute a command associated with the fifth buffer descriptor (BD5).

[0096] Meanwhile, the processing device may fail to perform an artificial intelligence operation based on data received from the host system or detect an error related to the artificial intelligence operation. In this case, the processing device may send an error message to the host system.

[0097] FIG. 8 is a diagram illustrating an example in which an error exists in an artificial intelligence operation associated with a first context, according to one embodiment of the present disclosure.

[0098] Referring to FIG. 8, the third processing device is shown to have completed the artificial intelligence operation associated with the fourth buffer descriptor (BD4). In this case, the third processing device may transmit the result of the operation, including a work completion report, to the host system. In response to receiving the work completion report, the host system may identify the work associated with the received work completion report and process the identified work as completed. In one embodiment, the third processing device may transmit the fourth buffer descriptor (BD4) along with the work completion report to the host system. Additionally, the host system may store the completed work in a work pending queue. Likewise, the host system may store the completed work in the work pending queue in the form of a buffer descriptor, but is not limited thereto.

[0099] Meanwhile, the processing unit may execute artificial intelligence operations based on data and / or commands received from the host system, and errors may occur during the execution of the artificial intelligence operations. Here, the artificial intelligence operations may be graphics-related operations. For example, the artificial intelligence operations may be rendering operations. Additionally, rendering may be a result primarily based on pixel operations. For example, rendering in a machine learning model (i.e., artificial intelligence operations related to rendering) may be a result of operations related to at least one of weight data or a multi-layer perceptron. When a graphics job is performed, significant overhead may not occur even if a single command buffer contains a program, constant data, input data, etc.

[0100] Referring to the example in FIG. 8, an error may exist in the artificial intelligence operation associated with the first context (CTX1). The first processing unit may detect the error while performing the artificial intelligence operation associated with the first buffer descriptor (BD1). If the first processing unit detects that an error has occurred in the artificial intelligence operation, it may send an error message to the host system. Meanwhile, an error may also exist in the third buffer descriptor (BD3) associated with the first context (CTX1), but since the second processing unit is normally executing the command associated with the second buffer descriptor (BD2), there is a possibility that it may not detect the error associated with the third buffer descriptor (BD3).

[0101] In response to receiving an error message from the first processing unit, the host system may determine that an error associated with at least one context has occurred. In this case, the host system may reset the pending task queue and the command queue. Here, resetting the pending task queue and the command queue may mean deleting data stored in the pending task queue and the command queue. In one embodiment, the host system may reset the command queue included in each processing unit by sending a reset command to each processing unit. As another example, the host system may reset the command queue by directly accessing memory shared with the processing unit and deleting data stored in the command queue.

[0102] FIG. 9 is a diagram illustrating the process of recovering a command queue after it has been initialized according to one embodiment of the present disclosure. Referring to FIG. 9, the first buffer descriptor (BD1) and the third buffer descriptor (BD3) associated with the context where the error occurred are not recovered, and the second buffer descriptor (BD2), the fourth buffer descriptor (BD4), and the fifth buffer descriptor (BD5) not associated with the context where the error occurred may be recovered.

[0103] In one embodiment, when the host system determines that an error associated with at least one context has occurred, it may identify the context associated with the error and determine a task to be recovered based on the identified context. Referring to FIGS. 8 and 9, the host system may receive an error message from a first processing unit. Based on the error message, the host system may determine that an error associated with a first context (CTX1) has occurred. The host system may determine a task not associated with the first context (CTX1) as a task to be recovered. Specifically, the host system may determine a task not associated with the first context (CTX1) among the tasks stored in the task pending queue as a first task to be recovered, and a task not associated with the first context among the tasks stored in the command queue as a second task to be recovered. Here, the first task to be recovered may refer to a priority task to be recovered, and the second task to be recovered may refer to a lower priority task to be recovered. That is, the host system may recover tasks stored in the task pending queue first and recover tasks stored in the command queue later.

[0104] In the example illustrated in FIG. 9, the host system may prioritize recovering the fourth buffer descriptor (BD4). Based on the node data contained in the fourth buffer descriptor (BD4), the host system may transmit the fourth buffer descriptor (BD4) to the third processing unit. Additionally, or alternatively, the host system may transmit work data extracted based on the fourth buffer descriptor (BD4) to the third processing unit. In response to receiving the fourth buffer descriptor (BD4) from the host system, the third processing unit may prioritize recovering the fourth buffer descriptor (BD4) to the third command queue. Subsequently, the third processing unit may prioritize recovering the fifth buffer descriptor (BD5) that was stored in the third command queue.

[0105] Since the second processing unit does not receive a priority recovery target task from the host system, it can recover the second buffer descriptor (BD2) that was stored in the second command queue.

[0106] FIG. 10 is a diagram illustrating examples of a task pending queue included in a host system (e.g., HS of FIG. 1) and a command buffer included in each of a processing device (e.g., 1 of FIG. 1) according to one embodiment of the present disclosure. Hereinafter, descriptions that overlap with the description above with reference to FIG. 7 are omitted.

[0107] A host system may distribute multiple work data associated with at least one context to multiple processing devices. Each processing device may generate a buffer descriptor based on the received work data and store the generated buffer descriptor in a command queue. In the example of FIG. 10, the first processing device may generate a first buffer descriptor (BD1) and a second buffer descriptor (BD2) and store them in a first command queue, and the second processing device may generate a third buffer descriptor (BD3) and a fourth buffer descriptor (BD4) and store them in a second command queue. Additionally, the first processing device may generate a fifth buffer descriptor (BD5) and a sixth buffer descriptor (BD6) and store them in a third command queue, and the fourth processing device may generate a seventh buffer descriptor (BD7) and an eighth buffer descriptor (BD8) and store them in a fourth command queue.

[0108] Additionally, there may be dependency relationships between buffer descriptors that have the same context data and task dependency data. In the example of FIG. 10, the first buffer descriptor (BD1) and the third buffer descriptor (BD3) may have a dependency relationship with each other. Also, the second buffer descriptor (BD2), the fourth buffer descriptor (BD4), the fifth buffer descriptor (BD5), and the seventh buffer descriptor (BD7) may have a dependency relationship with each other. Additionally, the sixth buffer descriptor (BD6) and the eighth buffer descriptor (BD8) may have a dependency relationship with each other.

[0109] Meanwhile, the processing unit can detect a timeout while performing artificial intelligence operations. Here, a timeout may be detected if the execution time of a command associated with the artificial intelligence operation exceeds a predetermined threshold time. When the processing unit detects a timeout, it may transmit a timeout report to the host system. Upon receiving the timeout report from the processing unit, the host system may provide a response to the user terminal associated with the received timeout report. The host system may provide a rapid response to the timeout detection to the user terminal before identifying whether an error has occurred. Through this configuration, the user can quickly recognize the situation and take necessary measures, the overall stability of the system can be improved, and the user experience can be enhanced.

[0110] FIG. 11 is a drawing illustrating an example in which a timeout is detected according to one embodiment of the present disclosure. Hereinafter, descriptions that overlap with the description above with reference to FIG. 8 are omitted.

[0111] Referring to FIG. 11, it is illustrated that a timeout is detected in the first to fourth processing devices. There may be various causes for the detection of a timeout. For example, a timeout may occur when data synchronization is delayed due to dependencies between tasks. For instance, in a multi-device environment, a timeout may be detected if there is no response while another device is waiting for a specific task to be completed because the task has not been completed. As another example, a timeout may occur when an unintended dependency relationship is established. For instance, in a multi-device environment, if the result of a specific task is transmitted to a device that the user did not intend, a hang may occur in which the originally intended device waits indefinitely for a response.

[0112] Information that a timeout has been detected by a processing unit does not necessarily mean that there is a problem with the context associated with the task where the timeout was detected. Therefore, even if a timeout is detected, it may be necessary to determine whether there is a problem with the corresponding context.

[0113] For example, in the example of FIG. 11, the dependency relationship between the first buffer descriptor (BD1) and the third buffer descriptor (BD3) associated with the first context (CTX1) may be set in a direction unintended by the user. The first processing unit and the second processing unit may detect a timeout by determining that the task associated with the first buffer descriptor (BD1) and the task associated with the third buffer descriptor (BD3) are not executed within a predetermined threshold time.

[0114] Meanwhile, the fifth buffer descriptor (BD5) and the seventh buffer descriptor (BD7) associated with the second context (CTX2) may have a dependency relationship with at least one of the second buffer descriptor (BD2) or the fourth buffer descriptor (BD4) associated with the second context (CTX2). Accordingly, the third processing unit and the fourth processing unit may wait for a response from at least one of the first processing unit or the second processing unit while executing commands contained in the fifth buffer descriptor (BD5) and the seventh buffer descriptor (BD7). In this way, even if there is no problem with the second context, a timeout may be detected while executing commands associated with the second context.

[0115] In one embodiment, the host system may receive a timeout report from at least one of a plurality of processing devices. In one embodiment, the host system may receive a buffer descriptor associated with the timeout report. In response to receiving the timeout report, the host system may identify the task associated with the received timeout report and process the identified task as completed. Additionally, the host system may store the buffer descriptor associated with the completed task in a task pending queue. Here, the timeout report may include information that a timeout has been monitored for a task assigned to the processing device, but is not limited thereto. In one embodiment, in response to receiving the timeout report, the host system may identify the context associated with the timeout report and transmit the timeout report to the entity that transmitted the identified context to the host system. For example, the host processor may transmit the timeout report to a user terminal or an external system associated with the identified context, but is not limited thereto.

[0116] In the example of FIG. 11, the host system may receive a timeout report from the first to fourth processing units. Additionally, the host system may receive a first buffer descriptor (BD1), a third buffer descriptor (BD3), a fifth buffer descriptor (BD5), and a seventh buffer descriptor (BD7) from the first to fourth processing units. Furthermore, based on the received timeout report, the host system may store the first buffer descriptor (BD1), the third buffer descriptor (BD3), the fifth buffer descriptor (BD5), and the seventh buffer descriptor (BD7) in a pending task queue. Additionally, in response to receiving the timeout report, the host system may transmit the timeout report to a user terminal associated with the first context (CTX1) and a user terminal associated with the second context (CTX2).

[0117] In one embodiment, the host processor may determine that an error associated with at least one context has occurred based on a timeout report received from at least one processing device. For example, the host processor may count the number of received timeout reports and determine that an error associated with a specific context has occurred if it determines that a timeout report has been received for all tasks included in a specific context.

[0118] In the example of FIG. 11, the host system may determine, based on the received timeout reports, that the number of times a timeout report associated with the first context (CTX1) has been received is 2, and the number of times a timeout report associated with the second context (CTX2) has been received is 2. Here, the host system may determine that no timeout report has been received for at least one task included in the second context (CTX2) and that no error associated with the second context (CTX2) has occurred. On the other hand, the host system may determine that a timeout report has been received for all tasks included in the first context (CTX1) and that an error associated with the first context (CTX1) has occurred. In this case, the host system may initialize the task pending queue and the command queue.

[0119] FIG. 12 is a diagram illustrating the process of recovering a command queue after it has been initialized according to one embodiment of the present disclosure. Hereinafter, descriptions that overlap with the description above with reference to FIG. 9 are omitted. Referring to FIG. 12, the first buffer descriptor (BD1) and the third buffer descriptor (BD3) associated with the context where the error occurred are not recovered, and the second buffer descriptor (BD2) and the fourth buffer descriptor (BD4) to the eighth buffer descriptor (BD8) not associated with the context where the error occurred can be recovered.

[0120] In one embodiment, the host system may determine a task not associated with the context in which the error occurred as a task to be recovered. In the example of FIG. 12, the host system may not determine the first buffer descriptor (BD1) and the third buffer descriptor (BD3) associated with the first context (CTX1) as tasks to be recovered. Additionally, the host system may determine the second buffer descriptor (BD2) and the fourth buffer descriptors (BD4) through eighth buffer descriptors (BD8) associated with the second context (CTX2) or the third context (CTX3) as tasks to be recovered.

[0121] In one embodiment, the host system may determine a task stored in a task pending queue among the tasks to be recovered as a first task to be recovered, and a task stored in a command queue among the tasks to be recovered as a second task to be recovered. In the example of FIG. 12, the host system may determine the fifth buffer descriptor (BD5) and the seventh buffer descriptor (BD7) as the first task to be recovered, and the second buffer descriptor (BD2), the fourth buffer descriptor (BD4), the sixth buffer descriptor (BD6), and the eighth buffer descriptor (BD8) as the second task to be recovered.

[0122] In one embodiment, the host system may recover a first recovery target task, which is a priority recovery target task, first, and recover a second recovery target task, which is a subordinate recovery target task, sequentially so that it has a sequence number after the first recovery target task. In the example of FIG. 12, the host system may transmit the fifth buffer descriptor (BD5) and the seventh buffer descriptor (BD7) to the third processing device and the fourth processing device based on the node data contained in the fifth buffer descriptor (BD5) and the seventh buffer descriptor (BD7). Additionally, or alternatively, the host system may transmit the extracted task data based on the fifth buffer descriptor (BD5) and the seventh buffer descriptor (BD7) to the third processing device and the fourth processing device. In response to receiving the fifth buffer descriptor (BD5) and the seventh buffer descriptor (BD7) from the host system, the third processing device and the fourth processing device may restore the fifth buffer descriptor (BD5) and the seventh buffer descriptor (BD7) to the third command queue and the fourth command queue, respectively, with priority. Subsequently, the third processing device and the fourth processing device may restore the sixth buffer descriptor (BD6) and the eighth buffer descriptor (BD8) that were stored in the third command queue and the fourth command queue with lower priority.

[0123] Since the first processing unit and the second processing unit do not receive priority recovery target tasks from the host system, the second buffer descriptor (BD2) and the fourth buffer descriptor (BD4) stored in the first command queue and the second command queue can be recovered.

[0124] FIG. 13 is a flowchart for explaining a task management method (1300) according to one embodiment of the present disclosure. The method (1300) illustrated in FIG. 13 may be performed by at least one host processor included in a host system (e.g., 510 of FIG. 5). For convenience of explanation, it will be described that each step illustrated in FIG. 13 is performed by the host processor illustrated in FIG. 5. The method (1300) according to FIG. 13 may be disclosed when the host processor receives at least one context including a plurality of tasks.

[0125] The processor may distribute a plurality of tasks extracted from at least one context to a plurality of processing devices (S1310). Here, a context refers to a unit of operation / command requested by a user terminal or another external system to the host system, and a single context may include at least one task. Additionally, a task refers to a unit of operation / command allocated to be performed by each processing device, and a single task may include at least one command. However, the present disclosure is not limited thereto, and the unit of operation / command allocated to the host system and / or processing device may be defined differently.

[0126] The processor may store the completed task in a task pending queue included in the host system (S1320). In one embodiment, the processor may receive a task completion report from at least one of a plurality of processing devices. In response to receiving the task completion report, the processor may identify the task associated with the received task completion report and process the identified task as completed. Here, the task completion report may include information that the task assigned to the processing device has been successfully completed, but is not limited thereto. In one embodiment, the processor may receive a timeout report from at least one of a plurality of processing devices. In response to receiving the timeout report, the processor may identify the task associated with the received timeout report and process the identified task as completed. Here, the timeout report may include information that a timeout has been monitored for the task assigned to the processing device, but is not limited thereto. In one embodiment, in response to receiving the timeout report, the processor may identify the context associated with the timeout report and transmit the timeout report to the entity that transmitted the identified context to the host system. For example, the processor may send a timeout report to a user terminal or external system associated with the identified context, but is not limited thereto.

[0127] The processor can determine whether an error associated with at least one context has occurred (S1330). In one embodiment, the processor may receive an error message from at least one of a plurality of processing devices. In response to receiving the error message, the processor may determine that an error associated with at least one context has occurred. In one embodiment, the processor may determine that an error associated with at least one context has occurred based on a timeout report received from at least one processing device. For example, the processor may count the number of received timeout reports and determine that a timeout report has been received for all tasks included in a specific context, thereby determining that an error associated with a specific context has occurred.

[0128] If the processor determines that no error has occurred, steps S340 through S370 related to command queue initialization and recovery may not be performed.

[0129] When the processor determines that an error has occurred, it can identify the context associated with the error (S1340). In one embodiment, the processor can identify the task in which the error occurred based on the received error message and determine the context associated with the identified task as the context associated with the error. In one embodiment, the processor can count the number of received timeout reports and determine the context in which a timeout report was received for all tasks as the context associated with the error. For convenience, the context associated with the error will be referred to as the first context.

[0130] The processor can initialize each of the multiple command queues included in the multiple processing devices (S1350). For example, the processor can initialize each of the multiple command queues included in the multiple processing devices by transmitting an initialization command to each processing device. As another example, the processor can initialize the command queue by directly accessing memory shared with the processing device and deleting data stored in the command queue. In addition, the processor may generate an interrupt signal to cause the processing device to perform the initialization of the command queue itself. In the present disclosure, the method by which the processor initializes the command queue is not limited thereto, and the initialization operation may be performed in various ways depending on the interface and / or communication protocol between the host system and the processing device. Additionally, the processor may initialize the pending task queue together with the command queue.

[0131] The processor may determine a task to be recovered based on the identified first context (S1360). In one embodiment, the processor may determine a task stored in a task pending queue that is not associated with the first context as the first task to be recovered. Additionally, the processor may determine a task stored in a command queue included in each processing unit that is not associated with the first context as the second task to be recovered. Here, the first task to be recovered may refer to a priority task to be recovered, and the second task to be recovered may refer to a junior task to be recovered.

[0132] The processor may recover the recovery target tasks to the command queue (S1370). In one embodiment, the first recovery target task, which is a priority recovery target task, may be recovered first, and the second recovery target task, which is a subordinate recovery target task, may be recovered sequentially to have a sequence number after the first recovery target task. In one embodiment, the processor may recover the recovery target tasks stored in the task pending queue first, and recover the recovery target tasks stored in each command queue in a subordinate order. In one embodiment, each task includes node data, and the node data may be associated with at least one processing device. Based on the node data included in each of the first recovery target tasks, the processor may transmit each of the first recovery target tasks to the processing device associated with each of the first recovery target tasks. Additionally, or alternatively, the processor may transmit the buffer descriptor associated with the first recovery target task to the processing device associated with each of the first recovery target tasks.

[0133] The processor may delete a task associated with a completed context from the task pending queue (S1380). In one embodiment, the processor may determine that the context is complete when it determines that all tasks included in the context are completed. For example, the second context may include a first task and a second task. In this case, the processor may determine whether the first task is completed and whether the second task is completed. In response to determining that the first task and the second task are completed, the processor may determine that the second context is complete. In one embodiment, the processor may count the number of completions of tasks associated with the second context and determine that the second context is complete in response to determining that the counted number of completions is equal to the number of tasks included in the second context. In one embodiment, when the processor deletes a specific context and / or a specific task from the task pending queue, it may send a trace data deletion command to the processing unit so that the address of the buffer descriptor and / or command buffer associated with the specific context and / or the specific task is deleted from the trace data.

[0134] The flowchart and description described above using FIG. 13 are merely examples and may be implemented differently in some embodiments. For example, in some embodiments, the order of each step may be changed, some steps may be repeated, some steps may be omitted, or some steps may be added.

[0135] FIG. 14 is a flowchart illustrating a task recovery method (1400) according to one embodiment of the present disclosure. The method (1400) illustrated in FIG. 14 may be performed by at least one device processor included in a processing device (e.g., at least one of 522, 524, and 526 of FIG. 5). For convenience of explanation, it will be described that each step illustrated in FIG. 14 is performed by the device processor illustrated in FIG. 5. The method (1400) according to FIG. 14 may be disclosed when the processor stores a descriptor in a command queue. In one embodiment, the descriptor may include a buffer descriptor.

[0136] The processor can extract a descriptor associated with a target job, which is to be executed among a plurality of jobs, from the command queue (S1410). Here, the target job is a job executed in the current cycle, and the descriptor associated with this job can be stored in the command queue in advance.

[0137] After that, the processor can execute at least one command associated with the extracted descriptor to perform an artificial intelligence operation associated with the target task (S1420). According to one embodiment, the at least one executed command may be associated with inference performed by the processing device.

[0138] In one embodiment, the processor can determine whether an error occurs while performing an artificial intelligence operation. If the processor determines that an error has occurred, it can send an error message to the host system.

[0139] In one embodiment, the processor may detect a timeout while performing an artificial intelligence operation. Here, the timeout may be detected when the execution time of a command associated with the artificial intelligence operation exceeds a predetermined threshold time. When the processor detects a timeout, it may send a timeout report to the host system.

[0140] Next, the processor can determine whether an initialization command has been received while performing artificial intelligence operations (S1430). If the processor determines that an initialization command has not been received, it can perform step S1470.

[0141] When the processor determines that an initialization command has been received, it may initialize the command queue (S1440). Here, initializing the command queue may mean deleting data stored in the command queue.

[0142] According to one embodiment, the processor identifies the context associated with the task that failed based on an initialization command and can control the command queue so that additional descriptors associated with the identified context are not stored in the command queue. Additionally, the processor identifies the context associated with the task that failed based on an initialization command, and if it determines that an additional task not associated with the identified context has occurred, it can store the descriptors associated with the additional task in the command queue before initializing the command queue. Here, the context may be associated with at least one of a specific user or a specific port.

[0143] Next, the processor may determine a recovery target descriptor (S1450). In one embodiment, the processor may receive a first recovery target task associated with a recovery target descriptor from the host system. Additionally, or otherwise, the processor may receive a descriptor associated with the first recovery target task from the host system. Here, the first recovery target task may refer to a task that requires recovery first. In one embodiment, the processor may receive a second recovery target task associated with a recovery target descriptor from the host system. Here, the second recovery target task may refer to a task that needs to be recovered later. Additionally, or otherwise, the processor may identify a descriptor associated with a task that has failed based on an initialization command, and determine the descriptor stored in the command queue, excluding the identified descriptor, as the second recovery target task. In one embodiment, the processor may determine the descriptor associated with the first recovery target task and the second recovery target task as the recovery target descriptor.

[0144] Subsequently, the processor can recover at least one determined recovery target descriptor to the command queue (S1460). In one embodiment, the processor can recover the descriptor associated with the first recovery target task first, and recover the descriptor associated with the second recovery target task second.

[0145] In one embodiment, the processor may obtain at least one descriptor stored in a command queue prior to initialization from trace data, and may recover a descriptor associated with a second recovery target task to the command queue based on the obtained at least one descriptor. In one embodiment, the processor may identify at least one command buffer associated with a command queue prior to initialization based on trace data, and may recover a descriptor associated with a second recovery target task to the command queue based on a command stored in the identified at least one command buffer.

[0146] If the initialized command queue is restored or if the initialization command is not received, the processor can determine whether there is a descriptor associated with a task that has not yet been executed (S1470). That is, the processor can determine whether a descriptor associated with a task that has not been executed is stored in the command queue.

[0147] In one embodiment, when the processor completes an artificial intelligence operation associated with at least one descriptor, it may delete the descriptor associated with the result of the operation from the trace data. Additionally, or alternatively, in response to receiving a trace data deletion command from the host system, the processor may delete the address of the buffer descriptor and / or command buffer included in the trace data deletion command from the trace data.

[0148] If the processor determines that there is a descriptor associated with an unexecuted task, it may determine the task associated with the next descriptor in sequence as the target task (S1480). Subsequently, the processor may repeat step S910 to extract the descriptor associated with the determined target task (i.e., the descriptor with the highest priority) from the command queue.

[0149] Meanwhile, if the processor determines that there is no descriptor associated with an unexecuted task, it may switch to a waiting mode and terminate the method according to FIG. 14. If a new descriptor is stored in the command queue, the processor may restart the method according to FIG. 14.

[0150] The flowchart and description described above using FIG. 14 are merely examples and may be implemented differently in some embodiments. For example, in some embodiments, the order of each step may be changed, some steps may be repeated, some steps may be omitted, or some steps may be added.

[0151] The method described above may be provided as a computer program stored on a computer-readable recording medium for execution on a computer. The medium may continuously store a computer-executable program, or temporarily store it for execution or download. Additionally, the medium may be various recording or storage means in the form of a single or multiple combined hardware, and may not be limited to a medium directly connected to a computer system but may exist distributed over a network. Examples of media may include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and media configured to store program instructions, including ROM and flash memory. Furthermore, other examples of media may include recording or storage media managed by app stores that distribute applications or sites and servers that supply or distribute various other software.

[0152] The methods, operations, or techniques of the present disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will understand that the various exemplary logical blocks, modules, circuits, and algorithmic steps described in connection with the disclosure herein may be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate such interchangeability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been generally described above in terms of their functional aspects. Whether such functions are implemented in hardware or in software depends on the design requirements imposed on the specific application and the overall system. Those skilled in the art may implement the functions described in various ways for each specific application, but such implementations should not be construed as departing from the scope of the present disclosure.

[0153] In a hardware implementation, the processing units used to perform the techniques may be implemented in one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described in this disclosure, computers, or a combination thereof.

[0154] Accordingly, the various exemplary logic blocks, modules, and circuits described in connection with the present disclosure may be implemented or performed by any combination of general-purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or those designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, for example, a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors coupled with a DSP core, or any other combination of configurations.

[0155] In firmware and / or software implementations, techniques may be implemented as instructions stored on a computer-readable medium such as read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or optical data storage devices, etc. The instructions may be executable by one or more processors, and may cause the processor(s) to perform specific aspects of the functions described in this disclosure.

[0156] When implemented in software, the techniques described above may be stored on a computer-readable medium as one or more instructions or code, or transmitted through a computer-readable medium. Computer-readable media include both computer storage media and communication media, including any medium that facilitates the transfer of a computer program from one place to another. Storage media may be any available medium accessible by a computer. As a non-limiting example, such computer-readable media may include ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium accessible by a computer that can be used to transfer or store desired program code in the form of instructions or data structures. Additionally, any connection is appropriately referred to as a computer-readable medium.

[0157] For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line, or wireless technologies such as infrared, radio, and microwave are included within the definition of a medium. The terms "disk" and "disc" as used herein include CDs, laser discs, optical discs, DVDs (digital versatile discs), floppy disks, and Blu-ray discs, wherein "disks" typically reproduce data magnetically, while "discs" reproduce data optically using a laser. The above combinations should also be included within the scope of computer-readable media.

[0158] The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other known form of storage medium. An exemplary storage medium may be connected to a processor so that the processor can read information from the storage medium or write information to the storage medium. Alternatively, the storage medium may be integrated into the processor. The processor and the storage medium may exist within an ASIC. The ASIC may exist within a user terminal. Alternatively, the processor and the storage medium may exist as separate components within the user terminal.

[0159] Although the embodiments described above have been described as utilizing aspects of the subject matter disclosed herein in one or more standalone computer systems, the present disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or a distributed computing environment. Furthermore, aspects of the subject matter in the present disclosure may be implemented in a plurality of processing chips or devices, and storage may be similarly affected across a plurality of devices. Such devices may include PCs, network servers, and portable devices.

[0160] Although the present disclosure has been described in relation to some embodiments, various modifications and changes may be made without departing from the scope of the present disclosure as understood by a person skilled in the art to which the invention of the present disclosure pertains. Furthermore, such modifications and changes should be considered to fall within the scope of the claims appended to this specification. Explanation of the symbols

[0161] PS: Processing system HS: Host System 1: Processing unit HIO: Host Interface

Claims

Claim 1 A method for managing a job, performed by at least one processor, comprising: distributing a plurality of jobs associated with at least one context to a plurality of processing devices, wherein the plurality of processing devices include a plurality of command queues for storing the distributed jobs; storing a job that has been processed as completed among the distributed jobs in a job pending queue; identifying a first context associated with an error in response to determining that an error associated with the at least one context has occurred; initializing each of the plurality of command queues included in the plurality of processing devices; determining a job to be recovered based on the identified first context; and recovering the determined job to be recovered to each of the initialized command queues. Claim 2 A method for managing tasks according to claim 1, wherein the determining step comprises: determining a task not associated with the first context among the tasks stored in the task pending queue as a first task to be recovered; and determining a task not associated with the first context among the tasks stored in each of the plurality of command queues as a second task to be recovered. Claim 3 A method for managing tasks according to paragraph 2, wherein the recovery step comprises: a step of recovering the first task to be recovered first; and a step of recovering the second task to be recovered sequentially so as to have a sequence number after the first task to be recovered. Claim 4 A method for managing tasks according to claim 3, wherein the task stored in the task pending queue includes node data associated with at least one processing device among the plurality of processing devices, and the step of prioritizing recovery includes the step of transmitting the first task to be recovered to a processing device associated with the first task to be recovered based on the node data included in the first task to be recovered. Claim 5 A method for managing tasks according to claim 1, further comprising, after the saving step, a step of identifying a second context that is completed among at least one context; and a step of deleting a task associated with the identified second context from the task pending queue. Claim 6 A method for managing a task according to claim 5, wherein the second context includes a first task and a second task, and the step of identifying the second context includes: a step of determining whether the first task is completed; a step of determining whether the second task is completed; and a step of determining that the second context is completed in response to determining that the first task and the second task are completed. Claim 7 A method for managing tasks according to claim 5, wherein the step of identifying the second context comprises: identifying the number of tasks associated with the second context; counting the number of completions of tasks associated with the second context; and determining that the second context is completed in response to determining that the counted number of completions is equal to the number of tasks. Claim 8 A method for managing a task according to claim 1, further comprising: a step of receiving an error message from at least one of the plurality of processing devices prior to the step of identifying the first context; and a step of determining that the error occurred in the at least one context in response to receiving the error message. Claim 9 A method for managing a task according to claim 8, wherein the step of identifying the first context comprises: identifying the task in which the error occurred based on the error message; and determining that an error occurred in the first context in response to determining that the first context is associated with the task in which the error occurred. Claim 10 A method for managing a task according to claim 1, further comprising: a step of receiving a time-out report from at least one of the plurality of processing devices prior to the step of identifying the first context; and a step of determining that the error occurred in the at least one context based on the time-out report. Claim 11 A method for managing tasks according to claim 10, wherein the step of identifying the first context comprises: identifying the number of tasks included in the first context; counting the number of times a timeout report associated with the first context is received; and determining that an error has occurred in the first context in response to determining that the number of times received is equal to the number of tasks. Claim 12 A method for managing a task according to claim 1, wherein the step of storing in the task pending queue comprises: receiving a task completion report from at least one processing device among the plurality of processing devices; identifying a task associated with the task completion report in response to receiving the task completion report; processing the task associated with the task completion report as completed; and storing the completed task in the task pending queue. Claim 13 A method for managing a task according to claim 1, wherein the step of storing in the task pending queue comprises: receiving a time-out report from at least one of the plurality of processing devices; identifying a task associated with the time-out report in response to receiving the time-out report; completing the task associated with the time-out report; and storing the completed task in the task pending queue. Claim 14 A method for managing an operation according to claim 13, wherein at least one context is associated with a specific user, and the method further comprises the steps of: identifying a context associated with the time-out report in response to receiving the time-out report; and transmitting the time-out report to a user associated with the identified context. Claim 15 A method for managing tasks according to claim 1, wherein the first context includes a third task and a fourth task, the third task includes a first command and a second command, the fourth task includes a third command and a fourth command, and at least one of the first command or the second command is associated with at least one of the third command or the fourth command. Claim 16 A computer program stored on a computer-readable recording medium for executing a method of managing work according to any one of paragraphs 1 through 15 on a computer. Claim 17 A computing system comprising: a task pending queue for storing completed tasks; and at least one host processor configured to manage the task pending queue, wherein the at least one host processor distributes a plurality of tasks associated with at least one context to a plurality of processing devices, wherein the plurality of processing devices include a plurality of command queues for storing the distributed tasks; stores a task that has been completed among the distributed tasks in the task pending queue; and, in response to determining that an error has occurred in the at least one context, identifies a first context associated with the error, initializes each of the plurality of command queues included in the plurality of processing devices, determines a task to be recovered based on the identified first context, and further configured to recover the determined task to be recovered to each of the initialized command queues. Claim 18 A computing system according to claim 17, wherein the above determination comprises determining a task not associated with the first context among the tasks stored in the task pending queue as a first task to be recovered, and determining a task not associated with the first context among the tasks stored in each of the plurality of command queues as a second task to be recovered. Claim 19 A computing system according to claim 18, wherein the recovery comprises recovering the first recovery target task first and recovering the second recovery target task sequentially to have a sequence number after the first recovery target task. Claim 20 A computing system according to claim 17, wherein the at least one host processor is further configured to identify a second context that has been completed among the at least one context and to remove a task associated with the identified second context from the task pending queue.

Images