Task processing method and chip
By using a combination of clear queue instructions and delete task instructions in the chip, the problem of incomplete task queue management is solved, and the complete clearing and state restoration of the task queue is achieved, thereby improving the reliability and efficiency of task execution.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING XINGYUN INTEGRATED CIRCUIT CO LTD
- Filing Date
- 2026-04-10
- Publication Date
- 2026-07-21
Smart Images
Figure CN122431820A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of chip processing task technology, and more specifically, to a task processing method and a chip. Background Technology
[0002] The chip can receive commands and create and execute corresponding tasks based on those commands.
[0003] In related technologies, chips (such as GPGPU (General-Purpose computing on Graphics Processing Units)) can receive commands from host devices (such as CPU (Central Processing Unit)) or other chips through interconnect interfaces, and parse and execute them. In order to better execute the tasks corresponding to the commands, corresponding command queues are set up, and multiple tasks can be in a command queue.
[0004] However, the relevant technologies cannot effectively manage the clearing of multiple tasks in the command queue. Summary of the Invention
[0005] The purpose of this application is to provide a task processing method and chip to address the shortcomings of the prior art, thereby solving the aforementioned technical problems in the related technologies.
[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: In a first aspect, embodiments of this application provide a method for obtaining a first command, the first command including a control command, the control command including at least one of a clear queue instruction and a delete task instruction, the clear queue instruction including a clear start instruction and a clear end instruction; If the first command is the clear queue instruction, modify all tasks in the target task queue according to the clear start instruction; The clearing process ends according to the clearing end command, restoring the state of the target task queue.
[0007] Optionally, if the first command is the clear queue instruction, modifying all tasks in the target task queue according to the clear start instruction includes: If the first command is the clear queue instruction, according to the clear start instruction, the termination process is triggered for the tasks being executed in the target task queue, and the first task in the target task queue that is in a waiting state is changed to a terminated state. If it is detected that the first task is at the head of the target task queue, the first task is discarded according to the terminated status. The clear start instruction is stored in the first command buffer unit, and the clear end instruction is stored in the second command buffer unit. The reading priority of the first command buffer unit is higher than that of the second command buffer unit.
[0008] Optionally, the method further includes: The second command creates a second task in the target task queue and updates the status information of the task queue list. The second task is launched based on the status information of the updated task team list and the information of the target task queue to execute the second task.
[0009] Optionally, the task queue includes a computation task queue and a transport task queue, and the number of task queues is multiple. The method further includes: The target task queue is determined based on the priority of the multiple task queues; According to the enqueue order of multiple tasks in the target task queue, the multiple tasks in the target task queue are executed sequentially.
[0010] Optionally, the first command is configured to obtain a read pointer and obtain a write pointer, and the first command includes: If the write pointer is not equal to the read pointer, then a read request is initiated to the memory according to the size of the command buffer unit used to store the first command and the read pointer, so as to read the target command from the command buffer unit of the memory; Update the read pointer according to the target command read.
[0011] Optionally, the method further includes: Create a waiting task in the task queue according to the waiting command; If it is detected that the waiting task has moved to the head of the target task queue, monitor whether the task on which the waiting task depends has been completed, and obtain the monitoring result; Based on the monitoring results, the waiting task is processed.
[0012] Optionally, the task on which the waiting task depends is a local task of the chip; if it is detected that the waiting task has moved to the head of the target task queue, monitoring whether the task on which the waiting task depends has been completed and obtaining the monitoring result includes: If it is detected that the waiting task has moved to the head of the target task queue, the remaining task queues are traversed, and the task that the waiting task depends on is monitored in the remaining task queues to obtain the monitoring results; The step of processing the waiting task based on the monitoring results includes: If the monitoring result indicates that the task the waiting task depends on is in the remaining task queue, then the waiting task is executed and enters a waiting state; If the monitoring result indicates that the task on which the waiting task depends has been completed, the waiting task is released, and the task following the waiting task is executed. Optionally, the task on which the waiting task depends is a task of an external chip; if the waiting task is detected to have moved to the head of the target task queue, monitoring whether the task on which the waiting task depends has been completed and obtaining the monitoring result includes: If it is detected that the waiting task has moved to the head of the target task queue, monitor whether the completion notification of the task on which the waiting task depends is received from the external chip, and obtain the monitoring result; The step of processing the waiting task based on the monitoring results includes: If the monitoring result indicates that the completion notification from the external chip has not been received, then the waiting task is executed and the system enters a waiting state. If the monitoring result indicates that the completion notification from the external chip has been received, the waiting task is released, and the task following the waiting task is executed.
[0013] If the completion notification sent by the external chip is received, and the waiting task has not moved to the head of the target task queue, the status of the waiting task is marked as terminated. If the waiting task moves to the head of the target task queue, the waiting task that has been aborted is discarded.
[0014] Secondly, embodiments of this application provide a chip, the chip including: a command scheduling module, the command scheduling module being used to execute the method described in any of the first aspects above.
[0015] Optionally, the chip includes computing units and data transport engines, wherein the number of computing task queues is greater than the number of computing units, and the number of data transport task queues is greater than the number of data transport engines.
[0016] The beneficial effects of this application are as follows: This application provides a task processing method, including: obtaining a first command, the first command including a control command, the control command including at least one of a clear queue command and a delete task command, thereby better managing task status and queue clearing through different command types; the clear queue command including a clear start command and a clear end command; if the first command is a clear queue command, modifying all tasks in the target task queue according to the clear start command; ending the clearing according to the clear end command, and restoring the state of the target task queue. Modifying all tasks in the target task queue based on the clear start command and ending the clearing according to the clear end command ensures effective and reliable clearing of all tasks between the clear start and clear end commands, reducing the possibility of tasks not yet in the queue being missed due to the queue clearing command being a single command. Furthermore, the state of the target task queue can be automatically restored after the clearing is completed based on the clear end command. Attached Figure Description
[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 A flowchart illustrating a task processing method provided in this application embodiment. Figure 1 ; Figure 2 A schematic diagram of the framework of a command processing unit provided in an embodiment of this application; Figure 3 A flowchart illustrating a task processing method provided in this application embodiment. Figure 2 ; Figure 4 A flowchart illustrating a task processing method provided in this application embodiment. Figure 3 ; Figure 5 A flowchart illustrating a task processing method provided in this application embodiment. Figure 4 ; Figure 6 A flowchart illustrating a task processing method provided in this application embodiment. Figure 5 ; Figure 7 A flowchart illustrating a task processing method provided in this application embodiment. Figure 6 ; Figure 8 This is a schematic diagram of the structure of a chip provided in an embodiment of this application. Detailed Implementation
[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of this application, but not all embodiments.
[0020] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.
[0021] In the description of this application, it should be noted that if the terms "upper", "lower", etc. appear to indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship that the product of this application is usually placed in, it is only for the convenience of describing this application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.
[0022] Furthermore, the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Additionally, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0023] The modules, units, command processors, etc. mentioned in this application may include one or more hardware circuits, including but not limited to: application-specific integrated circuits, digital signal processors, field-programmable gate arrays, discrete logic circuits, state machines, and any combination of the aforementioned circuits; these circuits are specifically designed, configured, and interconnected to perform one or more specific functions disclosed in this application.
[0024] It should be noted that, where there is no conflict, the features in the embodiments of this application can be combined with each other.
[0025] This application provides a task processing method applied to a chip. The following is an explanation of the task processing method provided by this application.
[0026] Figure 1 A flowchart illustrating a task processing method provided in this application embodiment. Figure 1 ,like Figure 1 As shown, the method may include: S101, Obtain the first command.
[0027] The first command includes control commands, which include at least one of a clear queue command and a delete task command. The clear queue command includes a clear start command and a clear end command. The control commands can include task control commands and queue control commands, corresponding to a specific task and all tasks in a specific queue, respectively.
[0028] In some implementations, the command receiving unit reads the first command from the first command buffer unit of the storage unit and parses the first command to obtain the parsing result of the first command; the command processing unit executes the subsequent processes S102 to S103 based on the parsing result of the first command.
[0029] It should be noted that if the first command is a delete task instruction, the command processing unit deletes the target task in the target task queue according to the delete task instruction.
[0030] In addition, in practical applications, control commands do not create tasks in the task queue; they are commands that are sent and executed immediately.
[0031] In this embodiment, the on-chip command processor can obtain command packets from the main CPU and determine whether the command packet is a normal command packet or a complex command packet. If the command packet is a complex command packet, it is decomposed into a fixed-length normal command packet, which is then parsed to obtain a parsed command packet. If the command packet is a normal command packet, it is directly parsed to obtain a parsed command packet. The on-chip command processor can send the parsed command packet to the command scheduling module, which can then receive the parsed command packet, which includes a first command.
[0032] S102. If the first command is a clear queue command, modify all tasks in the target task queue according to the clear start command.
[0033] In this embodiment of the application, if the first command is a clear queue instruction, the state of all tasks in the target task queue is modified according to the clear start instruction. The whole process can only modify the state of the tasks without clearing the data of the tasks, which can reduce power consumption.
[0034] This includes modifying all tasks in the target task queue, including modifying tasks currently being executed and tasks that are waiting to be executed. Modifications can also be made to tasks that will be added to the target task queue later.
[0035] S103. The clearing process ends according to the clearing end command, and the state of the target task queue is restored.
[0036] In some implementations, after the clearing end command concludes the process for the target task queue, the state of the target task queue is restored to the state before the clearing start command was executed. Tasks that are subsequently to be added to the target task queue are then added normally.
[0037] It should be noted that the clear queue instruction and the clear end instruction can be triggered in a single operation. For example, when a user performs a clear queue operation, the command processor in the chip can send a clear start instruction and a clear end instruction to the HyperQueue mapped to the user process. The command processor and the command scheduling module are connected.
[0038] In this embodiment of the application, the target task queue enters the clearing state according to the clearing start command, and is restored to the state before entering the clearing state according to the clearing end command. For example, the target task queue was in the paused or running state before entering the clearing state, and is restored to the paused or running state after the clearing is completed.
[0039] In summary, this application provides a task processing method, including: obtaining a first command, the first command including a control command, the control command including at least one of a clear queue command and a delete task command, to better manage task status and queue clearing through different command types; the clear queue command including a clear start command and a clear end command; if the first command is a clear queue command, modifying all tasks in the target task queue according to the clear start command; ending the clearing according to the clear end command and restoring the state of the target task queue. Modifying all tasks in the target task queue based on the clear start command and ending the clearing according to the clear end command ensures effective and reliable clearing of all tasks between the clear start and clear end commands, reducing the possibility of tasks not yet in the queue being missed due to a single queue clear command. Furthermore, the state of the target task queue can be automatically restored after the clearing is completed based on the clear end command.
[0040] Figure 2 This is a schematic diagram of the framework of a command processing unit provided in an embodiment of this application. Figure 2As shown, in one embodiment of this application, the task queue can be set within the command processing unit, specifically including: a task queue list and a task queue buffer. The task queue list is used to store the queue's metadata and management information, and can specifically be a register array stored in the chip logic circuit. The memory corresponding to the task queue can be composed of two parts: Hyper Queue SRAM (a dedicated static random access memory).
[0041] The task queue list can be divided into two dimensions: fine and coarse. The fine dimension includes an instance identifier array, a task priority array, a task function pointer array, and a task status array, while the coarse dimension includes a multi-task flow parallel queue status array, a tail pointer array, and a head pointer array.
[0042] Among them, the instance identifier array is used to record the unique instance ID of each enqueued task, the task priority array is used to record the priority of each enqueued task, the task function pointer array is used to record the entry address of each enqueued task in the task queue buffer, and the task status array is used to record the status information of each enqueued task.
[0043] It should be noted that these detailed dimensions of information consist of a total of Hyper Queue number × Hyper Queue number of entries, and each corresponds one-to-one with the task body information stored in the task queue cache.
[0044] In addition, the multi-task flow parallel queue state array is used to record the state of each task queue, and the tail pointer array and head pointer array are used to record the tail pointer and head pointer of each task queue, respectively. These coarse-dimensional information have a total of Hyper Queue number items, which correspond to each Hyper Queue.
[0045] Hyper Queue has four states: idle, running, paused, and empty.
[0046] It's worth noting that in the idle state, the corresponding Hyper Queue is empty. When the Hyper Queue is not empty, the state changes to running. In the running state, the corresponding Hyper Queue is not empty and is operating normally. When a hyperq halt (multi-task flow parallel queue stop) signal is received, it will change to paused state; if a flush start (clear start) signal is received, it will change to cleared state. Specifically, if a task in a Hyper Queue encounters a problem, the user can send a hyperq halt signal. The on-chip command processor needs to pause all task transmissions in that Hyper Queue until the problem is resolved.
[0047] While paused, the HyperQueue can still accept new tasks, but will pause sending. Upon receiving the `hyperq resume` (multitasking stream parallel queue resume) signal, it will either enter a running or idle state depending on whether there are currently any valid tasks. If a clear start command is received, the HyperQueue will enter a cleared state. The `hyperqresume` signal is either generated after the problem is resolved or sent by the user.
[0048] When in the flush state, all existing content in the Hyper Queue will be cleared, and commands sent to this Hyper Queue will be discarded. Upon receiving the flush finish signal, the Hyper Queue will return to the idle state.
[0049] In addition, there are four states in the task state array: idle state, queued state, executing state, and terminated state.
[0050] In the queueing state, the corresponding task is being queued. When it reaches the front of the queue and is selected for launch, the task status changes to "in execution." If a task abort signal is received before selection for launch, the task status changes to "aborted." If the entire queue is cleared, the task status returns directly to "idle." In the aborted state, the corresponding task is marked as canceled and will be discarded when selected for launch. If the entire queue is cleared, the task status returns directly to "idle."
[0051] Additionally, in the idle state, there is no corresponding task or the data is invalid (when a task is completed or cleared, considering power consumption, dirty data is not reset; a new task overwrites it). When a task creation signal is received, the task state changes to the queued state. In the executing state, the corresponding task is being executed; the execution target may be a SIMT Cluster (compute unit), HDMA, dependency resolution unit, etc. When a task completion signal is received, the task state will directly return to the idle state; if the GPU launch task being executed at this time receives a task replacement signal, the task state will return to the queued state.
[0052] Optionally, Figure 3 A flowchart illustrating a task processing method provided in this application embodiment. Figure 2 ,like Figure 3 As shown, in S102 above, if the first command is a queue clearing instruction, the process of modifying all tasks in the target task queue according to the clearing start instruction may include: S201. If the first command is a clear queue command, according to the clear start command, trigger the termination process for the tasks being executed in the target task queue, and change the first task in the target task queue that is in a waiting state to a terminated state.
[0053] Among them, the first task in the waiting state is the task in the queue, and the task that has been suspended is the task in the queue.
[0054] In some implementations, for tasks currently being executed in the target task queue, a configuration to terminate execution is sent by the command scheduling module according to the type of the first task. Then, the execution unit is waited for feedback of an interruption due to termination. In this way, the tasks currently being executed in the target task queue will change directly from the executing state to the idle state, thereby triggering the termination process of the tasks currently being executed in the target task queue.
[0055] In practical applications, waiting tasks can establish dependencies between multiple task queues. For example, task queue 1 and task queue 2 are performing calculations, while task queue 3 is handling the results. Based on these dependencies, a monitoring mechanism can be established using waiting tasks, meaning task queue 3 will wait for task queues 1 and 2 to complete before executing. In multi-card scenarios, cross-card synchronization is achieved by the main CPU triggering commands via notifications. These notification triggers can include feedback mechanisms to prevent the failure to find the corresponding waiting task.
[0056] S202. If the first task is detected to be at the head of the target task queue, the first task is discarded according to its aborted status.
[0057] The clear start instruction is stored in the first command buffer unit, and the clear end instruction is stored in the second command buffer unit. The reading priority of the first command buffer unit is higher than that of the second command buffer unit.
[0058] In this embodiment, the priority of the clear start instruction is higher than the priority of the clear end instruction. The high-priority clear start instruction is given priority to establish the clear state, and the low-priority clear end instruction ends the clear state last. This ensures that all tasks between the start and end can be accurately captured and terminated.
[0059] It should be noted that if the first task is detected to be at the head of the target task queue and is found to be in an aborted state, it means that the first task has been cleared in advance. Therefore, the first task will be discarded directly.
[0060] In summary, the first task in the target task queue that is in a waiting state is changed to a stopped state; if the first task is detected to be at the head of the target task queue, it is discarded based on its stopped state. This prevents the first task in a waiting state from being missed during the clearing process, thus avoiding invalid execution of the first task in a waiting state.
[0061] Optionally, Figure 4 A flowchart illustrating a task processing method provided in this application embodiment. Figure 3 ,like Figure 4 As shown, the method may further include: S301. Create a second task in the target task queue according to the second command, and update the status information of the task queue list.
[0062] like Figure 2 As shown, the command processing unit may further include a task / queue control unit and a task creation unit, wherein the task / queue control unit is used to change the status of tasks or task queues. The task creation unit determines the target task queue corresponding to the second command, and then creates a second task in the target task queue according to the second command.
[0063] Additionally, the task creation unit can update the status information of the task queue list, that is, update the status information in the Hyper Queue Table. The task queue list can be used to indicate whether a second task is valid.
[0064] S302. Launch the second mission based on the status information of the updated mission team list and the information of the target mission queue to execute the second mission.
[0065] It should be noted that the second command is the command to create a task, also known as a task command. In this embodiment, all commands can be divided into control commands and task commands, all of which are fixed-length commands. Both control commands and task commands are stored in fixed bits to reduce the complexity of design and parsing.
[0066] In some implementations, such as Figure 2 As shown, the command processing unit may further include a task launching unit, which retrieves a second task from the target task queue and launches it based on the status information of the second task, the updated task queue list, and the information of the target task queue in the task queue buffer.
[0067] If the second task is a computation task, the task sending unit will send the computation task to the computation unit on the chip for computation. If the second task is a data transfer task, the task sending unit will send the data transfer task to the data transfer engine on the chip for data transfer.
[0068] In this embodiment, the task queue buffer is a read-write dual-port SRAM memory used to store various task entities, capable of storing multiple tasks. The storage unit occupied by each data transfer initiation task and computation initiation task is larger than that occupied by each waiting and notification type task. For each command to be created, the entry address of the buffer needs to be updated in the task queue list, and simultaneously, the corresponding task entity is stored in the SRAM.
[0069] For example, the task queue list records the priority and status of each task. The Hyper Queue Table pointed to by the head pointer is monitored. The priority of all tasks in the running Hyper Queue is sent to the arbitrator, with higher values indicating higher priority. If values are the same, a round-robin strategy is used. The status of the selected task is changed to "in execution," and then the content pointed to by the entry address of the task queue buffer is read. As long as there are valid tasks in the task queue buffer, they will be prefetched onto the prefetch buffer built by DFFs (D-flip-flops).
[0070] If the task type is found to be a computation task after reading, the configuration information storage unit will be retrieved, and then the APB master device will be invoked to send these configurations to the chip's subsystem on-chip network. If the task type is found to be waiting and dependent on the local GPU, a monitoring counter will be initialized. This counter will decrement by 1 each time a task is popped from the target queue. When the counter reaches 0, it indicates that the target task has been completed, and the waiting tasks will be released. Besides tasks in the task queue triggering APB transactions, task abort commands will also be triggered directly, with higher priority than the operations triggered by retrieving content from the task queue.
[0071] like Figure 2 As shown, the command processing unit may also include a dependency resolution unit, which can be used to detect the execution status of the second task.
[0072] In summary, this ensures that the second task strictly adheres to the creation-then-launch processing order, avoiding out-of-order execution. Launching the second task based on the updated task queue list's state information guarantees that the execution unit can acquire and execute the new task immediately, reducing pipeline downtime. Dividing the task queue into a task queue list and a task queue buffer allows modification of the task / queue state to be done only by updating the task queue list, while task launch requires considering the task state within the task queue list. This ensures that tasks are processed according to the latest state during launch, achieving low-power task / queue state modification and control.
[0073] Optionally, Figure 5 A flowchart illustrating a task processing method provided in this application embodiment. Figure 4 ,like Figure 5 As shown, the task queue includes a computation task queue and a transport task queue, and there are multiple task queues. The method also includes: S401. Determine the target task queue based on the priority of multiple task queues; Specifically, for multiple task queues of the same type, i.e., multiple computation task queues or multiple transport task queues, the task queue with the highest priority among the multiple task queues is taken as the target task queue.
[0074] In addition, for multiple task queues of different types, check the number of idle computing units and data transfer engines. If there are idle computing units and data transfer engines at the same time, multiple task queues of different types can be executed at the same time, that is, computing task queues and multiple transfer task queues can be executed at the same time.
[0075] Furthermore, multiple task queues with the same priority can be executed in a round-robin fashion.
[0076] S402. Execute multiple tasks in the target task queue in sequence according to the enqueue order of the multiple tasks in the target task queue.
[0077] In this embodiment of the application, multiple tasks in the same target task queue do not have priority and can be executed sequentially according to the order in which they were enqueued.
[0078] It should be noted that when multiple tasks are enqueued, tasks with dependencies can be placed in the same task queue, while tasks without dependencies can be placed in different task queues. This enables low-cost and efficient parallel execution and reduces the need to set up specific interrupt and synchronization mechanisms for tasks with dependencies.
[0079] In summary, it can be ensured that tasks with higher priority are executed first. For multiple tasks in the target task queue, it can be ensured that tasks added to the target task queue first are executed first, and tasks added later are executed later.
[0080] Optionally, Figure 6 A flowchart illustrating a task processing method provided in this application embodiment. Figure 5 ,like Figure 6 As shown, the first command acquisition setting includes acquiring the read pointer and acquiring the write pointer. The process of acquiring the first command in S101 above may include: S501. If the write pointer is not equal to the read pointer, then based on the size of the command buffer unit used to store the first command and the read pointer, a read request is initiated to the memory to read the target command from the command buffer unit of the memory.
[0081] S502. Update the read pointer according to the target command being read.
[0082] Among them, the command buffer unit that stores the first command is the first command buffer unit.
[0083] In practical applications, the first command buffer unit uses a ring buffer data structure. The data structure resembles a ring, initially with both the read and write pointers pointing to address 0. When a command is written, the write pointer is incremented by 1; when the command receiving unit reads a command, the read pointer is incremented by 1. The read pointer always follows the write pointer. The user side does not need to concern itself with the address organization. If the write pointer is not equal to the read pointer, it means there is still space in the command buffer to write commands. If the write pointer catches up with the read pointer, i.e., the write pointer equals the read pointer, it means the command buffer is full.
[0084] In some implementations, the command receiving unit may include a command reading module and a command decoder. The command reading module can maintain multiple read pointer counters locally, with a default value of 0. When the write pointer of the first command buffer unit is not equal to the read pointer, it will drive the command reading module to initiate a read request starting from the address N×size + read pointer. For each command read, the first command buffer unit updates the corresponding read pointer counter by one, until the read pointer counter = N×Size-1, at which point it is reset to zero. Both the first and second command buffer units can be managed in a first-in-first-out manner and each has its own read and write pointers.
[0085] In summary, when the write pointer and read pointer are not equal, it indicates that there are commands awaiting processing in the command buffer unit. A read request is initiated based on the size of the command buffer unit and the position of the read pointer, the correct target command is read, and the read pointer is updated. This ensures that commands are consumed in order, without duplication or omission, maintains the dynamic balance of the circular buffer, and improves processing efficiency.
[0086] Optionally, Figure 7 A flowchart illustrating a task processing method provided in this application embodiment. Figure 6 ,like Figure 7 As shown, the method may further include: S601. Create a waiting task in the task queue according to the waiting command.
[0087] In some implementations, the KMD (Kernel Mode Driver) can send a wait command to the command processing unit; the command processing unit then creates a wait task in the task queue based on the wait command.
[0088] S602. If it is detected that the waiting task has moved to the head of the target task queue, monitor whether the task on which the waiting task depends has been completed and obtain the monitoring result.
[0089] Among them, the tasks that the waiting tasks depend on are either local tasks of the chip, which can be executed locally on the chip, or tasks of external chips, which can be executed on the external chip.
[0090] In this embodiment of the application, if it is detected that the waiting task has moved to the head of the target task queue, the dependency resolution unit monitors whether the task on which the waiting task depends has been completed and obtains the monitoring result. The monitoring result is used to indicate whether the task on which the waiting task depends has been completed or not.
[0091] S603. Based on the monitoring results, process the waiting tasks.
[0092] Based on the monitoring results, the task may either enter a waiting state, terminate the waiting task and execute the task following it, or mark the waiting task as terminated.
[0093] In summary, by creating waiting tasks and monitoring the status of the tasks that the waiting tasks depend on, frequent CPU polling or intervention can be avoided. This allows the chip to autonomously process the waiting tasks based on the monitoring results of the tasks they depend on, thereby improving the parallelism of command execution and the utilization of hardware resources, and ensuring the smoothness and stability of multi-task collaborative work.
[0094] Optionally, the task on which the waiting task depends is a local task of the chip; the process in S602 above, which detects that the waiting task has moved to the head of the target task queue, and monitors whether the task on which the waiting task depends has been completed, and obtains the monitoring result, may include: If it is detected that the waiting task has moved to the head of the target task queue, the remaining task queues are traversed, and the task that the waiting task depends on is monitored in the remaining task queues to obtain the monitoring results.
[0095] In this embodiment of the application, if the task on which the waiting task depends is a local task of the chip, that is, both the waiting task and the task on which the waiting task depends are local tasks of the chip, KMD and UMD (User Mode Driver) will ensure that the task on which the waiting task depends is submitted to the command scheduling module first, thereby guaranteeing the basis of the execution order.
[0096] When a waiting task is scheduled to the head of the target task queue, the dependency resolution unit will actively traverse the other task queues to monitor whether the tasks that the waiting task depends on are in the other task queues, and obtain the monitoring results to check whether the tasks that the waiting task depends on have been completed.
[0097] The process of handling waiting tasks based on monitoring results in S603 above may include: If the monitoring results indicate that the task the waiting task depends on is in the remaining task queue, then the waiting task will be executed and enter the waiting state. If the monitoring results indicate that the task the waiting task depends on is in the other task queues, it means that the task the waiting task depends on has not been completed, and the waiting task will enter the waiting state.
[0098] Furthermore, after the tasks that the waiting task depends on have completed, the subsequent operation is determined based on the feedback type of those tasks. If no feedback is required, the waiting task is released directly, and the task following it is executed. If feedback is required, an interrupt must first be sent to the command processor, and the waiting task is released only after the command processor has processed the feedback event.
[0099] If the monitoring results indicate that the task the waiting task depends on has been completed, release the waiting task and execute the task that follows the waiting task.
[0100] In practical applications, once the tasks that the waiting task depends on have completed their execution, subsequent tasks can begin, so the waiting task can be released directly.
[0101] In summary, since the tasks on which the waiting task depends are local tasks of the chip, it ensures that the waiting task is released only after the tasks on which it depends have been completed, allowing the tasks that follow the waiting task to execute. This ensures that the execution order of multiple tasks is accurate and reliable.
[0102] Optionally, the task on which the waiting task depends is a task of an external chip; the process in S602 above, which detects that the waiting task has moved to the head of the target task queue, and monitors whether the task on which the waiting task depends has been completed, and obtains the monitoring result, may include: If it is detected that the waiting task has moved to the head of the target task queue, monitor whether a completion notification of the task that the waiting task depends on is received from the external chip, and obtain the monitoring result.
[0103] The process of handling waiting tasks based on monitoring results in S602 above may include: If the monitoring results indicate that no completion notification has been received from the external chip, the waiting task will be executed and the system will enter a waiting state.
[0104] If the monitoring results indicate that no completion notification has been received from the external chip, it means that the task on which the waiting task depends has not been completed, so the waiting task enters a waiting state.
[0105] If the monitoring results indicate that a completion notification has been received from an external chip, the waiting task is released, and the task following the waiting task is executed.
[0106] If the monitoring results indicate that no completion notification has been received from the external chip, it means that the task on which the waiting task depends has been completed. Therefore, the waiting task can be released directly, and the task following the waiting task can be executed.
[0107] If a completion notification is received from an external chip, and the waiting task has not moved to the head of the target task queue, the status of the waiting task will be marked as aborted.
[0108] If a waiting task moves to the head of the target task queue, then waiting tasks that have been aborted are discarded.
[0109] In this embodiment, if a completion notification is received from an external chip, it indicates that the task on which the waiting task depends has not been completed; if the waiting task has not moved to the head of the target task queue, it means that the waiting task cannot be executed at present, and it is marked as a stopped task. When the waiting task moves to the head of the target task queue and it is the turn of the waiting task to be executed, its stopped state is found, and it is discarded directly.
[0110] In some embodiments, the completion notification sent by the external chip is required to include a feedback request. This is to ensure that the sender can confirm whether the target waiting task has been successfully found and processed. If the waiting task has been aborted or does not exist, the feedback mechanism can prevent notification loss or state asynchrony, ensuring the reliability of cross-chip synchronization.
[0111] In summary, the chip automatically handles cross-GPU task dependencies, avoiding software polling and complex synchronization overhead. Even when the dependent task is an external chip, it can ensure that multiple tasks are executed in sequence.
[0112] This application also provides a chip. Figure 8 This is a schematic diagram of the structure of a chip provided in an embodiment of this application, such as... Figure 8 As shown, the chip includes a command scheduling module, which is used to execute any of the task processing methods described above.
[0113] like Figure 8 As shown, the command scheduling module may include: a storage unit, a command receiving unit, and a command processing unit; the command receiving unit is connected to the storage unit, and the storage unit includes at least: a first command buffer unit and a second command buffer unit, the first command buffer unit has a higher read priority than the second command buffer unit, the first command buffer unit is used to buffer control commands, and the second command buffer unit is used to buffer low-priority commands, including ordinary task commands.
[0114] The command receiving unit is used to read commands from the storage unit and parse the commands to obtain the parsing results; the command processing unit is connected to the command receiving unit and is used to create or process tasks based on the parsing results.
[0115] The storage unit further includes: a configuration information storage unit; the configuration information storage unit stores the computational configuration information required for executing computational tasks, including the system-mode entry address and the system-mode packet address; the command processing unit reads the computational configuration information from the configuration information storage area of the storage unit when processing computational tasks. The storage unit also includes an interrupt storage unit; the command processor reads message interrupt vectors from the message interrupt vector table area in the interrupt storage unit and sends the message interrupt vectors to the bus interface unit.
[0116] Optionally, the chip includes computing units and data transport engines, with the number of computing task queues being greater than the number of computing units and the number of data transport task queues being greater than the number of data transport engines.
[0117] The computing unit includes multiple computing cores.
[0118] It should be noted that the computation task queue is used to cache computation commands, while the data transport command queue is used to cache transport commands. Computation commands are executed by the computation unit, and data transport commands are executed by the on-chip data transport unit (such as HDMA, Direct Memory Access). If there are no dependencies, the computation task queue and the transport task queue can execute in parallel. If multiple computation units and HDMA units exist, task queues of the same type can execute in parallel.
[0119] The computation task queue and the transport task queue can each be a multiple Hyper Queue (multi-task flow parallel queue). Hyper Queues follow a first-in, first-out (FIFO) principle, allowing dependent tasks to be placed in the same Hyper Queue. Multiple Hyper Queues can be used to switch between computation and transport resources; that is, when one task finishes execution or needs to wait, the queue can be quickly switched to seize resources. Different Hyper Queues have priorities; after one Hyper Queue finishes execution, the highest priority Hyper Queue is executed first, and if priorities are the same, they are executed in a round-robin fashion.
[0120] In this embodiment, if there are x copies of computing resources and y copies of transport resources, the number of Hyper Queues can be greater than x + y. If the HDMA on the chip is implemented through a bus interface unit, regardless of hardware limitations, the more the better. The bus interface unit may include a PCIe (Peripheral Component Interconnect Express) unit and a UCI-Ie unit.
[0121] In practical applications, taking large-scale model inference as an example, suppose multiple users access the chip simultaneously. If the number of HyperQueues is too small, such as only one, multiple users' tasks can only be placed in the same HyperQueue. Since tasks are executed sequentially within the HyperQueue, the user who asks the question first receives a quick response, while subsequent users have to wait for the previous user's question to finish before they can start generating a token (the smallest text unit output by the large model when generating an answer), resulting in a poor user experience. This application's embodiment provides multiple HyperQueues, allowing users to switch between them to ensure a roughly consistent user experience across multiple scenarios.
[0122] The command receiving unit can be used to execute S101 in the above embodiments, the command processing unit can be used to execute S102 and S103 in the above embodiments, the command processing unit can be used to execute S201 and S202 in the above embodiments, the command processing unit can also be used to execute S301 and S302 in the above embodiments, and the command processing unit can also be used to execute S401 and S402 in the above embodiments.
[0123] Alternatively, the command receiving unit can also be used to execute S501 and S502 in the above embodiments. The command processing unit can be used to execute S601, S602, and S603 in the above embodiments.
[0124] In this embodiment, the command scheduling module executes any of the task processing methods described in the previous application. Based on the clear start command, it modifies all tasks in the target task queue and ends the clearing according to the clear end command. In this way, all tasks can be effectively and reliably cleared between the clear start command and the clear end command, reducing the possibility of tasks that have not yet entered the queue being missed due to the queue clearing command being a single command. Moreover, based on the clear end command, the state of the target task queue can be automatically restored after the clearing is completed.
[0125] Optionally, the chip includes computing units and data transport engines, with the number of computing task queues being greater than the number of computing units and the number of data transport task queues being greater than the number of data transport engines.
[0126] In this embodiment, the computation task queue is used to cache computation commands, and the data transfer command queue is used to cache transfer commands. There can be multiple task queues for computation and transfer. Multiple task queues can be used to switch between computation resources and transfer resources. That is, when one task is completed or needs to wait, the queue can be quickly switched to seize resources.
[0127] Assuming multiple users access the chip simultaneously, if the number of task queues is too small, such as only one, tasks from multiple users will be placed in the same queue. Within the task queue, the user who asks the question first will receive a quick response, while later users will have to wait. This embodiment provides multiple task queues, allowing for seamless switching between them to ensure a similar user experience across multiple scenarios. Preferably, the number of task queues is greater than the sum of the number of data transfer engines and computing units, but less than twice that sum, balancing chip area, power consumption design requirements, and overall efficiency.
[0128] In some implementations, before calling a function to perform a calculation, the data required for the calculation needs to be prepared in advance near the computing core of the chip provided in this application embodiment. Therefore, it is sometimes necessary to move remote data (such as stored data on other chips). After the calculation is completed, if it is necessary to integrate with the calculation results on other chips, data movement is also required. In addition, if there are dependencies between the calculation tasks and the data movement tasks, waiting tasks can be inserted to establish dependencies in order to ensure that the execution order meets expectations.
[0129] In some application scenarios, when a user submits a series of tasks, these tasks can be encapsulated into a predefined format using software and submitted from the main CPU to the chip provided in this embodiment. The command scheduling module obtains the task commands, parses and extracts them according to the format, and obtains a task queue. This allows for automatic control of the data transfer engine and computing unit startup through task launch. Once the computing and data transfer tasks are completed normally, the command scheduling module automatically pops the task and launches the next task.
[0130] The above process, employing the solution described in this application, reduces software involvement and significantly lowers latency. If it's necessary to obtain the progress of batch task execution, the command processor can set a quota for task execution feedback reporting for each task queue, enabling the command processor to add more tasks to the command scheduling module.
[0131] Furthermore, if a computational unit encounters an anomaly in its task, the command scheduling module can be notified via an interrupt, which in turn notifies the command processor. The task queue containing that task can be paused from issuing. The command processor then retrieves information about the anomaly and sends it back to the user. While this feedback is being sent, other task queues can continue issuing tasks to schedule the computational unit for execution, preventing idle computational resources and waste. Once the cause of the anomaly is identified and corrected, the corrected task can be resubmitted to the task queue. The head of the queue is replaced, and a task queue control command is sent to resume issuance. Afterward, scheduling returns to normal.
[0132] In some embodiments of this application, the chip's subsystem on-chip network can receive and store commands. In most cases, task packets and control packets are stored in a low-priority second command buffer unit, unless there are urgent control or task packets that need to be queued, in which case they are stored in a high-priority second command buffer unit. The chip's subsystem on-chip network can read and parse the stored commands, and put the parsed task packets into a queue, wherein the queue includes a queue list and a task queue storage area. The same task queue storage area can simultaneously store computation tasks, transport tasks, or waiting tasks, etc.; the corresponding resources are matched according to the task type to execute the task; when the resources have finished executing a queue, the next task to be launched is determined according to the priority of the task at the head of all currently pending queues. During task execution, the status of the task or queue is adjusted and updated according to the control instructions of the control packet, such as aborting or clearing. In other embodiments, the same task queue storage area may only store tasks of the same type.
[0133] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.
Claims
1. A task processing method, characterized in that, Applied to a chip, the method includes: Obtain a first command, the first command including a control command, the control command including at least one of a clear queue command and a delete task command, the clear queue command including a clear start command and a clear end command; If the first command is the clear queue instruction, modify all tasks in the target task queue according to the clear start instruction; The clearing process ends according to the clearing end command, restoring the state of the target task queue.
2. The method according to claim 1, characterized in that, If the first command is the clear queue instruction, modifying all tasks in the target task queue according to the clear start instruction includes: If the first command is the clear queue instruction, according to the clear start instruction, the termination process is triggered for the tasks being executed in the target task queue, and the first task in the target task queue that is in a waiting state is changed to a terminated state. If it is detected that the first task is at the head of the target task queue, the first task is discarded according to the terminated status. The clear start instruction is stored in the first command buffer unit, and the clear end instruction is stored in the second command buffer unit. The reading priority of the first command buffer unit is higher than that of the second command buffer unit.
3. The method according to claim 1, characterized in that, The method further includes: The second command creates a second task in the target task queue and updates the status information of the task queue list. The second task is launched based on the status information of the updated task team list and the information of the target task queue to execute the second task.
4. The method according to claim 1, characterized in that, The task queue includes a computation task queue and a transport task queue, and there are multiple task queues. The method further includes: The target task queue is determined based on the priority of the multiple task queues; According to the enqueue order of multiple tasks in the target task queue, the multiple tasks in the target task queue are executed sequentially.
5. The method according to claim 1, characterized in that, The first command is configured to retrieve a read pointer and a write pointer, and the first command includes: If the write pointer is not equal to the read pointer, then a read request is initiated to the memory according to the size of the command buffer unit used to store the first command and the read pointer, so as to read the target command from the command buffer unit of the memory; Update the read pointer according to the target command read.
6. The method according to claim 1, characterized in that, The method further includes: Create a waiting task in the task queue according to the waiting command; If it is detected that the waiting task has moved to the head of the target task queue, monitor whether the task on which the waiting task depends has been completed, and obtain the monitoring result; Based on the monitoring results, the waiting task is processed.
7. The method according to claim 6, characterized in that, The task on which the waiting task depends is a local task of the chip; if the waiting task is detected to have moved to the head of the target task queue, the monitoring of whether the task on which the waiting task depends has been completed is performed, and the monitoring results are obtained, including: If it is detected that the waiting task has moved to the head of the target task queue, the remaining task queues are traversed, and the task that the waiting task depends on is monitored in the remaining task queues to obtain the monitoring results; The step of processing the waiting task based on the monitoring results includes: If the monitoring result indicates that the task the waiting task depends on is in the remaining task queue, then the waiting task is executed and enters a waiting state; If the monitoring result indicates that the task on which the waiting task depends has been completed, the waiting task is released, and the task following the waiting task is executed.
8. The method according to claim 6, characterized in that, The task on which the waiting task depends is a task from an external chip; if the waiting task is detected to have moved to the head of the target task queue, the monitoring of whether the task on which the waiting task depends has been completed is performed, and the monitoring results are obtained, including: If it is detected that the waiting task has moved to the head of the target task queue, monitor whether the completion notification of the task on which the waiting task depends is received from the external chip, and obtain the monitoring result; The step of processing the waiting task based on the monitoring results includes: If the monitoring result indicates that the completion notification from the external chip has not been received, then the waiting task is executed and the system enters a waiting state. If the monitoring result indicates that the completion notification from the external chip has been received, then the waiting task is released, and the task following the waiting task is executed. If the completion notification sent by the external chip is received, and the waiting task has not moved to the head of the target task queue, the status of the waiting task is marked as terminated. If the waiting task moves to the head of the target task queue, the waiting task that has been aborted is discarded.
9. A chip, characterized in that, The chip includes a command scheduling module, which is used to execute the method described in any one of claims 1-8.
10. The chip according to claim 9, characterized in that, The chip includes a computing core unit and a data transfer engine. The number of computing task queues is greater than the number of computing core units, and the number of data transfer task queues is greater than the number of data transfer engines.