Artificial intelligence system and command queue scheduling method

Abstract

An artificial intelligence system and a method for scheduling command queues are proposed. The method for scheduling command queues includes: causing a device driver on the system side to perform management actions on multiple command queue descriptors; causing the device driver to transmit a list of command queue descriptors corresponding to the command queue descriptors to a command processor; and causing the command processor to adjust the mapping relationship between each command queue descriptor and one of the hardware command queue descriptors based on a preset scheduling mechanism and the list of command queue descriptors.

Description

[Technical Field]

[0001] This invention relates to the field of artificial intelligence, and more particularly to an artificial intelligence system and a method for scheduling command queues that can improve the computing efficiency of artificial intelligence chips. [Previous Technology]

[0002] In related technologies, command queue descriptors and hardware command queue descriptors in artificial intelligence systems always exist in a one-to-one manner. This severely limits the operational performance of artificial intelligence systems. Increasing the number of command queues to improve system performance may also lead to oversubscription of system resources, additional processing time delays, and idle resources, thereby reducing the efficiency of system resource utilization. [Summary of the Invention]

[0003] This invention provides an artificial intelligence system and a scheduling method for command queues, which can improve the computing efficiency of artificial intelligence chips.

[0004] The command queue scheduling method of the present invention includes: causing a device driver on the system side to perform management actions of multiple command queue descriptors; causing the device driver to transmit a list of command queue descriptors corresponding to the command queue descriptors to the command processor; and causing the command processor to adjust the mapping relationship between each command queue descriptor and one of the hardware command queue descriptors according to the list of command queue descriptors based on a preset scheduling mechanism.

[0005] The artificial intelligence system of the present invention includes a system terminal and a command processor. The system terminal has a device driver and system memory. The command processor is coupled to the system terminal, wherein the device driver of the system terminal is used to execute management actions of a plurality of command queue descriptors, the device driver transmits a list of command queue descriptors corresponding to the plurality of command queue descriptors to the command processor, and the command processor, based on a preset scheduling mechanism, adjusts the mapping relationship between each of the plurality of command queue descriptors and one of the plurality of hardware command queue descriptors according to the list of command queue descriptors.

[0006] Based on the above, the command queue scheduling method of the present invention adjusts the mapping relationship between each command queue descriptor and one of the hardware command queue descriptors based on a preset scheduling mechanism. In this way, the mapping relationship between multiple command queue descriptors and the hardware command queue descriptors can be dynamically adjusted, effectively improving the computing performance of the artificial intelligence system.

Implementation Method

[0008] Reference will now be made in detail to exemplary embodiments of the invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same element symbols are used in the drawings and description to denote the same or similar parts.

[0009] Please refer to Figure 1, which shows a flowchart of a command queue scheduling method according to an embodiment of the present invention. The command queue scheduling method of the embodiment in Figure 1 can be applied to an artificial intelligence system. In step S110, the system-side device driver can perform management actions on multiple command queue descriptors. The system-side device driver can create one or more command queues on the system-side memory. The command queue can be used to store command queue descriptors. In this embodiment, the command queue descriptor can be a memory command queue description (MCQD). Additionally, the system-side device driver can also perform modification and removal actions on command queues and command queue descriptors. It is worth noting that in this embodiment, even when there are no data packets in the command queue, the removal of the command queue and command queue descriptor can only be performed by the user through the system-side device driver. In this embodiment, the system side can be the Central Processing Unit (CPU).

[0010] In step S120, the device driver transmits a list of command queue descriptors corresponding to the generated command queue descriptors to the command processor. The command processor may be located in an artificial intelligence chip, such as a graphics processing chip. In this embodiment, the device driver stores the created list of command queue descriptors in system memory.

[0011] In step S130, the command processor can adjust the mapping relationship between each command queue descriptor and one of the multiple hardware command queue descriptors based on a preset scheduling mechanism and the command queue descriptor list. The command queue descriptors are memory command queue descriptors, maintained by software and stored in memory. The hardware command queue descriptors are maintained by hardware. Each hardware command queue descriptor points to a corresponding hardware processing function. In practice, the number of command queue descriptors is much higher than the number of hardware command queue descriptors. Therefore, a scheduling mechanism is needed to adjust the mapping relationship between each hardware command queue descriptor and the command queue descriptors. Specifically, in the preset scheduling mechanism, the command processor can establish a mapping relationship between each command queue descriptor and one of the hardware command queue descriptors, or the command processor can remove the established mapping relationship between each command queue descriptor and the hardware command queue descriptor. In this embodiment, the command processor can be a graphics processing unit (GPU).

[0012] In this embodiment, the command queue descriptor is a complete list of service queue descriptors for all users on the CPU side. The hardware queue descriptor is a queue descriptor provided by the GPU hardware resources. Typically, the number of command queue descriptors far exceeds the number of hardware queue descriptors, and there is over-booking, thus requiring the scheduling and binding work of this embodiment.

[0013] Furthermore, the scheduling mechanism described above can be, for example, a round-robin scheduling mechanism or a priority scheduling mechanism. Under a round-robin scheduling mechanism, the command processor can define an execution order (run list) and, based on the set execution order, determine whether there are data packets in the command queues within the corresponding current time slot, and establish a mapping relationship between the command queues and the hardware command queue descriptors accordingly. Under a priority scheduling mechanism, the command processor can select the first command queue with relatively high priority to establish a mapping relationship with the hardware command queue descriptor. If the hardware command queue descriptor already has a mapping relationship with a second command queue with relatively low priority, the command processor can remove the mapping relationship between the hardware command queue descriptor and the second command queue, and establish a mapping relationship between the first command queue and the hardware command queue descriptor. The aforementioned time slot is also a scheduled time slice, and each scheduled time slice processes one queue bound to that time slice.

[0014] Incidentally, in this embodiment, the command processor is also used to monitor the hardware command queue, handle the dependencies between queue scheduling and command packets, and send ready tasks to the command dispatcher. The command dispatcher is used to distribute tasks to the corresponding hardware and enable the hardware to execute the corresponding tasks.

[0015] Please refer to Figure 2, which shows a schematic diagram of an artificial intelligence system according to an embodiment of the present invention. The artificial intelligence system 200 includes a system terminal 210 and a command processor 220. The system terminal 210 has a device driver 211 and a system memory 212. The system memory 212 can be any type of memory. The device driver 211 can be used to perform management actions of command queue descriptors. The device driver 211 can plan multiple command queues QA~QC in the system memory 212. The command queues QA~QC can store different command queue descriptors A~C respectively, and a command queue descriptor list Qlist is established based on the planned command queues QA~QC. The device driver 211 can store the command queue descriptor list Qlist in the system memory 212. The command queue descriptor list Qlist records position information A*~C* corresponding to the command queues QA~QC. The position information A*~C* can be different queue indices.

[0016] The command processor 220, based on a preset scheduling mechanism, adjusts the mapping relationship between each command queue descriptor and one of the multiple hardware command queue descriptors according to the command queue descriptor list QList. In this embodiment, the command processor 220 has a driver 221, a hardware scheduler 222, and a packet processor 223. The driver 221 can access the system memory 212 and read or write the command queue descriptor list QList. The hardware scheduler 222 is coupled to the system memory 212 and the packet processor 223, and can establish the mapping relationship between the command queue descriptors and the hardware command queue descriptors according to the command queue descriptor list QList, and perform data packet processing operations.

[0017] It is worth mentioning that, in this embodiment of the invention, the command processor can set or unmap the mapping relationship between each command queue descriptor and one of the hardware command queue descriptors through firmware. Firmware is embedded software and an indispensable part of the hardware device. Through the execution of firmware, the basic functions of the hardware device can be controlled and managed. The firmware is loaded and executed when the hardware device starts up, providing low-level control code and interfaces to enable the hardware device to operate normally.

[0018] Here, the establishment of the mapping relationship between the command queue descriptor and the hardware command queue descriptor can be carried out through the aforementioned round robin scheduling mechanism or priority scheduling mechanism, so that each command queue descriptor can generate a corresponding mapping relationship with one of the multiple hardware command queue descriptors.

[0019] Please refer to Figure 3 here, which shows a schematic diagram of an embodiment of the command queue scheduling method of the present invention. The command queue scheduling method of the present invention can be implemented by executing firmware or software. However, performing scanning operations on a large number of command queue descriptors will result in a large number of memory read operations. Based on the above, in this embodiment, the command processor can, through software or firmware, set the active bit stream ACTBIT corresponding to the command queue descriptors MCQD0~MCQD1023 during system initialization. The active bit stream ACTBIT includes multiple activity data corresponding to the command queue descriptors MCQD0~MCQD1023 respectively. The activity data corresponding to each command queue descriptor MCQD0~MCQD1023 can be used to indicate whether the corresponding command queue descriptor is active, on going, or inactive. In this embodiment, each activity data point may have two bits to represent three states of each command queue descriptor. For example, activity data can be 1, 1 to represent that the corresponding command queue descriptor is active; activity data can be 1, 0 to represent that the corresponding command queue descriptor is continuously executing; and activity data can be 0, 1 to represent that the corresponding command queue descriptor is inactive. Of course, the correspondence between the logical values ​​of the activity data and the states of the command queue descriptors can be arbitrarily set by the designer. The above description is only an example for illustrative purposes and is not intended to limit the scope of the present invention.

[0020] Regarding the status setting of activity data, the command processor can detect whether command packets are written to the command queue, and when a command packet is detected being written to the command queue, set the activity data corresponding to the command queue descriptor to "active". Furthermore, before starting a scheduling cycle, the command processor can detect whether command packets in the command queue have been removed. If the command packet has not been removed, the command processor can set the activity data of the corresponding command queue descriptor to "continuously executing". If the command packet has been removed, the command processor can set the activity data of the corresponding command queue descriptor to "inactive".

[0021] In the above operation, when the operation is based on the firmware, the command processor can set a scheduling timer operation and start the scheduling work of the queue according to the scheduling timer operation.

[0022] On the other hand, during the initialization of the scheduling action, based on software operation, the command processor can initialize multiple bits in the active bit stream ACTBIT through the Kernel Mode Drive (KMD) program during the initialization action. Alternatively, based on firmware operation, the command processor can transmit a pseudo-binding command to the corresponding hardware through the KMD program during the initialization action, and enable multiple bits in the active bit stream ACTBIT by executing the pseudo-binding command through the firmware without binding any hardware. Here, the so-called pseudo-binding command refers to notifying the hardware and enabling multiple bits in the active bit stream ACTBIT through a command without performing any actual binding action on the hardware. In this way, it is not necessary to define a new notification command.

[0023] On the other hand, the command processor can release multiple command queues in system memory via software or firmware. Before releasing the command queues, the command processor can use a kernel-mode driver to check whether each command queue still contains command packets. Based on software operation, the command processor can use a kernel-mode driver to reset the active command queue descriptors MCQD0~MCQD1023 and the corresponding active bit stream ACTBIT's active data. Under firmware operation, the command processor can directly reset the command queue descriptors MCQD0~MCQD1023 via a kernel-mode driver.

[0024] Furthermore, the command processor can send release commands to the hardware via the kernel-mode driver, thereby releasing the corresponding hardware command queue descriptors. Specifically, in software-based operation, the command processor can execute the release command sent by the kernel-mode driver to reset the bound command queue descriptors MCQD0~MCQD1023. In firmware-based operation, the command processor executes the release command sent by the kernel-mode driver to reset the bound command queue descriptors MCQD0~MCQD1023 and the corresponding multiple activity data ACTBITs of the activity bit stream.

[0025] According to the above, the command queue scheduling method of the present invention enables a relatively large number of command queue descriptors and a relatively small number of hardware command queue descriptors to establish or demap between each command queue descriptor and the hardware command queue descriptor through software or firmware operations. In this way, the mapping relationship between multiple command queue descriptors and hardware command queue descriptors can be dynamically adjusted, effectively improving the computing performance of the artificial intelligence system.

[0026] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention. [Simplified Explanation of the Diagram]

[0007] Figure 1 shows a flowchart of the command queue scheduling method according to an embodiment of the present invention. Figure 2 shows a schematic diagram of an artificial intelligence system according to an embodiment of the present invention. Figure 3 shows a schematic diagram of one implementation of the command queue scheduling method according to an embodiment of the present invention.

Claims

1. A method for scheduling command queues, comprising: This enables the device driver on the system side to perform management actions on multiple command queue descriptors; The device driver transmits a list of command queue descriptors corresponding to the plurality of command queue descriptors to the command processor; And the command processor, based on a preset scheduling mechanism, adjusts the mapping relationship between each of the plurality of command queue descriptors and one of the plurality of hardware command queue descriptors according to the list of command queue descriptors, wherein the plurality of command queue descriptors are maintained by software and each of the plurality of hardware command queue descriptors points to the corresponding hardware processing routine.

2. The scheduling method as described in Request 1, wherein the scheduling mechanism is a round-robin scheduling mechanism or a priority scheduling mechanism.

3. The scheduling method as described in claim 1, wherein the step of adjusting the mapping relationship between each of the plurality of command queue descriptors and one of the plurality of hardware command queue descriptors includes: The command processor enables the firmware to set or remove the mapping relationship between each of the plurality of command queue descriptors and one of the plurality of hardware command queue descriptors.

4. The scheduling method as described in claim 1, wherein the step of causing the device driver on the system side to perform management actions of the plurality of command queue descriptors includes: The device driver can plan multiple command queues in the system memory to store the multiple command queue descriptors respectively.

5. The scheduling method as described in claim 4 further includes causing the device driver to store the list of command queue descriptors into the system memory.

6. The scheduling method as described in request item 1 further includes: During system initialization, the command processor sets an activity bit stream corresponding to the plurality of command queue descriptors via software or firmware. The activity bit stream includes a plurality of activity data corresponding to the plurality of command queue descriptors.

7. The scheduling method as described in claim 6, wherein each of the plurality of activity data is used to indicate whether the corresponding plurality of command queue descriptors are active, continuously executing, or inactive.

8. The scheduling method as described in claim 7 further includes: Check if any command packets have been written to the command register; If so, then set the corresponding multiple command queue descriptors as active; Before starting the scheduling cycle, check whether the command packets in the command register have been discharged; If the command packet portion is discharged, the corresponding multiple command queue descriptors are set to be continuously executed. And if all the command packets are ejected, the corresponding command queue descriptors are set to inactive.

9. The scheduling method as described in claim 1, wherein the command processor is configured to set a scheduling timing operation, and the scheduling method is initiated based on the scheduling timing operation.

10. The scheduling method as described in claim 6 further includes: Based on the software, during the initialization process, multiple bits in the active bit stream are initialized through the kernel mode driver.

11. The scheduling method as described in claim 6 further includes: Based on the firmware, during the initialization process, a pseudo-binding command is transmitted to the corresponding hardware through the kernel mode driver. Without binding any hardware, the firmware enables multiple bits in the active bit stream by executing the pseudo-binding command.

12. The scheduling method as described in request item 1 further includes: The command processor can release multiple command queues in system memory via software or firmware.

13. The scheduling method as described in claim 12 further includes: Before releasing the plurality of command queues, the kernel mode driver is used to confirm whether each of the plurality of command queues still contains command packets.

14. The scheduling method as described in claim 12 further includes: Based on the software, the kernel mode driver is used to reset the plurality of command queue descriptors and the plurality of activity data of the corresponding activity bit streams in the activity. Alternatively, based on the firmware, the multiple command queue descriptors can be reset via a kernel-mode driver.

15. The scheduling method as described in claim 12 further includes: Based on the software, the release command sent by the core mode driver is executed to reset the bound command queue descriptors. Alternatively, based on the firmware, a release command sent by the kernel mode driver can be executed to reset the bound command queue descriptors and the corresponding activity data of the activity bit streams.

16. An artificial intelligence system, comprising: The system side includes device drivers and system memory; A command processor is coupled to the system. The device driver on the system is used to perform management actions on multiple command queue descriptors. The device driver transmits a list of command queue descriptors corresponding to the multiple command queue descriptors to the command processor. The command processor, based on a preset scheduling mechanism, adjusts the mapping relationship between each of the multiple command queue descriptors and one of the multiple hardware command queue descriptors according to the list of command queue descriptors. The multiple command queue descriptors are maintained by software, and each of the multiple hardware command queue descriptors points to a corresponding hardware processing routine.

Images