Command queue scheduling method and artificial intelligence system
By dynamically adjusting the mapping relationship between command queue descriptor and hardware command queue descriptor in artificial intelligence systems, the problem of system performance in the one-to-one mapping of command queues and hardware command queues in the prior art is solved, and more efficient computing performance.
Patent Information
- Application Number
- CN202411124482.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-15
- Publication Date
- 2024-11-26
AI Technical Summary
The one-to-one mapping between command queues and hardware command queues in existing artificial intelligence systems seriously limits the system's operational efficiency. Adding command queues may lead to oversubscriptions and time delays, reducing the efficiency of system resources usage.
By managing multiple command queue descriptors on the system side device driver and transmitting their list to the command processor, adjusting the mapping relationship between the command queue descriptor and the hardware command queue descriptor according to the preset scheduling mechanism, and dynamically adjusting the mapping Relationships to improve computing efficiency.
It effectively improves the computing efficiency of the artificial intelligence system, avoids resource oversubscription and time delay, and improves the efficiency of system resources use.
Smart Images

Figure CN119025241A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence, and in particular to a command queue scheduling method and an artificial intelligence system that can improve the computing efficiency of an artificial intelligence chip. Background Art
[0002] In the related art, the command queue descriptor in the artificial intelligence system and the hardware command queue descriptor always exist in a one-to-one manner. This situation severely limits the operational efficiency of the artificial intelligence system. Improving system performance by adding multiple command queues may also cause oversubscription of system resources, generate additional processing time delays, and generate idle resources, which in turn reduces the efficiency of system resource utilization. Summary of the invention
[0003] The present invention is directed to a command queue scheduling method and an artificial intelligence system, which can improve the computing efficiency of an artificial intelligence chip.
[0004] According to an embodiment of the present invention, a scheduling method for a command queue includes: enabling a device driver on the system side to execute management actions of multiple command queue descriptors; enabling the device driver to transmit a command queue descriptor list corresponding to the multiple command queue descriptors to a command processor; and enabling the command processor to adjust the mapping relationship between each command queue descriptor and one of the multiple hardware command queue descriptors according to the command queue descriptor list based on a preset scheduling mechanism.
[0005] According to an embodiment of the present invention, an artificial intelligence system includes a system end and a command processor. The system end has a device driver and a system memory. The command processor is coupled to the system end, wherein the device driver of the system end is used to perform management actions of multiple command queue descriptors, the device driver transmits a command queue descriptor list corresponding to the multiple command queue descriptors to the command processor, and the command processor adjusts the mapping relationship between each command queue descriptor and one of the multiple hardware command queue descriptors based on a preset scheduling mechanism and the command queue descriptor list.
[0006] According to the above, the command queue scheduling method of the present invention enables the command processor to adjust the mapping relationship between each command queue descriptor and one of the multiple hardware command queue descriptors based on a preset scheduling mechanism. Thus, the multiple command queue descriptors can dynamically adjust the mapping relationship with the hardware command queue descriptors, which can effectively improve the computing performance of the artificial intelligence system. BRIEF DESCRIPTION OF THE DRAWINGS
[0007] Figure 1 A flowchart of a command queue scheduling method according to an embodiment of the present invention is shown.
[0008] Figure 2 A schematic diagram showing an artificial intelligence system according to an embodiment of the present invention.
[0009] Figure 3 A schematic diagram showing an implementation of a command queue scheduling method according to an embodiment of the present invention. DETAILED DESCRIPTION
[0010] Reference will now be made in detail to exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals are used in the drawings and the description to refer to the same or like parts.
[0011] Please refer to Figure 1 , Figure 1 The flowchart of the command queue scheduling method of the embodiment of the present invention is shown. Figure 1 The scheduling method of the command queue of the embodiment can be applied to an artificial intelligence system. In step S110, the device driver of the system end can perform management actions of multiple command queue descriptors. Among them, the device driver of the system end can execute the establishment of one or more command queues on the memory of the system end. The command queue can be used to store command queue descriptors. In this embodiment, the command queue descriptor can be a memory command queue descriptor (Memory Command Queue Description, MCQD). In addition, the device driver of the system end can also perform modification of command queues and command queue descriptors, as well as removal actions. It is worth mentioning that in this embodiment, even if there is no data packet in the command queue, the removal of the command queue and the command queue descriptor can only be performed by the user through the device driver of the system end. In this embodiment, the system end can be the central processing unit (CPU) side.
[0012] In step S120, the device driver transmits a command queue descriptor list corresponding to the plurality of command queue descriptors to a command processor. The command processor may be disposed in an artificial intelligence chip, such as a graphics processing chip. In this embodiment, the device driver stores the established command queue descriptor list in a system memory.
[0013] In step S130, the command processor may adjust the mapping relationship between each command queue descriptor and one of the multiple hardware command queue descriptors based on the command queue descriptor list based on the preset scheduling mechanism. Among them, the command queue descriptor is a memory command queue descriptor and is maintained by software. It can be stored in the memory. The hardware command queue descriptor is maintained by hardware. Among them, each hardware command queue descriptor points to a corresponding hardware handler. In implementation, the number of command queue descriptors is much higher than the hardware command queue descriptor. Therefore, it is necessary to adjust the mapping relationship between each hardware command queue descriptor and each command queue descriptor through a scheduling mechanism. In detail, in the preset scheduling mechanism, the command processor may generate a mapping relationship between each command queue descriptor and one of the hardware command queue descriptors, or the command processor may release the established mapping relationship between each command queue descriptor and each hardware command queue descriptor. In this embodiment, the command processor may be a graphics processing unit (GPU) side.
[0014] The command queue descriptor in this embodiment is a complete service queue descriptor for all users on the CPU side. The hardware command queue descriptor is a queue descriptor provided by the GPU hardware resources. Usually, the number of command queue descriptors will far exceed the number of hardware command queue descriptors, and there will be overbooking, so the scheduling and binding work of this embodiment is required.
[0015] In addition, the above-mentioned scheduling mechanism may be, for example, a round robin scheduling mechanism or a priority scheduling mechanism. Under the round robin scheduling mechanism, the command processor may formulate a certain execution order (run list), and determine whether there is a data packet in the command queue corresponding to the current time slot according to the set execution order, and accordingly establish a mapping relationship between the command queue and the hardware command queue descriptor. Under the priority scheduling mechanism, the command processor may select the first command queue with a relatively high priority in the command queue to establish a mapping relationship with the hardware command queue descriptor. If the hardware command queue descriptor has a mapping relationship with the second command queue with a relatively low priority, the command processor may release the mapping relationship between the hardware command queue descriptor and the second command queue, and establish a mapping relationship between the first command queue and the hardware command queue descriptor. The above-mentioned time interval is also a scheduling time slice, and each scheduling time slice will process a queue bound to the time slice.
[0016] Incidentally, in this embodiment, the command processor is also used to monitor the hardware command queue, process the dependency between queue scheduling and command packets, and transmit ready tasks to the command dispatcher. The command dispatcher is used to dispatch tasks to corresponding hardware and make the hardware execute the corresponding tasks.
[0017] Please refer to Figure 2 , Figure 2 A schematic diagram of an artificial intelligence system according to an embodiment of the present invention is shown. The artificial intelligence system 200 includes a system end 210 and a command processor 220. The system end 210 has a device driver 211 and a system memory 212. The system memory 212 can be a memory in any form. The device driver 211 can be used to perform management actions of command queue descriptors. Among them, the device driver 211 can plan a plurality of command queues QA~QC in the system memory 212, and the command queues QA~QC can store different command queue descriptors A~C respectively, and establish a command queue descriptor list Qlist according to the planned command queues QA~QC. Among them, the device driver 211 can store the command queue descriptor list Qlist in the system memory 212. The command queue descriptor list Qlist records the position information A*~C* corresponding to the command queues QA~QC. The position information A*~C* can be different queue pointers.
[0018] The command processor 220 adjusts the mapping relationship between each command queue descriptor and one of the plurality of hardware command queue descriptors according to the command queue descriptor list QList based on a preset scheduling mechanism. In the present embodiment, the command processor 220 has a driver 221, a hardware scheduler 222, and a packet processor 223. The driver 221 can access the system memory 212 and read or write the command queue descriptor list Qlist. The hardware scheduler 222 is coupled to the system memory 212 and the packet processor 223, and can establish a mapping relationship between the command queue descriptor and the hardware command queue descriptor according to the command queue descriptor list Qlist, and perform processing actions on the data packets.
[0019] It is worth mentioning that in an embodiment of the present invention, the command processor 220 can set or release the mapping relationship between each command queue descriptor and one of the multiple hardware command queue descriptors through firmware. Among them, firmware is an embedded software and an indispensable part of the hardware device. Through the execution of firmware, the basic functions of the hardware device can be controlled and managed. The firmware is loaded and executed when the hardware device is started, providing the underlying control code and interface so that the hardware device can operate normally.
[0020] Here, the mapping relationship between the command queue descriptor and the hardware command queue descriptor can be established through the aforementioned round robin scheduling mechanism or priority scheduling mechanism, so that each command queue descriptor can generate a corresponding mapping relationship with one of the multiple hardware command queue descriptors.
[0021] Please refer to this Figure 3 , Figure 3 A schematic diagram of an implementation of a method for scheduling a command queue in an embodiment of the present invention is shown. Among them, in the method for scheduling a command queue in an embodiment of the present invention, it can be implemented by executing firmware or software. However, performing a scanning action on a large number of command queue descriptors will cause a large number of memory read operations. Based on the above, in this embodiment, the command processor can set the activity bit stream ACTBIT corresponding to the command queue descriptors MCQD0~MCQD1023 through software or firmware when the system is initialized. The activity bit stream ACTBIT includes a plurality of activity data corresponding to the command queue descriptors MCQD0~MCQD1023 respectively. Among them, the respective activity data corresponding to each command queue descriptor MCQD0~MCQD1023 can be used to indicate that the corresponding command queue descriptor is active (in active), continuously executing (on going) or inactive (non active). In this embodiment, each activity data may have 2 bits to represent three states of each command queue descriptor. For example, the activity data may be 1, 1 to represent that the corresponding command queue descriptor is active; the activity data may be 1, 0 to represent that the corresponding command queue descriptor is continuously executing; the activity data may be 0, 1 to represent that the corresponding command queue descriptor is inactive. Of course, the corresponding relationship between the logic value of the above activity data and the state of the command queue descriptor can be arbitrarily set by the designer. The above description is only an example for explanation and is not intended to limit the scope of the present invention.
[0022] In terms of setting the status of the activity data, the command processor can detect whether a command packet is written into the command queue, and when it is detected that a command packet is written into the command queue, the activity data corresponding to the command queue descriptor is set to active. In addition, before starting the scheduling cycle, the command processor can detect whether the command packet in the command queue is discharged. If the command packet is partially discharged, the command processor can set the activity data of each corresponding command queue descriptor to continue to execute. If the command packet is completely discharged, the command processor can set the activity data of each corresponding command queue descriptor to inactive.
[0023] In the above operation, when operating based on firmware, the command processor may set a scheduling timing operation, and start the scheduling work of the queue according to the scheduling timing operation, that is, the scheduling method of the command queue of the embodiment of the present invention.
[0024] On the other hand, in the initialization action of the scheduling action, based on the operation of the software, the command processor can initialize multiple bits in the active bitstream ACTBIT through the kernel mode driver (KMD) program in the initialization action. In addition, if it is based on the operation of the firmware, the command processor can transmit a pseudo-binding command to the corresponding hardware through the kernel mode driver in the initialization action, and enable multiple bits in the active bitstream ACTBIT by executing the pseudo-binding command through the firmware without binding any hardware. Here, the so-called pseudo-binding command means that no actual binding action is performed on the hardware, but only the hardware is notified through the command, and multiple bits in the active bitstream ACTBIT are enabled. In this way, there is no need to define a new notification command.
[0025] On the other hand, the command processor can release multiple command queues in the system memory through software or firmware. Before releasing the command queue, the command processor can confirm whether there are still command packets in each command queue through the kernel mode driver. Based on software operation, the command processor can reset each active command queue descriptor MCQD0~MCQD1023 and each activity data of the corresponding activity bitstream ACTBIT through the kernel mode driver. Under firmware-based operation, the command processor can directly reset the command queue descriptors MCQD0~MCQD1023 through the kernel mode driver.
[0026] In addition, the command processor can send a release command to the hardware end through the kernel mode driver, and release the corresponding hardware command queue descriptor. In the software-based operation, the command processor can execute the release command sent by the kernel mode driver to reset the bound command queue descriptors MCQD0-MCQD1023. In the firmware-based operation, the command processor resets the bound command queue descriptors MCQD0-MCQD1023 and the corresponding activity data ACTBIT of the activity bit stream by executing the release command sent by the kernel mode driver.
[0027] According to the above, the command queue scheduling method of the present invention can enable a relatively large number of command queue descriptors and a relatively small number of hardware command queue descriptors to establish or release a mapping relationship between each command queue descriptor and a hardware command queue descriptor through the operation of software or firmware. In this way, multiple command queue descriptors can dynamically adjust the mapping relationship with the hardware command queue descriptor, which can effectively improve the computing performance of the artificial intelligence system.
[0028] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for scheduling a command queue, characterized in that: include: Enable a device driver on the system side to perform management actions of a plurality of command queue descriptors; causing the device driver to transmit a command queue descriptor list corresponding to the plurality of command queue descriptors to a command processor; as well as The command processor is enabled to adjust the mapping relationship between each of the command queue descriptors and one of the plurality of hardware command queue descriptors according to the command queue descriptor list based on a preset scheduling mechanism.
2. The scheduling method according to claim 1, characterized in that: The scheduling mechanism is a polling scheduling mechanism or a priority scheduling mechanism.
3. The scheduling method according to claim 1, characterized in that: The step of adjusting the mapping relationship between each of the command queue descriptors and one of the plurality of hardware command queue descriptors comprises: The command processor is enabled to set or release the mapping relationship between each of the command queue descriptors and one of the plurality of hardware command queue descriptors through firmware.
4. The scheduling method according to claim 1, characterized in that: The step of causing the device driver of the system end to perform the management action of the plurality of command queue descriptors comprises: The device driver is enabled to plan a plurality of command queues in the system memory to respectively store different command queue descriptors.
5. The scheduling method according to claim 4, characterized in that: The device driver is caused to store the command queue descriptor list in the system memory.
6. The scheduling method according to claim 4, characterized in that: Also includes: When the system is initialized, the command processor is enabled to set an activity bit stream corresponding to the plurality of command queue descriptors through software or firmware, wherein the activity bit stream includes a plurality of activity data respectively corresponding to the plurality of command queue descriptors.
7. The scheduling method according to claim 6, characterized in that: Each of the activity data is used to indicate whether the corresponding command queue descriptor is active, continuously executing, or inactive.
8. The scheduling method according to claim 7, characterized in that: Also includes: Check whether a command packet is written into the command queue; If yes, setting the activity data corresponding to the command queue descriptor to be in the activity; Before starting the scheduling cycle, detecting whether the command packet in the command queue is discharged; If the command packet part is discharged, setting the activity data of each corresponding command queue descriptor to the continuous execution; as well as If all the command packets are discharged, the activity data of the corresponding command queue descriptors are set to inactive.
9. The scheduling method according to claim 1, characterized in that: The command processor is enabled to set a scheduling timing operation, and the scheduling method is started according to the scheduling timing operation.
10. The scheduling method according to claim 6, characterized in that: Also includes: Based on the software, in an initialization action, a plurality of bits in the active bitstream are initialized by a kernel mode driver.
11. The scheduling method according to claim 6, characterized in that: Also includes: Based on the firmware, in an initialization action, a pseudo-binding command is transmitted to corresponding hardware through a kernel mode driver; Under the premise of not binding any hardware, the firmware executes the pseudo binding command to enable a plurality of bits in the active bitstream.
12. The scheduling method according to claim 6, characterized in that: Also includes: The command processor is caused to release the plurality of command queues in the system memory through the software or the firmware.
13. The scheduling method according to claim 12, characterized in that: Also includes: Before releasing the plurality of command queues, a kernel mode driver is used to confirm whether each of the command queues still has a command packet.
14. The scheduling method according to claim 12, characterized in that: Also includes: Based on the software, resetting each of the command queue descriptors in the activity and each of the activity data of the corresponding activity bitstream through a kernel mode driver; or Based on the firmware, the plurality of command queue descriptors are reset by a kernel mode driver.
15. The scheduling method according to claim 12, characterized in that: Also includes: Based on the software, executing a release command sent by a kernel mode driver to reset each of the bound command queue descriptors; or Based on the firmware, a release command sent by a kernel mode driver is executed to reset each of the bound command queue descriptors and each of the activity data of the corresponding activity bitstream.
16. An artificial intelligence system comprising: The system side has device drivers and system memory; as well as A command processor coupled to the system end, Among them, the device driver on the system side is used to perform management actions of multiple command queue descriptors. The device driver transmits a command queue descriptor list corresponding to the multiple command queue descriptors to the command processor. The command processor adjusts the mapping relationship between each of the command queue descriptors and one of the multiple hardware command queue descriptors based on the command queue descriptor list based on a preset scheduling mechanism.
