Maintaining quality of service for client applications with workloads executed by compute cores or hardware accelerators.

Abstract

This document describes a system and method for maintaining quality of service for client applications having workloads executed by compute cores or hardware accelerators. It also describes a method for operating a hardware accelerator configured to process commands submitted to it by a client application, wherein the workload is executable by compute cores or hardware accelerators. This method includes queuing commands for the execution of a workload in a first command queue set and queuing commands for the execution of a workload in a second command queue set. The method includes the workload processor executing the workloads specified by the commands in the first and second command queue sets in an execution order determined based on the output of a set of trackers configured to track one or more criteria for the selected command queue sets.

Claims

1. A method for operating a hardware accelerator configured to process commands directly submitted to the hardware accelerator by a client application, wherein the workload corresponding to the command is executable by a compute core accessible by the client application or by the hardware accelerator, and the method is In the first command queue set, commands for executing the workload are queued, In the second command queue set, commands for executing the workload are queued, The workload processor associated with the hardware accelerator executes the workload specified by the commands in the first command queue set and the second command queue set in an execution order determined based on the output of a set of trackers configured to track one or more criteria for a command queue set selected from the first command queue set and the second command queue set. Methods that include...

2. The method according to claim 1, wherein a first subset of the selected command queue set has an adjustable priority, and a second subset of the selected command queue set has a fixed priority.

3. The method according to claim 2, further comprising adjusting the priority associated with the first subset of the selected command queue set based on the output of one or more trackers in the set of trackers.

4. The method according to claim 1, wherein the output from a separate tracker in a first subset of the tracker set is used in a first stage of arbitration for each command queue in the subset of selected command queues.

5. The method according to claim 4, wherein, for each virtual function or group of virtual functions, a separate tracker from the second subset of the tracker set is used in the second stage of arbitration for each physical function associated with the hardware accelerator.

6. The method according to claim 5, wherein the output from a separate tracker in the set of trackers for the hardware accelerator is used for managing the workload at the chip level.

7. The method according to claim 1, wherein the one or more criteria include at least one of the following: (1) the absolute or relative use of resources by one or more command queues included in the selected command queue set; (2) the minimum performance of executing commands included in the selected command queue set; or (3) the absolute or relative use of the hardware accelerator by the selected command queue set.

8. An accelerator portal for enabling multiple client applications to access one or more shared hardware accelerators, wherein each of the multiple client applications can execute a workload using compute cores or using one of the multiple shared hardware accelerators; From among the multiple shared hardware accelerators, a hardware accelerator configured to queue commands for executing the workload in the first command queue set, The hardware accelerator is further configured to queue commands for executing workloads in a second command queue set, A workload processor associated with the hardware accelerator and configured to execute the workloads specified by the commands in the first command queue set and the second command queue set in an execution order determined based on the output of a set of trackers configured to track one or more criteria for a command queue set selected from the first command queue set and the second command queue set, A system that includes this.

9. The system according to claim 8, wherein a first subset of the selected command queue set has an adjustable priority, and a second subset of the selected command queue set has a fixed priority.

10. The system according to claim 9, further configured to allow adjustment of the priority associated with the first subset of the selected command queue set based on the output of one or more trackers in the set of trackers.

11. The system according to claim 8, wherein the output from a separate tracker in a first subset of the tracker set is used in the first stage of arbitration for each command queue in the subset of selected command queues.

12. The system according to claim 11, wherein, for each virtual function or group of virtual functions, a separate tracker from a second subset of the tracker set is used in a second stage of arbitration for each physical function associated with the hardware accelerator.

13. The system according to claim 12, wherein the output from a separate tracker in the set of trackers for the hardware accelerator is used for managing the workload at the chip level.

14. The system according to claim 8, wherein the one or more criteria include at least one of the following: (1) the absolute or relative use of resources by one or more command queues included in the selected command queue set; (2) the minimum performance of executing commands included in the selected command queue set; or (3) the absolute or relative use of the hardware accelerator by the selected command queue set.

15. A method for operating a hardware accelerator configured to process commands directly submitted to the hardware accelerator by a client application, wherein the workload corresponding to the command is executable by a compute core accessible by the client application or by the hardware accelerator, and the method is Queueing commands for workload execution in a first command queue set, wherein each command queue in the first command queue set is associated with one or more virtual functions from a set of virtual functions for physical functions associated with the hardware accelerator, and a first tracker set configured to track one or more criteria from a first set of criteria for a first selected command queue set from the first command queue set is allocated by the virtual machine to a first stage of arbitration. Queueing commands for workload execution in a second command queue set, wherein a first subset of the second command queue set is associated with the virtual function set for the physical function, and a second tracker set configured to track one or more criteria from a second set of criteria for a second selected command queue set from the second command queue set is assigned to a second stage of arbitration by a system agent. The workload processor associated with the hardware accelerator executes the workload specified by the commands in the first command queue set and the second command queue set in an execution order determined based on tracking by the first tracker set and a subset of the second tracker set. Methods that include...

16. The method according to claim 15, wherein a first subset of the second selected command queue set has an adjustable priority, and a second subset of the second selected command queue set has a fixed priority.

17. The method according to claim 16, further comprising adjusting the priority associated with the first subset of the second selected command queue set based on the output of one or more trackers from the second set of trackers.

18. The method according to claim 15, wherein the first reference set includes absolute or relative resource usage by command queues included in the first selected command queue set.

19. The method according to claim 16, wherein the second reference set includes the maximum total resource usage or maximum relative resource usage by virtual functions included in the second selected command queue set.

20. The method according to claim 15, wherein the first set of trackers is for tracking arbitration criteria in a first stage of arbitration, the second set of trackers is for tracking arbitration criteria in a second stage of arbitration, and the execution order is determined based on the first stage of arbitration and the second stage of arbitration.

Images