Device comprising accelerator and method for controlling accelerator

Abstract

An apparatus including an accelerator and a method for controlling an accelerator are disclosed. The interface may receive a frequency control signal from the processor. The control circuit may set a frequency of the circuit based at least in part on a frequency control signal from the processor. The circuitry may process data based at least in part on a data processing command from the processor.

Description

[0001] This application claims the benefit of U.S. Provisional Patent Application Serial No. 63 / 606,091, filed on December 4, 2023, and U.S. Application No. 18 / 795,159, filed on August 5, 2024, which are incorporated herein by reference for all purposes. Technical Field

[0002] The disclosure relates generally to accelerators, and more particularly to managing loads of artificial intelligence (AI) accelerators. Background Art

[0003] The use of accelerators may be particularly involved when processing large data models for artificial intelligence (AI) or machine learning (ML). The processing operations may involve processing large amounts of data, or performing multiple calculations on specific data. Due to varying workloads, efficient operation of accelerators may be difficult to achieve.

[0004] There is still a need to manage accelerator loads efficiently. Summary of the invention

[0005] The accelerator may receive a frequency control signal from software. The accelerator may use the frequency control signal to control the frequency of the accelerator.

[0006] According to a disclosed embodiment, there is provided an apparatus including an accelerator, the accelerator including: an interface for receiving a frequency control signal from a processor; a circuit for processing data based at least in part on a data processing command from the processor; and a control circuit for setting a frequency of the circuit based at least in part on the frequency control signal from the processor.

[0007] According to a disclosed embodiment, a method for controlling the frequency of an accelerator is provided, the method comprising: receiving a frequency control signal from a processor at a control circuit of the accelerator, the processor being outside the accelerator; and applying the frequency control signal to a circuit of the accelerator through the control circuit, the circuit of the accelerator being configured to process data based at least in part on a data processing command from the processor.

[0008] According to a disclosed embodiment, a system is provided that includes a non-transitory storage medium that stores instructions on the non-transitory storage medium, which, when executed by a machine, causes: receiving a frequency control signal from a processor at a control circuit of an accelerator, the processor being external to the accelerator; and applying the frequency control signal to a circuit of the accelerator through the control circuit, the circuit of the accelerator being configured to process data based at least in part on a data processing command from the processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The drawings described below are examples of how the disclosed embodiments may be implemented and are not intended to limit the disclosed embodiments. The various embodiments disclosed may include elements not shown in a particular drawing and / or may omit elements shown in a particular drawing. The drawings are intended to provide illustrations and may not be to scale.

[0010] Figure 1 A machine including an accelerator according to a disclosed embodiment is shown.

[0011] Figure 2 Showing the embodiment according to the disclosure Figure 1 Details of the machine.

[0012] Figure 3 Showing the embodiment according to the disclosure Figure 1 Details of the accelerator.

[0013] Figure 4 According to the disclosed embodiment, Figure 1 The software running on the processor receives the signal Figure 1 accelerator.

[0014] Figure 5 Showing the embodiment according to the disclosure Figure 1 Accelerator usage Figure 4 Flowchart of an example processing of a frequency control signal.

[0015] Figure 6 Showing the embodiment according to the disclosure Figure 1 Accelerator usage Figure 4 The frequency control signal is used to override the Figure 3 The proportional-integral-derivative (PID) control loop Figure 4 Flowchart of an example processing of an internal frequency control signal.

[0016] Figure 7 Showing the embodiment according to the disclosure Figure 1 The accelerator applies Figure 4 Flowchart of an example processing of a frequency control signal. DETAILED DESCRIPTION

[0017] Reference will now be made in detail to the disclosed embodiments, examples of which are shown in the accompanying drawings. In the following detailed description, many specific details are set forth to enable a thorough understanding of the disclosure. However, it should be understood that one of ordinary skill in the art may practice the disclosure without these specific details. In other cases, well-known methods, processes, components, circuits, and networks are not described in detail to avoid unnecessarily obscuring aspects of the embodiments.

[0018] It will be understood that although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the scope of the disclosure, a first module may be referred to as a second module, and similarly, a second module may be referred to as a first module.

[0019] The terms used in the disclosed descriptions herein are only used for the purpose of describing specific embodiments and are not intended to limit disclosure. As used in the disclosed descriptions and the appended claims, unless the context clearly indicates otherwise, the singular form is intended to also include the plural form. It will also be understood that the term "and / or" as used herein represents and encompasses any and all possible combinations of one or more of the associated listed items. It will also be understood that the terms "including" and / or "comprising" when used in this specification specify the presence of stated features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their groups. The components and features of the accompanying drawings are not necessarily drawn to scale.

[0020] Processing large amounts of data, such as for artificial intelligence (AI) or machine learning (ML) models, can involve hardware using various accelerators. The workloads on these accelerators can vary: some workloads may be processing-intensive, while others may be data-intensive.

[0021] Operating an accelerator at its maximum frequency may produce the fastest results from the accelerator. However, operating the accelerator at its maximum frequency may also mean that the accelerator consumes the most power, which may be inefficient from a power management perspective. In addition, the accelerator may not need to operate at maximum power all the time. It may be expected that two or more accelerators will end their processing at the same time. When operating at maximum frequency, one accelerator may complete before another accelerator, thereby idling the accelerator until the other accelerator completes its processing. If the first accelerator is operated at a lower frequency, the first accelerator may consume less power and complete its operation in a manner roughly consistent with the other accelerator, thereby avoiding accelerator idling.

[0022] The accelerator may include a proportional integral derivative (PID) control loop that can adjust the operating frequency of the accelerator. Depending on how busy the accelerator is currently and what is considered a target busy level, the PID control loop may suggest smoothly adjusting the accelerator's frequency up or down to reach the target busy level. But the PID control loop is unaware of other accelerators and when they may complete their data processing: the PID control loop works for the accelerator in isolation. Furthermore, the PID control loop has no information about what the workload of the accelerator may be for the next iteration.

[0023] The disclosed embodiments solve these problems by introducing a frequency control signal from an AI workload runtime software running under an operating system. The AI ​​workload runtime software (or simply the AI ​​software) can handle the division and scheduling of work, and therefore can have information about the current and future workloads of both the accelerator and other accelerators. The AI ​​software can send frequency control signals to the accelerators to manage their power consumption and coordinate when the accelerators complete their processing. The disclosed embodiments can send frequency control signals based on the type of processing that the accelerator will perform, the number of vectors that the accelerator will process, or the dimension of the vectors that the accelerator will process (the number of coordinates in the vector). The disclosed embodiments can use the frequency control signal to override the internal frequency control signal from the PID, or to override the minimum or maximum frequency established for the accelerator.

[0024] Figure 1 A machine including an accelerator according to a disclosed embodiment is shown. Figure 1 In the example, machine 105 (machine 105 may also be referred to as a host or system) may include a processor 110 , a memory 115 , and a storage device 120 .

[0025] Processor 110 may be any type of processor. (For ease of illustration, processor 110 is shown outside the machine along with other components discussed below: the disclosed embodiments may include three components within the machine.) Although Figure 1 A single processor 110 is shown, but machine 105 may include any number of processors, each of which may be a single-core processor or a multi-core processor, each of which may execute a reduced instruction set computer (RISC) architecture or a complex instruction set computer (CISC) architecture (among other possibilities), and which may be mixed in any desired combination.

[0026] The processor 110 may be coupled to a memory 115. The memory 115 (memory 115 may also be referred to as main memory) may be any kind of memory (such as flash memory, dynamic random access memory (DRAM), static random access memory (SRAM), permanent random access memory, ferroelectric random access memory (FRAM), or non-volatile random access memory (NVRAM) (such as magnetoresistive random access memory (MRAM)), etc.). The memory 115 may also be any desired combination of different memory types and may be managed by a memory controller 125. The memory 115 may be used to store data that may be referred to as "short-term": that is, data that is not expected to be stored for a long period of time. Examples of short-term data may include temporary files, data used locally by an application (which may have been copied from other storage locations), etc.

[0027] The processor 110 and the memory 115 may also support an operating system under which various applications may run. These applications may issue requests (which may also be referred to as commands) to read data from or write data to the memory 115 or the storage device 120. The storage device 120 may be accessed using a device driver 130.

[0028] The storage device 120 may be associated with an accelerator (the accelerator may be the accelerator 135 or may be a different accelerator), which may also be referred to as a computing storage device, a computing storage unit, or a computing device. The storage device 120 and the accelerator may be designed and manufactured as a single integrated unit, or the accelerator may be separated from the storage device 120. The phrase "associated with..." is intended to cover a single integrated unit including both the storage device and the accelerator, as well as both the accelerator and the storage device that are paired but not manufactured as a single integrated unit. In other words, when the storage device and the accelerator are physically separate devices but are connected in a manner that enables them to communicate with each other, they may be referred to as "paired." In addition, in the remainder of this document, any reference to the storage device 120 may be understood as referring to the device as physically separated but paired (and therefore may include other devices), or integrating the two devices into a single component as a computing storage unit.

[0029] Furthermore, the connection between the storage device and the paired accelerator may enable the two devices to communicate, but may not enable one (or both) devices to work with different partners: that is, the storage device may not be able to communicate with the other accelerator, and / or the accelerator may not be able to communicate with the other storage device. For example, the storage device and the paired accelerator may be connected to the fabric serially (in either order) so that the accelerator accesses information from the storage device in a way that the other accelerator may not be able to achieve.

[0030] Although Figure 1 The generic term "storage device" is used, but the disclosed embodiments may include any storage device format that may be associated with a computing storage device, examples of which may include hard disk drives and solid-state drives (SSDs). Any reference below to a specific type of storage device (such as, "SSD") should be understood to include such other embodiments disclosed.

[0031] The processor 110 and the storage device 120 may be cross-structured ( Figure 1105) to communicate. The structure may be any structure along which information can be passed. Such a structure may include a structure that may be internal to the machine 105 and may use interfaces such as Peripheral Component Interconnect Express (PCIe), Serial Advanced Technology Attachment (SATA), or Small Computer System Interface (SCSI), as well as other interfaces. Such a structure may also include a structure that may be external to the machine 105 and may use interfaces such as Ethernet, InfiniBand or Fibre Channel, as well as other interfaces. In addition, such a structure may support one or more protocols such as Non-Volatile Memory Express (NVMe), NVMe over Fabric (NVMe-oF), Simple Service Discovery Protocol (SSDP), or Cache Coherent Interconnect Protocol (such as Compute Express Link Protocol® (CXL®)), as well as other protocols. (Compute Express Link and CXL are registered trademarks of the Compute Express Link Alliance in the United States.) Therefore, such a structure may be considered to cover both internal network connections and external network connections through which commands can be sent directly or indirectly to the storage device 120. In the disclosed embodiment where such a structure supports an external network connection, the storage device 120 may be located outside the machine 105, and the storage device 120 may receive requests from a processor remote from the machine 105.

[0032] Figure 1 An accelerator 135 is also shown. The accelerator 135 may be a stand-alone element, or the accelerator 135 may be included as part of another device, such as the storage device 120, a network interface card, or any other desired element. As with the processor 110, the accelerator 135 may implement a reduced instruction set computer (RISC) architecture or a complex instruction set computer (CISC) architecture (among other possibilities), and the accelerator 135 may be implemented using a central processing unit (CPU), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SoC), a graphics processing unit (GPU), a general purpose GPU (GPGPU), a neural processing unit (NPU), or a tensor processing unit (TPU), but the accelerator 135 may often implement a neural network. The accelerator 135 may be an accelerator designed to perform any processing that typically requires specialized processing, such as a neural network processor used with artificial intelligence (AI) computing tasks such as natural language processing. However, although Figure 1 An accelerator 135 is shown, but the disclosed embodiments may include any accelerator 135 that processes data, whether or not a neural network is used, and may be used to solve any desired problem, whether or not natural language processing is processed. For example, the disclosed embodiments may include an accelerator 135 designed to use a transfer model with repetition for image processing.

[0033] Although Figure 1Only one accelerator 135 is shown, but the disclosed embodiments may include any number (one or more) of accelerators 135. For example, two or more accelerators 135 may be configured to process data serially, so that the output of one accelerator 135 may be input to the next accelerator 135. Alternatively, the accelerators 135 may be configured to process data in parallel, and each accelerator 135 operates independently of the other accelerators. In either case, it may be more efficient for each accelerator 135 or all accelerators 135 to complete their processing at approximately the same time, in which case management of the accelerators 135 may be beneficial.

[0034] Figure 2 Showing the embodiment according to the disclosure Figure 1 Details of the machine. Figure 2 In the embodiment of the present invention, the machine 105 generally includes one or more processors 110, which may include a memory controller 125 and a clock 205, which may be used to coordinate the operation of the components of the machine. The processor 110 may also be coupled to a memory 115, which may include, by way of example, a random access memory (RAM), a read-only memory (ROM), or other state preservation medium. The processor 110 may also be coupled to a storage device 120 and a network connector 210, which may be, for example, an Ethernet connector or a wireless connector. The processor 110 may also be connected to a bus 215, to which a user interface 220 and input / output (I / O) interface ports may be attached, which may be managed using an I / O engine 225 and other components.

[0035] Figure 3 Showing the embodiment according to the disclosure Figure 1 Details of the accelerator 135. Figure 3 In the embodiment, the accelerator 135 may include an interface 305, a receiver 310, a proportional integral derivative (PID) control loop 315, a control circuit 320, and a processing circuit 325. The interface 305 may be used to connect the accelerator 135 to Figure 1 A processor 110, and Figure 1 Receiver 310 may receive a signal via interface 305 and may determine what the signal represents: for example, a command to process data, data to be processed, or (as described below with reference to Figure 4 PID control loop 315 (PID control loop 315 may also be referred to as a PID circuit, a PID controller, or simply PID) may generate an internal frequency control signal. Control circuit 320 may use the internal frequency control signal generated by PID control loop 315 (and as described below with reference to Figure 4The frequency control signal discussed above) determines (sets) the frequency at which the processing circuit 325 can operate. The processing circuit 325 (processing circuit 325 may also be referred to as a circuit) may determine (set) the frequency at which the processing circuit 325 can operate according to the frequency control signal from the accelerator 135. Figure 1 The processor 110 receives the data processing command to process the data.

[0036] The frequency (i.e., clock cycle) of processing circuit 325 is important for a number of reasons. First, there is a correlation between frequency and power consumption. The higher the frequency of processing circuit 325, the more power is consumed. Since minimizing the amount of power used by accelerator 135 can be a goal, it can be considered more efficient for accelerator 135 to operate at a lower frequency, even if operating accelerator 135 at a lower frequency means that calculations may take slightly longer. Second, if the frequency of processing circuit 325 is higher, the accelerator 135 will be more efficient. Figure 1 If there is more than one accelerator 135 in the machine 105, it may be useful to synchronize the operation of two accelerators 135 or all accelerators 135, with each accelerator starting and ending their processing of data processing commands at approximately the same time. This choice may be particularly important in the case where the accelerators 135 are in a producer-consumer relationship, as it is inefficient for a producer or consumer to sit idle waiting for another accelerator to become idle. That is, a producer should not generate data for a consumer faster than the consumer is ready to consume the data (in which case the producer may waste time idle), and a consumer should not consume data faster than the producer is ready to produce the data (in which case the consumer may waste time idle). By coordinating the operation of the two accelerators 135, no accelerator 135 may spend too much time idle, resulting in more efficient operation. Third, different types of data processing commands may represent different workloads. For example, a workload that may be described as memory-bound may involve processing data from a processor or a processor. Figure 1 The data processing command to read data from the memory 115 may depend to some extent on the Figure 1 The time required for accessing data from memory 115 is as follows. Since accessing data may require accessing data across structures Figure 1 The memory 115 is therefore Figure 1 Accessing data from memory 115 may be a relatively slow operation, so it may be more efficient for accelerator 135 to operate at a lower frequency. On the other hand, since the only limitation on operation is the speed of processing circuit 325, the computational constraints on accelerator 135 (e.g., Figure 3 The data processing commands that perform calculations on data in some local memory (not shown in the figure) can operate relatively quickly. Therefore, when the calculation is limited, it is more efficient for the accelerator 135 to operate at a higher frequency.

[0037] In addition, the actual frequency to be used by the accelerator 135 may depend on other factors. For example, the dimensionality of the data (i.e., the number of coordinates in the data vector) and the number of vectors in the data may be factors in calculating the appropriate frequency control signal. Similarly, the architecture of the accelerator 135 may be relevant to calculating the frequency control signal for the accelerator 135. Because the details of how the frequency control signal affects the accelerator 135 may depend on many factors, how a specific example frequency may be calculated for a specific accelerator 135 is not described herein.

[0038] Figure 4 According to the disclosed embodiment, Figure 1 The software running on the processor 110 receives the signal Figure 1 The accelerator 135. Figure 4 In the PID control loop 315, Figure 1 The current busyness level of the accelerator 135 together with Figure 1 The PID control loop 315 may then generate an internal frequency control signal 405 which may be sent to the control circuit 320. Note that the internal frequency control signal 405 may also be fed back to the Figure 1 The current frequency of the processing circuit 325 may affect the busyness level of the accelerator 135. Figure 1 The busy level of the accelerator is 135.

[0039] The control circuit 320 may then compare the internal frequency control signal 405 with a frequency minimum value (or minimum frequency or frequency lower limit) 410 and a frequency maximum value (or maximum frequency or frequency upper limit) 415. The frequency minimum value 410 and the frequency maximum value 415 may represent Figure 1 The lower and upper limits of the frequency of the accelerator 135 are determined by the control circuit 320. If the internal frequency control signal 405 is within the limits of the frequency minimum value 410 and the frequency maximum value 415, the control circuit 320 may apply the internal frequency control signal 405 as specified by the PID control loop 315; otherwise, the control circuit 320 may apply the frequency minimum value 410 or the frequency maximum value 415 as appropriate. The frequency minimum value 410 and / or the frequency maximum value 415 may be specified in any desired manner: for example, the frequency minimum value 410 and / or the frequency maximum value 415 may be specified from Figure 1 The firmware or other storage device in the accelerator 135 is read. Once the internal frequency control signal 405 has been adjusted to comply with the frequency minimum value 410 and the frequency maximum value 415, the control circuit 320 can send a frequency signal to the processing circuit 325 to determine the frequency that the processing circuit 325 should use.

[0040] Although Figure 1The operation of the accelerator 135 is appropriate, but Figure 1 The accelerator 135 lacks some useful information. For example, Figure 1 The processor 110 may support an operating system that may include AI workload runtime software (or referred to as AI workload software) 420, which may also be referred to as software 420. Software 420 may be responsible for dispatching data processing commands to Figure 1 accelerators 135. In this way, the software 420 may have information about the workload of the data processing commands: what data processing commands may be memory-bound versus compute-bound, what data processing commands may be part of a producer-consumer relationship with another accelerator 135, the dimensionality of the data and the number of vectors in the data, etc. The information may be available Figure 1 accelerator 135, so the software 420 may have additional information that can be used to determine the frequency control signal.

[0041] Therefore, if Figure 4 As shown in FIG. 4 , the software 420 may send a frequency control signal 425-1 to the control circuit 320. (The software 420 may also send another frequency control signal 425-2 to the control circuit 320.) Figure 1 4. The two frequency control signals 425-1 and 425-2 may specify the same frequency for the two accelerators, or the two frequency control signals 425-1 and 425-2 may be different. ) The control circuit 320 may apply the frequency upon receiving the frequency control signal 425-1. Thus, such application of the frequency control signal 425-1 may override the internal frequency control signal 405. Note that in some disclosed embodiments, the frequency control signal 425-1 may also exceed the limits of the frequency minimum 410 and / or the frequency maximum 415. For example, the frequency control signal 425-1 may be based at least in part on a first workload of the accelerator 135 and one of the first implementations of the accelerator 135. For example, the frequency control signal 425-1 may also be based at least in part on a second workload of the additional accelerator 135 and one of the second implementations of the additional accelerator 135. For example, a first workload of the accelerator 135 includes a first dimension of first data to be processed using the accelerator 135 or a first number of vectors of first data to be processed using the accelerator 135, and a second workload of the additional accelerator 135 includes a second dimension of second data to be processed using the additional accelerator 135 or a second number of vectors of second data to be processed using the additional accelerator 135. For example, the frequency control signal 425-1 may coordinate the accelerator 135 and the additional accelerator 135.

[0042] Although Figure 4Software 420 is included, but disclosed embodiments may include hardware circuitry that may send frequency control signals 425-1 and 425-2 to control circuitry 320. Assuming such hardware circuitry has access to the workload of the accelerator that the data processing commands may require, the hardware circuitry may be able to send frequency control signals 425-1 and 425-2 to control circuitry 320.

[0043] In some disclosed embodiments, the control circuit 320 may apply the frequency specified in the frequency control signal 425-1 for the duration of the data processing command being executed by the processing circuit 325. In other disclosed embodiments, the control circuit 320 may apply the frequency specified in the frequency control signal 425-1 until a new frequency is specified in a new frequency control signal 425-1, or until the software 420 de-asserts the frequency control signal 425-1. Once the control circuit 320 no longer applies the frequency control signal 425-1, the internal frequency control signal 405 from the PID control loop 315 may be applied instead, and the control circuit 320 may apply the frequency limits (e.g., limits on the frequency minimum 410 and / or the frequency maximum 415) specified in the frequency control signal 425-1 to the internal frequency control signal 405.

[0044] Figure 5 Showing the embodiment according to the disclosure Figure 1 The accelerator 135 uses Figure 4 A flow chart of an example processing of a frequency control signal 425-1 (eg, a method for controlling an accelerator). Figure 5 In block 505, Figure 1 The accelerator 135 can be obtained from Figure 4 Software 420 Receiving Figure 4 The frequency control signal 425-1. In block 510, Figure 3 The control circuit 320 can Figure 4 The frequency control signal 425-1 is applied to Figure 3 The processing circuit 325.

[0045] Figure 6 Showing the embodiment according to the disclosure Figure 1 The accelerator 135 uses Figure 4 The frequency control signal 425-1 is used to override the frequency Figure 3 The PID control loop 315 Figure 4 Flow chart of an example processing of the internal frequency control signal 405. Figure 6 In block 605, Figure 3 The PID control loop 315 can generate Figure 4 The internal frequency control signal 405. At block 610, Figure 3The PID control loop 315 can Figure 4 The internal frequency control signal 405 is sent to Figure 3 The control circuit 320. At block 615, Figure 3 The control circuit 320 can be used Figure 4 The frequency control signal 425-1 overrides Figure 4 The internal frequency control signal 405.

[0046] Figure 7 Showing the embodiment according to the disclosure Figure 1 The accelerator 135 is applied Figure 4 Flow chart of an example processing of the frequency control signal 425-1. Figure 7 In block 705, Figure 3 The control circuit 320 may apply Figure 4 The frequency control signal 425-1 lasts for the duration of the data processing command: when the data processing command ends, Figure 3 The control circuit 320 may then apply the Figure 3 The PID control loop 315 Figure 4 The internal frequency control signal 405. Optionally, at block 710, Figure 3 The control circuit 320 may continue to apply Figure 4 The frequency control signal 425-1 is Figure 3 The control circuit 320 from Figure 4 Software 420 received Figure 4 The new frequency control signal 425-1 is obtained. At this time point, Figure 3 The control circuit 320 can be used Figure 4 The new frequency control signal 425-1 overrides Figure 4 Optionally, at block 715, Figure 3 The PID control loop 315 can be based on the Figure 4 The software 420 signal to make Figure 4 The frequency control signal 425-1 is invalid.

[0047] exist Figures 5 to 7 In the figure, some embodiments of the disclosure are shown. However, those skilled in the art will recognize that other embodiments of the disclosure are also possible by changing the order of the blocks, by omitting blocks, or by including connections not shown in the drawings. All such variations of the flow charts are considered to be disclosed embodiments, whether explicitly described or not.

[0048] Disclosed embodiments may include software for generating a frequency control signal for an accelerator. The accelerator may include a control circuit that may receive the frequency control signal and may override an internal frequency control signal generated, for example, internally by a proportional-integral-derivative (PID) control loop. Because the software has access to information about the workload that the PID control loop may not have access to, the frequency control signal from the software may result in more efficient operation of the accelerator, thereby providing a technical advantage.

[0049] Artificial intelligence (AI) accelerators require a precise frequency and performance management architecture. The time to complete a unit of work is difficult to predict. The disclosed embodiments provide a hardware / software-based solution for load management of AI accelerators.

[0050] The disclosed embodiments enable independent demand-based frequency control for accelerators. Software-driven boost signals can provide tight hardware / software coordination. The disclosed embodiments can be scalable for multiple accelerators. Each accelerator can be independently frequency controlled using the following parameters:

[0051] The disclosed embodiments may support independent demand-based frequency control, which may enable hardware-based workload balancing, power consumption control via frequency management, programmable frequency minimum, frequency maximum, and responsiveness to allow for better heterogeneous solutions. The disclosed embodiments may support fast responsiveness but power hungry accelerators, medium responsiveness accelerators for balanced performance, and economical accelerators for maximum power efficiency. The disclosed embodiments may help resolve complex producer / consumer relationships within multi-accelerator topologies.

[0052] The following discussion is intended to provide a brief, general description of one or more suitable machines that can implement specific aspects of the disclosure. One or more machines may be controlled at least in part by input from conventional input devices (such as keyboards, mice, etc.) and by instructions received from another machine, interaction with a virtual reality (VR) environment, biometric feedback, or other input signals. As used herein, the term "machine" is intended to broadly cover a single machine, a virtual machine, or a system of communicatively combined machines, virtual machines, or devices operating together. Exemplary machines include computing devices (such as personal computers, workstations, servers, portable computers, handheld devices, phones, tablets, etc.), and transportation devices (such as private or public transportation (e.g., cars, trains, taxis, etc.)).

[0053] One or more machines may include embedded controllers (such as programmable or non-programmable logic devices or arrays, application specific integrated circuits (ASICs), embedded computers, smart cards, etc.). One or more machines may utilize one or more connections to one or more remote machines (such as through a network interface, modem, or other communication combination). Machines may be interconnected through physical networks and / or logical networks (such as intranets, the Internet, local area networks, wide area networks, etc.). Those skilled in the art will understand that network communications may utilize various wired and / or wireless short-range or long-range carriers and protocols (including radio frequency (RF), satellite, microwave, Institute of Electrical and Electronics Engineers (IEEE) 802.11, Bluetooth®, fiber optic, infrared, cable, laser, etc.).

[0054] Embodiments of the present disclosure may be described by reference to or in conjunction with associated data including functions, processes, data structures, applications, etc., which, when accessed by a machine, produce machine-executed tasks or define abstract data types or low-level hardware contexts. The associated data may be stored, for example, in volatile memory and / or non-volatile memory (e.g., RAM, ROM, etc.), or in other storage devices and their associated storage media (including hard drives, floppy disks, optical storage devices, tapes, flash memory, memory sticks, digital video disks, biometric storage devices, etc.). The associated data may be delivered in the form of packets, serial data, parallel data, propagated signals, etc. through a transmission environment including a physical network and / or a logical network, and may be used in a compressed format or an encrypted format. The associated data may be used in a distributed environment and stored locally and / or remotely for machine access.

[0055] The disclosed embodiments may include a tangible, non-transitory, machine-readable medium including instructions executable by one or more processors, including instructions for performing elements of the disclosure as described herein.

[0056] The various operations of the above methods may be performed by any suitable means capable of performing the operations (such as various hardware and / or one or more software components, circuits and / or one or more modules). Software may include an ordered list of executable instructions for implementing logical functions, and may be implemented in any "processor-readable medium" for use by or in conjunction with an instruction execution system, device or apparatus (such as a single-core processor or a multi-core processor or a system containing a processor).

[0057] The blocks or steps of the methods or algorithms and functions described in conjunction with the embodiments disclosed herein may be implemented directly in hardware, in software modules executed by a processor, or in a combination of the two. If implemented in software, the functions may be stored on or transmitted through a tangible, non-transitory computer-readable medium as one or more instructions or codes. The software module may reside in a random access memory (RAM), flash memory, read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, removable disk, compact disk (CD) ROM, or any other form of storage medium known in the art.

[0058] Having described and illustrated the principles of the disclosure with reference to the illustrated embodiments, it will be appreciated that the illustrated embodiments may be modified in arrangement and detail without departing from such principles, and may be combined in any desired manner. Also, although the foregoing discussion has focused on specific embodiments, other configurations are contemplated. Specifically, even though expressions such as "according to the disclosed embodiments" and the like are used herein, these phrases are intended to refer generally to embodiment possibilities, and are not intended to limit the disclosure to specific embodiment configurations. As used herein, these terms may refer to the same or different embodiments that may be combined into other embodiments.

[0059] The foregoing illustrative embodiments should not be construed as limiting the disclosure thereof. Although some embodiments have been described, it will be readily appreciated by those skilled in the art that many modifications may be made to those embodiments without substantially departing from the novel teachings and advantages of the present disclosure. Therefore, all such modifications are intended to be included within the scope of the present disclosure as defined in the claims.

[0060] The disclosed embodiments extend to the following statements without limitation: Statement 1. The disclosed embodiments include an accelerator, the accelerator comprising: An interface for receiving a frequency control signal from a processor; circuitry for processing data based at least in part on data processing commands from the processor; and A control circuit is provided for setting a frequency of the circuit based at least in part on a frequency control signal from the processor.

[0061] Statement 2. A disclosed embodiment includes the accelerator of statement 1, wherein the interface is configured to receive the frequency control signal from software executing on the processor.

[0062] Statement 3. A disclosed embodiment includes the accelerator of statement 2, wherein the interface is configured to receive the frequency control signal from artificial intelligence (AI) workload software executing on the processor.

[0063] Statement 4. A disclosed embodiment includes the accelerator of statement 1, the accelerator further comprising: a proportional integral derivative (PID) controller for generating an internal frequency control signal for the accelerator that is sent to the control circuit.

[0064] Statement 5. A disclosed embodiment includes the accelerator of statement 4 wherein the control circuit applies the frequency control signal.

[0065] Statement 6. A disclosed embodiment includes the accelerator of statement 5, wherein the control circuit applies the frequency control signal to override an internal frequency control signal from the PID controller.

[0066] Statement 7. A disclosed embodiment includes the accelerator of statement 1 wherein the control circuit includes a maximum frequency for the circuit.

[0067] Statement 8. A disclosed embodiment includes the accelerator of statement 7, wherein the accelerator includes firmware, the firmware including the maximum frequency.

[0068] Statement 9. A disclosed embodiment includes the accelerator of statement 7, wherein the frequency control signal is greater than a maximum frequency for the circuit.

[0069] Statement 10. A disclosed embodiment includes the accelerator of statement 1 wherein the control circuit includes a minimum frequency for the circuit.

[0070] Statement 11. A disclosed embodiment includes the accelerator of statement 10, wherein the accelerator includes firmware, the firmware including the minimum frequency.

[0071] Statement 12. A disclosed embodiment includes the accelerator of statement 10 wherein the frequency control signal is less than a minimum frequency for the circuit.

[0072] Statement 13. A disclosed embodiment includes the accelerator of statement 1, wherein the frequency control signal is based at least in part on one of a first workload of the accelerator and a first implementation of the accelerator.

[0073] Statement 14. A disclosed embodiment includes the accelerator of statement 13, wherein the frequency control signal is further based at least in part on one of a second workload of the second accelerator and a second implementation of the second accelerator.

[0074] Statement 15. A disclosed embodiment includes the accelerator of statement 14, wherein: The first workload of the accelerator comprises a first memory-bound workload or a first compute-bound workload; and The second workload of the second accelerator includes a second memory-bound workload or a second compute-bound workload.

[0075] Statement 16. A disclosed embodiment includes the accelerator of statement 14, wherein: A first workload of the accelerator includes a first dimension of first data to be processed using the accelerator or a first number of vectors of first data to be processed using the accelerator; and The second workload of the second accelerator includes a second dimension of second data to be processed using the second accelerator or a second number of vectors of second data to be processed using the second accelerator.

[0076] Statement 17. A disclosed embodiment includes the accelerator of statement 14, wherein the frequency control signal is calculated to coordinate the accelerator and the second accelerator.

[0077] Statement 18. A disclosed embodiment includes the accelerator of statement 14, wherein the frequency control signal coordinates the accelerator and the second accelerator.

[0078] Statement 19. A disclosed embodiment includes the accelerator of statement 1 wherein the control circuit applies a frequency control signal to the data processing command.

[0079] Statement 20. A disclosed embodiment includes the accelerator of statement 1 wherein the control circuit applies a frequency control signal that overrides the second frequency control signal.

[0080] Statement 21. A disclosed embodiment includes the accelerator of statement 1 wherein the frequency control signal deactivates the second frequency control signal.

[0081] Statement 22. The disclosed embodiments include a method comprising: receiving a frequency control signal from a processor at a control circuit of the accelerator, the processor being external to the accelerator; and The frequency control signal is applied by the control circuit to circuitry of the accelerator, the circuitry being configured to process data based at least in part on the data processing commands from the processor.

[0082] Statement 23. A disclosed embodiment includes the method of statement 22, wherein the step of receiving the frequency control signal from the processor at the control circuit of the accelerator includes: receiving the frequency control signal from software executing on the processor at the control circuit of the accelerator.

[0083] Statement 24. The disclosed embodiments include the method of statement 23, wherein the step of receiving a frequency control signal at the control circuit of the accelerator from software executing on the processor comprises: receiving a frequency control signal at the control circuit of the accelerator from artificial intelligence (AI) workload software executing on the processor.

[0084] Statement 25. A disclosed embodiment includes the method of statement 22, further comprising: generating an internal frequency control signal at a proportional-integral-derivative (PID) controller of the accelerator; and sending an internal frequency control signal from the PID controller to the control circuit; and The step of applying the frequency control signal to the circuit of the accelerator through the control circuit includes: overriding the internal frequency control signal with the frequency control signal.

[0085] Statement 26. A disclosed embodiment includes the method of statement 22 wherein the control circuit includes a maximum frequency.

[0086] Statement 27. A disclosed embodiment includes the method of statement 26 wherein the frequency control signal is greater than the maximum frequency.

[0087] Statement 28. A disclosed embodiment includes the method of statement 26 further comprising: reading the maximum frequency from firmware of the accelerator.

[0088] Statement 29. A disclosed embodiment includes the method of statement 22 wherein the control circuit includes a minimum frequency.

[0089] Statement 30. A disclosed embodiment includes the method of statement 29 wherein the frequency control signal is less than a minimum frequency.

[0090] Statement 31. A disclosed embodiment includes the method of statement 29 further comprising: reading the minimum frequency from firmware of the accelerator.

[0091] Statement 32. A disclosed embodiment includes the method of statement 22 wherein the frequency control signal is based at least in part on one of a first workload of the accelerator and a first implementation of the accelerator.

[0092] Statement 33. A disclosed embodiment includes the method of statement 32 wherein the frequency control signal is further based at least in part on one of a second workload of the second accelerator and a second implementation of the second accelerator.

[0093] Statement 34. A disclosed embodiment includes the method of statement 33, wherein: The first workload of the accelerator comprises a first memory-bound workload or a first compute-bound workload; and The second workload of the second accelerator includes a second memory-bound workload or a second compute-bound workload.

[0094] Statement 35. A disclosed embodiment includes the method of statement 33, wherein: A first workload of the accelerator includes a first dimension of first data to be processed using the accelerator or a first number of vectors of first data to be processed using the accelerator; and The second workload of the second accelerator includes a second dimension of second data to be processed using the second accelerator or a second number of vectors of second data to be processed using the second accelerator.

[0095] Statement 36. A disclosed embodiment includes the method of statement 33 wherein the frequency control signal is calculated to coordinate the accelerator and the second accelerator.

[0096] Statement 37. A disclosed embodiment includes the method of statement 33 wherein the frequency control signal coordinates the accelerator and the second accelerator.

[0097] Statement 38. A disclosed embodiment includes the method of statement 22 wherein the step of applying the frequency control signal to the circuit of the accelerator via the control circuit comprises: applying the frequency control signal to the circuit of the accelerator via the control circuit in response to the data processing command.

[0098] Statement 39. A disclosed embodiment includes the method of statement 22 wherein applying, by the control circuit, the frequency control signal to the circuit of the accelerator comprises: overriding, by the control circuit, the second frequency control signal based at least in part on the frequency control signal.

[0099] Statement 40. A disclosed embodiment includes the method of statement 22 wherein applying, by the control circuit, the frequency control signal to the circuit of the accelerator includes deactivating the second frequency control signal based at least in part on the frequency control signal.

[0100] Statement 41. The disclosed embodiments include a system comprising a non-transitory storage medium storing instructions on the non-transitory storage medium that, when executed by a machine, cause: receiving a frequency control signal from a processor at a control circuit of the accelerator, the processor being external to the accelerator; and The frequency control signal is applied by the control circuit to circuitry of the accelerator, the circuitry being configured to process data based at least in part on the data processing commands from the processor.

[0101] Statement 42. A disclosed embodiment includes the system of statement 41 wherein the step of receiving the frequency control signal from the processor at the control circuit of the accelerator comprises: receiving the frequency control signal from software executing on the processor at the control circuit of the accelerator.

[0102] Statement 43. The disclosed embodiments include a system according to statement 42, wherein the step of receiving a frequency control signal at the control circuit of the accelerator from software executing on the processor includes: receiving a frequency control signal at the control circuit of the accelerator from artificial intelligence (AI) workload software executing on the processor.

[0103] Statement 44. A disclosed embodiment includes the system of statement 41 wherein: The non-transitory storage medium stores further instructions on the non-transitory storage medium, which when executed by the machine cause: generating an internal frequency control signal at a proportional-integral-derivative (PID) controller of the accelerator; and sending an internal frequency control signal from the PID controller to the control circuit; and The process of applying the frequency control signal to the circuit of the accelerator through the control circuit includes: overriding the internal frequency control signal with the frequency control signal.

[0104] Statement 45. A disclosed embodiment includes the system of statement 41 wherein the control circuit includes a maximum frequency.

[0105] Statement 46. A disclosed embodiment includes the system of statement 45 wherein the frequency control signal is greater than the maximum frequency.

[0106] Statement 47. A disclosed embodiment includes the system of statement 45, a non-transitory storage medium storing further instructions on the non-transitory storage medium, the further instructions, when executed by the machine, causing: reading a maximum frequency from firmware of an accelerator.

[0107] Statement 48. A disclosed embodiment includes the system of statement 41 wherein the control circuit includes a minimum frequency.

[0108] Statement 49. A disclosed embodiment includes the system of statement 48 wherein the frequency control signal is less than a minimum frequency.

[0109] Statement 50. A disclosed embodiment includes the system of statement 48, a non-transitory storage medium storing further instructions on the non-transitory storage medium, the further instructions when executed by the machine causing: reading a minimum frequency from firmware of an accelerator.

[0110] Statement 51. A disclosed embodiment includes the system of statement 41 wherein the frequency control signal is based at least in part on one of a first workload of the accelerator and a first implementation of the accelerator.

[0111] Statement 52. A disclosed embodiment includes the system of statement 51 wherein the frequency control signal is further based at least in part on one of a second workload of the second accelerator and a second implementation of the second accelerator.

[0112] Statement 53. A disclosed embodiment includes the system of statement 52, wherein: The first workload of the accelerator comprises a first memory-bound workload or a first compute-bound workload; and The second workload of the second accelerator includes a second memory-bound workload or a second compute-bound workload.

[0113] Statement 54. A disclosed embodiment includes the system of statement 52, wherein: A first workload of the accelerator includes a first dimension of first data to be processed using the accelerator or a first number of vectors of first data to be processed using the accelerator; and The second workload of the second accelerator includes a second dimension of second data to be processed using the second accelerator or a second number of vectors of second data to be processed using the second accelerator.

[0114] Statement 55. A disclosed embodiment includes the system of statement 52 wherein the frequency control signal is calculated to coordinate the accelerator and the second accelerator.

[0115] Statement 56. A disclosed embodiment includes the system of statement 52 wherein the frequency control signal coordinates the accelerator and the second accelerator.

[0116] Statement 57. A disclosed embodiment includes the system of statement 41 wherein the step of applying the frequency control signal to the circuit of the accelerator via the control circuit comprises: applying the frequency control signal to the circuit of the accelerator via the control circuit in response to the data processing command.

[0117] Statement 58. A disclosed embodiment includes the system of statement 41 wherein applying, by the control circuit, the frequency control signal to the circuit of the accelerator comprises: overriding, by the control circuit, the second frequency control signal based at least in part on the frequency control signal.

[0118] Statement 59. A disclosed embodiment includes the system of statement 41 wherein applying, by the control circuit, the frequency control signal to the circuit of the accelerator includes deactivating the second frequency control signal based at least in part on the frequency control signal.

[0119] Therefore, in view of the various arrangements of the embodiments described herein, this detailed description and accompanying material are intended to be illustrative only and should not be taken as limiting the scope of the disclosure. Therefore, what is claimed as disclosed is all such modifications as may come within the scope and spirit of the appended claims and their equivalents.

Claims

1. A device comprising an accelerator, the accelerator comprising: An interface for receiving a frequency control signal from a processor; circuitry for processing data based at least in part on data processing commands from a processor; as well as A control circuit is provided for setting a frequency of the circuit based at least in part on a frequency control signal from the processor.

2. The device according to claim 1, wherein: The interface is configured to receive a frequency control signal from software executing on the processor.

3. The device according to claim 1, wherein: The accelerator further includes a proportional-integral-derivative controller for generating an internal frequency control signal sent to the control circuit.

4. The device according to claim 3, wherein: The control circuit applies a frequency control signal.

5. The device according to claim 1, wherein: The frequency control signal is based at least in part on a first workload of the accelerator.

6. The device according to claim 5, wherein: The frequency control signal is also based at least in part on a second workload of a second accelerator of the device.

7. The device of claim 5, wherein: The frequency control signal is also based at least in part on a second workload of a second accelerator of the device; A first workload of the accelerator includes a first dimension of first data to be processed using the accelerator or a first number of vectors of first data to be processed using the accelerator; and The second workload of the second accelerator includes a second dimension of second data to be processed using the second accelerator or a second number of vectors of second data to be processed using the second accelerator.

8. The device according to claim 5, wherein: The frequency control signal is also based at least in part on a second workload of a second accelerator of the device, the frequency control signal coordinating the accelerator with the second accelerator.

9. The device according to any one of claims 1 to 8, wherein: The control circuit applies the frequency control signal for a duration to execute the data processing command.

10. The device according to any one of claims 1 to 8, wherein: The frequency control signal invalidates the second frequency control signal.

11. A method for controlling an accelerator, comprising: receiving a frequency control signal from a processor at a control circuit of the accelerator, the processor being external to the accelerator; as well as The frequency control signal is applied to circuitry of the accelerator by the control circuitry, the circuitry of the accelerator being configured to process data based at least in part on the data processing commands from the processor.

12. The method of claim 11, further comprising: generating an internal frequency control signal at a proportional-integral-derivative controller of the accelerator; as well as sending an internal frequency control signal from a proportional-integral-derivative controller to a control circuit; and The step of applying the frequency control signal to the circuit of the accelerator through the control circuit includes: overriding the internal frequency control signal with the frequency control signal.

13. The method of claim 11, wherein: The step of applying the frequency control signal to the circuit of the accelerator through the control circuit includes: applying the frequency control signal to the circuit of the accelerator through the control circuit in response to the data processing command.

14. The method of claim 11, wherein: Applying, by the control circuit, the frequency control signal to the circuit of the accelerator includes overriding, by the control circuit, the second frequency control signal based at least in part on the frequency control signal.

15. The method of claim 11, wherein: Applying, by the control circuit, the frequency control signal to the circuit of the accelerator includes deactivating the second frequency control signal based at least in part on the frequency control signal.

16. A system comprising a non-transitory storage medium storing instructions on the non-transitory storage medium that, when executed by a machine, cause: receiving a frequency control signal from a processor at a control circuit of the accelerator, the processor being external to the accelerator; and The frequency control signal is applied to circuitry of the accelerator by the control circuitry, the circuitry of the accelerator being configured to process data based at least in part on the data processing commands from the processor.

17. The system of claim 16, wherein: The frequency control signal is based at least in part on a first workload of the accelerator.

18. The system of claim 17, wherein: The frequency control signal is also based at least in part on a second workload of the second accelerator.

19. The system of claim 16, wherein: The step of applying the frequency control signal to the circuit of the accelerator through the control circuit includes: applying the frequency control signal to the circuit of the accelerator through the control circuit in response to the data processing command.

20. The system of claim 16, wherein: Applying, by the control circuit, the frequency control signal to the circuit of the accelerator includes overriding, by the control circuit, the second frequency control signal based at least in part on the frequency control signal.

Images