Method and apparatus for providing non-computing unit power control within an integrated circuit - Patent Application 20070122997

Abstract

The method and apparatus employ a plurality of heterogeneous compute units and a plurality of non-compute units operably coupled to the plurality of compute units. A power management logic (PML) determines a memory bandwidth level associated with a respective workload executed on each of the plurality of heterogeneous compute units on the IC, and adjusts a power level of at least one non-compute unit of a memory system on the IC from a first power level to a second power level based on the determined memory bandwidth level. In some examples, the power level of the non-compute unit is adjusted, also taking into account memory access latency.

Description

[Background technology]

[0001] Integrated circuit (IC) systems, such as systems on chip (SOC), employ computational units, also referred to as core components, such as central processing unit cores, graphics processing unit cores, video coders and decoders (codecs), and other computational units that access memory. Such systems on chips also include non-computational units, also referred to as non-computational components, that do not perform memory access requests, such as read and write requests, but instead perform other tasks on the integrated circuit. Examples of non-computational units include data fabrics, memory controllers for memories such as dynamic random access memories (DRAMs), physical interface circuits (PHYs) to DRAM memories, interconnect logic such as PCI Express interfaces, Universal Serial Bus interconnects, and other types of non-computational units.

[0002] Some (SOC) architectures include multiple-core central processing units (CPUs) and / or multiple-core graphics processing units (GPUs) connected to a data fabric that connects to different types of memory. Such systems attempt to save power and / or improve performance by monitoring bandwidth across sockets to other SOCs and to the data fabric within the SOC, and increasing or decreasing the clock frequency for the data fabric in response to the detected bandwidth needs of the CPUs. However, when latency-sensitive workloads are running on the CPUs, latency issues can degrade system performance and cause increased power consumption.

[0003] With increasing data center performance needs and rapidly growing video content consumption, SOC power consumption is becoming difficult to reduce given the increasing performance requirements. For example, power management units on SOCs for server systems-on-chips, such as those used in data centers and other server products, are known to use memory bandwidth monitors, such as counters and associated logic, to attempt to identify the amount of bandwidth required for similar workloads, such as those executed by a CPU. In some systems, non-compute unit clock frequencies, such as the data fabric and other non-compute unit clock frequency (FCLK), are increased when it is detected that more bandwidth is needed for the CPU, and voltage levels are associated with voltage islands that provide the non-compute units to improve the performance state of the CPU within the SOC. For example, to increase memory bandwidth, the data fabric, memory controller, and physical interface (PHY) to DRAM memory, which is a function of memory bandwidth, are used. However, such operation can unnecessarily cause high power consumption by the non-compute units, adversely affecting SOC product performance. The increased power consumption of non-computing units can have a significant impact on the overall total power budget available to the computing units, which can reduce the performance of the computing units under a given power budget.

[0004] Non-computing units or non-computing units of a memory system include, for example, the data fabric, memory controller, and physical interface (PHY), which in combination can significantly impact memory bandwidth. Standby power leakage of unused non-computing units or input / output components such as multiple displays, PCI Express, and USB interfaces that are not connected or in use also consumes power.

[0005] Different workloads on the system-on-chip compute units (e.g., memory clients) place different demands on the data fabric and memory system. In some cases, such as high-resolution gaming, memory bandwidth can become a bottleneck for the compute engine. Other applications in many CPU workloads, such as low-resolution gaming, are sensitive to the latency of retrieving requests from memory. Legacy games (e.g., 1080p resolution games) are also sensitive to memory latency. Other workloads are sensitive to memory bandwidth, such as content creation software for video editors.

[0006] Also, current memory performance state algorithms are based on course bandwidth requirements. Each performance state supports a bandwidth cutoff. If the requirement is not met, the next higher appropriate state is selected by the power management unit. Higher performance states run at higher frequencies and voltages. For example, based on the bandwidth requirements of homogeneous compute units (e.g., central processing units), the data fabric frequency is increased, the memory controller frequency is increased, and the voltages of non-compute units are all increased or decreased. This requires more power. Bandwidth is a function of the number of parallel data buses used to communicate data, the data bus width, and the frequency of the elements. Existing solutions use frequency as a lever to provide more bandwidth, but higher frequency implies higher voltage and power costs to deliver such bandwidth.

[0007] It is desirable to improve power management systems for integrated circuits to improve power consumption and performance.

[0008] The embodiments will be more readily understood in view of the following description when taken in conjunction with the following figures, in which like numerals represent like elements. [Brief explanation of the drawings]

[0009] [Figure 1]FIG. 1 is a block diagram illustrating an integrated circuit including power management with heterogeneous workload type detection and non-computing unit power control, according to one example described in this disclosure. [Figure 2] 1 is a flowchart illustrating a method for providing power management for one or more integrated circuits, according to an example described in this disclosure. [Figure 3] 1 is a block diagram illustrating an integrated circuit that provides latency and bandwidth-based non-computing unit power control for power management, according to one example described in this disclosure. [Figure 4] 1 is a flowchart illustrating a method for providing power management to one or more integrated circuits, according to one or more examples described below. [Figure 5] FIG. 2 is a block diagram illustrating an apparatus including power management control with memory latency-based non-compute unit power control, according to an example described in this disclosure. [Figure 6] 1 is a block diagram illustrating an apparatus using non-computing unit power control according to an example described in this disclosure. [Figure 7] FIG. 10 illustrates a memory performance status table according to an example described in the present disclosure. [Figure 8] FIG. 2 is a block diagram illustrating power management logic according to one example described in this disclosure. [Figure 9] FIG. 1 is a state diagram illustrating memory performance state changes according to one example described in this disclosure. [Figure 10] FIG. 1 is a flow diagram illustrating a method for providing power management according to one example described in this disclosure. [Figure 11] FIG. 10 is a timing diagram illustrating detected latency delays according to an example described in this disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0010] In the following description, the use of the same reference numerals in different figures indicates similar or identical items. Unless otherwise stated, the word "coupled" and its related verb forms include both direct and indirect electrical connections by means known in the art, and unless otherwise stated, any description of a direct connection also refers to an alternative embodiment using a suitable form of indirect electrical connection.

[0011] In some embodiments, the method and apparatus provide control of power levels to non-compute units, also referred to as non-core units, by controlling one or more of a data fabric clock, a memory controller clock, and an interconnect clock in a workload-aware manner. In some embodiments, the power levels of the non-core units are grouped in a manner within memory performance states (P-states). In some examples, run-time power management operations determine the non-compute unit operating state based on the memory bandwidth and / or memory latency characteristics of the workload.

[0012] In some embodiments, the method and apparatus distinguish between state and application-specific requirements for latency and / or bandwidth. In some instances, a particular data fabric frequency is increased only if the use case is latency-sensitive. In certain embodiments, for bandwidth-centric applications, the frequency is maintained at a minimum level sufficient to deliver maximum bandwidth through the parallel interface, such as by using a larger number of ports available to the data fabric by the compute unit than in conventional systems, while maintaining frequency at a minimum level. In some embodiments, the method and apparatus determine the fabric frequency, taking into account the latency requirements of coherent traffic between multiple sockets in a server system, as in a SOC. Non-compute units often have dependencies between them, such as clock frequency and other dependencies.

[0013] In a particular embodiment, a method for providing power management to one or more integrated circuits (ICs) includes determining a memory bandwidth level associated with a respective workload executing on each of a plurality of heterogeneous compute units on the IC, and adjusting a power level of at least one non-compute unit of a memory system on the IC from a first power level to a second power level based on the determined memory bandwidth level.

[0014] In some examples, the memory bandwidth level includes monitoring memory access traffic associated with each of a plurality of heterogeneous compute units on the IC, using at least one non-compute unit thereof to access memory used by the plurality of heterogeneous compute units.

[0015] In a particular example, the non-computing units include a data fabric that transfers data between the heterogeneous computing units and the memory, and the power level of the non-computing units of the memory system includes selecting a memory performance state that provides a maximum level memory data transfer rate for a specified power budget using a reduced frequency clock and reduced voltage level of the data fabric compared to another memory performance state that provides the same maximum level memory data transfer rate using a higher data fabric clock frequency setting and a higher voltage level.

[0016] In some examples, adjusting the power level includes accessing a memory performance state table, the memory performance state table including a plurality of memory performance states, each state including data representing an allowable memory data transfer rate, a non-computational memory system voltage setting, a data fabric clock frequency setting, and a memory clock frequency setting for the performance state, at least a first performance state and a second performance state including the same maximum level memory data transfer rate, and the first performance state having a lower data fabric frequency setting and a lower non-computational memory system voltage setting compared to the second performance state.

[0017] In a particular embodiment, an integrated circuit includes a plurality of heterogeneous compute units and a plurality of non-compute units operably coupled to the plurality of compute units, and power management logic (PML) determines a memory bandwidth level associated with a respective workload executing on each of the plurality of heterogeneous compute units on the IC, and adjusts a power level of at least one non-compute unit of a memory system on the IC from a first power level to a second power level based on the determined memory bandwidth level.

[0018] In some examples, the power management logic determines the memory bandwidth level by at least monitoring memory access traffic associated with each of a plurality of heterogeneous compute units on the IC, and at least one non-compute unit is used to access memory used by the plurality of heterogeneous compute units.

[0019] In a particular example, at least one non-computing unit includes a data fabric that transfers data between the heterogeneous computing units and the memory, and the power management logic adjusts the power level of the non-computing units of the memory system by selecting a memory performance state that provides a maximum level memory data transfer rate for a specified power budget using a reduced frequency clock and reduced voltage level of the data fabric compared to at least another memory performance state that provides the same maximum level memory data transfer rate using a higher data fabric clock frequency setting and a higher voltage level.

[0020] In some examples, the power management logic adjusts the power level by accessing a memory performance state table, the memory performance state table including a plurality of memory performance states, each state including data representing an allowable memory data transfer rate, a non-computational memory system voltage setting, a data fabric clock frequency setting, and a memory clock frequency setting for the performance state, at least a first performance state and a second performance state including the same maximum level memory data transfer rate, the first performance state having a lower data fabric frequency setting and a lower non-computational memory system voltage setting compared to the second performance state.

[0021] In a particular embodiment, an integrated circuit includes a plurality of compute units and a plurality of non-compute units operably coupled to the plurality of compute units, wherein power management logic detects memory access latency associated with a workload executing on the plurality of compute units and alters a memory performance state associated with the plurality of non-compute units based on the detected memory access latency.

[0022] In some examples, the apparatus includes a data fabric configured to communicate data with at least another integrated circuit, and the power management logic detects memory access latency associated with a compute unit of the other IC and modifies memory associated with the plurality of non-compute units based on the detected memory access latency associated with the other integrated circuit.

[0023] In a particular example, the power management logic determines a memory bandwidth level associated with each workload executing on each of a plurality of heterogeneous compute units on the IC, and selects a memory performance state based on both the detected memory access latency and the determined memory bandwidth level.

[0024] In some examples, the plurality of non-compute units includes a data fabric, one or more memory controllers, and memory control logic such as an associated PHY interface to the memory, and the power management logic selects the memory performance state by arbitrating between data representing a detected memory latency level associated with the central processing core compute unit and a determined memory bandwidth level associated with the graphics processing core compute unit.

[0025] In a particular example, the power management logic includes memory latency monitor logic that detects memory access latency associated with memory load requests issued by the central processing computing unit during runtime, and memory bandwidth monitor logic that detects memory bandwidth levels associated with other computing units of the plurality of computing units.

[0026] In some examples, the power management logic includes cross integrated circuit memory bandwidth monitor logic configured to detect memory bandwidth associated with a compute unit on another integrated circuit, and the power management logic increases the memory performance state to a highest power state including increasing the data fabric clock frequency to a highest performance state level based on the detected memory bandwidth level from the other integrated circuit.

[0027] In certain examples, the power management logic prioritizes improving latency of at least one computing unit over improving bandwidth of at least another computing unit.

[0028] In some examples, the power management logic includes memory latency detection logic that detects memory latency for a workload associated with at least a first compute unit and provides a first memory performance state based on the detected memory latency. In particular embodiments, the power management logic includes memory bandwidth detection logic that detects a memory bandwidth level used by at least a second compute unit and provides a second memory performance state based on the detected memory bandwidth level. In some embodiments, the power management logic includes arbitration logic that selects a final memory performance state based on the first and second memory performance states and based on available power headroom.

[0029] In a particular example, the plurality of computing units includes a plurality of heterogeneous computing units, and the power management logic determines a memory bandwidth level associated with a respective workload executing on each of the plurality of heterogeneous computing units on the IC, and adjusts a power level of at least one non-computing unit of a memory system on the IC from a first power level to a second power level based on the determined memory bandwidth level.

[0030] In some examples, the power management logic determines the memory bandwidth level by at least monitoring memory access traffic associated with each of a plurality of heterogeneous compute units on the IC, and at least one non-compute unit is used to access memory used by the plurality of heterogeneous compute units.

[0031] In a particular embodiment, an apparatus includes a memory system, a plurality of compute units operably coupled to the memory system, and a plurality of memory non-compute units of the memory system, the memory non-compute units including a data fabric, a memory controller, and memory interface logic such as a PHY interface to the memory. The power management logic detects memory access latency associated with a workload executing on at least one of the plurality of compute units, determines a memory bandwidth level associated with each workload executing on multiple of the plurality of compute units, and alters a memory performance state associated with the plurality of non-compute units based on the detected memory access latency and the determined memory bandwidth level.

[0032] In some examples, the power management logic changes the memory performance state by controlling the operating state of the data fabric to: increase the clock frequency of the data fabric to provide a maximum level memory data transfer rate for the memory in response to determining that a memory latency sensitive workload is being executed by at least one of the plurality of compute units; and, in response to an increased bandwidth level being needed for the compute unit, select a memory performance state that provides a maximum level memory data transfer rate for a specified power budget using a reduced frequency clock and a reduced voltage level of the data fabric compared to another memory performance state that provides the same maximum level memory data transfer rate with a higher data fabric clock frequency setting and a higher voltage level.

[0033] In particular examples, the power management logic includes memory latency detection logic that detects memory latency for a workload associated with at least a first compute unit and provides a first memory performance state based on the detected memory latency. In some embodiments, the power management logic includes bandwidth detection logic that detects a memory bandwidth level used by at least a second compute unit and provides a second memory performance state based on the detected memory bandwidth level. In some embodiments, the power management logic includes arbitration logic that selects a final memory performance state based on the first and second memory performance states and based on available power headroom. In some examples, the power management logic prioritizes improving latency of at least one compute unit over improving bandwidth of at least another compute unit.

[0034] In a particular example, the apparatus includes a coherent data fabric socket that interfaces with the data fabric and interfaces with the compute units in both ICs, and the power management logic uses the coherent data fabric socket to detect memory access latencies associated with the compute units and alter memory performance states associated with the multiple non-compute units based on the detected memory access latencies associated with the other integrated circuits.

[0035] In some examples, the power management logic adjusts the power level by accessing a memory performance state table including a plurality of memory performance states, each state including data representing an allowable memory data transfer rate, a non-computational memory system voltage setting, a data fabric clock frequency setting, and a memory clock frequency setting for the performance state, at least a first performance state and a second performance state including the same maximum level memory data transfer rate, the first performance state having a lower data fabric frequency setting and a lower non-computational memory system voltage setting compared to the second performance state.

[0036] 1 illustrates an example of an integrated circuit 100, such as a system-on-chip or any other suitable integrated circuit, that includes multiple computational units 102, each of which processes data and accesses memory, such as cache memory, shared memory such as volatile or non-volatile memory, or any suitable memory. In some embodiments, the computational units are accessible through multiple non-computational units 104. In one embodiment, the non-computational units are components of a memory system, which in some embodiments includes memory 106, such as DRAM or any other suitable memory. This memory 106 is integrated within the integrated circuit 100 in some embodiments and is off-chip in other embodiments. In this example, the computational units 102 are heterogeneous computational units, although homogeneous computational units can also be used.

[0037] Integrated circuit 100 includes power management logic 108 that controls power settings for multiple heterogeneous compute units and multiple non-compute units via suitable communication paths 110. In this embodiment, power management logic 108 includes heterogeneous workload type detection and non-compute unit power control for controlling non-compute power consumption, as described further below. In this example, memory 112, such as cache memory, register memory, or any other suitable memory, provides memory performance state tables 114 for power management logic 108 to control clock frequency and voltage settings for non-compute units 104. In some embodiments, power management logic 108 generates the tables. In particular embodiments, memory performance state tables 114 are stored in memory and accessible by the power management logic. If tables are used, a separate table for each specified maximum power allocation allows different control depending on the type of IC or device being used, such as whether the IC is a mobile or non-mobile device. As is known in the art, memory 112 may include other performance state tables as well, if desired, for controlling the performance states of heterogeneous computing units 102 .

[0038] In some embodiments, integrated circuit 100 is a system-on-chip, where the heterogeneous computing units are each a different type of computing unit, including a central processing core 116, a graphics core 118, and one or more multimedia engines 120, such as a video codec. Such integrated circuits are used in some examples in data center servers, laptops, desktops, game consoles, printers, smartphones, wearables, and any other suitable devices. Thus, in some examples, the integrated circuit is coupled to a battery in the case of a mobile device, and in other embodiments is used in servers, desktops, game consoles, or other devices where the power source is a power outlet or other unlimited source.

[0039] In this and other embodiments, power management logic 108 is implemented as a microcontroller executing firmware, although any suitable configuration may be used, such as, but not limited to, one or more state machines, a field programmable gate array, or other implementation. Additionally, power management logic 108 in this example includes a memory bandwidth monitor, as described further below, configured to detect the memory bandwidth level used by each of heterogeneous compute units 102 to memory 106 or any other suitable memory.

[0040] The power management logic 108 monitors memory access traffic associated with each of the multiple heterogeneous compute units on the IC to determine memory bandwidth levels. detection In this example, non-compute units are used to access memory used by multiple heterogeneous compute units, and in one example, a non-compute unit includes a data fabric, another non-compute unit includes a memory controller, and another non-compute unit includes a physical memory interface (PHY). In some embodiments, the non-compute units include a data fabric that transfers data between the heterogeneous compute units and memory 106. In some embodiments, while both the GPU and the CPU can benefit from maximum bandwidth and minimum latency under power-constrained conditions, the power management logic makes static assumptions, for example, that CPU traffic should be optimized for latency and graphics traffic should be optimized for performance / watt. Multimedia unit (MM) traffic is sensitive to both latency and bandwidth, and upon detecting an MM traffic request, the power management logic transitions the non-core units to memory state P0, as also described below. transfer To make.

[0041] In some embodiments, one of the non-compute units includes a data fabric, another non-compute unit includes a memory controller in communication with the data fabric, another non-compute unit includes a physical (PHY) memory interface between the memory controller and memory 106, and in some embodiments, the non-compute unit includes interconnect circuitry and input / output interconnect circuitry, such as PCIE, Universal Serial Bus, and other interconnect circuitry. As described further below, the data fabric transfers data between the heterogeneous compute units 102 and memory 106.

[0042] 2, a flowchart illustrates a method for providing power management to one or more integrated circuits, which in some embodiments is performed by power management logic 108. However, it will be appreciated that any other suitable logic may perform the operations, and that the operations may be performed in any suitable order. As indicated by block 200, the method includes determining a memory bandwidth level associated with a respective workload executing on each of the plurality of heterogeneous compute units 102. As indicated by block 202, the method includes adjusting a power level of at least one of the non-compute units 104 of the memory system from a first power level to a second power level based on the determined memory bandwidth level associated with the respective workload executing on each of the plurality of heterogeneous compute units 102.

[0043] In other words, integrated circuit 100 includes power management logic 108 that determines which of the plurality of heterogeneous compute units 102, the plurality of memory system non-compute units 104, and the plurality of heterogeneous types of workloads is being executed by one or more of the heterogeneous compute units on the IC. This is done by using a bandwidth detector for each compute unit and evaluating the bandwidth usage of each compute unit, as described further below. Power management logic 108 adjusts the operating state of at least one memory system non-compute unit, such as a data fabric, to provide an increased power allocation for the determined type of heterogeneous workload. For example, non-compute unit power is reduced so that more power is available to apply to compute unit power. For example, the computation voltage and / or operating frequency may be increased by power management logic 108 using known techniques, such as through the use of a PState table for the compute unit.

[0044] FIG. 3 illustrates a bandwidth monitoring-based non-compute unit power control similar to that shown in FIG. 1, but also includes a bandwidth monitoring-based non-compute unit power control for the CPU cores 116, etc. Plan 1 is a block diagram illustrating another example of an integrated circuit 300 that uses power management logic 302 to monitor memory latency associated with the computing units. In some embodiments, the power management logic 302 provides memory latency monitoring for one or more of the computing units without providing bandwidth monitoring. For illustrative purposes, the power management logic 302 in this example uses both memory latency monitoring and bandwidth monitoring to control non-computing unit power levels. It will be appreciated that in some embodiments, the computing units 102 are all of the same type (i.e., homogeneous), such as all CPU cores or all CPU complexes on-chip, and therefore are not heterogeneous. However, in this example, the computing units 102 are shown to be heterogeneous computing units. Heterogeneous computing units are units of different types, whereas homogeneous computing units are all of the same type, meaning, for example, all CPUs, all GPUs, or all multimedia units.

[0045] The power management logic 302 in this example is similar to the power management logic 108 but also includes memory latency detection, such that both memory latency and memory bandwidth monitoring are used to determine suitable power levels for non-compute blocks. Additionally, this embodiment illustrates the use of an inter-integrated circuit socket used to interconnect with another integrated circuit 306, such as another SOC. This socket interconnects with, for example, a data fabric. The performance state table 114 includes memory performance state settings for the non-compute units for different specified maximum power levels allocated for a given mode of operation. For example, one example memory PState table may top out at 15 watts for a laptop configuration, while another memory PState table may allow for a power allocation of 45 watts for a desktop or server embodiment; however, any suitable power allocation may be used with any suitable number of performance state tables.

[0046] Referring to Figure 4 、1 A method for providing power management for one or more integrated circuits is illustrated. It will be appreciated that the operations may occur in any suitable order and be performed by any suitable structure. In a particular embodiment, the method is performed by power management logic 302. In one embodiment, power management logic 302 is configured to perform only memory access latency detection for homogeneous units, such as CPU core 116. In another embodiment, power management logic 302 performs both memory latency detection and memory bandwidth level detection for the compute units.

[0047] In some embodiments, as shown in block 402, the method includes detecting memory access latency associated with a workload executing on at least one of the plurality of compute units. For example, a workload executing on a CPU core is monitored through the use of a memory latency detector that evaluates memory latency experienced by the CPU core 116. As shown in block 404, the method includes altering a memory performance state associated with a non-compute unit, such as a CPU, based on the detected memory access latency. For example, if a latency error is detected for the CPU, the power management logic increases the memory power state to its lowest level (highest power state), so that the CPU does not experience latency interruptions while executing the workload. Thus, high-latency workloads are prioritized over bandwidth-intensive workloads.

[0048] In certain embodiments, the method also takes into account detected bandwidth levels for the same or other compute units to attempt to provide the best memory performance state for a given scenario. As shown in block 406, the method includes determining a memory bandwidth level associated with each workload executing on at least one of a plurality of compute units, which in this example includes a GPU core, a multimedia engine, and a CPU core. As shown in block 408, the power management logic takes into account both the detected memory access latency from block 402 and the determined memory bandwidth levels associated with the various compute units, and modifies the memory performance state based on the detected memory access latency and based on the determined bandwidth levels from block 406. For example, for threads that are sensitive to in-socket or on-chip local latency, the power management logic 302 prioritizes latency optimization over bandwidth optimization. However, if the bandwidth requirements of multiple compute units exceed a certain level and bandwidth usage becomes a bottleneck, the power management logic may modify the latency optimization setting. OverridesAlso addressed are latency sensitivities across sockets across multiple integrated circuits, such as the latency for coherent data transfer between sockets. For example, workloads such as transactional databases require large amounts of memory space in a data center and may be sensitive to latency issues, even though they are not necessarily completely localized to each socket. In such situations, For example, if memory bandwidth cannot be increased any further, Power management logic increases fabric frequency and voltage 。

[0049] 5 illustrates a block diagram of an integrated circuit (IC) 300, such as an APU, suitable for use in a data processing system. However, IC 300 may be any suitable IC, including, for example, a SOC for use in a desktop computer, a gaming console, a laptop, a mobile device such as a tablet, a wearable, or other suitable device. In this example, IC 300 generally includes a central processing unit (CPU) core complex 210, a graphics core 118, a set of display or multimedia engines 120, a memory management hub 240, a data fabric 518, a set of peripheral controllers 260, a set of peripheral bus controllers 270, a system management unit (SMU) 280, and a set of memory controllers 290 (memory controller 522 and memory controller 520).

[0050] CPU core complex 210 includes CPU core 116. In this example, CPU core complex 210 includes two CPU cores, but in other embodiments, CPU core complex 210 can include any number of CPU cores. Each of CPU cores 116 and 214 is bidirectionally connected to a system management network (SMN) forming a control fabric and to data fabric 518 and can provide memory access requests to data fabric 518. Each of CPU cores 116 and 214 may be a single core, or may be a core complex having two or more single cores that further share certain resources such as caches.

[0051] Graphics core 118 is a high-performance graphics processing unit (GPU) capable of performing graphics operations such as vertex processing, fragment processing, shading, texture blending, etc. in a highly integrated and parallel manner. Graphics core 118 is bidirectionally connected to the SMN and to data fabric 518 and can provide memory access requests to data fabric 518. In this regard, the APU can support either a unified memory architecture in which CPU core complex 210 and graphics core 118 share the same memory space, or a memory architecture in which CPU core complex 210 and graphics core 118 share a portion of the memory space, but graphics core 118 also uses private graphics memory that is inaccessible by CPU core complex 210.

[0052] The multimedia engine 120, or display engine, renders and rasterizes objects generated by the graphics core 118 for display on a monitor. The graphics core 118 and display engine are bidirectionally connected to a common memory management hub 240 for uniform translation to suitable addresses within the memory system, which in turn is bidirectionally connected to a data fabric 518 for generating such memory accesses and receiving read data back from the memory system.

[0053] Data fabric 518 includes a crossbar switch for routing memory access requests and responses between any memory accessing agent and memory controller 290 (memory controller 522 and memory controller 520). The data fabric also includes a system memory map defined by the BIOS, as well as buffers for each virtual connection, for determining the destination of memory accesses based on the system configuration.

[0054] The peripheral controllers 260 include a USB controller 506 and a SATA interface controller 264, each of which is bidirectionally connected to the system hub 266 and to the SMN bus. These two controllers are merely examples of peripheral controllers that may be used within the APU.

[0055] Peripheral bus controllers 270 include a system controller or "southbridge" (SB) 272 and a PCIe controller 504, each of which is bidirectionally connected to an input / output (I / O) hub 508 and to the SMN bus. I / O hub 508 is also bidirectionally connected to system hub 266 and to data fabric 518. Thus, for example, a CPU core can program registers in USB controller 506, SATA interface controller 264, SB 272, or PCIe controller 504 with accesses routed by data fabric 518 through I / O hub 508.

[0056] The SMU 280 is a local controller that controls the operation of resources on the APU and synchronizes communication between them. The SMU 280 manages the power-up sequencing of the various processors on the APU and controls multiple off-chip devices via reset, enable, or other signals. The SMU 280 includes one or more clock sources, such as a phase-locked loop (PLL), not shown in FIG. 2, to provide clock signals to each of the APU's components. The SMU 280 also manages power for the various processors and other functional blocks and can receive measured power consumption values ​​from the CPU cores 116 and 214 and the graphics core 118 to determine the appropriate power state.

[0057] The APU also implements various system monitoring and power saving functions. One system monitoring function in particular is thermal monitoring. For example, if the APU gets too hot, the SMU 280 can reduce the frequency and voltage of the CPU cores 116 and 214 and / or the graphics core 118. If the APU gets too hot, the SMU can be shut down entirely. Thermal events can also be received from external sensors by the SMU 280 via the SMN bus, and the SMU 280 can reduce the clock frequency and / or power supply voltage accordingly.

[0058] 5 also shows multiple integrated circuits 300 and 306 interconnected via socket 308, such as a socket connecting a coherent data fabric between SOCs. Such an embodiment may be used in server systems and data centers, as well as any other device using multiple SOCs. Thus, in this example, an IC includes a coherent data fabric socket coupled to a data fabric and to some of the multiple compute units, and power management logic 302 uses the coherent data fabric socket to detect memory access latency and / or bandwidth associated with the compute units from another IC and alter memory performance states associated with the multiple non-compute units based on the detected memory access latency associated with the other integrated circuit. In one example, this is done by increasing the clock frequency and operating voltage of the data fabric to improve bandwidth through the data fabric when more bandwidth is needed within the SOC or for SOCs in other sockets.

[0059] As also shown in this example, integrated circuit 300 includes several different non-compute units, such as PCIe controllers 504, 506, interconnected via input / output hub 508. Bandwidth detector 510 is configured to detect memory bandwidth levels passed by the I / O hub to the data fabric for clients using USB and PCI Express interconnections, e.g., accessing memory 106. Independent bandwidth detectors 512, 514, 516 are configured to detect memory bandwidth levels by monitoring memory access traffic associated with heterogeneous compute units, which in this example are a display engine, graphics core, and CPU core, via data ports different from data fabric 518. Data fabric 518 transfers data between heterogeneous compute units 120, 116, 118 and for interconnecting non-compute units 506, 504 in this example. Power management logic (also referred to as a power management controller) 302 determines memory bandwidth levels for each of the compute units by monitoring memory access traffic associated with each of the compute units. The type of workload is inferred from the type of compute units accessing memory 106 via data fabric 518. For example, a high bandwidth level detected by bandwidth detector 514 monitoring memory bandwidth usage of a graphics core may be used to determine that a gaming application is being executed by the graphics core as a workload. Similarly, bandwidth detector 516 configured to monitor bandwidth levels used by CPU cores may infer a CPU-intensive workload. Data fabric 518 provides data flow to respective memory controllers 520 and 522, which, together with physical interfaces 524 and 526, respectively, form a memory system. In some embodiments, the memory system also includes memory 106. Thus, in some embodiments, non-compute units 518-526 are used to access memory 106 and are used by the compute units to access memory.

[0060] In this embodiment, data fabric 518 is configured to communicate data with at least another integrated circuit 306 via socket 308. Power management logic 302 detects memory access latency via a socket latency detector and / or detects bandwidth via socket bandwidth detector 530. Socket bandwidth detector 530 in one embodiment is a lane width detector that detects the number of lanes being used by SOC 306. The more lanes being used, the greater the detected bandwidth usage by SOC 306. In some embodiments, socket bandwidth detector 530 includes an inter-socket latency detection mechanism. Latency for another SOC can be estimated based on workload type, such as whether a CPU from the other SOC is requesting memory, or latency can be calculated using latency counters from each SOC, as described further below. Thus, the power management logic detects memory access latencies associated with the compute units of other ICs 306 and modifies memory performance states associated with non-compute units, such as data fabric 518, memory controller, and non-compute units 520-526, to modify memory performance states associated with the non-compute units based on the detected memory access latencies associated with SOC 306.

[0061] The power management logic 302 determines a memory bandwidth level associated with each workload executing on each of the compute units 120-118 and selects a memory performance state based on both the detected memory access latency and the determined memory bandwidth. The latency detector 540, also referred to as memory latency detection logic, is configured to measure memory access latency for the CPU in association with memory accesses to the memory 106, such as DRAM. Each of the bandwidth detector and latency detector uses programmable thresholds, e.g., stored in configuration registers, to set thresholds indicative of an undesirable amount of latency or undesirable bandwidth usage by a particular compute unit. The power management logic 302 includes a latency detector 540 that detects memory access latency associated with memory load requests issued by the central processing compute core 116 (compute unit) during execution time. The power management logic 302 also includes bandwidth detectors 516, 514, 512, 510 that detect memory bandwidth levels associated with each of the respective compute units. It will be appreciated that the location of the bandwidth detector may be any suitable location within the integrated circuit, such as, for example, as part of a data port to the data fabric for detecting incoming and outgoing data associated with memory access requests (e.g., reads and writes) to memory 106, or at any other suitable interface as desired. Bandwidth detector 512 detects die-to-die bandwidth through a data port of the data fabric. Bandwidth detector 510 detects bandwidth from I / O devices, such as PCIe-connected devices.

[0062] In some embodiments, socket bandwidth detector 530 functions as inter-integrated circuit memory bandwidth monitor logic that detects memory bandwidth associated with compute units within integrated circuit 306. Power management logic 302 increases the memory performance state to the highest power state, including increasing the data fabric clock frequency to the highest performance state level, based on the detected memory bandwidth levels from other integrated circuits. For example, if socket bandwidth detector 530 detects a bandwidth usage level above a threshold, power management logic 302 increases the data fabric clock frequency and non-compute unit voltage, as well as increasing the clock frequency for other non-compute units, such as memory controllers, from their previous performance state levels if desired to increase the amount of bandwidth the data fabric and memory system can handle.

[0063] The power management logic 302 prioritizes improving the latency of a computing unit, such as the CPU core 116, over improving the bandwidth of another computing unit, such as the graphics core 118. For example, if an undesirable amount of latency is detected for the CPU, a higher power level (a lower power state, such as P1-P0) is implemented by the power management logic 302.

[0064] Each of the bandwidth detectors provides a bandwidth metric, such as bandwidth level data 550, to the power management logic 302. Similarly, the latency detector 540 provides measured information in the form of a metric, such as count data indicating the number of delayed loads encountered, shown as latency count data 552. In some embodiments, the SOC 306 has a similar latency and bandwidth detector configuration to the SOC 304, and therefore the socket bandwidth detector 530 block represents a latency detector within the SOC 306 that provides measured latency information from the SOC 306 to the PML so that it can be compared with latency information from the latency detector 540, as described further below. The power management logic 302 also provides operating state power control data 560 to the non-computing units to, for example, control the clock frequency and voltage level used by each of the non-computing units. The power control data settings in one embodiment are stored in the performance state table 114.

[0065] Generally, the power management logic will push the non-compute unit memory P-state to the lowest level (P0) for computationally intensive workloads, allowing for more power allocation to the computational resources. For bandwidth optimization, which is heavily influenced by the memory controller frequency (UCLK), the power management logic will move to the highest memory frequency state. By way of example, the main bandwidth paths that can trigger memory P-state changes are DRAM read and write traffic, interconnect bandwidth usage (e.g., die-to-die communication links on sockets, links and inter-socket links), and input / output PCIe traffic.

[0066] For threads that are sensitive to intra-socket local latency, the power management logic prioritizes latency optimization over BW optimization. However, if the BW requirement exceeds a certain level, the BW becomes a bottleneck, and therefore the power management logic overrides any latency-optimizing decisions. Furthermore, workloads such as transactional databases that require large memory regions and are therefore not necessarily completely localized to each socket can be sensitive to the latency for coherent data transfers between sockets. The power management logic detects such cases by inspecting the latency requirements across sockets and increases the fabric frequency even if the DRAM bandwidth cannot be increased any further (e.g., constrained by the maximum speed of the connected memory devices).

[0067] Regarding power optimization, this can be divided into two stages: power optimization when in the active state and power optimization when in the standby state.

[0068] Particularly in power-limited 15W notebook applications, power (pj / bit) during activity is a priority, such as in a graphics gaming workload with high bandwidth and optimal pj / bit requirements. Therefore, PState P2 is used to reduce non-compute unit power consumption and increase the clock frequency of compute units such as the GPU, thus enabling higher power allocation. The PML uses a compute unit PState table, as known in the art, to set the GPU clock to a higher speed. For power consumption during standby conditions (low BW), which is important for battery life workloads, the PML selects non-compute units to operate in the lowest PState (P3) with the lowest non-compute voltage setting (Vmin).

[0069] 6-7, the use of memory performance state data, such as memory performance state tables, will be described. It will be appreciated that any suitable data structure may be used. FIG. 6 is a block diagram of an example portion of an integrated circuit 300, illustrating that power management logic 302 controls one or more clock frequencies and voltages for non-compute units. In this example, FCLK is the clock for data fabric 518, MEMCLK is the clock for memory physical interface (PHY) 524, and UCLK is the clock for memory controller 520. Interconnect non-compute blocks, such as an I / O hub or PCIE block, use the interconnect clock. Voltage levels, in this example, are shared among memory non-compute units and vary with clock frequency depending on the memory performance state selected by the power management logic (PML). Register 600 stores programmable thresholds used by the PML to compare predicted or actual latency and bandwidth metrics obtained by latency detector 540 and bandwidth detectors 510-516.

[0070] 7, a memory performance state (PState) table 700 shows four power levels, referred to as PStateP0-P03, for operating settings for non-compute units, with P0 being the highest performance state and P3 being the lowest performance state. The power management logic 302 adjusts power levels by accessing the memory performance state table 700. This memory performance state table 700 includes multiple memory performance states (P0-P3), each of which includes data representing an allowable memory data transfer rate for the performance state (e.g., 3200 MT / s, 1600 MT / s), a non-compute memory system voltage setting (e.g., a non-core voltage used for the data fabric, memory controller, and PHY), a data fabric clock frequency setting (FCLK), a memory clock frequency setting (MEMCLK), and a memory controller (UCLK) setting. Performance states P2 and P0 have the same maximum level memory data transfer rate (e.g., 3200 MT / s), with state P2 having a lower data fabric frequency setting and a lower non-compute memory system voltage setting compared to state P0 so as to leave more power budget available for the compute units. For example, by reducing the power level for non-compute units, PML allows better performance options to be applied to the compute units, such as by allowing the compute units to operate at higher frequencies / voltages than would be available without immediate non-compute unit power control.

[0071] In this example, the non-compute units are part of the memory system and include the data fabric, memory controller, and PHY interface. PState P0 is for workload types requiring high bandwidth and low latency, such as CPU workloads and multimedia playback workloads, including live video playback, such as high-definition and standard-definition video / audio playback with video codecs. PState P1 is used for power-constrained workloads and is optimized to provide optimal levels of bandwidth and latency for the workload while using a lower power level than P0. Such workload types include multi-threaded CPU workloads. PState P2 is a lower non-compute unit power level setting that provides high bandwidth and is optimized for picojoule-per-bit (pj / bit) operation. Such workload types include graphics gaming workloads. PState P3 is a battery-saving level optimized to provide low-power operation.

[0072] As can be seen, all PStates halve the data fabric frequency compared to MEMCLK and use the same frequency for all non-compute units, except for PState P2, which has a lower voltage than P0 and P1. The memory transfer rate of 3200 MT / s is the same for P2 as it is for P0. The data fabric uses wider data ports for the compute units, allowing those compute units to use a lower data fabric clock and utilize higher bandwidth. For example, the data path to the data fabric is wider for engines that require a wider data path to optimize the power, BW, and latency of a particular compute unit. For example, the GPU 118 to memory interface is twice as wide as some other compute unit paths, resulting in the data fabric running at half speed, saving power at the expense of latency. GPU engines can tolerate more latency than other engines, such as CPUs. However, a higher GPU clock is required when power is limited, even if there is a performance loss (GPU performance increases with higher FCLK), which is fully overcome by the significantly reduced non-compute power. P03 is used to save battery life and is used for battery life workloads.

[0073] The power management logic 302 changes the memory performance state by controlling the operating state of the data fabric, as indicated by PState P0, to increase the clock frequency (FCLK) of the data fabric in response to a memory latency sensitive workload being executed by at least one of the plurality of compute units, thereby providing a maximum level memory data transfer rate (e.g., 3200 MT / s) to the memory 106. In response to an increased bandwidth level required for the compute unit, PML 302 selects a memory performance state (P2) that provides a maximum level memory data transfer rate (e.g., 3200 MT / s) for a specified power allocation, such as a 15W power state table for battery-powered devices or a 40W specified power allocation for non-battery-powered devices, using a reduced frequency clock (e.g., 800 MHz for P2 vs. 1600 MHz for P0) and voltage level (7V for P2 vs. 1.05V for P0) for the data fabric, compared to another memory performance state (P0) that provides the same maximum level memory data transfer rate using a higher data fabric clock frequency setting and a higher voltage level.

[0074] FIG. 8 is a block diagram illustrating an example of power management logic using an arbiter 900, also referred to as arbitration logic, that selects a final memory performance state based on memory performance states provided by each of the latency and bandwidth detectors, and bandwidth detectors 510-514, 530, and 540, and based on available power headroom data, such as generated by the PM. For example, traditional headroom calculations are performed based on the SOC thermal design power (PDP) and information from power monitors across the SOC to calculate current power consumption and determine available power headroom. The PML generates a non-compute unit power increase when bandwidth is needed or increases the data fabric frequency when a latency-sensitive workload is detected. Memory latency detection logic, shown in this example as latency detector 540, detects memory latency for workloads associated with particular compute units, such as CPU cores, depending on where the request originates, including within the SOC or across the SOC, and provides a memory performance state 902 based on the detected memory latency. Similarly, memory bandwidth detection logic, shown as bandwidth detectors 510, 512, 514, and 530, each provide a determined memory performance state based on the detected memory bandwidth level from each respective compute unit (including compute unit requests from the entire socket).

[0075] For example, each of the latency detector and bandwidth detector functions as an independent optimization unit, measuring actual latency data or data traffic metrics and / or predicted latency and / or predicted bandwidth usage and seeking its own performance state. In some embodiments where actual execution time data is monitored, the detector compares the monitored metrics 906 to one or more thresholds, shown generally as 904, to select a non-compute unit memory performance state. The controller 908 selects one of four memory PStates and outputs it to the arbiter 900 as the memory performance state 902. The arbiter 900 then selects the maximum state among all received PStates, where maximum refers to the highest power output provided by the non-compute unit. For example, if latency detector 540 detects that a large amount of latency is being experienced because the latency delay exceeds threshold 904, controller 908 selects maximum PState P0 to be output to the DPM arbiter, which will cause a change in non-compute unit memory state as shown by 910 if the memory performance state is not already at P0 because that is the highest latency state. Separate units monitor memory bandwidth, memory latency, and inter-socket communication.

[0076] In some embodiments, inter-socket communication is measured, for example, by a bandwidth detector 530 between sockets. For example, to optimize remote latency between two sockets, the bandwidth detector 530 monitors the bandwidth between the sockets and selects a non-compute unit performance state that will increase the data fabric frequency based on the inter-socket activity. For example, if the link goes to x16, the power level is increased by going to the highest PState (P0). Thus, remote latency optimization takes precedence over pure bandwidth optimization. For example, the bandwidth detector 514 associated with the graphics core can detect a high-bandwidth state and select PState P2, a state lower than P0 output by the inter-integrated circuit bandwidth detector 530, which prioritizes inter-socket activity.

[0077] 8, the detectors operate individually and select a state taking into account the metrics used by each detector. The arbiter 900 makes the final decision based on the activity monitor and forces a change of PState every decision time interval if the PState differs from the currently set PState.

[0078] In some embodiments, non-compute PState operation can be disabled. For example, based on the maximum DDR speed supported in the platform (e.g., via user settings or DRAM capabilities) and the maximum FCLK frequency setting fused for the IC, the power management logic selects and trains four memory system non-compute unit P-states for run-time power management. Two of the states (P2 and P1) are used for run-time active power management, one is a fixed state when run-time latency / bandwidth detection operations are disabled, and the other is a state for standby power management or efficiency mode (for performance optimization).

[0079] As described above, the PML selects the non-compute unit PState based on bandwidth and / or latency monitoring of the workload by the compute unit. The PML defaults to the lowest PState (e.g., P2) when active and controls the non-compute engine PStates (e.g., P1, P0) to be pushed higher based on detected (including predicted) workload characteristics.

[0080] In certain scenarios where low latency is desirable, a higher compute clock (CCLK) for the compute unit, controlled by the PML, can still provide better performance. In some embodiments, the PML prevents the PState change mechanism from aggressively switching to a higher P-State, lowering latency, removing power from the core, lowering CCLK, and adversely affecting performance. In some embodiments, the user sets a CCLK threshold in a register via a user interface, and the PML uses that threshold as a gate to prevent aggressively switching the PState higher even if the monitor indicates a low latency condition.

[0081] In some embodiments, hysteresis is provided between PState transitions to prevent PStates from transitioning too quickly. For example, in some embodiments, programmable hysteresis thresholds are used to provide up and down hysteresis. For up hysteresis, a single observation across a programmable threshold (e.g., across a threshold of 50% of the current DPM state bandwidth) is used, although any suitable threshold can be used. For down hysteresis, multiple consecutive observations below the next lower PState threshold are used to prevent dithering, although any suitable threshold and hysteresis mechanism can be used.

[0082] For multiple socket embodiments where each IC contains its own data fabric, the FCLK requirements can be evaluated independently to obtain different FCLKs on the two sockets, in some instances maintaining a desired ratio between the fabric frequency and the socket-to-socket interconnect frequency.

[0083] Figure 9 is a state diagram showing the memory performance states that are ordered. Memory performance states P0-Pn are the performance states of the memory data fabric and other non-core units used to access memory. P0 is the highest performance state, and Pn is the lowest performance state. PL represents the power threshold mapped to each state. For AC power settings or high performance settings, non-core performance is maximized, and for battery DC settings or energy efficiency settings, fabric performance / watt is maximized. The SMU tracks the power consumed by different non-compute units based on power monitors and leakage vs. temperature curves. This state diagram is based on the performance state table shown in Figure 7.

[0084] FIG. 10 illustrates an integrated circuit using multiple core groups 1100 and 1102, with each core 1104 having a memory latency counter 1106. The PML in some embodiments aggregates latency measurements for the groups of cores. In a particular embodiment, the memory latency counters are aggregated because a single SoC has multiple core groups (CCDs) 1100 and 1102. The power management logic reads them and calculates the median latency metric for each group, as shown in Equation 1. The PML then finds the maximum value among all cores in that group. Next, the PML calculates the maximum value among all CCDs. The PML then multiplies the result by the memory busy value, as described in Equation 2. Finally, that value is compared against a threshold to select a non-compute unit memory PState. In FIG. 11, MP5 represents the power management unit for the group of cores, and MP1 represents the SoC power management unit. In some embodiments, the method is also applied to inter-socket latency where aggregated latency counters for each SoC are compared, and if the SoC requires a higher power PState to compensate for the latency, a memory PState that provides a higher fabric frequency and improves inter-socket latency is selected.

[0085] As shown, the maximum value is taken because voltage scaling of the SoC frequency can flatten beyond a certain FCLK. Therefore, the amount of power and performance core that can be gained for a compute range scenario by running in a lower PState is small. On the other hand, the amount of performance core that can be lost by not running in a higher PState is large for workloads that are sensitive to memory latency (and memory bandwidth). In another embodiment, instead of the maximum value of all cores, the aggregated latency metric can be calculated as a weighted sum of the distance of each core's latency metric from a threshold t across all active cores. If the weighted aggregated latency metric is still positive, the P0 state is taken as the global system-level optimum. If the weighted metric is negative, a lower PState is selected as the global system-level optimum. This is shown in the following equation for the alternative final latency metric:

number

number

[0086] (alternative final latency metric) In certain embodiments, a metric is used to estimate memory latency workloads. The memory latency metric is used to identify when a workload is sensitive to memory latency (such as a workload being executed by a CPU complex). In this case, the SoC needs to transition to a higher PState to provide best performance. This metric is based on three events: microinstructions per cycle, the average latency of sampled previous load hits in DRAM or IO as seen by the load storage unit, and the memory busy signal.

[0087] The calculation is done in two steps: 1) reading core counters and deriving intermediate latency metrics, 2) aggregating the intermediate metrics (by computing the maximum) for each core and multiplying the result by the memory busy signal. The following equation gives the formal definition of the metric:

[0088] First, the intermediate metric mc for each core i i Calculate where IPC i is the microinstructions per cycle, and CoreMemLat i is the average latency of sampled preceding load hits in DRAM or IO as seen by the load storage unit.

number

[0089] Next, we aggregate the mc for all cores in the socket and determine the SoC P state for the socket based on the ms value.

number

[0090] The rationale behind multiplying by the memory busy signal is that workloads with infrequent memory accesses can experience long latencies for single accesses, but these infrequent accesses do not add significant penalty to the workload. Therefore, the memory busy signal is used to qualify the latency metric and amortize this impact. When both metrics are combined, the metric attempts to capture workloads that are neither sensitive to memory bandwidth nor extremely sensitive to latency; these should run in a lower PState. All others should run in higher and lower PStates. Thresholds are used to separate the spaces.

number

[0091] Referring to Figure 11, with regard to detecting memory latency, the preceding load latency can be defined as the latency of the first load missed in the cache / DRAM memory. See Figure 12, which shows an example of a preceding load. After the first load miss (load miss A), any consecutive load misses are not considered as preceding loads (load miss B) because the delays caused by those misses are hidden by the preceding loads. Only after the first preceding load is returned will the very next load be considered as a preceding load (load miss C).

[0092] Among various technical solutions, in some embodiments, the structure and operation distinguish between state and application-specific requirements for latency and / or bandwidth. In some instances, a particular fabric frequency is increased only if the use case is latency-sensitive, while for bandwidth-centric scenarios, the frequency is maintained at a minimum level sufficient to deliver maximum bandwidth to the data fabric through the parallel interface. In certain embodiments, the power management logic examines the latency requirements of coherent traffic between multiple sockets in the server system and adjusts the fabric frequency accordingly. detection This allows for inter-socket scaling when applied to multi-integrated circuit (e.g., die or chip) configurations, as it takes into account data traffic across the coherent fabric between sockets. In some embodiments, the power management system may be configured to provide a power management solution for various heterogeneous types of workloads. detectionThe system adjusts the power consumption of the non-compute units based on the type of heterogeneous workload, such as which compute units have detected bandwidth requirements above a threshold. This power consumption adjustment is done through adjusting from one performance state to another. Each performance state changes the frequency and / or voltage for the non-compute units. One advantage is that the system leaves more power budget available for the compute units. Heterogeneous types of workloads are those that are processed by heterogeneous compute units.

[0093] In some examples, determining the various heterogeneous types of workloads is performed using, for example, predictive operating system hints, driver hints, and / or an activity monitor that monitors memory access traffic for each of the compute units or memory access engines during runtime. The compute units may include video codecs, GPU cores, CPU cores, display engines, or other engines or units that access memory.

[0094] Although features and elements are described above in particular combinations, each feature or element can be used alone without the other features and elements, or in various combinations, with or without the other features and elements. In some embodiments, the apparatus described herein may be implemented in a computer program, software, or firmware embodied in a non-transitory computer-readable storage medium for execution by a general-purpose computer or processor. Examples of computer-readable storage media include read-only memory (ROM), random-access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media (e.g., internal hard disks and removable disks), magneto-optical media, and optical media (e.g., CD-ROM disks and digital versatile disks (DVDs)).

[0095] In the foregoing detailed description of various embodiments, reference has been made to the accompanying drawings which form a part hereof, and which show, by way of illustration, specific preferred embodiments in which the invention may be practiced. These embodiments have been described in sufficient detail to enable those skilled in the art to practice the invention, it being understood that other embodiments may be utilized, and that logical, mechanical, and electrical changes may be made without departing from the scope of the invention. To avoid detail not necessary to enable those skilled in the art to practice the invention, the description may omit specific information known to those skilled in the art. Moreover, many other various embodiments incorporating the teachings of the present disclosure may be readily constructed by those skilled in the art. Therefore, the present invention is not intended to be limited to the particular forms described herein, but on the contrary, is intended to cover all such alternatives, modifications, and equivalents that may reasonably be included within the scope of the invention. Therefore, the foregoing detailed description is not to be construed in a limiting sense, and the scope of the present invention is defined only by the appended claims. The above detailed description of the embodiments and examples described herein has been presented for purposes of illustration and description only, and not limitation. For example, the described operations may be performed in any suitable order or manner. It is therefore contemplated that the present invention encompasses any modifications, variations or equivalents that fall within the scope of the basic underlying principles disclosed above and claimed herein.

[0096] The foregoing detailed description and examples described therein have been presented for purposes of illustration and description only and not by way of limitation.

Claims

1. 1. A method for providing power management to one or more integrated circuits (ICs), comprising: determining a memory bandwidth level associated with a workload executed on each of a plurality of heterogeneous compute units on the IC; adjusting a power level of at least one non-computational unit of a memory system on the IC from a first power level to a second power level in response to the determined memory bandwidth level; The at least one non-computing unit: a data fabric that transfers data between the plurality of heterogeneous computing units and a memory; adjusting the power level of the at least one non-computing unit selecting a memory performance state that provides a maximum level memory data transfer rate using a reduced frequency clock and reduced voltage level of the data fabric compared to another memory performance state that provides the same maximum level memory data transfer rate using a higher data fabric clock frequency setting and a higher voltage level; method.

2. determining the memory bandwidth level includes monitoring memory access traffic associated with each of the plurality of heterogeneous compute units on the IC, wherein the at least one non-compute unit is used to access memory used by the plurality of heterogeneous compute units; 10. The method of claim 1.

3. adjusting the power level includes accessing a memory performance state table; The memory performance status table includes: a plurality of memory performance states, each state including data representing an allowable memory data transfer rate for said memory performance state, a non-computational memory system voltage setting, a data fabric clock frequency setting, and a memory clock frequency setting; at least a first performance state and a second performance state include the same maximum level memory data transfer rate, the first performance state having a lower data fabric frequency setting and a lower non-computational memory system voltage setting compared to the second performance state; 10. The method of claim 1.

4. 1. An integrated circuit (IC), comprising: a plurality of heterogeneous computational units; a plurality of non-computing units operably coupled to the plurality of heterogeneous computing units; The power management logic determining a memory bandwidth level associated with a workload executed on each of a plurality of heterogeneous compute units on the IC; adjusting a power level of at least one non-computational unit of a memory system on the IC from a first power level to a second power level in response to the determined memory bandwidth level; and The at least one non-computing unit: a data fabric that transfers data between the plurality of heterogeneous computing units and a memory; adjusting the power level of the at least one non-computing unit selecting a memory performance state that provides a maximum level memory data transfer rate using a reduced frequency clock and reduced voltage level of the data fabric compared to another memory performance state that provides the same maximum level memory data transfer rate using a higher data fabric clock frequency setting and a higher voltage level; Integrated circuit.

5. The power management logic: Operable to determine the memory bandwidth level by at least monitoring memory access traffic associated with each of the plurality of heterogeneous compute units on the IC; the at least one non-computing unit is used to access a memory used by the plurality of heterogeneous computing units; The IC of claim 4.

6. the power management logic is operable to adjust the power level by accessing a memory performance state table; The memory performance status table includes: a plurality of memory performance states, each state including data representing an allowable memory data transfer rate for said memory performance state, a non-computational memory system voltage setting, a data fabric clock frequency setting, and a memory clock frequency setting; at least a first performance state and a second performance state include the same maximum level memory data transfer rate, the first performance state having a lower data fabric frequency setting and a lower non-computational memory system voltage setting compared to the second performance state; The IC of claim 5.

7. 1. An integrated circuit (IC), comprising: a plurality of computational units; a plurality of non-computing units operably coupled to the plurality of computing units via a data fabric; The power management logic detecting a memory access latency associated with a workload executed on at least one of the plurality of computing units; modifying a memory performance state associated with at least one of the plurality of non-computing units based on the detected memory access latency; and memory control logic operably coupled to the plurality of compute units via the data fabric; the power management logic is operable to select the memory performance state by arbitrating between data representing a detected latency level associated with a central processing core computing unit and a determined memory bandwidth level associated with a graphics processing core computing unit; Integrated circuit.

8. a data fabric configured to communicate data with at least another integrated circuit (IC); the power management logic is operable to detect memory access latency associated with a compute unit of the other IC and to modify the memory performance state associated with the plurality of non-compute units based on the detected memory access latency associated with the other IC. The IC of claim 7.

9. the power management logic is operable to determine a bandwidth level associated with a respective workload executing on each of a plurality of heterogeneous compute units on the IC, and to select a memory performance state based on both the detected memory access latency and the determined memory bandwidth level. The IC of claim 7.

10. The power management logic: memory latency monitor logic operable to detect memory access latencies associated with memory load requests issued by the central processing computing unit during run time; memory bandwidth monitoring logic operable to detect memory bandwidth levels associated with other of the plurality of computing units; The IC of claim 7.

11. the power management logic comprises inter-integrated circuit memory bandwidth monitor logic configured to detect memory bandwidth associated with a compute unit on the other IC; the power management logic is operable to increase a memory performance state to a highest power state including increasing a data fabric clock frequency to a highest performance state level based on the detected memory bandwidth level of the other IC. The IC of claim 7.

12. the power management logic is operable to prioritize improved latency of at least one computing unit over improved bandwidth of at least another computing unit; The IC of claim 7.

13. The power management logic: memory latency detection logic operable to detect a memory latency of a workload associated with at least a first computing unit and to provide a first memory performance state based on the detected memory latency; memory bandwidth detection logic operable to detect a memory bandwidth level used by at least a second computing unit and to provide a second memory performance state based on the detected memory bandwidth level; arbitration logic operable to select a final memory performance state based on the first memory performance state and the second memory performance state and based on available power headroom data. The IC of claim 7.

14. the plurality of computing units comprises a plurality of heterogeneous computing units; The power management logic: determining a memory bandwidth level associated with a respective workload executing on each of a plurality of heterogeneous compute units on the IC; adjusting a power level of at least one non-computational unit of a memory system on the IC from a first power level to a second power level based on the determined memory bandwidth level; and The IC of claim 7.

15. The power management logic: Operable to determine the memory bandwidth level by at least monitoring memory access traffic associated with each of the plurality of heterogeneous compute units on the IC; the at least one non-computing unit is used to access a memory used by the plurality of heterogeneous computing units; 15. The IC of claim 14.

16. An apparatus including an integrated circuit (IC) according to any one of claims 7 to 15, A memory system; the plurality of computing units operably coupled to the memory system; memory interface logic operably coupled to the data fabric and to memory of the memory system; The power management logic: detecting memory access latencies associated with workloads executed on the plurality of computing units; determining a memory bandwidth level associated with a respective workload executed on each of the plurality of computing units; modifying a memory performance state associated with the plurality of non-computing units based on the detected memory access latency and the determined memory bandwidth level; and Device.

17. The power management logic: increasing a clock frequency of the data fabric to provide a maximum level memory data transfer rate for the memory in response to a memory latency sensitive workload determination performed by at least one of the plurality of compute units; In response to an increased bandwidth level being needed for the compute unit, selecting a memory performance state that provides a maximum level memory data transfer rate for a specified power budget using a reduced frequency clock and a reduced voltage level for the data fabric compared to another memory performance state that provides the same maximum level memory data transfer rate using a higher data fabric clock frequency setting and a higher voltage level; and operable to change the memory performance state by controlling an operational state of the data fabric to perform 17. The apparatus of claim 16.

Images