Resource manager collocated with IOMMU

Abstract

Devices, methods and systems are provided for managing resources in a computing device. Information regarding resource usage is obtained. Based on the information, a prediction is generated that usage of the resource by a processor will exceed a threshold during a future time period. Based on the prediction, an operating parameter of the processor is adjusted. In some embodiments, information regarding memory bandwidth is obtained. Based on the information, a prediction is generated that a memory region stored in a first memory device will be addressed by a memory-intensive instruction during a future time period. Based on the prediction, data stored in the memory region is moved to a second memory device.

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of U.S. Patent Application No. 17 / 565,336, filed December 29, 2021, the contents of which are incorporated herein by reference. [Background technology]

[0002] Modern computing systems typically include more than one type of processor. For example, a personal computer commonly includes both a CPU and a GPU. Such systems also typically include more than one type of memory. For example, a personal computer may include a main memory and a graphics memory. A personal computer's main memory is typically comprised of dynamic random-access memory (DRAM), which offers a relatively large capacity and a relatively lower bandwidth, and the graphics memory is typically comprised of high bandwidth memory (HBM), which has a relatively smaller capacity.

[0003] Typically, the CPU performs calculations on data stored in main memory, and the GPU performs calculations on data stored in graphics memory. In most cases, however, the CPU can also access the graphics memory. Furthermore, memory bus speeds are becoming more common that allow the CPU to take advantage of the higher memory bandwidth of the graphics memory without losing an unacceptable amount of bandwidth due to the memory bus speed. Similarly, the GPU can, in some cases, access the main memory for calculations. In this manner, various types of memory in a computer system can be considered as resources shared between processors (e.g., the CPU and the GPU). In addition to memory, other types of resources, such as power and thermal budgets, can also be considered as resources shared between processors.

[0004] A more detailed understanding may be had from the following description, given by way of example in conjunction with the accompanying drawings, in which: [Brief description of the drawings]

[0005] [Figure 1] 1 is a block diagram of an example device capable of implementing one or more features of the present disclosure. [Diagram 2] 2 is a block diagram of the device of FIG. 1 showing additional details. [Diagram 3] FIG. 1 is a block diagram illustrating an example device capable of implementing one or more features of the present disclosure. [Figure 4] 4 is a flowchart illustrating an example method for managing memory bandwidth in a computing device, such as the computing device shown and described with respect to FIG. 3. [Diagram 5] 1 is a block diagram illustrating an example method for predicting accesses to a region of memory by bandwidth intensive instructions during a particular period of time. [Figure 6]1 is a line graph illustrating application of an autocorrelation function to predict accesses to a region of memory by bandwidth-intensive instructions during a particular period of time. [Figure 7] 4 is a flowchart illustrating an example method for managing energy consumption in a computing device, such as the computing device shown and described with respect to FIG. 3. [Figure 8] 4 is a flowchart illustrating an example method for managing temperature in a computing device, such as the computing device shown and described with respect to FIG. 3. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0006] Some embodiments provide a device, method and system for managing resources in a computing device. Information regarding memory bandwidth utilization is captured. Based on the information, a prediction is generated that a memory region stored in a first memory device will be addressed by a memory-intensive instruction during a future time period. Based on the prediction, data stored in the memory region is moved to a second memory device.

[0007] In some embodiments, the method is implemented in a resource management device. In some embodiments, the resource management device comprises an input output memory management unit (IOMMU), a processor in communication with the IOMMU, and / or a processor integrated with the IOMMU. In some embodiments, the information indicates whether the previously executed instruction is a memory intensive instruction, whether the previously executed instruction is a specified instruction, and / or whether the previously executed instruction is a particular type of instruction. In some embodiments, the prediction is generated by applying an autocorrelation function (ACF) and / or an artificial neural network (ANN) to the information.

[0008] In some embodiments, the first memory device comprises a relatively lower bandwidth memory and the second memory device comprises a relatively higher bandwidth memory. In some embodiments, the first memory device comprises synchronous dynamic random-access memory (SDRAM), double data rate (DDR) SDRAM, and / or low power double data rate (LPDDR) SDRAM. In some embodiments, the second memory comprises high bandwidth memory (HBM) and / or cache memory.

[0009] Some embodiments provide further devices, methods and systems for managing resources in a computing device. Information regarding resource usage is captured. Based on the information, a prediction is generated that resource usage by a processor will exceed a threshold during a future time. Based on the prediction, an operating parameter of the processor is adjusted.

[0010] In some embodiments, the resource management device comprises an input / output memory management unit (IOMMU), a processor in communication with the IOMMU, and / or a processor integrated with the IOMMU. In some embodiments, the resource comprises a power and / or thermal budget. In some embodiments, the operating parameter comprises a voltage, a current and / or a clock frequency. In some embodiments, the operating parameter is adjusted for a portion of the processor.

[0011] In some embodiments, the information indicates whether resource usage by the processor has exceeded a threshold during a previous time period. In some embodiments, the prediction is generated by applying an autocorrelation function (ACF) and / or an artificial neural network (ANN) to the information.

[0012] FIG. 1 is a block diagram of an example device 100 capable of implementing one or more features of the present disclosure. The device 100 may include a computer, a gaming device, a handheld device, a set-top box, a television, a mobile phone, a server, a tablet computer, or other type of computing device. The device 100 includes a processor 102, a memory 104, a storage 106, one or more input devices 108, and one or more output devices 110. The device 100 may also include an input path 112 and an output path 114. Aspects of the input and output paths may include, but are not limited to, circuitry and electrical connections within or between dies, as well as software components such as drivers. It should be understood that the device 100 may include additional components not shown in FIG. 1.

[0013] In various alternatives, the processor 102 includes a central processing unit (CPU), a graphics processing unit (GPU), a CPU and a GPU located on the same die, or one or more processor cores, each of which may be a CPU or a GPU. In various alternatives, the memory 104 is located on the same die as the processor 102 or is located separately from the processor 102. The memory 104 may include volatile or non-volatile memory (e.g., random access memory (RAM), dynamic RAM, cache).

[0014] Storage 106 includes fixed or removable storage devices (e.g., hard disk drives, solid state drives, optical disks, flash drives). Input devices 108 include, but are not limited to, a keyboard, a keypad, a touch screen, a touch pad, a detector, a microphone, an accelerometer, a gyroscope, a biometric scanner, or a network connection (e.g., a wireless local area network card for transmitting and / or receiving wireless IEEE 802 signals). Output devices 110 include, but are not limited to, a display, a speaker, a printer, a haptic feedback device, one or more optical, antennae, or network connections (e.g., a wireless local area network card for transmitting and / or receiving wireless IEEE 802 signals).

[0015] The input path 112 couples the processor 102 and the input device 108, allowing the processor 102 to receive input from the input device 108. The output path 114 communicates with the processor 102 and the output device 110, allowing the processor 102 to send output to the output device 110. The output path 116 includes an accelerated processing device (APD) 116 coupled to a display device 118. The APD accepts computational and graphics rendering commands from the processor 102, processes the computational and graphics rendering commands, and provides pixel output to the display device 118 for display. As described in more detail below, the APD 116 includes one or more parallel processing units that perform computations according to the single-instruction-multiple-data (SIMD) paradigm. Thus, although various functions are described herein as being performed by or in conjunction with APD 116, in various alternatives, the functions described as being performed by APD 116 are additionally or alternatively performed by other computing devices having similar capabilities that are not driven by a host processor (e.g., processor 102) and that do not provide graphical output to display device 118. For example, it is contemplated that any processing system that performs processing tasks according to the SIMD paradigm may perform the functions described herein. Alternatively, it is contemplated that computing systems that do not perform processing tasks according to the SIMD paradigm may perform the functions described herein.

[0016] 2 is a block diagram of device 100 showing additional details regarding the execution of processing tasks on APD 116. Processor 102 maintains, in system memory 104, one or more control logic modules for execution by processor 102. The control logic modules include operating system 120, kernel mode driver 122, and applications 126. These control logic modules control various aspects of the operation of processor 102 and APD 116. For example, operating system 120 communicates directly with hardware and provides an interface to the hardware for other software executing on processor 102. Kernel mode driver 122 controls the operation of APD 116, for example, by providing an application programming interface (API) to software executing on processor 102 (e.g., applications 126) to access various features of APD 116. Kernel mode driver 122 may also include a just-in-time compiler that compiles programs for execution by processing components of APD 116 (such as SIMD unit 138, described in more detail below).

[0017] APD 116 executes commands and programs for selected functions, such as graphics and non-graphics operations that are or may be suitable for parallel processing. APD 116 may be used to perform graphics pipeline operations, such as pixel operations, geometry calculations, and rendering of images to display device 118, based on commands received from processor 102. APD 116 also performs computational operations not directly related to graphics operations, such as operations related to video, physics simulations, computational fluid dynamics, or other tasks, based on commands received from processor 102.

[0018] The APD 116 includes a computation unit 132 that includes one or more SIMD units 138 that perform operations in a parallel manner according to the SIMD paradigm at the request of the processor 102. The SIMD paradigm is one in which multiple processing elements share a single program control flow unit and program counter, and thus execute the same program, but can execute the program on or using different data. In one example, each SIMD unit 138 includes 16 lanes, each lane executes the same instruction simultaneously with other lanes in the SIMD unit 138, but can execute the instruction on different data. Lanes can be switched off with prediction when not all lanes need to execute a given instruction. Prediction can also be used to execute programs with branching control flows. More specifically, for programs with conditional branches or other instructions where the control flow is based on calculations made by individual lanes, prediction of lanes corresponding to currently not executed control flow paths and serial execution of different control flow paths allows arbitrary control flow.

[0019] The basic unit of execution within the compute units 132 is the work item. Each work item represents a single instantiation of a program that executes in parallel on a particular lane. Work items can execute simultaneously as a "wavefront" on a single SIMD processing unit 138. One or more wavefronts are included in a "workgroup," which includes a collection of work items designated to execute the same program. A workgroup can be executed by executing each of the wavefronts that make up the workgroup. In the alternative, a wavefront executes sequentially on a single SIMD unit 138, or partially or fully in parallel on different SIMD units 138. A wavefront can be thought of as the largest collection of work items that can execute simultaneously on a single SIMD unit 138. Thus, if commands received from the processor 102 indicate that a particular program should be parallelized to the extent that the program cannot be run simultaneously on a single SIMD unit 138, the program is split into wavefronts that are either parallelized on two or more SIMD units 138, or serialized on the same SIMD unit 138 (or both parallelized and serialized, as appropriate). The scheduler 136 performs operations related to scheduling the various wavefronts on the different compute units 132 and SIMD units 138 .

[0020] The parallel processing provided by the computation units 132 is well suited to graphics-related operations, such as pixel value calculations, vertex transformations, and other graphics operations. Thus, in some cases, the graphics pipeline 134, which accepts graphics processing commands from the processor 102, provides computation tasks to the computation units 132 for execution in parallel.

[0021] Computation units 132 are also used to perform computational tasks that are not related to graphics or that are not performed as part of the "normal" operation of graphics pipeline 134 (e.g., custom operations performed to supplement the processing performed on the operation of graphics pipeline 134). Applications 126 or other software executing on processor 102 send programs defining such computational tasks to APD 116 for execution.

[0022] 3 is a block diagram of an example device 300 capable of implementing one or more features of the present disclosure. Device 300 may include a computer, a gaming device, a handheld device, a set-top box, a television, a mobile phone, a server, a tablet computer, or other type of computing device, etc. In some embodiments, device 300 is implemented using some or all of the aspects of device 100 shown and described with respect to FIG.

[0023] Device 300 includes a CPU 302, a GPU 304, a main memory 306, a graphics memory 308, a memory management unit (MMU) 310, an input / output memory management unit (IOMMU) 312, a bus 314, and a resource manager 316. In this example, resource manager 316 manages memory bandwidth.

[0024] CPU 302 includes any suitable processing device. In some embodiments, CPU 302 is implemented using processor 102 as shown and described with respect to FIG. 1. GPU 304 includes any suitable graphics processing device. In some embodiments, GPU 304 is implemented using APD 116 as shown and described with respect to FIG. 1.

[0025] Main memory 306 includes any suitable memory device. In some embodiments, main memory 306 is implemented using memory 104 as shown and described with respect to Figure 1. In this example, main memory 306 is implemented using LPDDR or any other suitable memory that provides a relatively large amount of storage compared to graphics memory and is relatively slow compared to graphics memory.

[0026] Graphics memory 308 includes any suitable graphics memory device. In some embodiments, graphics memory 308 is implemented using memory 104 as shown and described with respect to FIGURE 1. In this example, main memory 306 is implemented using HBM or any other suitable memory that is relatively fast compared to main memory and provides a relatively small amount of storage compared to main memory.

[0027] MMU 310 manages access to main memory 306. For example, requests from CPU 302 or other devices to read from or write to main memory 306 are managed by MMU 310. IOMMU 312 manages access to GPU 304 and graphics memory 308. For example, requests from GPU 304 (or CPU 302, or in some embodiments other devices) to read from or write to graphics memory 308 are managed by IOMMU 312. IOMMU 312 also manages a virtual address space (i.e., translates between virtual and physical memory addresses) covering CPU 302, GPU 304 (and in some embodiments other accelerator devices), main memory 306, and graphics memory 308. IOMMU 312 is positioned "between" CPU 302 and GPU 304 (and in some embodiments, other accelerator devices), although it should be noted that there may be more than one accelerator connected to IOMMU 312. IOMMU 312 handles address transaction requests from GPU 304 (or CPU 302, or in some embodiments, other devices).

[0028] Bus 314 comprises a suitable communications infrastructure for communication between the components of device 300. In some embodiments, bus 314 comprises one or more of a Peripheral Component Interconnect (PCI) bus, a PCI enhanced (PCIe) bus, an Advanced Microcontroller Bus Architecture (AMBA) bus, an Accelerated Graphics Port (AGP) bus, or other suitable communications infrastructure. CPU 302 communicates with main memory 306 over bus 314 via MMU 310 and with GPU 304 over bus 314 via IOMMU 312. CPU 302 and GPU 304 each have access to main memory 306 and graphics memory 308 over bus 314.

[0029] The CPU 302 executes various instructions that access data stored in memory. It should be noted that executing different instructions requires more memory bandwidth than other instructions. For example, the following instruction families may present increased BW requirements: Streaming SIMD Extensions (SSE), Advanced Vector Extensions (AVX), AVX2, AVX-512, and AVX-512BW may result in varying bandwidth requirements. SSE is a single instruction, multiple data (SIMD) instruction set extension to the x86 architecture. AVX is an extension of the x86 instruction set architecture for microprocessors. AVX2 extends most integer commands to 256 bits and introduces fused multiply-accumulate (FMA) operations. AVX-512 extends AVX to 512-bit support using EVEX prefix encoding. AVX extensions such as AVX-512BW mask vector accesses resulting in data-dependent access patterns across intervals and gather / scatter instructions such as VGATHERDPD / VSCATTERDPD. These are just examples of instructions that may occupy significantly more memory bandwidth (e.g., more memory bandwidth above a desired threshold) than other instructions during execution. Such instructions are referred to herein as "bandwidth-intensive" instructions.

[0030] Resource manager 316 is a processing element that manages memory bandwidth for requests issued by a workload from GPU 304 (or APD 304). In some embodiments, resource manager 316 is an additional execution path of IOMMU 312 and may be implemented using a dedicated FPGA or ASIC processing element, or other suitable hardware. In some embodiments, resource manager 316 is a device that alternatively or additionally communicates with MMU 310. In some embodiments, aspects of resource manager 316 may be implemented in operating system software.

[0031] The resource manager 316 determines data placement for CPU instructions. For example, in some embodiments, the resource manager 316 determines where to place data corresponding to CPU instructions executed in a future time interval (e.g., the next minute). In some embodiments, the resource manager 316 takes as input a time series profile of the number of accesses A from BW-intensive instructions to each of a number of physical regions of a particular size. This is described in more detail with respect to FIG. 4. In general, the resource manager 316 determines the appropriate type of memory for data stored in a particular region of the shared address space to be stored in a future time period, taking into account predictions made based on the time series profile.

[0032] In the present example, system 300 includes main memory 306 implemented using LPDDR (low bandwidth memory) and graphics memory 308 implemented using HBM (high bandwidth memory). Thus, resource manager 316 determines whether data stored in a particular region of the shared address space should be stored in low bandwidth memory or high bandwidth memory in a future time period, taking into account a prediction of whether and / or how many bandwidth-intensive instructions will access the region during the future time period. In response to this determination, resource manager 316 moves the data to a new physical memory location having suitable memory bandwidth capabilities, for example using operating system commands.

[0033] It should be understood that device 300 may include additional components not shown in FIG.

[0034] Figure 4 is a flow chart illustrating an example method 400 for managing memory bandwidth in a computing device, such as device 300 shown and described with respect to Figure 3. In some embodiments, method 400 is implemented by a resource manager, such as resource manager 316 shown and described with respect to Figure 3.

[0035] In step 402, the resource manager captures information about instructions accessing an X-sized region of virtual memory in the system over a time period (sample length) Y. For example, in some embodiments, the resource manager 316 captures information about instructions accessing a 1 gigabyte (GB) region of memory 306 and 308 over a one minute (min) time period, where the values ​​X=1 GB and Y=1 min are exemplary and in other embodiments other values ​​of X and Y are available and / or these values ​​are dynamically variable and / or user configurable.

[0036] In this example, for each memory region, the resource manager records whether the region is accessed by a bandwidth-intensive instruction during that time period and / or the number of times A that it is accessed. The resource manager also records whether each region is accessed by a bandwidth-intensive instruction during subsequent time periods over a sample size of Z time periods, thus accumulating a time series of data reflecting how frequently each memory region is accessed by bandwidth-intensive instructions.

[0037] The resource manager predicts, for each memory region, whether it is likely to be accessed by a bandwidth-intensive instruction during the next (or later) time period based on the accumulated time series information, The prediction is made in any suitable manner, such as by applying an autocorrelation function (ACF) or an artificial neural network (ANN) to the time series data.

[0038] For each memory region, on condition 404 that an address in the memory region is predicted to be accessed by a bandwidth-intensive instruction (or to be accessed by at least a threshold number of bandwidth-intensive instructions), the resource manager moves the data in the virtual memory region to the HBM (i.e., graphics memory 308 in this example) if it is not already in the HBM. On condition 404 that an address in the memory region is predicted to not be accessed by a bandwidth-intensive instruction (or to be accessed by fewer than a threshold number of bandwidth-intensive instructions), the resource manager moves the data in the virtual memory region to the LBM (i.e., main memory 306 in this example) if it is not already in the LBM. These particular types of memory are merely exemplary. Note that in some embodiments, other kinds of memory are used and more than two different types of memory may be distinguished by their bandwidth intensiveness.

[0039] The resource manager moves data in the virtual memory region from one physical memory to another in any suitable manner. For example, in some embodiments, the resource manager calls the operating system (OS) to move the data. In some embodiments, the resource manager invokes standard OS calls, such as Linux move_pages() or mbind() with the MPOL_MF_MOVE flag, to change the mapping of virtual addresses to physical addresses in order to move the data transparently to applications running on the system. In some embodiments, this has the advantage of moving data without having to modify the application's source code.

[0040] It should be noted that although method 400 is described as being implemented using a resource manager located similarly to resource manager 316 as shown and described with respect to FIG. 3 (e.g., within IOMMU 312 or in communication with IOMMU 312), method 400 can be implemented using any suitable hardware capable of tracking accesses by bandwidth-intensive instructions to multiple memory regions (e.g., across a unified address space covering heterogeneous memory types).

[0041] 5 is a block diagram illustrating an example method 500 for predicting whether and / or how many times a region of virtual memory is likely to be accessed by a bandwidth-intensive instruction during a particular period of time. The prediction is made in any suitable manner, such as using an autocorrelation function (ACF) or an artificial neural network (ANN). In some embodiments, the method 500 is implemented in the resource manager 316 or any other suitable hardware.

[0042] In this example, it is determined whether and / or how many times A an address in each of a number of memory regions (n, m in this example) is accessed by a bandwidth-intensive instruction during a time period i having length Y. This information is stored for use as input data to the prediction.

[0043] During a subsequent time period i+1, also having length Y, it is again determined whether and / or how many times A an address in memory region n, m is accessed by a bandwidth-intensive instruction. This information is also stored for use as input to the prediction.

[0044] After a suitable amount of historical data (for sample number Z) regarding bandwidth instruction accesses to memory regions has been accumulated, a prediction is generated regarding whether and / or how many times an address within each memory region will be accessed by a bandwidth-intensive instruction. In the example shown, based on data collected during time period i and during time period i+1 (Z=2 in this example), a prediction is made that memory region m will be accessed by y bandwidth-intensive instructions during time period i+2.

[0045] The prediction is made based on any suitable technique, such as by applying an ACF to the collected data or by inputting the collected data into a suitably trained ANN. Based on the prediction, the resource manager may move data stored at virtual addresses in region m to a different type of physical memory that coincides with the y bandwidth-intensive instructions expected in the future time period.

[0046] It should be noted that different variables can be user selectable and / or dynamically controllable. For example, the region size X, sample length Y, and sample size Z can all be adjusted, for example, by the user or by a dynamic mechanism. In some embodiments, dynamic control is advantageous when workload characteristics change at run-time. For example, a particular workload may require a smaller or larger sample length Y, or a smaller or larger sample size Z. In some embodiments, such as when a multi-programmed workload is used, these variables are dynamically adjusted, for example, due to the OS adjusting resource allocation for each process at run-time. In some embodiments, such a dynamic mechanism inputs one or more workload characteristics (e.g., bandwidth requirements or access granularity) and correlates these input variables with the output variables (X, Y, Z). In some embodiments, the correlation is generated based on regression or by another correlation mechanism.

[0047] FIG. 6 shows a line graph illustrating application of an autocorrelation function to collected data to predict whether and / or how many bandwidth-intensive instructions will access a memory region during a future time period based on the collected data.

[0048] Autocorrelation is a method for time series analysis that measures the correlation between a signal and a delayed copy of itself as a function of a delay, called a lag. The analysis of autocorrelation is a mathematical tool for finding repeating patterns, such as the presence of a periodic signal obscured by noise. A plot of the autocorrelation of a time series by lag is called the autocorrelation function (ACF).

[0049] The graph above is a plot of the Pearson correlation coefficient for 100 samples of the number of times an address in a memory region, A, is accessed by a bandwidth-intensive instruction. The Pearson correlation coefficient is a number between -1 and 1 that describes a negative or positive correlation, respectively. A value of 0 indicates no correlation. In this example, the series of 100 samples hides a sine function.

[0050] If the data follow a trend, the autocorrelation for small lags tends to be large and positive, since observations close in time are also close in size. Thus, the ACF of a trended time series tends to have positive values ​​that decrease slowly as lag increases. If the data are periodic, the autocorrelation will be larger for seasonal lags (at multiples of the periodic frequency) than for other lags.

[0051] In this example, for each 1 minute (Y) time interval, the CPU resource manager generates an ACF function for the 100 values ​​of A measured in the last 100 (Z) intervals for each memory region of 1 GB (X).

[0052] The graph below is a plot of the ACF function for 100 values ​​of A measured over 100 time intervals. In some embodiments, the resource manager analyzes the ACF to predict the value of A in the next interval (i.e., the 101st interval in this example where Z=100).

[0053] An increase in ACF as lag decreases indicates a trend that is likely to continue. In terms of the example above, this is detected by the resource manager by measuring the delta (i.e., change) between adjacent lags in the ACF.

[0054] If the delta is within a certain range (e.g., configurable via a memory-mapped register in the resource manager, such as |0.1| on the ACF by default), then a gradual change (and thus a trend) is detected. In this example, if the current value of A is low (e.g., the current BW from issued CPU instructions is low, e.g., BW is below a threshold BW), the resource manager moves the region to LPDDR for the next interval (anticipating that the trend will continue). If the current A is high, the region is moved to HBM. These memory types are merely examples, and the region may be moved to (or kept in) any suitable memory type based on the prediction.

[0055] A "scalloped" or periodic shape in the ACF indicates seasonality. In some embodiments, a scalloped or periodic shape is identified by identifying spikes (i.e., values ​​above a threshold, which is configurable in some embodiments) and measuring the interval between the spikes. A scalloped or periodic shape exists if the interval is consistent for adjacent spikes (e.g., between spikes 1 and 2 is 4, between spikes 2 and 3 is also 4, and between spikes 3 and 4 is also 4). In some embodiments, the resource manager detects periodic intervals, where the interval is the distance between periodic spikes in the correlation. Spikes are detected based on a threshold (e.g., configurable via a memory-mapped register in the resource manager, such as |0.5| on the ACF by default).

[0056] In some embodiments, the resource manager proactively moves the region to a preferred memory (e.g., to LPDDR if A is low (e.g., below threshold A) and / or to HBM if A is high (e.g., below threshold A) before the next spike occurs. These memory types are merely examples, and the region may be moved to (or kept in) any preferred memory type based on the prediction.

[0057] It should be noted that autocorrelation is merely an exemplary technique for predicting whether a region is likely to be accessed by a bandwidth-intensive instruction during a particular time period. For example, in some embodiments, historical data is input into an ANN, and the ANN outputs as an inference the probability that a region is likely to be accessed by a bandwidth-intensive instruction during a particular time period.

[0058] In some embodiments, the techniques described herein are applicable to resources other than shared memory. For example, in some embodiments, the resource manager manages power consumption in a computing device, similar to method 400 for managing memory bandwidth, or manages a thermal budget of the computing device. In some implementations, any suitable resource (e.g., a resource that is shared or affected by both the CPU and GPU, or by more than one processor) can be managed by the resource manager.

[0059] Figure 7 is a flow chart illustrating an example method 700 for managing energy consumption in a computing device, such as device 300 shown and described with respect to Figure 3. In some embodiments, method 700 is implemented by a resource manager located similarly to resource manager 316 shown and described with respect to Figure 3.

[0060] In step 702, the resource manager captures data regarding power consumption by the CPU 302 and GPU 304 over a time period Y. For example, in some embodiments, a resource manager located similar to the resource manager 316 captures power consumption by the CPU 302 and GPU 304 over a time period of 1 minute (min), where the value Y=1 min is exemplary and in other embodiments other values ​​of Y can be used and / or these values ​​are dynamically variable and / or user configurable.

[0061] For each processor (e.g., CPU 302, GPU 304), the resource manager records the power consumption (e.g., average, cumulative, etc.) during a period of time. The resource manager also records the power consumption of each processor during subsequent periods of time, thus accumulating a time series of data reflecting the power consumption of each processor.

[0062] Based on the accumulated time series data, the resource manager predicts the expected power consumption for each processor during the next (or later) time period, using any suitable method, such as using an autocorrelation function (ACF) or an artificial neural network (ANN).

[0063] If the total power consumption of the processors is predicted to exceed the threshold 704, the resource manager, in step 706, reduces the power supplied to the system (e.g., reduces the voltage) for the predicted period of time if the supplied power is currently above the minimum power.

[0064] If the condition 704 is that the total power consumption of the processors is predicted not to exceed the threshold, then the resource manager, in step 708, increases the power supplied to the system (e.g., increases the voltage) for the predicted period of time if the supplied power is currently below the maximum power.

[0065] These particular power interventions (i.e., power gating or scaling, for example, the entire system, the CPU and / or GPU, the CPU and individual cores or computational units of the CPU, etc.) are merely exemplary. Note that in some embodiments, other types of interventions (e.g., increasing and decreasing clock frequencies) may be used and / or more than two different levels of frequency scaling and / or power gating / scaling may be used.

[0066] It should be noted that although method 700 is described as being implemented using a resource manager located similarly to resource manager 316 as shown and described with respect to FIG. 3 (e.g., within IOMMU 312 or in communication with IOMMU 312), method 700 may be implemented using any suitable hardware capable of tracking the power of multiple processors.

[0067] In another example, Figure 8 is a flow chart illustrating an example method 800 for managing temperature in a computing device, such as device 300 shown and described with respect to Figure 3. In some embodiments, method 800 is performed by a resource manager located similarly to resource manager 316 shown and described with respect to Figure 3 (e.g., within IOMMU 312 or in communication with IOMMU 312).

[0068] In step 802, the resource manager captures data regarding the temperatures of the CPU 302 and GPU 304 over a time period Y. For example, in some embodiments, a resource manager located similar to the resource manager 316 captures the temperatures of the CPU 302 and GPU 304 over a time period of 1 minute (min), where the value Y=1 min is exemplary and in other embodiments other values ​​of Y can be used and / or these values ​​are dynamically variable and / or user configurable.

[0069] For each processor (e.g., CPU 302, GPU 304), the resource manager records the temperature (e.g., average, cumulative, etc.) during a time period. The resource manager also records the temperature of each processor during subsequent time periods, thus accumulating a time series of data reflecting the temperature of each processor.

[0070] Based on the accumulated time series data, the resource manager predicts for each processor the likely temperature during the next (or later) time period. The prediction is made in any suitable manner, such as using an autocorrelation function (ACF) or an artificial neural network (ANN).

[0071] On condition 804 that the average processor temperature is predicted to exceed the threshold, the resource manager, in step 806, reduces the system clock frequency for the predicted period of time if the system clock frequency is currently above the minimum frequency.

[0072] If the average processor temperature is predicted not to exceed the threshold 804, then the resource manager, in step 808, increases the system clock frequency for the predicted period of time if the system clock frequency is below the maximum frequency.

[0073] These particular thermal interventions (i.e., increasing and decreasing clock frequency) are merely exemplary. Note that in some embodiments, other types of interventions (e.g., power gating or scaling the entire system, the CPU and / or GPU, the CPU and individual cores or compute units of the CPU, etc.) may be used and / or more than two different levels of frequency scaling and / or power gating / scaling may be used.

[0074] It should be noted that although method 800 is described as being implemented using a resource manager located similarly to resource manager 316 as shown and described with respect to FIG. 3 (e.g., within IOMMU 312 or in communication with IOMMU 312), method 800 may be implemented using any suitable hardware capable of tracking the temperatures of multiple processors.

[0075] It should be understood that many variations are possible based on the disclosure herein, and although features and elements have been described above in particular combinations, each feature or element can be used alone without the other features and elements, or in various combinations with or without the other features and elements.

[0076] Various functional units shown in the figures and / or described herein, including but not limited to the processor 102, input paths 112, input devices 108, output paths 114, output devices 110, acceleration processing devices 116, scheduler 136, graphics processing pipeline 134, computational unit 132, SIMD unit 138, may be implemented as a general purpose computer, processor or processor core, or as a program, software or firmware stored in a non-transitory computer readable storage medium or another medium executable by the general purpose computer, processor or processor core. The provided methods may be implemented in a general purpose computer, processor or processor core. Suitable processors include, by way of example, a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) circuit, any other type of integrated circuit (IC), and / or a state machine. Such a processor may be manufactured by configuring a manufacturing process using the results of processed hardware description language (HDL) instructions and other intermediate data, including netlists (instructions that may be stored on a computer-readable medium). The result of such processing may be a mask work that is used in subsequent semiconductor manufacturing processes to produce a processor that implements features of the present disclosure.

[0077] The methods or flow charts provided herein may be implemented in a computer program, software, or firmware embodied in a non-transitory computer-readable storage medium for execution by a general purpose computer or processor. Examples of non-transitory computer-readable storage media include read only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital versatile disks (DVDs).

Claims

1. A resource management device, a circuit configured to obtain information regarding memory bandwidth utilization; a circuit configured to generate a prediction that a memory region stored in the first memory device will be addressed by a memory-intensive instruction in a future time period based on the information; and a circuit configured to move data stored in the memory region to a second memory device based on the prediction. Resource management device.

2. the resource management device comprises an input / output memory management unit (IOMMU), a processor in communication with the IOMMU, and / or a processor integrated with the IOMMU; The resource management device of claim 1.

3. The information indicates whether the previously executed instruction is a memory intensive instruction. The resource management device of claim 1.

4. The information indicates whether the previously executed instruction is a specified instruction. The resource management device of claim 1.

5. The information indicates whether the previously executed instruction was a particular type of instruction. The resource management device of claim 1.

6. a circuit configured to generate the prediction by applying an autocorrelation function (ACF) to the information; The resource management device of claim 1.

7. a circuit configured to generate the prediction by applying an artificial neural network (ANN) to the information; The resource management device of claim 1.

8. the first memory device comprises a relatively lower bandwidth memory and the second memory device comprises a relatively higher bandwidth memory; The resource management device of claim 1.

9. the first memory device comprises a synchronous dynamic random access memory (SDRAM), a double data rate (DDR) SDRAM, and / or a low power double data rate (LPDDR) SDRAM; The resource management device of claim 1.

10. the second memory device comprises a high bandwidth memory (HBM) and / or a cache memory; The resource management device of claim 1.

11. A resource management device, comprising: a circuit configured to obtain information regarding resource usage; a circuit configured to generate a prediction that a usage of the resource by the processor will exceed a threshold value in a future time period based on the information; and a circuit configured to adjust an operating parameter of the processor based on the prediction. Resource management devices.

12. The resources include power. The resource management device of claim 11.

13. the resources include a thermal budget; The resource management device of claim 11.

14. The operating parameter includes a voltage or a current. The resource management device of claim 11.

15. The operating parameters include a clock frequency. The resource management device of claim 11.

16. the operating parameters are adjusted for a portion of the processor; The resource management device of claim 11.

17. the resource management device comprises an input / output memory management unit (IOMMU), a processor in communication with the IOMMU, and / or a processor integrated with the IOMMU; The resource management device of claim 11.

18. the information indicating whether the processor's usage of a resource exceeded the threshold during a previous period of time. The resource management device of claim 11.

19. The prediction is generated by applying an autocorrelation function (ACF) to the information. The resource management device of claim 11.

20. The prediction is generated by applying an artificial neural network (ANN) to the information. The resource management device of claim 11.

21. 1. A method implemented in a resource management device, comprising: Obtaining information regarding memory bandwidth; generating a prediction based on the information that a memory region stored in the first memory device will be addressed by a memory-intensive instruction during a future time period; and migrating data stored in the memory region to a second memory device based on the prediction. method.

22. the resource management device comprises an input / output memory management unit (IOMMU), a processor in communication with the IOMMU, and / or a processor integrated with the IOMMU; 22. The method of claim 21.

23. The information indicates whether the previously executed instruction is a memory intensive instruction.

22. The method of claim 21.

24. The information indicates whether the previously executed instruction is a specified instruction.

22. The method of claim 21.

25. The information indicates whether the previously executed instruction was a particular type of instruction.

22. The method of claim 21.

26. The prediction is generated by applying an autocorrelation function (ACF) to the information.

22. The method of claim 21.

27. The prediction is generated by applying an artificial neural network (ANN) to the information.

22. The method of claim 21.

28. the first memory device comprises a relatively lower bandwidth memory and the second memory device comprises a relatively higher bandwidth memory; 22. The method of claim 21.

29. the first memory device comprises a synchronous dynamic random access memory (SDRAM), a double data rate (DDR) SDRAM, and / or a low power double data rate (LPDDR) SDRAM; 22. The method of claim 21.

30. the second memory device comprises a high bandwidth memory (HBM) and / or a cache memory; 22. The method of claim 21.

31. 1. A method implemented in a resource management device, comprising: Obtaining resource usage information; generating a prediction based on said information that utilization of the resource by the processor will exceed a threshold value in a future time period; and adjusting operating parameters of the processor based on the prediction. method.

32. The resources include power.

32. The method of claim 31.

33. the resources include a thermal budget; 32. The method of claim 31.

34. The operating parameter includes a voltage or a current.

32. The method of claim 31.

35. The operating parameters include a clock frequency.

32. The method of claim 31.

36. the operating parameters are adjusted for a portion of the processor; 32. The method of claim 31.

37. the resource management device comprises an input / output memory management unit (IOMMU), a processor in communication with the IOMMU, and / or a processor integrated with the IOMMU; 32. The method of claim 31.

38. the information indicating whether the processor's usage of a resource exceeded the threshold during a previous period of time.

32. The method of claim 31.

39. The prediction is generated by applying an autocorrelation function (ACF) to the information.

32. The method of claim 31.

40. The prediction is generated by applying an artificial neural network (ANN) to the information.

32. The method of claim 31.

Images