Thermal throttle mechanisms for a processing-in-memory architecture

Abstract

Methods, systems, and apparatus, including computational instructions / programs encoded on non-transitory computer-readable media, are disclosed for implementing thermal throttle mechanisms for a Processing-in-Memory (PiM) block of a memory device. A system obtains data indicating a temperature in the PiM block to a remote SoC as computations are executed at the PiM block and enable the remote SoC to detect an adverse thermal condition in the PiM block based on indications that the temperature exceeds a threshold temperature. The remote SoC determines a real-time thermal throttling response based on the extent to which the temperature exceeds the threshold temperature and generates a command to implement the thermal throttling response at the PiM block in real-time to alleviate the adverse thermal condition. The system executes the command and implements the thermal throttling response block by adjusting a clock signal, clock division ratio, or PiM command rate passed to the PiM block.

Description

THERMAL THROTTLE MECHANISMS FOR A PROCESSING-IN-MEMORY ARCHITECTUREBACKGROUND

[0001] This specification generally relates to thermal throttling in memory devices used to execute computations.

[0002] Modem computing systems often incorporate a wide variety of compute processing units that each offer different computing capabilities and trade-offs. Efficient execution of a given compute job often involves parsing computations into meaningful subtasks or workloads that are mapped to available processor cores of a computing system. The computations may be parsed and mapped based on suitability criteria, such as processor capability', performance, and power. Generally, this overall process of allocating portions of a computation to appropriate processor resources is referred to as a heterogeneous computation.

[0003] At least one processor core of the computing system can be an Intellectual Property block (“IP block”) that executes a respective portion of a computational operation for different multimedia workloads. Example use cases can involve processing image or speech data captured respectively by a camera or microphone on the mobile device as well as performing computations for generative artificial intelligence (“GenAI”) applications. The system can use a heterogeneous computing operation to process input samples derived from image data, speech data, a text corpus, or a combination of these. An example step in the heterogeneous computation can include processing data associated with the input samples using a memory device that provides in-memory processing or computing capabilities.SUMMARY

[0004] This specification describes techniques for detecting and responding to a range of thermal conditions in a memory device of an integrated system that includes data processing resources in the memory device being communicatively coupled to a system-on-chip (“SoC”) of the integrated system. The memory device can be a dynamic random-access memory (DRAM) device that includes a Processing-in-Memory (PiM) architecture configured to perform memor -bounded and compute-bounded computations for executing an inference workload or task in the memory device.

[0005] The PiM architecture defines one or more PiM blocks of the memory device and each PiM block includes computing resources / elements, such as a processor unit, moderegisters, and one or more computational units, e.g., arithmetic logic units (ALUs) or related addition and multiplication circuitry. The PiM blocks are used to execute computations for an example workload that originates at the SoC. The computations can be segmented into respective portions that are allocated between the SoC and the memory device that includes the PiM blocks. The workload can be for a machine-learning (“ML”) model, such as a large language model (“LLM") used to implement a GenAI application as well as other types of ML and non-ML workloads.

[0006] In some examples, the SoC cooperates with the memory device to implement (or run) an ML model and perform computations for an ML workload. A portion of the computations may be assigned to the PiM block, which includes integrated switching and arithmetic circuitry that generates heat when power and clock signals are applied to circuitry to execute the computations. The disclosed thermal throttling techniques can be used to alleviate or preclude occurrences of hotspots and other adverse thermal conditions that can degrade performance of circuitry at the PiM block and across the SoC. The disclosed throttling mechanisms can detect and respond to a range of PiM and system-level thermal conditions for effective thermal management.

[0007] A real-time throttling controller of the SoC generates thermal throttling responses that allow for dynamically controlling PiM compute loads and data accesses for memory' device and system-level thermal management. The real-time throttling controller can generate control signals that configure mode register values of the memory device, where the values are used to adjust or reduce an operating frequency at the PiM block. For example, the frequency can be reduced by adjusting a PiM clock signal at a clock divider of the PiM block. The PiM block performs the computations at the reduced operating frequency- established by the control signals. Operating the PiM block at the reduced frequency alleviates adverse thermal conditions from elevated PiM temperatures that were triggered by the higher PiM operating frequency.

[0008] One aspect of the subject matter described in this specification can be embodied in a method implemented using an integrated circuit comprising an SoC and a memory device coupled to the SoC. The method includes obtaining temperature data indicating a temperature in a PiM block of the memory device as computations are being executed at the PiM block and detecting an adverse thermal condition in the PiM block based on an indication in the temperature data that the temperature has exceeded a threshold temperature.

[0009] The method includes determining a thermal throttling response to alleviate the adverse thermal condition. The thermal throttling response is determined based on the extentto which the temperature exceeds the threshold temperature. The method also includes: i) generating (e.g., at the SoC) a command to implement the thermal throttling response at the PiM block in real time and ii) executing the command and implementing the thermal throttling response at the PiM block by adjusting a clock signal, clock division ratio, or PiM command rate passed from the SoC to the PiM block.

[0010] These and other implementations can each optionally include one or more of the following features. For example, in some implementations, determining a thermal throttling response includes determining a thermal throttling response rate in accordance with the temperature of the PiM block such that the thermal throttling response corresponds to a delta between the threshold temperature and the temperature at the PiM block. The SoC obtaining the temperature data in a PiM block includes reading the temperature data from a mode register in the memory device that receives the temperature data from one or more temperature sensors in the PiM block. The method can further include controlling an operating voltage of the PiM block to alleviate the adverse thermal condition by implementing an additional measure of thermal reduction at the PiM block.

[0011] Adjusting a clock signal can include: i) writing a clock control value to the mode register in the memory device; and ii causing, based on the clock control value, an adjustment to the clock signal provided to the PiM block. In some implementations, i) the clock control value written to the mode register specifies a clock division ratio; and ii) the clock control value causes a clock divider circuit of the memory device to adjust the clock signal provided to the PiM block based on the clock division ratio specified by the clock control value. In some implementations, each of the one or more temperature sensors in the PiM block writes temperature data to the mode register based on a respective write frequency of the corresponding temperature sensor in the PiM block. In some cases, each respective write frequency of the corresponding one or more temperature sensors is dynamically adjustable by the SoC.

[0012] The method can further include dynamically adjusting a respective write frequency of a corresponding temperature sensor in the PiM block based on a compute load. Determining a thermal throttling response can include i) computing a temperature compensation value associated with obtaining the temperature in the PiM block; and ii) determining a thermal throttling response based on the temperature compensation value. In some implementations, the temperature compensation value is computed based on: i) a temperature slope associated with obtaining the temperature in the PiM block; and ii) a portion of a temperature gradient associated with obtaining the temperature in the PiM block.

[0013] In some implementations, the temperature gradient is a measure of time that includes: i) a read interval for reading the temperature data from the mode register, and ii) a system response delay for implementing the thermal throttling response at the PiM block. Determining a thermal throttling response can include determining a real-time thermal throttling response rate using a real-time throttling controller of a memory controller on the SoC. Determining a thermal throttling response can also include determining a thermal throttling response using a system-level thermal balancer on the SoC. In some implementations, the system-level thermal balancer determines thermal throttling responses based on: i) the temperature at the PiM block; ii) system-level thermal condition data for distinct heat sources; and iii) system-level power monitoring and thermal monitoring information.

[0014] In some implementations, the method further includes determining multiple temperature values representing a range of temperatures for the PiM block, where each temperature in the range of temperatures exceeds the threshold temperature; and for each temperature in the range of temperatures: determining a respective thermal throttling response for the corresponding temperature. In some implementations, the method further includes determining a mapping between: i) a particular temperature in the range of temperatures for the PiM block; ii) a corresponding temperature value of the multiple temperature values that represent the particular temperature; and iii) a corresponding thermal throttling response that was determined for the particular temperature.

[0015] The adverse thermal condition can be a hot spot represented by a high temperature in a localized area of the memory device. In some implementations, detecting an adverse thermal condition in the PiM block includes: i) computing a function of temperature data obtained from at least two different temperature sensors in the memory device; ii) generating an estimated temperature value of the high temperature at the localized area as a result from the computed function; and iii) detecting the adverse thermal condition in the PiM block based on the estimated temperature value.

[0016] One aspect of the subject matter described in this specification can be embodied in an integrated circuit comprising: SoC’?); a memory device coupled to the SoC; and a processor and a non-transitory machine-readable storage medium of the SoC (or memory device) for storing instructions that are executable by the processor to cause performance of various operations. The operations can include obtaining temperature data indicating a temperature in a PiM block of the memory device concurrent with computations being executed at the PiM block and detecting an adverse thermal condition in the PiM block basedon an indication in the temperature data that the temperature has exceeded a threshold temperature.

[0017] The operations include determining a thermal throttling response to alleviate the adverse thermal condition. The thermal throttling response is determined based on the extent to which the temperature exceeds the threshold temperature. The method also includes: i) generating (e.g., at the SoC) a command to implement the thermal throttling response at the PiM block in real time and ii) executing the command and implementing the thermal throttling response at the PiM block by adjusting a clock signal, clock division ratio, or PiM command rate passed from the SoC to the PiM block.

[0018] Other implementations of this and other aspects include corresponding systems, apparatus, and computer programs, configured to perform the actions of the methods, encoded on computer storage devices. A system of one or more computers can be so configured by virtue of software, firmware, hardware, or a combination of them installed on the system that in operation causes the system to perform the actions. One or more computer programs can be so configured by virtue of having instructions that, when executed by a data processing apparatus, cause the apparatus to perform the actions.

[0019] The subject matter described in this specification can be implemented in particular embodiments to realize one or more of the following advantages.

[0020] In contrast to prior / conventional approaches for integrated system-level controls, the disclosed techniques provide hardware and software mechanisms for accurately detecting and responding to thermal conditions and hotspots of computing resources in a memory device. The disclosed techniques provide multiple comprehensive thermal management response options to reduce, or prevent, potential occurrences of thermal runaway or degraded performance. For example, based on the detected hotspots and thermal conditions, the techniques can be used to dynamically and / or collaboratively control a computational load of a PiM block in the memory device.

[0021] The proposed techniques provide granular thermal management controls for implementing a range of real-time throttling responses at the PiM block, where the responses can correspond directly to the types of thermal conditions (e.g.. hot spots) and the amount of thermal load detected. In some instances, the real-time control for throttling compute loads in a memory device are combined with techniques for dynamic voltage & frequency scaling (“DVFS”) of an integrated system. This combined approach provides enhanced thermal management options that leverage real-time throttling controls with operating voltage andfrequency scaling to alleviate certain thermal conditions in memory cells, banks, and computing circuits of a memory device.

[0022] The details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other potential features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Fig. 1 is a block diagram of an example computing system with at least one SoC.

[0024] Fig. 2 shows an example PiM architecture with corresponding computing and data processing resources.

[0025] Fig. 3 illustrates an example implementation of temperature detection and thermal throttling at the system of Fig. 1.

[0026] Fig. 4 illustrates examples of sensors and temperature indications for PiM operations with memory cell transactions.

[0027] Fig. 5 shows examples of different mode features used for thermal management.

[0028] Fig. 6 illustrates example graphical data for temperature compensation predictions.

[0029] Fig. 7 shows example parameter tables for distinct thermal throttle modes.

[0030] Fig. 8 is a first example process for temperature detection and thermal throttling in a memory device.

[0031] Fig. 9 is a second example process for temperature detection and thermal throttling in a memory device.

[0032] Like reference numbers and designations in the various drawings indicate like elements.DETAILED DESCRIPTION

[0033] Fig. 1 is a block diagram of an example computing system 100 that includes a system-on-chip 102 (“SoC 102”). The SoC 102 includes a central processing unit 104 (“CPU 104”), a memory controller 105, a shared memory 106 (“memory 106”), a resource manager 108, and an IP / circuit block 110. In some implementations, system 100 can include multiple SoCs and any descriptions for the SoC 102 will apply equally to each of the multiple SoCs that may be included at system 100.

[0034] The CPU 104 can be a general-purpose CPU (e.g.. a single or multi-core CPU). The CPU 104 generates one or more indicators, such as an app-launch indicator or a function call that is triggered in response to executing or launching an application at a user device. For example, the application can be a camera application that uses an imaging sensor to generate image data or a gaming application that requires substantial memory and graphics processing resources to render graphical content of the game. The CPU 104 also generates one or more application values, such as pixel values or frame rate. The application values may be associated with a function call, may be descriptive of an event that occurs during execution of the application, or both.

[0035] The memory 106 is a system memory, shared memory , or both. In the example of Fig. 1, memory 106 is depicted external to circuit block 110. However, memory 106 can include portions of memory that are: i) specific to circuit block 110, ii) external to circuit block 110, or iii) both. The memory7106 can be random access memory of the SoC 102, such as static random-access memory (SRAM), dynamic random-access memory7(DRAM), a synchronous DRAM (SDRAM), or double data rate (DDR) SDRAM.

[0036] In some implementations, aspects of memory 106 are configured as a shared scratchpad memory that supports parallel access of its memory resources by two or more processors of the circuit 110. The memory7106 can also include various other ty pes of memory, such as high bandwidth memory (HBM), narrow memory (e.g., for storing 8-bit values), wide memory (e.g., for storing 16-bit or 32-bit values), or a combination of these.

[0037] The resource manager 1 8 is implemented in hardware and software. Aspects of the resource manager 108 can be also implemented as firmware of the SoC 102 or firmware of a device of the SoC 102, such as a DRAM memory device or the CPU 104. The resource manager 108 is a processor-in-memory (PiM) resource manager (“PiM resource manager 108”) that includes control logic implemented in hardware, software, or both. For example, the PiM resource manager 108 can include resources such as flip-flops, registers, buffers, cache units, or a combination of these, which may be implemented in hardw are, along with related control logic (e.g., programmed code), which may be implemented in software as well as hardware.

[0038] The circuit block 1 10 generally includes individual IP devices such as processors, processor cores, or special-purpose processing devices. For example, the circuit block 110 can include an image signal processor (ISP) 112, a host (or special purpose) processing unit (HPU) 114, a digital signal processor (DSP) 116, and a graphics processing unit (GPU) 118. The circuit block 110 is referred to alternatively as an IP block 110, where the IP block caninclude one or more proprietary hardware elements. For example, each of the ISP 112, HPU 114, DSP 116, and GPU 118 can be a respective proprietary IP block (or IP device) of a particular entity or device manufacturer.

[0039] One or more aspects of the PiM resource manager 108 can be implemented as a software routine (or module) of the CPU 104, which uses one or more hardware resources of the CPU 104, such as registers, buffers, etc. The CPU 104 can be configured as an instruction and vector data processing engine that processes data obtained from a system memory of the SoC 102, such as memory 106. In some implementations, each processor (e.g., ISP 112, DSP 116, HPU 114, GPU 118) of the SoC 102 includes multiple cores and the CPU 104 and / or the PiM resource manager 108 can generate control signaling 124 to manage and distribute memory intensive compute operations to a memory device 122 (e.g.. DRAM) to minimize the processing load at each core of the processors. The control signaling 124 is routed at system 100 using an example bus 120 of the SoC 102. The control signaling 124 can include commands, requests, data, instructions, or combination of these.

[0040] The PiM resource manager 108 cooperates with the CPU 104. memory controller 105 and storage controller 107 to dynamically control and manage one or more compute-inmemory (CiM) operations. In some implementations, the CiM operations are executed at the SoC 102 in support of a heterogeneous compute operation between two or more processing units that are included among the IP block 110, the CPU 104, or both. More specifically, the PiM resource manager 108 is configured to generate control signaling 124 and use one or more discrete signal values of the control signaling 124 to manage, configure, and / or boost data access operations at the memory device 122.

[0041] In general, techniques implemented at system 100 can include generating data and control signaling at the SoC 102, which are used to communicate with the PiM architecture and memory device 122 by way of a memory controller 105 of the SoC 105. For example, the data and control signaling 124 generated at the SoC 102 are passed to, and processed by, compute elements of the PiM blocks 202 within a given PiM architecture 200 to execute the computations for an example workload. In some implementations, the data / control signaling 124 are processed at the PiM architecture 200 to trigger execution of certain data processing and computing operations and thermal throttling operations using compute intervals that represent different pipeline stages of the PiM blocks 202. Control signals can also be generated locally at the memory device 122, which may be external to, or separate from, the SoC 102.

[0042] The system 100 includes an example memory device 122. The memory device 122 can include multiple memory dies. For example, the memory device 122 can include N memory die, where N is an integer greater than 1. The memory device 122 can be a dynamic random-access memory (DRAM) or Double Data Rate (DDR) synchronous DRAM (SDRAM). The memory device 122 is configured to perform or support various ty pes of PiM operations, CiM operations, and memory-near-computing operations (“MnC operations’7). The memory device 122 performs or supports these operations using data processing resources and / or compute elements of its PiM architecture, which are described below with reference to at least Fig. 2.

[0043] The SoC 102 cooperates with the memory device 122 to perform computations across one or more bank groups of the memory device 122. The computations can be for operations or workloads that involve one or more of the processors at IP block 110. Additionally, the computations can be for a heterogenous operation that spans multiple processors of IP block 110, multiple IP blocks 110, or both. In at least one example the memory device 122 may be external to the SoC 102, whereas in another example the memory device 122 may be internal to the SoC 102. In some implementations, the SoC 102 is integrated (or co-located) with the memory device 122, for example, as distinct circuit die(s) that are co-located in a single integrated circuit package.

[0044] In the example of Fig. 1, system 100 and the SoC 102 is an integrated circuit of an example user / client device 130, consumer electronic device, or mobile device, where each of these devices can include items such as a smartphone 130a, tablet 130b, laptop 130c, smartwatch or wearable device 130d. The devices 130 may also include other items such as an eNotebook, Netbook, smart speaker, or mobile computer. In some implementations, the system 100 and the SoC 102 are integrated circuits of a desktop computer, network server, or related cloud-based asset.

[0045] Fig. 2 shows an example processor-in-memory (PiM) architecture 200 for boosting PiM data access performance based on control signals generated using the SoC 102, the memory device 122, or both. In the example of Fig. 2, the memory device 122 includes a first memory die-1 with a first bank group that has multiple memory banks, where each memory bank includes one or more memory arrays and a second memory' die-2 with a second bank group that has multiple memory banks, where each memory' bank includes one or more memory arrays. In some implementations, the PiM architecture 200 includes multiple bank groups, multiple memory die, or both. For example, a single memory’ die can includemultiple bank groups and / or multiple bank groups can be distributed across multiple memory7die.

[0046] The PiM architecture 200 includes multiple PiM blocks, where each PiM block includes multiple compute elements. For example, a first PiM block of PiM architecture 200 includes mode register 204-1 and process unit 206-1, whereas a second, different PiM block of PiM architecture 200 includes mode register 204-2 and process unit 206-2. Each process unit 206-1. 206-2 can include a processor, a processor unit, or a processor core, such as a CPU. Each process unit 206-1, 206-2 can also include an example computation unit such as an arithmetic logic unit (ALU) or multiply-accumulate cell (MAC).

[0047] In some implementations, the PiM architecture 200 is included in the memon' device 122 as multiple discrete integrated circuits, where each integrated circuit is local to a given memory die (e.g., die-1 and die-2) and interacts or communicates with arrays of memory cells at that memory die. For example, the PiM architecture 200 can include compute elements that are replicated and distributed across each of the memory die in the memory device 122. In some other implementations, the PiM architecture 200 is included in the memory device 122 as a single integrated circuit that interacts or communicates with each memory die of the memory device 122, including the arrays of memory cells at each memory die.

[0048] To boost PiM data access performance as described in this specification, the PiM blocks or process units in the PiM architecture 200 are located within the memory device 122 but outside of a section of the memory device 122 that includes the bank groups. The section may be defined as a discrete memory die or defined in some other way (e.g., a portion of a memory' die). Irrespective of the hardware configuration or layout of PiM architecture 200, the PiM blocks are sufficiently external to the bank groups such that the PiM blocks communicate with the bank groups based on a particular timing constraint that can be leveraged to boost PiM data access performance with cross bank group data aggregation. A data channel / interconnection can be established between the individual memory7arrays of a memory bank and a corresponding processor unit and / or mode register of a PiM block 202.

[0049] The PiM operations can be managed and executed at the memory device 122 using a processor device that provides functionality similar to a central processor, such as CPU 104. The CiM operations and MnC operations can include standard arithmetic operations, such as computations normally performed by an ALU or MAC. The CiM operations and MnC operations can also include computational functions of a HPU 114, such as multiplication and addition operations for matrix math, vector computations, linearalgebra, and dot-product accumulations. In some implementations, each of the PiM operations. CiM operations, and MnC operations are performed in support of machinelearning computations, neural network computations, or both.

[0050] In some implementations, the PiM operations are an extension of the computational functions of the HPU 114. For example, a PiM block can generate accumulated values from sets of weight values / inputs and activation inputs obtained from memory banks of different bank groups in the memory device 122. The accumulated values are generated based on neural network computations performed using a computational array of the PiM block. The computational array can be a matrix multiplication unit with compute cells that are arranged as a systolic array. The accumulated values can be dot products of the sets of weight values and the activation inputs. That is, for a set of weights, the PiM block multiplies each weight with each activation input and sums the products together to form an accumulated value.

[0051] The PiM architecture 200 can include a register or other portion of memory for storing data for a respective memory die or group of memory banks. For example, the data can be mode / configuration / throttling values. The data can also describe errors that occurred during a compute operation at a corresponding PiM block of the memory device 122, or both. In some implementations, the register or other portion of memory' is used to store thermal throttling information, or associated instructions, for configuring aspects of a PiM block, or respective memory die, group of memory banks, or a combination of these.

[0052] For example, the mode registers 204-1 , 204-2 can be used to control or trigger selection of a particular mode in a PiM architecture, such as an error-capture mode, throttle mode, interleave configuration mode, multi-batch processing mode, etc. In some implementations, a particular mode is selected based on bit values of the mode registers 204- 1 , 204-2. For example, to trigger or select a thermal throttling mode(s) or multi-batch processing mode(s), a single bit, or a sequence of bits, can be defined for use in the mode register. This is described in more detail below with reference to the example of Fig. 7.

[0053] In some implementations, data access operations can be optimized at the memory device 122 by reading memory cells of bank groups at a frequency that exceeds other read operations that are subject to certain delay constraints for executing successive reads against banks of the memory' device 122. For example, an internal controller of the PiM block (1), (2) can execute successive read commands at a frequency that is based on a clock cycle generated by the memory device 122. The internal controller can operate based on aparticular clock frequency, e.g., a 200 or 800 MHz clock or 1000 MHz clock. Other clock frequencies are also within the scope of this disclosure.

[0054] Fig. 3 illustrates an example implementation of temperature detection and thermal throttling at the system of Fig. 1. In the example of Fig. 3, the SoC 102 and memory device 122 are integrated in an example mobile / client device 130. In some implementations, the SoC 102 and memory device 122 can be configured in a stacked configuration, in a single integrated circuit package, such that a bottom surface of the memory device 122 is mounted adjacent to a top surface of one or more circuit die that represent the SoC 102. The device 130 includes an integrated system 300 with example system components 320. The integrated system 300 can be a sub-system of system 100 and is described alternatively as system 300.

[0055] Circuit / system 300 includes the SoC 102, memory controller 105. and memory device 122 (described above), and a memory physical layer (PHY) interface that is intermediate the memory controller 105 and memory device 122. As indicated at Fig. 3, a device of the SoC 102 can be configured as a host device (e.g., HPU 114) that communicates with the memory device 122 to execute an example heterogeneous operation. For example, heterogeneous operations can include computations for vector math or matrix multiplication to execute certain ML workloads.

[0056] The host device 114 of the SoC 102 can be a special-purpose neural network processor or hardware ML accelerator configured to accelerate computations for generating different types of data processing and ML outputs. In some implementations, one or more of the PiM operations, CiM operations, or MnC operations are performed by the memory device 122 to support or enable accelerating computations for generating different types of data processing outputs.

[0057] Each of the host device, system fabrics and memory controller 105, and memory PHY interface 107 include and / or operate within a frequency & voltage domain of the system 100. Additionally, each of these components can be configured to support and / or implement thermal throttling controls and / or mechanism at the system 100, 300. The memory controller 105 is configured for data communication with mode registers 302. 304 in the memory device 122. The mode register 302 stores data indicating a self-refresh rate for refreshing or pre-charging data stored at memory cells of the memory device 122, whereas mode register 304 stores temperature data 310, which indicates PiM temperatures that are detected by localized temperature sensors in the PiM block 202.

[0058] The memory controller 105 uses the mode register 304 to implement thermal throttling and thermal management at the PiM block 202 based on the temperature data 310.The memory controller 105 includes control logic for implementing a real-time throttling controller 306 for determining and establishing thermal throttling response values at the memory device 122. In some implementations, the host device 1 14 includes a system-level thermal balancer 308 for determining and establishing thermal throttling response values across system 100. Each of real-time throttling controller 306 and thermal balancer 308 can be implemented in hardware, software, or both.

[0059] The memory controller 105 uses the memory PHY interface 107 to convey thermal throttling control and / or command signals 312, 314 from the SoC 102 to the memory device 122 in accordance with temperature data read or obtained from the mode register 304 in the memory device 122. For example, the signals 312, 314 can represent a DRAM operating frequency 312 and a PiM block operating frequency 314. In some implementations, the signals 314 are data signals for writing certain thermal throttling values to a new mode register 304. In some other implementations, the signals 314 are clock signals for establishing or adjusting PiM operating points (e.g., target operating frequency) based at least on the temperature data captured from the mode register 304.

[0060] Relatedly, the signals 314 can also be data / control signals that configure mode register values of the memory device 122, where the values configured at the mode register 304 are used to adjust or reduce an operating frequency at the PiM block 202. For example, the frequency can be reduced by adjusting a PiM clock signal 314 at a clock divider 318 of the PiM block 202 to produce an adjusted or reduced clock signal 320. Thus, the control / command signals 314 can include adjusted or reduced operating frequency and operating voltage controls for configuring thermal throttling responses at the PiM block 202.

[0061] The memory device 122 processes signal communications from the SoC 102 in accordance with a clock signal 312. The signal communications 312, 314 can include controls, requests, and / or command signals from the SoC 102, more specifically, the memory controller 105. For example, the request signals can also include or correspond to read / write commands for reading data from, or writing data to, registers, memory7cells / banks of the memory device 122. Relatedly, and as indicated above, the signal communications 312, 314 also include thermal throttling control signals 314 that can represent commands or a division ratio value from the SoC 102 for establishing a certain clock signal at the PiM block 202.

[0062] In some implementations, the clock signal 312 can be a master or global DRAM clock signal that drives memory access and PiM operations at the memory device 122. In some implementations, the clock signal 312 is generated by, or using, the SoC 102 and provided to the memory device 122 from the SoC 102. In some other implementations, theclock signals 312, 314 are generated locally at the memory device 122 based on power signals routed via traces / lines of the integrated circuit 300. For example, the clock signal 314 may be generated independent of the SoC 102.

[0063] The memory device 122 can be configured to include or generate one or more local clock signals that trigger functions of components in the PiM architecture. The system 100, 300 can include an example clock divider circuit 318 that generates one or more local PiM clock signals 320. In some implementations, the PiM clocks (or clock signals) 320 are derived or generated based on clock signal 312. For example, the PiM clock signals 320 can be generated from a branch signal of a DRAM clock(s) of the memory device 122. In some implementations, memory device 122 includes an independent clock source that is local to one or more PiM blocks 202.

[0064] Fig. 4 illustrates a memory device 122 with examples of sensors and temperature indications for PiM operations with DRAM memory cell transactions. The memory device 122 includes a DRAM block 402 with a DRAM temperature sensor 404 and a PiM block 202 with a PiM temperature sensor 406. In the example of Fig. 4. each PiM temperature sensor 406 is inside a corresponding processing unit or PiM block 202 and is configured to measure or obtain a temperature of the PiM block 202. To detect hotspots 408 the memory device 122 can include at least one PiM temperature sensor 406 per one processing unit of a PiM block 202.

[0065] In some implementations, the memory controller 105 includes temperature estimation logic for estimating the temperature of DRAM cells or locations 410 that are near or within a threshold proximity to a PiM temperature sensor 406. The system 100 can compute the temperature estimations to obtain a more reliable refresh rate for refreshing or pre-charging data stored at memory cells of the memory device 122. As shown at Fig. 4, the DRAM temperature sensor 404 can be placed or located in DRAM cells or banks to capture local DRAM temperatures and temperature data (416). In some instances, DRAM cells that are near or within a threshold proximity of a PiM temperature sensor 406 can have higher temperatures, corresponding to hot spots 408, than local DRAM temperatures that are captured by DRAM temperature sensor (418).

[0066] In some implementations, the temperature values of the hot spots 408 can be estimated by computing a function of temperature readings from at least two temperature sensors of the memory device 122, namely, the DRAM temperature sensor 404 and the PiM temperature sensor 406. Thus, the real-time throttling controller 306 can compute a function of temperature data obtained from at least two different temperature sensors in the memorydevice 122 and generate an estimated temperature value of a high temperature at the localized area as a result from the computed function. The real-time throttling controller 306 can then detect the adverse thermal condition (e.g., hot spot 408) in the PiM block 202 based on the estimated temperature value.

[0067] As indicated above, the mode register 304 is configured to store the temperature data generated by the PiM temperature sensors 406. In some implementations, the PiM temperature sensors 406 write or transmit data to the mode register 304 at a particular frequency. The write frequency of the PiM temperature sensors 406 can be dynamically increased or decreased based on a compute load, operating frequency, or both. Thus, the SoC 102 and / or the memory device 122 can dynamically adjust a respective write frequency of a corresponding temperature sensor in the PiM block 202 based on a compute load at the PiM block.

[0068] For example, a PiM temperature sensor 406 can be configured to increase its write frequency to record more PiM temperature values when the PiM operating frequency exceeds a threshold frequency. Relatedly, the PiM temperature sensor 406 can be configured to decrease its write frequency to record fewer PiM temperature values when the PiM operating frequency falls below a threshold frequency. The PiM block 202 can cause a corresponding PiM temperature sensor 406 to perform similar write frequency adjustments based on a compute load at the PiM block 202. In some implementations, the compute load can be determined based on the number of instructions or opcodes in an instruction buffer or mode register of the PiM block 202.

[0069] In some implementations, the system-level thermal balancer determines thermal throttling responses based on: i) the temperature at the PiM block(s) 202; ii) system-level thermal condition data for distinct heat sources corresponding to hot spots 408; iii) systemlevel power monitoring and thermal monitoring information; or iv) a combination of these. The system-level power monitoring and thermal monitoring information can include thermal conditions and hotspots of computing resources within the PiM architecture 200 of the memory device 122 as well as within the DRAM memory banks and memory cells of the memory device 122. In some implementations, the system-level power monitoring and thermal monitoring information can include DVFS data corresponding to operating frequency and voltages settings across the integrated compute cells and memon cells of memory7device 122. The thermal monitoring information includes the corresponding thermal response that results from a particular DVFS setting applied at the memory device 122. As describedherein, the thermal response can be indicated by temperature sensors integrated across memory device 122.

[0070] The memory controller 105 periodically reads the PiM temperature data from the mode register 304 and processes the temperature data to detect or identify any hot spots, exceedingly high temperatures, and / or adverse thermal conditions at the PiM block 202. In some implementations, the memory controller 105 can adjust its frequency based on the write frequency of the PiM temperature sensor 406. As described below with reference to example of Fig. 6, the system 100 is configured to apply a compensation temperature adjustment based on a mode register read latency of the memory controller 105 and the recent temperature slope of the PiM temperature sensor 406.

[0071] Fig. 5 shows examples of different mode features used for thermal management. More specifically, op-code bit definitions 502, 504 are shown for mode register features 506, 508 corresponding to a potential refresh rate (506) that can be written to mode register 302 and corresponding to a potential PiM temperature (508) that can be written to mode register 304, respectively.

[0072] Fig. 6 illustrates example graphical data 600 for temperature compensation predictions. The SoC 102 is configured to implement compensation and prediction logic when reading or obtaining temperature data from the mode registers. For example, to reduce reading overheads between the SoC 102 and a PiM block 202, each of the host device 114 and / or the memory controller 105 is configured to read or obtain temperature data from the mode registers 204, 304, based on a recent temperature slope 602 and a corresponding latency time 604. The recent temperature slope can be tied to a trip level of a mode register 204, 304 in the memory device 122.

[0073] In some implementations, the trip level represents a threshold temperature that triggers indication of an adverse thermal condition, such as a hot spot 408, when a PiM temperature value that is written to the mode register indicates a PiM temperature exceeds the threshold temperature. The SoC 102 determines a thermal throttling response to alleviate the adverse thermal condition. For example, the real-time throttling controller 306 determines thermal throttling response at least by computing a temperature compensation value 606 associated with obtaining the temperature in the PiM block 202. The SoC 102 then determines the thermal throttling response based on the temperature compensation value 606.

[0074] The temperature compensation value 606 can be computed based on: i) a temperature slope associated with obtaining the temperature in the PiM block; and ii) a portion of a temperature gradient associated with obtaining the temperature in the PiM block.The temperature gradient is a measure of time that includes: i) a read interval 608 for reading the temperature data from the mode register, and ii) a system response delay 610 for implementing the thermal throttling response at the PiM block. The expression “temperature slope” may also be referred to as a rate of change (e.g., increase) of the temperature in the PiM block. The expression “temperature gradient” may also be referred to as a latency time associated with obtaining the temperature in the PiM block.

[0075] Hence, the thermal throttling mechanisms applied by the real-time throttling controller 306 (or thermal balancer 308) are tuned to apply thermal throttling responses that also compensate for incremental temperature increases corresponding to temperature gradients in the graphical data 600.

[0076] Fig. 7 shows example parameter tables for distinct thermal throttle modes. In the example of Fig. 7, a first throttle mode 702 includes PiM temperatures 704, thermal throttling rates or percentages 706, clock signals 708, and corresponding PiM operating frequencies 710. Relatedly, a second throttle mode 720 also includes PiM temperatures 722, thermal throttling rates or percentages 724, clock signals 726, clock division ratios 728, and corresponding PiM operating frequencies 730.

[0077] When the first throttle mode 702 is selected the system 100 controls a DRAM clock (CK) frequency to throttle the PiM block operating frequency, whereas when the second throttle mode 720 is selected the system 100 controls or adjusts a clock control division ratio to throttle the PiM block operating frequency. The clock control division ratio can be represented as a CK-to-PiM clock ratio, which indicates a division of the DRAM CK frequency. In some implementations, CK stands for the clock signal in a low-power DDR (“LPDDR”) memory, such as memory device 122. For example, LPDDR devices operate using a differential clock, which includes CK and CK#, where the crossing of CK going HIGH and CK# going LOW is referred to as the positive edge of CK.

[0078] As shown at Fig. 7, for each of modes 702, 720, the SoC 102 can determine or select a representative set of temperature values 704, 722 that represent a range of PiM temperatures for the PiM block(s) 202. For each temperature in the range of temperatures, the real-time throttling controller 306 can determine a respective thermal throttling response rate 706, 724 for the corresponding temperature.

[0079] As indicated at Fig. 7, the system 100 can determine a mapping between: i) a particular PiM temperature in the range of temperatures 704, 722 for the PiM block; ii) a corresponding temperature value that represents the particular temperature; iii) a corresponding thermal throttling response 706, 724 that was determined for the particulartemperature; and iv) a corresponding PiM operating frequency 710. 730 that was determined for implementing the thermal throttling response. For clarity, a corresponding temperature value that represents the particular temperature is an actual temperature obtained using a temperature sensor within a PiM block. The temperature sensor can be also at or near a compute cell (or circuit) of the PiM architecture 200. For example, the corresponding temperature value that represents the temperature can be a PiM temperature value that is written to the mode register after being obtained via the temperature sensor. In some cases, temperatures in the range of temperatures may exceed a corresponding threshold temperature.

[0080] The mode registers 204, 304 can be used to control or trigger selection of a particular mode in the PiM architecture of the memory device 122. The particular modes include first throttle mode 702 or second throttle mode 704. In some implementations, a particular mode 702, 704 is selected based on bit values of the mode registers 204, 304. For example, to trigger or select a thermal throttling mode(s), a single bit, or a sequence of bits, can be defined for use in the mode register.

[0081] Fig. 8 is a first example process for temperature detection and thermal throttling in a memory device. Process 800 is implemented or executed at system 100 using at least the SoC 102 and memory device 122 described above, including the integrated system / circuit 300 described above with reference to Fig. 3. Hence, descriptions of process 800 will reference the above-mentioned computing resources of system 100, 300. In some examples, the steps or actions of process 800 are enabled by programmed software instructions, firmware instructions, or both. Each type of instruction may be stored in a non-transitory machine-readable storage device and is executable by one or more of the processors or other resources described in this specification.

[0082] Referring again to process 800, the system 100 obtains temperature data indicating a temperature in a PiM block 202 of the memory device 122 as computations are being executed at the PiM block (802). For example, the memory controller 105 obtains the temperature data by periodically reading temperature values from the mode register 304 of the memory device 122. In some implementations, the computations are matrix multiplications that are performed at PiM block 202 using operands accessed from DRAM banks of the memory device 122.

[0083] The system 100 detects an adverse thermal condition in the PiM block 202 based on an indication in the temperature data that the temperature has exceeded a threshold temperature (804). In some implementations, the adverse thermal condition is a hot spot represented by a high temperature in a localized area of the memory device. The hot spot canbe local or near to a PiM temperature sensor 406. For example, the detected hot spot can be in a DRAM cell that is sufficiently close to a PiM block 202 such that heat generated in the PiM block 202 contributes, or substantially contributes, to the occurrence of the hot spot in the DRAM cell.

[0084] The system 100 determines a real-time thermal throttling response to alleviate the adverse thermal condition (806). For example, the real-time throttling controller 306 determines the thermal throttling response based on the extent to which the temperature at the PiM block 202 exceeds the threshold temperature. The real-time throttling controller 306 determines a thermal throttling %, e.g., 100%: full speed, 0%: complete stop, according to the PiM temperatures obtained by the PiM temperature sensor 406. The real-time throttling controller 306 implemented in the memory controller 105 is configured to execute its compensations much faster (e.g., tens of microseconds faster) than thermal management software that runs or executes on an example host device (e.g., CPU 104, GPU 118, or S / HPU 114), which may require tens of milliseconds to execute.

[0085] In some implementations, the real-time throttling controller 306 determines the thermal throttling response with reference to minimum thresholds for power efficient operating points that can achieve a processing, compute, and emory access speed required to generate computation results, such as ML outputs, that satisfy certain user-experience criteria. For example, the user-experience criteria can include a certain compute latency for generating a result or a rate of displaying information on a display of a client device 130.

[0086] The system 100 generates a command to implement the thermal throttling response at the PiM block 202 of the memory device (808). For example, the memory7controller 105 can generate the command based on thermal throttling determinations computed by the real-time throttling controller 306. The system 100 executes the command and implements the thermal throttling response at the PiM block 202 by adjusting a clock signal that is passed from the SoC 102 to the PiM block (810). The system 100 performs computations at the PiM block 202 in response to implementing the thermal throttling response based on the adjusted clock signal.

[0087] In some implementations, the SoC 102 adjusts the clock signal by writing a clock control value to the mode register 304 in the memory device 122. For example, the SoC 102 can use the memon' controller 105 to write or otherwise pass the clock control value to a mode register 304 or other configuration register of the PiM block 202. The PiM architecture 200 triggers or causes an adjustment to the clock signal provided to the PiM block 202 based on the clock control value. For example, a processing unit 206 or other related control logicof the PiM block 202 can read that clock control value and triggers the adjustment of the clock signal based on the control value. In some implementations, the clock control value written to the mode register specifies a clock division ratio. The clock control value causes a clock divider circuit 318 of the memory device 122 to adjust the clock signal provided to the PiM block(s) 202 based on the clock division ratio (e.g., !4, 1 / 3, or 1 / 5) specified by the clock control value.

[0088] The computations can be performed after memory-intensive data access operations are executed at the memory device 122 to obtain operands from memory banks of the memory device 122. The computations are performed at a lower frequency as a result of the adjusted clock signal. In some implementations, the steps of process 800 are performed at a hardware integrated circuit as part of a larger compute operation to generate an ML output, including an output for a neural network layer of a neural network that implements one or more ML models. For example, the output can be a portion of a computation for a ML task or inference workload to generate an image processing, speech processing, or image recognition output.

[0089] Fig. 9 is a second example process 900 for temperature detection and thermal throttling in a memory device. Much like process 800, process 900 is also implemented or executed at system 100 using at least the SoC 102 and memory device 122 described above, including the integrated system / circuit 300 described above with reference to Fig. 3. In some examples, the steps or actions of process 900 are enabled by programmed software instructions, firmw are instructions, or both. Each type of instruction may be stored in a non- transitory machine-readable storage device and is executable by one or more of the processors or other resources described in this specification.

[0090] At process 900. temperature sensors inside processing units of a PiM block of the memory device 122 measure temperature using at least one temperature sensor per processing unit to detect a hotspot at least by estimating a temperature of DRAM cells near the PiM block (902). In some implementations, the SoC 102 couples the PiM block temperatures with the estimated temperatures of DRAM cells near the PiM block to achieve a more reliable refresh rate for the DRAM memory units / cells. A new (or additional) Mode Register (MR) can be configured to and / or used to store temperature data obtained from, or measured by, temperature sensors in the PiM block (904). The memory controller (MC) (e.g., memory' controller 105) periodically reads temperature data stored in the MR and compensates the measured PiM temperature based on a read latency and a recent temperate slope (906).

[0091] A real-time throttling control logic (or controller) of the memory controller 105 determines a throttling percentage (%) (e.g.. 100%: full speed - 0%: complete stop) in accordance with the measured PiM temperature (908). In some implementations, this throttling determination of the memory controller 105 is much faster (e.g., tens of microseconds (ps)) than example thermal management software that may be running locally on a host device, such as a CPU 104, GPU 118, or HPU 114). which may only be capable of determining the throttling percentage at a speed of tens of milliseconds, rather than the microsecond speeds of the memory controller 105.

[0092] The real-time throttling control logic (or controller) of the memory controller 105 changes the WCK & CK (e.g., clocks to the DRAM that are linked to the PiM frequency) or writes a new value at the MR to change the clock division ratio (CK-to-PiM clock ratio) (910). In some implementations, one or both of these methods can be used to throttle the PiM execution rate or can be used combination with throttling or adjusting the PiM command rate or command execution rate. For example, the SoC 102 can adjust a rate which a processing unit 206 within a PiM block 202 executes PiM commands to perform computations for a given workload assigned to the PiM architecture 200 within the memory device 122.

[0093] For instance, the memory' controller 105 can use its real-time throttling control logic to adjusts (or throttles) the PiM command rate based on a command rate control value stored in the MR of the PiM block. In some implementations, the throttling control logic adjusts the PiM command rate based on the command rate control value, the adjusted clock signal, the clock division ratio, or combination of these. In some other implementations, one or more of these methods can be used to control the operating voltage of the PiM block to realize further thermal reduction advantages.

[0094] Optionally, the system 100 can also use an SoC + PIM thermal balancer to program throttle percentage control values in throttle registers of the memory' controller 105 to update the throttling % settings according to the PiM block temperature (912). In some implementations, the thermal balancer is software module or control logic that runs in a host device of the SoC 102. such as a host represented by the CPU 104 or HPU 114.

[0095] Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry', in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as oneor more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non transitory program carrier for execution by, or to control the operation of, data processing apparatus.

[0096] Alternatively or in addition, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

[0097] The term “computing system” encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). The apparatus can also include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g.. code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0098] A computer program (which may also be referred to or described as a program, software, a software application, a module, a software module, a script, or code) can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0099] A computer program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub programs, or portions of code. A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

[0100] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array), an ASIC (application specific integrated circuit), or a GPGPU (General purpose graphics processing unit).

[0101] Computers suitable for the execution of a computer program include, by way of example, can be based on general or special purpose microprocessors or both, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read only memory or a random access memory or both. Some elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g.. magnetic, magneto optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few.

[0102] Computer readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by. or incorporated in, special purpose logic circuitry.

[0103] To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user’s client device in response to requests received from the web browser.

[0104] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes afront end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (“LAN”) and a wide area network (“WAN”), e.g., the Internet.

[0105] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0106] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0107] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0108] Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As oneexample, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In certain implementations, multitasking and parallel processing may be advantageous.

Claims

What is claimed is:

1. A method implemented using an integrated circuit comprising a System-on-Chip (‘’SoC”) and a memon device coupled to the SoC, the method comprising: obtaining temperature data indicating a temperature in a Proccssing-in-Memory (PiM) block of the memory device as computations are being executed at the PiM block; detecting an adverse thermal condition in the PiM block by the SoC based on an indication in the temperature data that the temperature has exceeded a threshold temperature; based on the extent to which the temperature exceeds the threshold temperature, determining a thermal throttling response to alleviate the adverse thermal condition; generating, at the SoC. a command to implement the thermal throttling response at the PiM block; and executing the command and implementing the thermal throttling response at the PiM block by adjusting a clock signal, clock division ratio, or PiM command rate passed from the SoC to the PiM block.

2. The method of claim 1, wherein determining a thermal throttling response comprises: determining a thermal throttling response rate in accordance with the temperature of the PiM block such that the thermal throttling response corresponds to a delta between the threshold temperature and the temperature at the PiM block.

3. The method of claim 1 or 2, wherein the SoC obtaining the temperature data in a PiM block comprises reading the temperature data from a mode register in the memory device that receives the temperature data from one or more temperature sensors in the PiM block.

4. The method of any preceding claim, wherein further comprising: controlling an operating voltage of the PiM block to alleviate the adverse thermal condition by implementing an additional measure of thermal reduction at the PiM block.

5. The method of any preceding claim as dependent from claim 3, wherein adjusting a clock signal comprises: writing a clock control value to the mode register in the memory' device; and causing, based on the clock control value, an adjustment to the clock signal provided to the PiM block.

6. The method of claim 5, wherein: the clock control value written to the mode register specifies a clock division ratio; and the clock control value causes a clock divider circuit of the memory device to adjust the clock signal provided to the PiM block based on the clock division ratio specified by the clock control value.

7. The method of any preceding claim as dependent from claim 3, wherein: each of the one or more temperature sensors in the PiM block writes temperature data to the mode register based on a respective write frequency of the corresponding temperature sensor in the PiM block; and each respective write frequency of the corresponding one or more temperature sensors is dynamically adjustable by the SoC.

8. The method of claim 7, further comprising: dynamically adjusting a respective write frequency of a corresponding temperature sensor in the PiM block based on a compute load.

9. The method of any preceding claim as dependent from claim 3, wherein determining a thermal throttling response comprises: computing a temperature compensation value associated with obtaining the temperature in the PiM block; and determining a thermal throttling response based on the temperature compensation value.

10. The method of claim 9, wherein the temperature compensation value is computed based on: i) a temperature slope associated with obtaining the temperature in the PiM block; and ii) a portion of a temperature gradient associated with obtaining the temperature in the PiM block.

11. The method of claim 10, wherein the temperature gradient is a measure of time that includes:i) a read interval for reading the temperature data from the mode register, and ii) a system response delay for implementing the thermal throttling response at the PiM block.

12. The method of any preceding claim, wherein determining a thermal throttling response comprises: determining a real-time thermal throttling response rate using a real-time throttling controller of a memon controller on the SoC.

13. The method of any preceding claim, wherein determining a thermal throttling response comprises: determining a thermal throttling response using a system-level thermal balancer on the SoC.

14. The method of claim 13, wherein the system-level thermal balancer determines thermal throttling responses based on: i) the temperature at the PiM block; ii) system-level thermal condition data for distinct heat sources; and iii) system-level power monitoring and thermal monitoring information.

15. The method of any preceding claim, further comprising: determining a plurality of temperature values representing a range of temperatures for the PiM block, wherein each temperature in the range of temperatures exceeds the threshold temperature; and for each temperature in the range of temperatures: determining a respective thermal throttling response for the corresponding temperature.

16. The method of claim 15, further comprising: determining a mapping between: i) a particular temperature in the range of temperatures for the PiM block, ii) a corresponding temperature value of the plurality of temperature values that represents the particular temperature, and iii) a corresponding thermal throttling response that was determined for the particular temperature.

17. The method of any preceding claim, wherein the adverse thermal condition is a hot spot represented by a high temperature in a localized area of the memory device.

18. The method of claim 17, wherein detecting an adverse thermal condition in the PiM block comprises: computing a function of temperature data obtained from at least two different temperature sensors in the memory device; generating an estimated temperature value of the high temperature at the localized area as a result from the computed function; and detecting the adverse thermal condition in the PiM block based on the estimated temperature value.

19. An integrated circuit comprising: a System-on-Chip (“SoC”); a memory device coupled to the SoC; and a processor and a non-transitory machine-readable storage medium of the SoC for storing instructions that are executable by the processor to cause performance of operations comprising: obtaining temperature data indicating a temperature in a Processing-in-Memory (PiM) block of the memory device as computations are being executed at the PiM block; detecting an adverse thermal condition in the PiM block based on an indication in the temperature data that the temperature has exceeded a threshold temperature; based on the extent to which the temperature exceeds the threshold temperature, determining a thermal throttling response to alleviate the adverse thermal condition; generating, at the SoC. a command to implement the thermal throttling response at the PiM block; and executing the command and implementing the thermal throttling response at the PiM block by adjusting a clock signal, clock division ratio, or PiM command rate passed from the SoC to the PiM block.

20. The integrated circuit of claim 19, wherein determining a thermal throttling response comprises: determining a thermal throttling response rate in accordance with the temperature of the PiM block such that the thermal throttling response corresponds to a delta between the threshold temperature and the temperature at the PiM block.

Images