Enhanced reliability of processing-in-memory architectures

Abstract

This specification describes techniques, systems, and methods for enhancing reliability of devices having a processing-in-memory architecture ("PIM architecture"). One of methods includes providing, by a host device to a PIM subsystem, a sequence of instructions to be executed by the PIM subsystem. The host device receives from the PIM subsystem over an error interface between the host device and the PIM subsystem, an error indication that indicates that the PIM subsystem determined that an error occurred before one or more compute elements of the PIM subsystem completed execution of the sequence of instructions.

Description

[0001] ENHANCED RELIABILITY OF PROCESSING-IN-MEMORY ARCHITECTURES

[0002] CROSS REFERENCE TO RELATED APPLICATIONS

[0003] This application claims priority to U.S. Provisional Application No. 63 / 558,986 filed on February 28, 2024. The disclosure of the prior application is considered part of and is incorporated by reference in the disclosure of this application.

[0004] BACKGROUND

[0005] This specification generally relates to an architecture for executing computations in a processing-in-memory (PIM) architecture.

[0006] Modem computing systems often incorporate a wide variety of compute processing units that each offer different computing capabilities and trade-offs. Efficient execution of a given compute job often involves parsing computations into meaningful sub-tasks or workloads that are mapped to available processor cores of a computing system. The computations may be parsed and mapped based on suitability criteria, such as processor capability, performance, and power. Generally, this overall process of allocating portions of a computing task to appropriate processor resources is referred to as heterogeneous compute.

[0007] For example, a system-on-a-chip (SOC) can include multiple different processing cores, e.g., an Intellectual Property block (“IP block”) that executes a respective portion of a computational operation for different use cases. An example use case can involve processing image or speech data captured respectively by a camera or microphone on the mobile device. The SoC can use a heterogeneous compute operation to process input samples derived from the image data, the speech data, or both. An example step in the heterogeneous compute operation can include providing input samples from the image data to a neural network processor or machine-learning (ML) engine of the SoC to generate an inference output.

[0008] Some systems support heterogeneous computing using PIM devices. A PIM device is a memory device that supports processing-in-memory operations. That is, a PIM device is a memory device that has PIM architecture. In other words, a PIM device has (1 ) memory arrays to support operating as a memory, e.g., a dynamic random access memory (DRAM), for a central processing unit (CPU) and (2) one or more integrated compute elements that are operable to execute instructions on data stored in the memory arrays and to store resulting outputs of executing the instructions in the memory arrays without transferring the data or the resulting outputs to registers belonging to the CPU. PIM devices thus operate to provide two primary services for a heterogeneous computing system: 1) memory service operations, and 2) PIM operations. In other words, a device in the computing system can provide a PIM device with an ordinary memory service request to fetch values stored in memory, e.g., as a result of executing a load instruction. A device in the computing system can also provide a PIM device with a PIM operation request to perform a sequence of computations on a range of data values, e.g., by specifying one or more instructions and a range of memory addresses. In that case, the PIM device can perform the requested computations on the range of memory addresses while also storing the results in the memory itself. The PIM device can then respond to the requesting device to indicate that the PIM operations have completed, to be distinguished from the PIM device providing data values as part of its memory service operations.

[0009] Current PIM devices suffer from reliability problems due to a variety of causes. PIM devices typically have a reduced instruction set and a smaller number of registers compared to the CPUs and other devices that they support. For example, a PIM device might not support branching or conditional instructions. PIM devices also typically have less robust error handling mechanisms than CPUs. For example, in some cases, an SOC may only discover that PIM operations have failed when a PIM device stops responding after a threshold period of time. In those situations, it is difficult or impossible for the SOC to discern the cause of the failure, and the SOC simply needs to request that the instructions be attempted again.

[0010] Therefore, modern PIM devices are often relatively unreliable, which leads to performance degradation and redundant computations due to operations needing to be re- executed.

[0011] SUMMARY

[0012] This specification describes techniques for improving the reliability of PIM operations performed by a PIM device, e.g., a memory device having a PIM architecture in a host system. More specifically, a host system can receive error data from the PIM device using an error interface that is separate from a primary PIM interface for requesting PIM operations. The PIM device can use the error interface to provide various items of information regarding a failure, e.g., a last instruction sequence executed, a history of executed instruction sequences, an error code, or some combination of these. The host system can optionally use a warm reset interface to reset the state of the PIM device and restart execution of the instruction sequences without having to completely reset the PIM device or itself. Particular embodiments of the subject matter described in this specification can be implemented so as to realize one or more of the following advantages. The inclusion of an error interface between the PIM device and the host device improves the reliability of PIM processing by allowing the host device to detect an error quickly after it occurs. The host device can then restart PIM operations from the exact sequence of instructions that caused the error as opposed to resetting the entire system, saving time and computation power. The inclusion of the error interface vastly reduces the number of redundant computations that are performed in the face of a system error. The error handling mechanisms described in this specification require only a small increase to the memory footprint of a PIM device and can be implemented using only small hardware changes between the devices, e.g., by adding only one or two additional pins to implement the additional interfaces.

[0013] The details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other potential features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.

[0014] BRIEF DESCRIPTION OF THE DRAWINGS

[0015] FIG. 1 is a block diagram of an example computing system that uses a PIM device.

[0016] FIG. 2 illustrates an example system that uses a processor-in-memory architecture with a separate error interface.

[0017] FIG. 3 is a sequence diagram of an example process for handling an error in a PIM architecture using an error interface and a warm reset interface.

[0018] FIG. 4 is a flow chart diagram of an example process for the computing system including receiving an error from the error interface of the PIM device.

[0019] Like reference numbers and designations in the various drawings indicate like elements.

[0020] DETAILED DESCRIPTION

[0021] FIG. 1 is a block diagram of an example computing system 100 that uses a PIM device 122, e.g., a memory device that supports processing-in-memory operations. In this example, the system can include a host device, which is referred to as a system-on-a-chip (“SOC”) 102 and a memory device 122 that has a PIM architecture. That is, the system can include a PIM subsystem that includes one or more memory devices that store data and manage memory operations and one or more integrated compute units that handle the processing-in-memory operations. In other words, the PIM subsystem, or the memory devices 122 can serve as DRAM devices for the system 100 and can also receive and perform PIM operations on data stored in the DRAM. In this particular example, the PIM subsystem is comprised of a singular memory device. The memory device 122 may be referred to as a processing-in-memory (PIM) subsystem in this specification. The PIM subsystem further comprises one or more integrated compute elements 126. The memory device 122 comprises one or more memory arrays, configured to store data. The memory device 122 may comprise one or more memory dies 123, each comprising one or more compute elements 126, and one or more memory arrays 124.

[0022] In general, the SOC 102 can issue PIM requests to the memory device 122 using a primary PIM interface 140 and can receive error data back from the memory device 122 using a separate error interface 142. This functionality is described in more detail below with reference to FIG. 2.

[0023] The system 100 is an example of a subsystem that can be installed on any appropriate user device. In the example of FIG. 1, the system 100 is an integrated subsystem of an example user device 130, which can be a consumer electronic device or mobile device, where can be, for example, a smartphone 130a, a tablet computer 130b, a laptop 130c, a smartwatch 130d, another type of wearable device, an eNotebook, a Netbook, a smart speaker, or a mobile computer, to name just a few examples. In some other implementations, the system 100 is an integrated subsystem of a desktop computer, a network server, or any other appropriate cloud-based computing system.

[0024] The SoC 102 can have one or more computing components, including a central processing unit 104 (“CPU 104”), a memory controller 105, a shared memory 106 (“memory 106”), a PIM resource manager 108, and a circuit block 110. In some implementations, system 100 can include multiple SoCs. Any descriptions for the SoC 102 will apply equally to each of the multiple SoCs that may be included at system 100.

[0025] The CPU 104 can be a general-purpose CPU (e.g., a single or multi-core CPU) that can execute the primary functionality of the user device 130. The CPU 104 can generate one or more indicators, such as an app-launch indicator or a function call that is triggered in response to executing or launching an application at a user device. For example, the application can be a camera application that uses an imaging sensor to generate image data or a gaming application that requires substantial memory and graphics processing resources to render graphical content of the game. The CPU 104 can also generate one or more application values, such as pixel values or frame rate. The application values may be associated with a function call, may be descriptive of an event that occurs during execution of the application, or both.

[0026] The shared memory 106 is a memory subsystem that can be shared by other components of the SOC 102. In the example of Fig. 1, the shared memory 106 is depicted as being external to the circuit block 110. However, the shared memory 106 can include portions of memory that are: i) specific to the circuit block 110, ii) external to the circuit block 110, or iii) both. The shared memory 106 can be random access memory of the SoC 102, such as static random access memory (SRAM), dynamic random access memory (DRAM), a synchronous DRAM (SDRAM), or double data rate (DDR) SDRAM.

[0027] In some implementations, aspects of the shared memory 106 are configured as a shared scratchpad memory that supports parallel access of its memory resources by two or more components of the SOC 102. The shared memory 106 can also include various other types of memory subsystems, such as high bandwidth memory (HBM), narrow memory (e.g., for storing 8-bit values), or wide memory (e.g., for storing 16-bit or 32-bit values), to name just a few examples.

[0028] The circuit block 110 can generally include individual IP devices such as processors, processor cores, or special-purpose processing devices. For example, the circuit block 1 10 can include an image signal processor (ISP) 112, a tensor processing unit (TPU) 114, a digital signal processor (DSP) 116, and a graphics processing unit (GPU) 118. The circuit block 110 can include one or more proprietary hardware elements. For example, each of the ISP 112, TPU 1 14, DSP 1 16, and GPU 1 18 can be a respective proprietary IP block (or IP device) of a particular entity or device manufacturer.

[0029] The PIM resource manager 108 can be implemented in hardware, software, or a combination of these. Aspects of the PIM resource manager 108 can be also implemented as firmware of the SoC 102 or firmware of a device of the SoC 102. For example, the PIM resource manager 108 can include resources such as flip-flops, registers, buffers, etc., that are implemented in hardware and can have control logic (e.g., programmed code) that is implemented in software. One or more aspects of the PIM resource manager 108 can be implemented as a software routine (or module) of the CPU 104, which uses one or more hardware resources of the CPU 104, such as registers, buffers, etc.

[0030] The CPU 104 can be configured as an instruction and vector data processing engine that processes data obtained from the shared memory 106, the memory device 122, or both. In some implementations, each processor (e.g., ISP 112, DSP 116, TPU 114, GPU 118) of the SoC 102 can use the PIM resource manager 108 to generate control signaling to manage and distribute PIM requests to perform memory-intensive compute operations to the memory device 122 to minimize the processing load at each core of the processors. The control signaling can be routed through the system 100 using an example bus 120 of the SoC 102. The control signaling can include commands, requests, data, instructions, or combination of these.

[0031] The PIM resource manager 108 can cooperate with the CPU 104, memory controller 105 and storage controller 107 to dynamically control and manage one or more PIM operations to be performed by the memory device 122. In some implementations, the system 100 can include one or more memory devices as part of the PIM subsystem. Any descriptions for the memory device 122 will apply equally to each of the multiple memory devices that may be included in system 100.

[0032] As described above, the PIM subsystem can include the memory device 122, a primary PIM interface 140, and an error interface 142. In some implementations, the PIM subsystem can also include a warm reset interface, which is explained in farther detail with reference to Fig. 2.

[0033] The memory device 122 can include multiple memory dies, each having one or more memory arrays for implementing the memory functionality of the memory device 122. For example, the memory device 122 can include N memory dies, where N is an integer greater than 1. The memory device 122 can implement a dynamic random-access memory (DRAM) or Double Data Rate (DDR) synchronous DRAM (SDRAM). The memory device 122 is configured to perform or support various types of PIM operations, which in this specification can also include compute-in-memory operations (CIM operations), and memory-near- computing operations (“MnC operations”). The memory device 122 can perform or support these operations using one or more PIM compute elements, such as a processor unit, mode registers and one or more computational units, e.g., arithmetic logic units (ALUs) or related addition and multiplication circuitry. For example, the memory device can include discrete processors, processor units, register devices, buffers, multiply accumulate cells (MACs), etc. that cooperate to form one or more PIM compute elements.

[0034] In some implementations, the PIM architecture, e.g., the compute elements, is included in the memory device 122 as multiple discrete integrated circuits, where each integrated circuit is local to a given memory die (e.g., die-1 and die-2) and interacts or communicates with arrays of memory cells at that memory die. For example, the PIM architecture can include compute elements that are replicated and distributed across each of the memory die in the memory device 122. In some other implementations, the PIM architecture is included in the memory device 122 as a single integrated circuit that interacts or communicates with each memory die of the memory device 122, including the arrays of memory cells at each memory die.

[0035] The SoC 102 can cooperate with the memory device 122 to perform PIM computations across the one or more memory dies 123 of the memory device 122. The computations can be for operations or workloads that involve processes executed by one or more of the processors in the circuit block 1 10. For example, the PIM computations can include matrix, multiplication, graph processing, performing convolution or activation functions for neural network inferencing, or any other appropriate computations. Alternatively, or in addition, the PIM computations performed by the memory device can be part of a heterogeneous operation that spans multiple processors of circuit block 110, multiple circuit blocks 110, or both. In at least one example the memory device 122 is external to the SoC 102, whereas in another example the memory device 122 can be internal to the SoC 102.

[0036] In basic operation, a component of the SoC 102 can initiate a request to perform PIM operations on the memory device 122. For example, the component can communicate with the PIM resource manager 108 to provide a sequence of one or more instructions to be performed over a range of memory addresses by compute elements 126 of the memory device 122. The SOC 102 can then provide data representing the sequence of instructions and the range of memory addresses to the memory device 122 over the primary PIM interface 140.

[0037] The memory device 122 can receive the data representing the sequence of instructions as well as the addresses over which to perform the PIM operations on the primary PIM interface 140. The memory device 122 then can distribute the instructions among the compute elements 126. Each of the compute elements 126 can be a separate processor that can perform instructions according to a PIM instruction set that defines supported PIM operations. In some implementations, the memory device 122 can distribute the sequence instructions among the multiple compute elements 126 in a SIMD fashion.

[0038] After the PIM operations have completed, the memory device 122 can provide an indication that the operations have completed back to the SoC 102 over the primary PIM interface 140. The PIM resource manager 108 can then notify the requesting processing component that the PIM operations have completed and that the corresponding process can continue.

[0039] If an error occurs while performing the PIM operations, the memory device 122 can provide an indication of the error over the separate error interface 142. The PIM resource manager 108 can receive the error data over the error interface 142 and can take an appropriate remedial action. The error data can include one or more of the following, the error indication, an error code, an error message, or any other appropriate error data. The error data, depending on the information included and the implementation of the error interface, can be transferred in any appropriate form, including, but not limited to, binary or hexadecimal codes, error flags, interrupt signals, and encoded data, e.g., encoded data bits.

[0040] In some implementations, the PIM resource manager 108 can reset the state of the memory device 122 using the error data received over the error interface 142. The PIM resource manager 108 can then restart the PIM operations from an appropriate starting place, e.g., from an appropriate instruction sequence, which can often avoid performing redundant computations in the face of an error.

[0041] The error interface 142 can be implemented by adding only one or two additional pins between the host device, e.g., the SOC 102, and the PIM subsystem, e.g., memory device 122. In some implementations, the error interface 142 is implemented using a single pin between the host device and the PIM subsystem. For example, the SOC 102 and the memory device 122 can be connected through a single dedicated wire. In this example, all communication and data can be transferred through the single wire. The data can be transferred in bits, where each bit is represented by specific timing on the wire, e.g., to write a 0, the data line is pulled low for 60 microseconds.

[0042] Overall, the system 100 can include a host device, e.g., SOC 102, that includes a CPU, among other computation elements, a PIM subsystem that includes one or more memory devices, e.g., memory device 122, and one or more integrated compute elements, a primary PIM interface between the host device, e.g., SOC 102 and the PIM subsystem, and an error interface between the host device and the PIM subsystem.

[0043] With regard to the primary PIM interface, the host device is configured to provide, by the primary PIM interface, a sequence of instructions to the PIM subsystem and the PIM subsystem is configured to execute, by the one or more compute elements, the instructions to perform operations on data stored in the one or more memory devices and to store a result of the operations in one or more of the one or more memory devices.

[0044] With regard to the error interface, the PIM subsystem is configured to provide, by the error interface, an error indication to the host device upon determining that an error occurred before the one or more compute elements completed execution of the sequence of instructions. FIG. 2 illustrates an example system 200 that uses a PIM device, e.g., a memory device with a processor- in-memory architecture with a separate error interface 242. The system 200 also optionally includes a warm reset interface 244.

[0045] The system 200 can include an SOC 202, which is an example of a host device that can issue requests to a memory device 222 having a PIM architecture. The SoC 202 includes a CPU 210, a shared SRAM 220, a memory controller 260, a storage controller 280, a PIM resource manager 250, and a DSP 230. The system 200 also includes a memory device 222 having a PIM architecture. That is, the memory device 222 has one or more compute elements (not shown) that operate directly on data stored in the memory arrays 21 Oa-b of dies 261 and 263. The SoC 202 provides requests for these compute elements to perform sequences of instructions over a primary PIM interface 240. Each sequence of instructions has an identifier, which may be referred to as a sequence identifier. The sequence identifier ensures that errors in the instructions can be accurately tracked, allowing the system to retry the failed instruction without re-executing the previous instructions.

[0046] In order to enhance the reliability of the memory device 222, each die 261 and 263 can have a collection of error handling registers. The error handling registers can include a last sequence register, a plurality of sequence history registers, and one or more error code registers. In this example, the die 261 includes a last sequence number register 252a, sequence history registers 254a, and an error code register 256a. The die 263 includes a last sequence number register 252b, sequence history registers 254b, and an error code register 256b.

[0047] As it executes the instructions, the memory device 222 can keep track of an identifier of the last sequence of instructions that was attempted to be executed. The memory device 222 can continually populate the last sequence number registers 252a-b with an identifier of the last sequence of instructions that was attempted to be executed. Then, if an error occurs, the last sequence number registers 252a-b can record the sequence of instructions over which the error occurred.

[0048] In addition, the memory device 222 can continually populate a set of sequence history registers 254a-b that record the recent history of sequences of instructions that executed successfully. For example, the sequence history registers 254a-b can have eight registers that store the identifiers for the eight most recently successfully executed instruction sequences.

[0049] Upon encountering an error, the memory device 222 can provide, over the error interface 242, an indication that an error has occurred. The SOC 202 can then take remedial action appropriate for the type of error that occurred. In some implementations, the SOC 202 can retrieve data from one or more of the error handling registers in order to determine what remedial action is appropriate.

[0050] For example, the memory device 222 can populate error code registers 256a-b with an error code that represents the type of error that occurred. For example, the system can populate different error codes for situations in which a compute element attempts to access a restricted memory location, for when there was an interface error, for when one or more compute elements fail to finish execution, to name just a few examples. The SOC 202 can retrieve the data from the error handling registers over the error interface 242 in implementations where the error interface 242 is configured to provide such data. In some implementations, the error interface 242 simply provides an indication of whether or not an error has occurred, and the SOC 202 can retrieve the data from the error handling registers using the primary PIM interface 240 if the memory device 222 is still functioning. After retrieving data from the error handling registers, in some situations, the SOC 202 can perform a warm reset using a warm reset interface 244 that is implemented between the host device and the memory device 222. A warm reset is a type of reset that allows parts of the SoC to be reset while certain parts retain their states and values, e.g., resetting logic of the PIM device, but not resetting the values of the error handling registers or any of the memory contents stored in the memory arrays, e.g., memory arrays 210a and 210b. The warm reset interface 244 can be used to restart execution of the sequences of instructions, starting from the instruction sequence that failed. For example, the SOC 202 can read the last sequence number stored in the last sequence number registers 252a-b and then reconfigure the logic of the compute elements of the memory device 222 to restart execution at the last sequence number.

[0051] Using the warm-reset interface 244 can avoid having to entirely reset the logic and registers of the memory device 222. Therefore, while performing a warm reset, the SOC 202 can reset the logic but maintain the data in the error handling registers 252, 254, and 256. That is, the memory device is able to reset and pick up where it left off in terms of computations without having to re-compute early instructions which saves resources and increases the reliability of the PIM device 222.

[0052] FIG. 3 is a sequence diagram of an example process for handling an error in a PIM architecture using an error interface and a warm reset interface. The diagram illustrates the operations performed by a host device, such as a SOC 302, and a PIM device 322 having an error interface and a warm reset interface that operate in accordance with this specification. The SOC (i.e. the host device) first provides instruction sequences 1 through N to the PIM device 304 (305). As described above, each instruction sequence can have an identifier and can define one or more operations to be performed over one or more memory addresses. For example, if the PIM operations perform matrix multiplication, the SOC 302 can provide instruction sequences that include one or more of the following operations, loading matrices into memory arrays, e.g., the memory arrays 210a of FIG. 2, initializing the result matrix, performing element-wise multiplication and accumulation, and storing the results.

[0053] As the PIM device 304 iterates over the instruction sequences, the SOC 302 can log sequence numbers of successfully completed instruction sequences (310). For example, the PIM device 304 can log the sequence numbers of the instruction sequence that represents loading the matrices into memory array and initializing the result matrix after the instructions have been successfully completed. The PIM device 304 can also store sequence numbers of executed instruction sequences (315), e.g., in error handling registers as described above. For example, the PIM device 304 can then store the sequence numbers of the instruction sequence that represents performing element-wise multiplication and accumulation as the PIM device is executing the instruction.

[0054] When an error occurs during execution of sequence number N (320), the PIM device 304 can store sequence number N in a last sequence number register (325). As an example, a memory access violation may occur during the matrix multiplication instruction sequence. As another example, an error may occur where the PIM device 304 gets stuck performing a computation in the matrix multiplication instruction and cannot finish executing the instruction sequence. When either of these errors occur, the PIM device 304 can store the sequence number for the matrix multiplication instruction in the last sequence number register (325).

[0055] The PIM device 304 can then provide an error indication 330 back to the SOC 302, e.g., using an error interface (330).

[0056] In some implementations, the SOC 302 can receive the error indication 330, e.g., using the error interface, and can restart execution of a last sequence of instructions attempted by the PIM device. The host device, e.g., the host device 202 of FIG. 2. can retrieve the sequence identifier of the last instruction sequence of instructions attempted by the PIM device by reading a value of an error handling register of the PIM subsystem.

[0057] In some implementations, when the error indication is received, the PIM device 304 may need to be reset in order to resume communications with the SOC 302. For example, if the PIM device gets stuck performing a particular computation, the PIM device 304 might become unresponsive.

[0058] Thus, the SOC 302 can provide a request to perform a warm reset (335) to the PIM device 304 over the warm reset interface, e.g., the warm reset interface 244 of FIG. 2. The PIM device 304 can then perform a warm reset operation (340). The warm reset operation can reset the logic of the PIM device 304 (e.g. the one or more compute elements) so that it becomes responsive again and able to communicate with the SOC 302. Because the primary PIM interface might become unavailable during a system error, the SOC 302 can use a separate warm reset interface to request the warm reset. In some implementations, the warm reset operation preserves the data in the error handling registers (as well as the memory' arrays) so that the SOC 302 can determine which remedial actions are appropriate.

[0059] The SOC 302 can initialize a PIM interface for retrieving data from the error handling registers. In some implementations, the PIM interface needs to be re-initialized after an error occurs, and thus the SOC 302 can re-initialize the interface to retrieve the error data.

[0060] The SOC 302 can provide a request for the error data (350) and can retrieve the error data (355) from an error handling register of the PIM device 304. As described above, the error data can include an error code, an identifier of a last attempted instruction sequence, a history' of successfully executed instruction sequences, or some combination of these.

[0061] For example, in the restricted memory access violation example, the SOC can retrieve the error data (354) from the error handling register of the PIM device 304. The error data can include an error code for a restricted memory access violation, as well as the sequence identifier for the matrix multiplication instruction sequence, and the history' of successfully executed instructions, e.g., loading matrices into the memory arrays and initializing the result matrix. As another example, for a failure to finish execution example, the error data can include an error code for a failure to execute violation, the sequence identifier for the matrix multiplication instruction sequence and the history of successfully executed instructions, e.g., loading matrices into the memory' arrays and initializing the result matrix.

[0062] The SOC 302 can then perform a remedial action (360). The remedial action can depend on the type of error that has occurred as indicated by the error code. By retrieving the error data from the PIM device 304 the SOC 302 is able to distinguish between fatal and non- fatal errors of the PIM device 304 through their error codes. One example of a remedial action is to restart execution of the instruction sequence that failed. For example, for a failure to execute violation, e.g., the PIM device gets stuck on a computation and cannot finish executing the instruction sequence, the PIM device can restart execution of the matrix multiplication instruction sequence that failed and maintain the data and values from the previous instruction sequences. Another example of a remedial action is triggering a system crash that resets the PIM device 304, the SOC 302, or both. In this example, the PIM device 304 would require a system reset, as opposed to a warm reset, of the device, and would include resetting the error handling registers and the memory arrays that are not reset during a warm reset. For example, for severe enough security’ or access control violations, the SOC 302 can trigger a reset of both the PIM device 304 and the SOC 302, including the error handling registers and the memory' arrays. As a specific example, a severe restricted memory' access violation can trigger a system rest of both the PIM device and the SOC 302 as it can be a fatal error and require a system reset. In some implementations, a system reset can include the error handling registers, but not the memory arrays when resetting the PIM device 304. As described above, this reset differs from a warm reset as a warm reset does not reset the error handling registers.

[0063] FIG. 4 is a flow chart diagram of an example process for detecting errors in a computing system 400.

[0064] The computing system 400 can include a host device including a CPU, a PIM subsystem including one or more memory’ devices and one or more integrated compute elements, a primary' PIM interface between the host device and the PIM subsystem, and an error interface between the host device and the PIM subsystem.

[0065] The host device, also referred to as a system-on-a-chip (SOC) can include one or more computing components including a CPU. For example, the SOC may include shared memory’, a PIM resource manager, a memory’ controller, and a circuit block that can include one or more IP blocks. The one or more IP blocks can include an image signal processor (ISP), a tensor processing unit (TPU), a digital signal processor (DSP), and a graphics processing unit (GPU).

[0066] In some implementations, the error interface can be implemented using a single pin between the host device and the PIM subsystem.

[0067] The system 400 can provide, by the host device over the primary PIM interface, a sequence of instructions to the PIM subsystem, and where the PIM subsystem is configured to execute, by the one or more compute elements, the instructions to perform operations on data stored in the one or more memory' devices and to store a result of the operations in one or more of the one or more memory devices (step 402).

[0068] The system can then receive, by the host device from the PIM subsystem over the error interface, an error indication that indicates that the PIM subsystem determined that an error occurred before the one or more compute elements completed execution of the sequence of instructions (step 404).

[0069] In some implementations, the host device can receive the error indication over the error interface and can restart execution of a last sequence of instructions attempted by the PIM subsystem. The host device can retrieve a sequence identifier of the last instruction sequence of instructions attempted by the PIM subsystem before an error occurred by reading a value of an error handling register of the PIM subsystem.

[0070] In some implementations, the host device can receive the error indication over the error interface and perform a warm reset of the PIM subsystem over the warm reset interface, e.g., resetting the logic of the PIM subsystem. As described above, the system 400 can include a warm reset interface between the host device and the PIM subsystem, over which a warm reset can be performed. The host device can then retrieve an error code from the PIM subsystem that can be stored in an error handling register of the PIM subsystem. Based on the error code, the system 400 can perform a remedial action., Performing the remedial action can include resetting the PIM subsystem, resetting the host device, or both. The host device can be configured to distinguish between fatal and non-fatal errors of the PIM subsystem and depending on the type of error, the system 400 can decide the best remedial action.

[0071] Other techniques for using PIM devices are described in commonly owned U.S. Patent Application No. 63 / 609,093, entitled “PROCESSING-IN-MEMORY ARCHITECTURE FOR MEMORY BOUND COMPUTE WORKLOADS,” filed Dec. 12, 2023, which is herein incorporated by reference.

[0072] Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non transitory program carrier for execution by, or to control the operation of. data processing apparatus.

[0073] Alternatively or in addition, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

[0074] The term '‘computing system” encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). The apparatus can also include, in addition to hardware, code that creates an execution environment for the computer program in question, e g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0075] A computer program (which may also be referred to or described as a program, software, a software application, a module, a software module, a script, or code) can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0076] A computer program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub programs, or portions of code. A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication netw ork.

[0077] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array), an ASIC (application specific integrated circuit), or a GPGPU (General purpose graphics processing unit).

[0078] Computers suitable for the execution of a computer program include, by way of example, can be based on general or special purpose microprocessors or both, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read only memory or a random access memory or both. Some elements of a computer are a central processing unit for performing or executing instructions and one or more memory' devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few.

[0079] Computer readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory', media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0080] To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. Tn addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user’s client device in response to requests received from the web browser.

[0081] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication netw ork. Examples of communication networks include a local area network (“LAN”) and a wide area network (“WAN”), e.g.. the Internet. The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0082] In addition to the embodiments described above, the following embodiments are also innovative:

[0083] Embodiment 1 is a system comprising: a host device comprising a central processing unit (CPU); a processing-in-memory (PIM) subsystem comprising one or more memory' devices and one or more integrated compute elements; a primary PIM interface between the host device and the PIM subsystem. wherein the host device is configured to provide, by the primary PIM interface, a sequence of instructions to the PIM subsystem, and wherein the PIM subsystem is configured to execute, by the one or more compute elements, the instructions to perform operations on data stored in the one or more memory’ devices and to store a result of the operations in one or more of the one or more memory devices; and an error interface between the host device and the PIM subsystem, wherein the PIM subsystem is configured to provide, by the error interface, an error indication to the host device upon determining that an error occurred before the one or more compute elements completed execution of the sequence of instructions.

[0084] Embodiment 2 is the system of embodiment 1 , wherein the host device is a system-on-a- chip having a plurality’ of computing components including the CPU.

[0085] Embodiment 3 is the system of any one of embodiments 1-2, wherein the error interface is implemented using a single pin between the host device and the PIM subsystem.

[0086] Embodiment 4 is the system of any one of embodiments 1-3, wherein the host device is configured to execute instructions to perform operations comprising: receiving the error indication over the error interface; and restarting execution of a last sequence of instructions attempted by the PIM subsystem.

[0087] Embodiment 5 is the system of embodiment 4, wherein the operations further comprise retrieving, by the host device, a sequence identifier of the last instruction sequence of instructions attempted by the PIM subsystem.

[0088] Embodiment 6 is the system of embodiment 5. wherein retrieving the sequence identifier comprises reading a value of an error handling register of the PIM subsystem. Embodiment 7 is the system of any one of embodiments 1-6, further comprising a warm reset interface between the host device at the PIM subsystem.

[0089] Embodiment 8 is the system of embodiment 7, wherein the host device is configured to execute instructions to perform operations comprising: receiving the error indication over the error interface; and performing a warm reset of the PIM subsystem over the warm reset interface.

[0090] Embodiment 9 is the system of embodiment 8. wherein performing the warm reset comprises resetting logic of the PIM subsystem.

[0091] Embodiment 10 is the system of any one of embodiments 1-5 or embodiment 8 or 9, wherein the host device is configured to perform operations comprising: retrieving, by the host device from the PIM subsystem, an error code stored in an error handling register of the PIM subsystem; and performing a remedial action based on the error code.

[0092] Embodiment 11 is the system of embodiment 10, wherein performing the remedial action comprises resetting the PIM subsystem, resetting the host device, or both.

[0093] Embodiment 12 is the system of embodiment 10 or 11, wherein the host device is configured to distinguish between fatal and non-fatal errors of the PIM subsystem.

[0094] Embodiment 13 is a method for detecting errors in a system comprising: a host device comprising a central processing unit (CPU), a processing-in-memory (PIM) subsystem comprising one or more memory devices and one or more integrated compute elements, a primary PIM interface between the host device and the PIM subsystem, and an error interface between the host device and the PIM subsystem, the method comprising: providing, by the host device over the primary PIM interface, a sequence of instructions to the PIM subsystem, and wherein the PIM subsystem is configured to execute, by the one or more compute elements, the instructions to perform operations on data stored in the one or more memory devices and to store a result of the operations in one or more of the one or more memory devices; and receiving, by the host device from the PIM subsystem over the error interface, an error indication that indicates that the PIM subsystem determined that an error occurred before the one or more compute elements completed execution of the sequence of instructions. Embodiment 14 is the method of embodiment 13, wherein the host device is a system-on- a-chip having a plurality of computing components including the CPU.

[0095] Embodiment 15 is the method of any one of embodiments 13-14, wherein the error interface is implemented using a single pin between the host device and the PIM subsystem.

[0096] Embodiment 16 is the method of any one of embodiments 13-1 , further comprising: receiving, by the host device, the error indication over the error interface; and restarting execution of a last sequence of instructions attempted by the PIM subsystem.

[0097] Embodiment 17 is the method of embodiment 16, further comprising retrieving, by the host device, a sequence identifier of the last instruction sequence of instructions attempted by the PIM subsystem.

[0098] Embodiment 18 is the method of embodiment 17, wherein retrieving the sequence identifier comprises reading a value of an error handling register of the PIM subsystem.

[0099] Embodiment 19 is the method of any one of embodiments 13-18, wherein the system further comprises a warm reset interface between the host device at the PIM subsystem.

[0100] Embodiment 20 is the method of embodiment 19, further comprising: receiving, by the host device, the error indication over the error interface; and performing a warm reset of the PIM subsystem over the warm reset interface.

[0101] Embodiment 21 is the method of embodiment 20, wherein performing the warm reset comprises resetting logic of the PIM subsystem.

[0102] Embodiment 22 is the method of embodiment 20 or 21, further comprising: retrieving, by the host device from the PIM subsystem, an error code stored in an error handling register of the PIM subsystem; and performing a remedial action based on the error code.

[0103] Embodiment 23 is the method of embodiment 22, wherein performing the remedial action comprises resetting the PIM subsystem, resetting the host device, or both.

[0104] Embodiment 24 is the method of claim 22 or 23, wherein the host device is configured to distinguish between fatal and non-fatal errors of the PIM subsystem.

[0105] Embodiment 25 is a computer storage medium encoded with instructions that are operable, when executed by data processing apparatus, to cause the data processing apparatus to perform the method of any one of embodiments 13 to 24.

[0106] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0107] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0108] Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In certain implementations, multitasking and parallel processing may be advantageous.

[0109] What is claimed is:

Claims

CLAIMS1. A system comprising: a host device comprising a central processing unit (CPU); a processing-in-memory (PIM) subsystem comprising one or more memory devices and one or more integrated compute elements; a primary PIM interface between the host device and the PIM subsystem. wherein the host device is configured to provide, by the primary PIM interface, a sequence of instructions to the PIM subsystem, and wherein the PIM subsystem is configured to execute, by the one or more compute elements, the instructions to perform operations on data stored in the one or more memory devices and to store a result of the operations in one or more of the one or more memory devices; and an error interface between the host device and the PIM subsystem, wherein the PIM subsystem is configured to provide, by the error interface, an error indication to the host device upon determining that an error occurred before the one or more compute elements completed execution of the sequence of instructions.

2. The system of claim 1, wherein the host device is a system-on-a-chip having a plurality of computing components including the CPU.

3. The system of any one of claims 1-2, wherein the error interface is implemented using a single pin between the host device and the PIM subsystem.

4. The system of any one of claims 1-3, wherein the host device is configured to execute instructions to perform operations comprising: receiving the error indication over the error interface; and restarting execution of a last sequence of instructions attempted by the PIM subsystem.

5. The system of claim 4, wherein the operations further comprise retrieving, by the host device, a sequence identifier of the last instruction sequence of instructions attempted by the PIM subsystem.

6. The system of claim 5, wherein retrieving the sequence identifier comprises reading a value of an error handling register of the PIM subsystem.

7. The system of any one of claims 1-6, further comprising a warm reset interface between the host device at the PIM subsystem.

8. The system of claim 7. wherein the host device is configured to execute instructions to perform operations comprising: receiving the error indication over the error interface; and performing a warm reset of the PIM subsystem over the warm reset interface.

9. The system of claim 8, wherein performing the warm reset comprises resetting logic of the PIM subsystem.

10. The system of any one of claims 1-5 or claim 8 or 9, wherein the host device is configured to perform operations comprising: retrieving, by the host device from the PIM subsystem, an error code stored in an error handling register of the PIM subsystem; and performing a remedial action based on the error code.

11. The system of claim 10, wherein performing the remedial action comprises resetting the PIM subsystem, resetting the host device, or both.

12. The system of claim 10 or 11, wherein the host device is configured to distinguish between fatal and non-fatal errors of the PIM subsystem.

13. A method for detecting errors in a system comprising: a host device comprising a central processing unit (CPU), a processing-in-memory (PIM) subsystem comprising one or more memory devices and one or more integrated compute elements, a primary PIM interface between the host device and the PIM subsy stem, and an error interface between the host device and the PIM subsystem, the method comprising: providing, by the host device over the primary PIM interface, a sequence of instructions to the PIM subsystem, and wherein the PIM subsystem is configured to execute, by the one or more compute elements, the instructions to perform operations on data stored in the one or more memory devices and to store a result of the operations in one or more of the one or more memory devices; and receiving, by the host device from the PIM subsystem over the error interface, anerror indication that indicates that the PIM subsystem determined that an error occurred before the one or more compute elements completed execution of the sequence of instructions.

14. The method of claim 13, wherein the host device is a system-on-a-chip having a plurality of computing components including the CPU.

15. The method of any one of claims 13-14, wherein the error interface is implemented using a single pin between the host device and the PIM subsystem.

16. The method of any one of claims 13-15, further comprising: receiving, by the host device, the error indication over the error interface; and restarting execution of a last sequence of instructions attempted by the PIM subsystem.

17. The method of claim 16, further comprising retrieving, by the host device, a sequence identifier of the last instruction sequence of instructions attempted by the PIM subsystem.

18. The method of claim 17, wherein retrieving the sequence identifier comprises reading a value of an error handling register of the PIM subsystem.

19. The method of any one of claims 13-18, wherein the system further comprises a warm reset interface between the host device at the PIM subsystem.

20. The method of claim 19, further comprising: receiving, by the host device, the error indication over the error interface; and performing a warm reset of the PIM subsystem over the warm reset interface.

21. The method of claim 20, wherein performing the warm reset comprises resetting logic of the PIM subsystem.

22. The method of claim 20 or 21, further comprising: retrieving, by the host device from the PIM subsystem, an error code stored in an error handling register of the PIM subsystem; and performing a remedial action based on the error code.

23. The method of claim 22, wherein performing the remedial action comprises resetting the PIM subsystem, resetting the host device, or both.

24. The method of claim 22 or 23, wherein the host device is configured to distinguish between fatal and non-fatal errors of the PIM subsystem.

Images