Processing-in-memory simultaneous processing system and method

Abstract

The processing system (100) includes processing modules (104, 106) and a memory device (112). The memory device includes a processing-in-memory (PIM) module (120) that performs processing operations on behalf of the processing modules. The instruction set architecture (ISA) of the PIM module has fewer instructions than the ISA of the processing modules. Instructions received from the processing modules are translated such that the processing resources of the PIM module are virtualized. As a result, the PIM module simultaneously performs processing operations for multiple threads or applications of the processing modules.

Description

[Background technology]

[0001] Processing systems often include local memory resources that can be accessed relatively quickly by a processor (e.g., a central processing unit (CPU) or a graphics processing unit (GPU)) and external memory resources that take a relatively long time to access. In some cases, the external memory resources are located on a different chip than the processor. However, in some cases, due to the longer access times, instructions involving data stored in the external memory resources take an undesirably long time to process. Additionally, in some cases, data communication between the external memory resources and the processor consumes an undesirable amount of interconnect capacity. In some cases, to reduce the number of accesses to the external memory resources, processing-in-memory (PIM) circuits are used to process data in the external memory resources rather than bringing the data to the processor. The PIM circuits perform limited processing applications on data stored in the external memory resources without the power consumption or area usage of more typical processors. As a result, in some cases, the number of communications between the processor and the external memory resources is reduced, and the processing time of some instructions involving data stored in the external memory resources is reduced. However, this limited architecture makes it difficult to exploit hardware parallelism to process multiple kernels simultaneously. For example, due to the lack of virtualization, the PIM circuitry only loads commands for one kernel at a time, potentially creating a serialization bottleneck due to data dependencies. Summary of the Invention [Means for solving the problem]

[0002] In one general aspect, a memory device includes a plurality of storage devices configured to store data on behalf of programs executed by the processing units, the processing units being external to the memory device, a processing-in-memory (PIM) module configured to process PIM commands in response to kernel instructions corresponding to the programs from the processing units, the instruction set architecture (ISA) including the PIM commands and implemented by the PIM module having fewer instructions than the ISA implemented by the processing units, and processing resources of the PIM module configured to be virtualized such that the PIM module processes PIM commands from multiple PIM kernels simultaneously.

[0003] According to another general aspect, a method includes, at a processing-in-memory (PIM) arithmetic logic unit (ALU), receiving kernel instructions from a processing unit for multiple kernels to be processed simultaneously by the PIM ALU, wherein an instruction set architecture (ISA) implemented by the PIM ALU has fewer instructions than the ISA implemented by the processing unit. The method further includes retrieving PIM ALU commands corresponding to the kernel instructions from a PIM command storage (PCS) module, wherein at least one PIM instruction corresponds to at least two kernels. The method also includes executing the PIM ALU commands, including storing data for at least two of the kernels in a register bank of a memory that includes the PIM ALU.

[0004] According to yet another general aspect, a system includes a processing unit configured to execute a plurality of programs. The system further includes a memory device configured to receive kernel instructions from the processing unit, the kernel instructions causing a processing-in-memory (PIM) module of the memory device to simultaneously process PIM commands for a plurality of PIM kernels, wherein simultaneously processing the PIM commands includes storing data for at least some of the PIM kernels in a plurality of storage devices of the memory device configured to operate as virtual register banks, the PIM commands included, and an instruction set architecture (ISA) implemented by the PIM module having fewer instructions than the ISA implemented by the processing unit.

[0005] The present disclosure may be better understood, and its numerous features and advantages made apparent to those skilled in the art by referencing the accompanying drawings, in which: The use of the same reference numbers in different drawings indicates similar or identical items. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a block diagram of a PIM concurrent processing system according to some embodiments. [Figure 2] FIG. 1 is a block diagram of a PIM architecture, according to some embodiments. [Figure 3] FIG. 1 is a block diagram of an exemplary virtual-to-physical kernel register mapping, according to some embodiments. [Figure 4] FIG. 1 is a block diagram of an exemplary kernel instruction merging and mapping, according to some embodiments. [Figure 5] FIG. 1 is a flow diagram of a method for processing kernel instructions for multiple kernels simultaneously using a PIM concurrent processing system, according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0007] The processing system includes a processing unit and a memory device external to the processing unit. The memory device includes a processing-in-memory (PIM) module that processes data on behalf of the processing system. The instruction set architecture (ISA) of the PIM module is, in some cases, smaller than the ISA of the processing unit, such that PIM commands are a subset of instructions in the ISA of the processing unit. For example, in some cases, the PIM module is an arithmetic logic unit (ALU). As a result, compared to a processor, such as a CPU, that sends instructions to another processor, such as a GPU, additional instructions are sent to the PIM module. In some embodiments, the additional instructions are sent as kernel instructions that indicate the PIM commands to be executed. In response to the kernel instructions, the PIM module processes the PIM commands using data stored in the memory device.

[0008] In some cases, a PIM module processes PIM commands for multiple applications or different threads of the same application simultaneously. Address virtualization is commonly used to prevent errors caused by multiple applications attempting to use a single address and to reduce potential security issues caused by an application receiving information about other applications through the use of physical addresses. However, address virtualization is difficult for some PIM modules due to their relative simplicity compared to processing units. As a result, these PIM modules serially provide access to resources (e.g., the PIM module and associated storage devices, such as register files). In some embodiments, the instant memory device further includes a PIM kernel scheduler that performs virtual-to-physical register mapping on behalf of the PIM module, enabling register address virtualization in the memory device and allowing the PIM module to process multiple kernels simultaneously.

[0009] PIM commands are stored in a PIM command storage (PCS) module. However, in some embodiments, storage space within the PCS module is limited. As a result, in some cases, the PIM kernel scheduler further analyzes incoming kernel instructions to identify whether multiple instruction sets request the same operation. In response to detecting that the same operation is requested, the PIM kernel scheduler combines kernel instructions and causes the PCS module to store only a single PIM command. As a result, resource utilization by the PCS module is increased. Furthermore, in some embodiments, PCS addresses are virtualized so that instructions for multiple kernels are stored in the PCS module simultaneously. Thus, in some cases, in response to detecting that the same operation is requested by multiple kernels, the PIM kernel scheduler combines kernel instructions from multiple kernels.

[0010] As used herein, "processing concurrently" refers to executing at least a portion of an application or kernel after the initiation of a portion of another application or kernel but before its completion. For example, a first kernel and a second kernel are considered to be "processed concurrently" if portions of the two kernels are executed in an interleaved manner, such that a portion of the second kernel executes following a first portion of the first kernel but before a second portion of the first kernel. As another example, a pipelined processor in which a portion of the first kernel is in a first portion of the pipeline and a portion of the second kernel is in a second portion of the pipeline is considered to be "processing concurrently" two kernels. Two applications or kernels do not need to be processed at the exact same time by the exact same hardware device to be considered "processed concurrently."

[0011] The techniques described herein may, in different embodiments, be employed using any of a variety of parallel processors (e.g., vector processors, graphics processing units (GPUs), general-purpose GPUs (GPGPUs), non-scalar processors, highly parallel processors, artificial intelligence (AI) processors, inference engines, machine learning processors, other multi-threaded processing units, etc.). For ease of explanation, reference is made herein to exemplary systems and methods in which processing modules are employed. However, it will be understood that the systems and techniques described herein apply equally to the use of other types of parallel processors, unless otherwise noted.

[0012] 1 illustrates a processing system 100 for concurrently processing applications in PIM modules, according to at least some embodiments. The processing system 100 includes a data fabric 102 that is used to interconnect various components of the processing system 100, including multiple processing modules, such as processing modules 104-106, one or more memory controllers 108, and one or more I / O hubs 110. Each memory controller 108 is itself coupled to a system memory 112, and each I / O hub 110 is coupled to one or more I / O devices, such as I / O devices 114 and 115. At least one system memory 112 includes a PIM module 120.

[0013] The processing modules 104-106 include one or more processor cores and a local cache hierarchy. The processor core may include, for example, a CPU core, a GPU core, a DSP core, a parallel processor core, or a combination thereof. The processing client's local cache hierarchy includes one or more levels of cache. In some embodiments, at least one of the processing modules 104-106 is distinct from at least one other of the processing modules 104-106 (i.e., the processing modules 104-106 are heterogeneous). In other embodiments, the processing modules 104-106 are homogeneous. In some cases, as further described below with reference to FIG. 2, one or more of the processing modules 104-106 includes one or more coprocessors that perform tasks on behalf of the processing module 104-106.

[0014] The memory controller 108 acts as an interface between the corresponding system memory 112 and other components of the processing system 100. Accordingly, data cached in a processing client's local cache hierarchy is typically manipulated as blocks of data, referred to as "cache lines," which are addressed or otherwise located in the memory hierarchy using physical addresses in the system memory 112. Cache lines are accessed from the system memory 112 by the memory controller 108 in response to access requests from the processing clients, and the cache lines are installed or cached in one or more caches of the processing clients. Similarly, when a cache line containing modified data needs to be evicted from a processing client's local cache hierarchy and therefore updated in the system memory 112, the memory controller 108 manages this write-back process. Additionally, as described below with reference to FIGS. 2-5, in some cases, the PIM module 120 processes data on behalf of one or more of the processing modules 104-106. In some embodiments, the processing system 100 includes multiple system memories 112. Additionally, in some embodiments, the system memory 112 includes multiple PIM modules 120 .

[0015] I / O devices 114, 115 operate to transfer data to and from processing system 100 using DMA access operations. For example, any of I / O devices 114, 115 may include a network interface card (NIC) for connecting the node to a network to receive and transmit data, or a hard disk drive (HDD) or other mass storage device for non-volatile storage of relatively large amounts of data used by processing modules 104-106. In at least one embodiment, I / O hub 110 manages I / O devices 114, 115 and serves as an interface between data fabric 102 and I / O devices 114, 115. To illustrate, in some embodiments, I / O hub 110 includes a Peripheral Component Interconnect Express (PCIe) root complex to operate as a PCIe interconnect between I / O devices 114, 115 and data fabric 102.

[0016] As a general operational overview, data fabric 102 transports commands, data, requests, status communications, and other signaling between other components of processing system 100 and between processing system 100 and other nodes 141. One such subset of these transport operations is the storage of data provided by I / O devices 114, 115 in system memory 112 for use by one or more of processing modules 104-106. As further described with respect to FIGS. 2-5, in some cases, data stored in system memory 112 is processed using PIM module 120 in response to kernel instructions sent from one or more of processing modules 104-106 via data fabric 102. I / O agent 140 acts as a coherent agent for I / O hub 110 and I / O devices 114, 115. Additionally, in some embodiments, the transport layer 130 is coupled to corresponding transport layers or processing modules 104-16 of one or more other nodes 141 via one or more bridge components or coherent agents (not shown). In various embodiments, the data fabric 102 is compatible with one or more standardized interconnect specifications, such as the HyperTransport™ specification or the Infinity Fabric™ specification.

[0017] FIG. 2 is a block diagram of a PIM architecture 200 of the processing system 100, according to some embodiments. In other embodiments, the PIM architecture 200 corresponds to a different processing system that includes additional or fewer components relative to the processing system 100. In the illustrated embodiment, the PIM architecture includes at least a processing module 104 and a system memory 112. The processing module 104 includes a co-processing module 202. The system memory 112 includes a memory module 210. The memory module 210 includes a PCS module 212, a PIM kernel scheduler 214, a storage device 216, a storage map 218, and a PIM module 120. While the illustrated embodiment shows a particular configuration of components, other combinations of components are contemplated in various embodiments. For example, in some embodiments, the system memory 112 includes only a single memory module 210. As another example, in some embodiments, the processing module 104 does not include a co-processing module 202. Furthermore, in some embodiments, additional components, such as a buffer or a data fabric 102, are contemplated but are not shown for ease of illustration.

[0018] The processing module 104 executes various programs. Data for at least some of these programs is stored in the system memory 112 (e.g., in the storage device 216). In some cases, a decision is made to perform at least some processing functions in the system memory 112 (e.g., using one or more of the PIM modules 120) rather than repeatedly transmitting data requests and responses between the processing module 104 and the system memory 112. Accordingly, the processing module 104, or possibly the co-processing module 202, sends kernel instructions 204 to the system memory 112. In the illustrated embodiment, the kernel instructions 204 correspond to multiple applications, different threads within a single application, or both, and are executed as separate PIM kernels in the system memory 112. In some embodiments, the processing module 104 also executes some kernel instructions (e.g., kernel instructions that are not part of the ISA of the PIM module 120 or that the PIM module 120 does not have the bandwidth to execute).

[0019] In the illustrated embodiment, kernel instructions 204 include one or more command indexes 206 and one or more bank local addresses 208. Command index 206 indicates a PIM command to be executed by a PIM module (e.g., PIM module 120-1). In some cases, command index 206 indicates a respective PIM command to be executed via a respective instruction identifier. Furthermore, as described below with reference to FIG. 3, in some cases, at least some of command indexes 206 include virtual register addresses that indicate registers in system memory 112 (e.g., in storage device 216-2) that are used as part of executing the PIM command. Bank local addresses 208 indicate one or more memory addresses in a storage device (e.g., storage device 216-1) of system memory 112 to access when the PIM command performs a memory operation (e.g., a load operation). In some embodiments, bank local addresses 208 are full memory addresses. In other embodiments, bank local addresses 208 are modified or generated by system memory 112 (e.g., by adding a base address of a memory array to a data element index). Because the PIM module 120 has a smaller ISA and fewer hardware resources compared to, for example, the processing modules 104-106, in some embodiments, a separate kernel instruction 204 is sent for each PIM command (e.g., one kernel instruction is sent to perform a load operation and a second kernel instruction is sent to perform an add operation). In some embodiments, multiple sets of kernel instructions 204 (e.g., five kernel instructions or fifty kernel instructions) for one application or multiple applications are sent together, and possibly multiple PIM commands are stored in respective entries of the PCS module 212 for processing by one or more of the PIM modules 120. However, in some embodiments, processing resources (e.g., storage space within the PCS module 212) are limited, and therefore smaller groups of kernel instructions 204 are sent.Additionally, as described above, in some cases, some kernel instructions are executed in the processing module 104. Furthermore, in some cases, at least some PIM commands differ from corresponding instructions executed in the processing module 104 to perform similar operations. As a result, in some cases, the command index 206 is translated into a corresponding PIM command in the processing module 104. In other embodiments, the PIM kernel scheduler 214 translates the command indicated by the kernel instruction 204 into a corresponding PIM command.

[0020] System memory 112 includes one or more memory modules 210. In various embodiments, memory modules 210 perform operations for the same application or different applications. In some embodiments, memory modules 210 are dynamic random access memory (DRAM) devices. However, in other embodiments, memory modules 210 are other types of memory devices (e.g., static random access memory (SRAM) devices, synchronous DRAM (SDRAM) memory devices, flash memory devices, etc.). In some embodiments, memory modules 210 are the same type of memory devices. In other embodiments, memory modules 210 are different memory devices. In some embodiments, one or more of memory modules 210 include fewer or additional components. For example, in some embodiments, PIM kernel scheduler 214-1 corresponds to both memory modules 210-1 and 210-2, and memory module 210-2 does not include PIM kernel scheduler 214-2. In some embodiments, memory modules 210 include respective registers organized (e.g., physically or virtually) as register banks in storage device 216. In other embodiments, the registers are part of one or more separate devices associated with one or more of memory modules 210.

[0021] The PCS module 212 stores the PIM commands and sends the PIM commands to one or more PIM modules 120 for execution. As further described in the example of Figure 4, in some embodiments, one or more of the PCS modules 212 are virtualized. For example, in some cases, the PCS module 212-2 stores a sequence of PIM commands to be executed by the PIM module 120-2, where the PIM commands are part of multiple PIM kernels corresponding to multiple respective applications.

[0022] The PIM kernel scheduler 214 manages the virtualization of one or more PCS modules 212. For example, as described above, a command index 206 indicates each PIM command to be executed. In some embodiments, the PIM kernel scheduler 214-2 populates the PCS modules 212-2, translating command indexes 206 addressed to the PIM modules 120-2 into locations within the PCS modules 212-2, each location storing a corresponding PIM command. Additionally, in some embodiments, one or more of the PIM kernel schedulers 214 maintain and populate a storage map 218-2, a virtual-to-physical mapping table that indicates the physical registers (e.g., of the storage device 216-2) in which data used by the PIM commands is stored. In some embodiments, one or more of the storage maps 218 are stored in a storage device physically close to (e.g., on the same chip as) each PIM module 120. In some embodiments, one or more of the storage maps 218 are separate from the memory modules 210 but are associated with one or more memory modules 210 .

[0023] In some embodiments, to reduce PCS resource usage while submitting PCS modules 212, the PIM kernel scheduler 214 combines portions of at least two kernel instructions 204 corresponding to at least two PIM kernels. For example, as further described below with reference to FIG. 4, if two kernel instructions request "LOAD R0←[ADDR]" from PIM module 120-1, the PIM kernel scheduler 214-1 submits only a single indication of "LOAD R0←[ADDR]" to PCS module 212-1, thus conserving storage space in PCS module 212-1 and reducing the number of times PCS module 212-1 is programmed. Additionally, in some embodiments, one or more of the PCS module 212, the PIM kernel scheduler 214, or both are shared among multiple PIM modules 120. Thus, in some embodiments, the PIM kernel scheduler detects that PIM modules 120-1 and 120-2 each execute "LOAD R0←[ADDR]" for a different PIM kernel and stores only a single indication of "LOAD R0←[ADDR]" in PCS module 212-1.

[0024] In various embodiments, the PIM kernel scheduler 214 further determines combinations of PIM kernels to schedule in the PIM module 120, potentially delaying execution of at least some PIM commands. These combinations are determined based on combination metrics indicative of various factors, such as commonality of respective kernel instructions among the PIM kernels, the respective priorities of the PIM kernels, expected resource usage of the PIM kernels (e.g., PIM module usage, memory bandwidth usage, communication bandwidth, etc.), and PIM kernel dependencies (e.g., as indicated in a dependency graph received from the processing module 104). In some embodiments, the combinations are indicated by the processing module 104 (e.g., from a compiler). In other embodiments, the combinations are determined by the PIM kernel scheduler 214 at runtime, such as by receiving a dependency graph of the PIM kernels. Additionally, the order of PIM commands is determined based on various factors, such as the commonality of respective kernel instructions among the PIM kernels, the respective priorities of the PIM kernels, the expected resource usage of the PIM kernels, the dependencies of the PIM kernels (e.g., as shown in a dependency graph received from the processing module 104), or the time at which the corresponding kernel instructions were received.

[0025] The PIM module 120 receives PIM commands from the PCS module 212 and processes those PIM commands, possibly using the storage device 216 and data stored in registers of the storage device 216. In some embodiments, one or more of the PIM modules 120 are arithmetic logic units (ALUs). Additionally, in some embodiments, the storage device 216 is a register file organized as a physical or virtual register bank for the PIM module 120. In some cases, at least some registers of the register bank are allocated to a first kernel and at least some registers of the register bank are allocated to a second kernel. In response to a request from the processing module 104 (e.g., via kernel instructions 204), the PIM module 120 returns data to the processing module 104. Thus, the processing resources of the memory module 210 are potentially virtualized such that the PIM module 120 processes multiple PIM kernels simultaneously.

[0026] In some embodiments, one or more storage maps 218 and the portions of the PIM kernel scheduler 214 corresponding to the one or more storage maps 218 are stored in a scratchpad on the processing module 104. As a result, in these embodiments, a translation is performed on the bank-local addresses 208 before the kernel instructions 204 are sent to the system memory 112.

[0027] FIG. 3 illustrates an example 300 virtual-to-physical kernel register mapping 310, according to at least some embodiments. As mentioned above, in some embodiments, the virtual-to-physical kernel register mapping 310 is performed by a PIM kernel scheduler 214, such as PIM kernel scheduler 214-1 of FIG. 2. In the illustrated example 300, a kernel 1 virtual register map 302 and a kernel 2 virtual register map 304 are received at the PIM kernel scheduler 214. The registers indicated by instructions in kernel 1 virtual register map 302 are R0, R1, and R2. The PIM kernel scheduler 214 translates the received virtual register addresses into physical register addresses [0], [2], and [7], respectively, where the register addresses correspond to addresses in a storage device associated with the PIM kernel scheduler 214, such as registers in storage device 216-1. Similarly, the registers indicated by instructions in kernel 2 virtual register map 304 are R0, R1, R2, and R3. The PIM kernel scheduler translates these received virtual register addresses into physical register addresses [3], [4], [8], [9], respectively. Thus, the PIM module can process kernel instructions that indicate virtual indexes. Additionally, in some cases, at least some registers are allocated to the first kernel and at least some registers are allocated to the second kernel.

[0028] FIG. 4 illustrates an example 400 of kernel instruction merging and mapping 420 according to at least some embodiments. As mentioned above, in some embodiments, merging and mapping 420 is performed by a PIM kernel scheduler, such as PIM kernel scheduler 214-2 of FIG. 2. In the illustrated example 400, kernel 1 instruction 402 and kernel 2 instruction 404 each correspond to a respective index. Before storing in a PCS module, such as PCS module 212-2, the instructions are merged to reduce storage utilization within the PCS module. Thus, the "LOAD R0←[ADDR]" in both kernel 1 instruction 402 and kernel 2 instruction 404 is merged to physical index 0 in kernel 1 instruction index 406 and kernel 2 instruction index 408. Additionally, the PIM kernel scheduler determines the order of the instructions, resulting in PCS-merged kernel content 410. As a result, kernel 1 instruction 402 and kernel 2 instruction 404 are processed concurrently by a PIM module, such as PIM module 120-2, in the order shown. The bank-local addresses for each PIM kernel are generated by the host instruction and sent along with the respective PCS module index, so that in some cases, the same PCS module entry is used by multiple kernels but uses different physical registers. Thus, the PIM module can process kernel instructions for multiple kernels simultaneously, reducing the resource consumption of the PCS module.

[0029] 5 is a flow diagram illustrating a method 500 for processing kernel instructions for multiple kernels simultaneously using a PIM concurrent processing system, according to some embodiments. In some embodiments, method 500 is initiated by one or more processors in response to one or more instructions stored by a computer-readable storage medium.

[0030] At block 502, kernel instructions for multiple kernels to be processed simultaneously by the PIM ALU are received. For example, kernel instructions 204 corresponding to multiple kernels to be processed by PIM module 120-1 are received from processing module 104, and the ISA implemented by PIM module 120-1 has fewer instructions than the ISA implemented by processing module 104. At block 504, virtual register addresses of commands indicated by the kernel instructions are translated into physical register addresses. For example, PIM kernel scheduler 214-1 translates the virtual register addresses indicated by the kernel instructions 204 into physical register addresses of registers in storage device 216-1.

[0031] At block 506, at least some of the PIM commands indicated by the kernel instructions are merged. For example, the PIM kernel scheduler 214-1 analyzes the PIM commands indicated by the kernel instructions 204 and merges at least some of the PIM commands based on a combination metric. At block 508, the converted and merged commands are stored in a PCS module. For example, the converted and merged commands are stored in the PCS module 212-1. At block 510, at least one merged command is retrieved and executed. For example, at least one PIM command corresponding to at least two PIM kernels is retrieved from the PCS module 212-1 and sent to the PIM module 120-1 for execution. Thus, a method for processing kernel instructions for multiple kernels simultaneously is illustrated.

[0032] In some embodiments, a computer-readable storage medium includes any non-transitory storage medium or combination of non-transitory storage media that can be accessed by a computer system during use to provide instructions and / or data to the computer system. Such storage media may include, but are not limited to, optical media (e.g., compact discs (CDs), digital versatile discs (DVDs), Blu-ray discs), magnetic media (e.g., floppy disks, magnetic tape, or magnetic hard drives), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or microelectromechanical systems (MEMS) based storage media. In some embodiments, the computer readable storage medium is incorporated into the computer system (e.g., system RAM or ROM), fixedly attached to the computer system (e.g., a magnetic hard drive), removably attached to the computer system (e.g., an optical disk or Universal Serial Bus (USB)-based flash memory), or coupled to the computer system via a wired or wireless network (e.g., network accessible storage (NAS)).

[0033] In some embodiments, certain aspects of the techniques described above are implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored or otherwise tangibly embodied in a non-transitory computer-readable storage medium. The software may include instructions and specific data that, when executed by one or more processors, operate the one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer-readable storage medium may include, for example, a magnetic or optical disk storage device, a solid-state storage device such as flash memory, a cache, a random access memory (RAM), or other non-volatile memory device(s). The executable instructions stored on the non-transitory computer-readable storage medium may be implemented as source code, assembly language code, object code, or other form of instructions that can be interpreted or otherwise executed by one or more processors.

[0034] In addition to the above, it should be noted that not all activities or elements described in the summary description are required, that some of the particular activities or devices may not be required, that one or more additional activities may be performed, and that one or more additional elements may be included. Furthermore, the order in which the activities are listed is not necessarily the order in which they are performed. Also, the concepts have been described with reference to specific embodiments. However, those skilled in the art will recognize that various modifications and variations can be made without departing from the scope of the invention as set forth in the claims. Accordingly, the specification and drawings should be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the present invention.

[0035] Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and features from which any benefit, advantage, or solution may arise or be manifested are not construed as critical, essential, or essential features of any or all claims. Moreover, the specific embodiments described above are illustrative only, since the disclosed invention may be modified and practiced in different, but similar manners apparent to those skilled in the art having the benefit of the teachings herein. No limitations are intended to the details of construction or design herein shown, other than as described in the appended claims. It is therefore apparent that the specific embodiments described above may be altered or modified, and that all such variations are considered within the scope of the disclosed invention. Accordingly, the protection sought herein is set forth in the appended claims.

[0036] Within this disclosure, different entities (which may be variously referred to as "components," "units," "devices," etc.) may, at times, be described or claimed as being "configured" to perform one or more tasks or operations. This express phrase "configured to perform one or more tasks" is used herein to refer to a structure (i.e., a physical thing, such as an electronic circuit). More specifically, this express phrase is used to indicate that the structure is adapted to perform one or more tasks during operation. A structure may be said to be "configured to" perform some task even if the structure is not currently being operated. A "memory device configured to store data," for example, is intended to encompass an integrated circuit having circuitry that stores data during operation, even if the subject integrated circuit is not currently being used (e.g., no power source is connected to it). Thus, an entity described or listed as being "configured to" perform some task refers to something physical, such as a device, circuit, or memory that stores executable program instructions to perform a task. This phrase is not used herein to refer to something intangible. Furthermore, the term "configured to" is not intended to mean "configurable to." For example, an unprogrammed field programmable gate array is not considered to be "configured to" perform any particular function, but may be "configurable to" perform that function after programming. Additionally, reciting in the appended claims that a structure is "configured to" perform one or more tasks is clearly not intended to be construed as having means-plus-function elements.

Claims

1. 1. A memory device comprising: a plurality of storage devices configured to store data in response to program requests received from an external processor; a processing-in-memory (PIM) circuit configured to process PIM commands in response to kernel instructions received from the external processor, wherein an instruction set architecture (ISA) implemented by the PIM circuit has fewer instructions than an ISA implemented by the external processor, and processing resources of the PIM circuit are configured to be virtualized such that the PIM circuit processes PIM commands from multiple PIM kernels simultaneously. Memory device.

2. a PIM command store (PCS) circuit configured to store the PIM commands and provide the PIM commands to the PIM circuit, the PCS circuit configured to be virtualized. The memory device of claim 1.

3. and a PIM kernel scheduler configured in the PCS circuit to store a single PIM command corresponding to a portion of at least two kernel instructions corresponding to each of the at least two PIM kernels. The memory device of claim 2.

4. the PIM kernel scheduler is configured to combine portions of the at least two kernel instructions based on a combination metric. The memory device of claim 3.

5. The combination metric is based on at least one of a commonality of respective kernel instructions among PIM kernels awaiting execution in the PIM module, a weighted combination of kernel instructions expected based on resource usage of corresponding PIM kernels, or a dependency graph of the PIM kernels received from a processing unit. The memory device of claim 4.

6. the plurality of storage devices further comprising a register file configured as a register bank; the PIM kernel scheduler is configured to translate virtual register addresses indicated by the kernel instructions into physical register addresses in the register file; The memory device of claim 3.

7. the plurality of storage devices further comprising a register file configured as a register bank; the processing resources of the PIM circuit include the register files, wherein a first register file is configured to be assigned to a first PIM kernel of the at least two PIM kernels, and a second register file is configured to be assigned to a second PIM kernel of the at least two PIM kernels; The memory device of claim 3.

8. the PIM kernel scheduler is configured to schedule the PIM commands for execution on the PIM circuit based on the kernel instructions from the external processor; The memory device of claim 3.

9. 1. A method comprising: receiving, at a processing-in-memory (PIM) arithmetic logic unit (ALU), kernel instructions for a plurality of PIM kernels from a processor, wherein an instruction set architecture (ISA) implemented by the PIM ALU has fewer instructions than an ISA implemented by the processor; retrieving PIM ALU commands corresponding to said kernel instructions from a PIM command storage (PCS) circuit, at least one PIM ALU command corresponding to at least two PIM kernels; executing the PIM ALU command, which includes storing data for the at least two PIM kernels in a register bank of a memory containing the PIM ALU; method.

10. The kernel instruction is: individual command indexes, each indicating a PIM ALU command; at least one individual bank local address; 10. The method of claim 9.

11. and, prior to retrieving the PIM ALU commands from the PCS circuit, storing indices of the at least two PIM ALU commands in a single entry of the PCS circuit in response to detecting that at least two of the PIM ALU commands corresponding to the kernel instruction request the same operation in the PIM ALU.

10. The method of claim 9.

12. fetching the PIM ALU commands includes delaying execution of at least one of the PIM ALU commands based on a dependency of a corresponding PIM kernel.

10. The method of claim 9.

13. 1. A system comprising: a processor; a memory device configured to receive kernel instructions from the processor, the kernel instructions causing a processing-in-memory (PIM) circuit of the memory device to simultaneously process PIM commands for multiple PIM kernels, wherein simultaneously processing PIM commands includes storing data for at least some of the PIM kernels in a virtual register bank of the memory device, and wherein an instruction set architecture (ISA) implemented by the PIM circuit has fewer instructions than an ISA implemented by the processor. system.

14. the memory device comprises a PIM command store (PCS) circuit configured to store the PIM commands and provide the PIM commands to the PIM circuit; the processor comprises a portion of a PIM kernel scheduler that translates application addresses into PCS addresses; The system of claim 13.

15. the memory device comprises a second PIM circuit configured to simultaneously process second PIM commands for a second plurality of PIM kernels; The system of claim 13.

16. the processor comprises a coprocessor configured to send the kernel instructions to the memory device; The system of claim 13.

Images