Processing-in-memory simultaneous processing system and method

Abstract

The processing system (100) includes processing modules (104, 106) and a memory device (112). The memory device includes a processing-in-memory (PIM) module (120) that performs processing operations on behalf of the processing modules. The instruction set architecture (ISA) of the PIM module has fewer instructions than the ISA of the processing modules. Instructions received from the processing modules are translated such that the processing resources of the PIM module are virtualized. As a result, the PIM module simultaneously performs processing operations for multiple threads or applications of the processing modules.

Description

[Background technology]

[0001] Processing systems often include local memory resources that can be accessed relatively quickly by a processor (e.g., a central processing unit (CPU) or a graphics processing unit (GPU)) and external memory resources that take a relatively long time to access. In some cases, the external memory resources are located on a different chip than the processor. However, in some cases, due to the longer access times, instructions involving data stored in the external memory resources take an undesirably long time to process. In addition, in some cases, data communication between the external memory resources and the processor consumes an undesirable amount of interconnect capacity. In some cases, to reduce the number of accesses to the external memory resources, processing-in-memory (PIM) circuits are used to process data in the external memory resources rather than bringing the data to the processor. The PIM circuits perform limited processing applications on data stored in the external memory resources without the power consumption or area usage of a more typical processor. As a result, in some cases, the number of communications between the processor and the external memory resources is reduced, and the processing time of some instructions involving data stored in the external memory resources is reduced. However, due to this limited architecture, it is difficult to exploit hardware parallelism to process multiple kernels simultaneously. For example, due to the lack of virtualization, the PIM circuitry only loads commands for one kernel at a time, potentially creating a serialization bottleneck due to data dependencies. Summary of the Invention [Means for solving the problem]

[0002] In one general aspect, a memory device includes a plurality of storage devices configured to store data on behalf of programs executed by a processing unit, the processing unit being external to the memory device, a processing-in-memory (PIM) module configured to process PIM commands in response to kernel instructions corresponding to the programs from the processing unit, the PIM commands included, an instruction set architecture (ISA) implemented by the PIM module having fewer instructions than the ISA implemented by the processing unit, and processing resources of the PIM module configured to be virtualized such that the PIM module processes PIM commands from multiple PIM kernels simultaneously.

[0003] According to another general aspect, a method includes receiving, at a processing-in-memory (PIM) arithmetic logic unit (ALU), kernel instructions from a processing unit for multiple kernels to be processed simultaneously by the PIM ALU, where an instruction set architecture (ISA) implemented by the PIM ALU has fewer instructions than the ISA implemented by the processing unit. The method further includes retrieving PIM ALU commands corresponding to the kernel instructions from a PIM command storage (PCS) module, where at least one PIM instruction corresponds to the at least two kernels. The method also includes executing the PIM ALU commands, including storing data for at least two of the kernels in a register bank of a memory that includes the PIM ALU.

[0004] According to yet another general aspect, a system includes a processing unit configured to execute a plurality of programs, the system further including a memory device configured to receive kernel instructions from the processing unit, the kernel instructions causing a processing-in-memory (PIM) module of the memory device to simultaneously process PIM commands for a plurality of PIM kernels, the simultaneously processing the PIM commands including storing data for at least some of the PIM kernels in a plurality of storage devices of the memory device configured to operate as virtual register banks, the PIM commands included, an instruction set architecture (ISA) implemented by the PIM module having fewer instructions than the ISA implemented by the processing unit.

[0005] The present disclosure may be better understood, and its numerous features and advantages made apparent to those skilled in the art by reference to the following drawings, in which: The use of the same reference numbers in different drawings indicates similar or identical items. [Brief description of the drawings]

[0006] [Figure 1] 1 is a block diagram of a PIM concurrent processing system according to some embodiments. [Diagram 2] FIG. 1 is a block diagram of a PIM architecture according to some embodiments. [Diagram 3] FIG. 2 is a block diagram of an example virtual-to-physical kernel register mapping, according to some embodiments. [Figure 4] FIG. 1 is a block diagram of an example kernel instruction merging and mapping in accordance with some embodiments. [Diagram 5] FIG. 1 is a flow diagram of a method for simultaneously processing kernel instructions for multiple kernels using a PIM concurrent processing system according to some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] The processing system includes a processing unit and a memory device external to the processing unit. The memory device includes a processing-in-memory (PIM) module that processes data on behalf of the processing system. An instruction set architecture (ISA) of the PIM module is, in some cases, smaller than the ISA of the processing unit such that the PIM commands are a subset of instructions of the ISA of the processing unit. For example, in some cases, the PIM module is an arithmetic logic unit (ALU). As a result, compared to a processor such as a CPU that sends instructions to another processor such as a GPU, additional instructions are sent to the PIM module. In some embodiments, the additional instructions are sent as kernel instructions indicating the PIM commands to be executed. In response to the kernel instructions, the PIM module processes the PIM commands using data stored in the memory device.

[0008] In some cases, the PIM module processes PIM commands for multiple applications or different threads of the same application simultaneously. Address virtualization is typically used to prevent errors from multiple applications attempting to use a single address and to reduce potential security issues from an application receiving information about other applications through the use of physical addresses. However, address virtualization is difficult for some PIM modules due to the relative simplicity of the PIM modules compared to the processing units. As a result, these PIM modules serially provide access to resources (e.g., the PIM modules and associated storage devices such as register files). In some embodiments, the instant memory device further includes a PIM kernel scheduler that performs virtual-to-physical register mapping on behalf of the PIM module, enables register address virtualization in the memory device, and enables the PIM module to process multiple kernels simultaneously.

[0009] PIM commands are stored in a PIM command storage (PCS) module. However, in some embodiments, storage space in the PCS module is limited. As a result, in some cases, the PIM kernel scheduler further analyzes the incoming kernel instructions to identify whether multiple instruction sets request the same operation. In response to detecting that the same operation is requested, the PIM kernel scheduler combines the kernel instructions and causes the PCS module to store only a single PIM command. As a result, resource utilization by the PCS module is increased. Furthermore, in some embodiments, PCS addresses are virtualized such that instructions of multiple kernels are stored in the PCS module simultaneously. Thus, in some cases, in response to detecting that the same operation is requested by multiple kernels, the PIM kernel scheduler combines kernel instructions from multiple kernels.

[0010] As used herein, "processing simultaneously" refers to executing at least a portion of an application or kernel after the initiation but before the completion of a portion of another application or kernel. For example, a first kernel and a second kernel are considered to be "processed simultaneously" if portions of the two kernels are executed in an interleaved manner such that a portion of the second kernel executes following a first portion of the first kernel but before a second portion of the first kernel. As another example, a pipelined processor in which a portion of the first kernel is in a first portion of the pipeline and a portion of the second kernel is in a second portion of the pipeline is considered to be "processing simultaneously" two kernels. Two applications or kernels do not have to be processed at the exact same time by the exact same hardware device to be considered to be "processed simultaneously."

[0011] The techniques described herein may, in different embodiments, be employed using any of a variety of parallel processors (e.g., vector processors, graphics processing units (GPUs), general-purpose GPUs (GPGPUs), non-scalar processors, highly parallel processors, artificial intelligence (AI) processors, inference engines, machine learning processors, other multi-threaded processing units, etc.). For ease of explanation, reference is made herein to exemplary systems and methods in which processing modules are employed. However, it will be understood that the systems and techniques described herein apply equally to the use of other types of parallel processors, unless otherwise noted.

[0012] 1 illustrates a processing system 100 for concurrently processing applications in a PIM module, according to at least some embodiments. The processing system 100 includes a data fabric 102 that is used to interconnect various components of the processing system 100, including multiple processing modules, such as processing modules 104-106, one or more memory controllers 108, and one or more I / O hubs 110. Each memory controller 108 is itself coupled to a system memory 112, and each I / O hub 110 is coupled to one or more I / O devices, such as I / O devices 114, 115. At least one of the system memories 112 includes a PIM module 120.

[0013] The processing modules 104-106 include one or more processor cores and a local cache hierarchy. The processor cores may include, for example, a CPU core, a GPU core, a DSP core, a parallel processor core, or a combination thereof. The processing client's local cache hierarchy includes one or more levels of cache. In some embodiments, at least one of the processing modules 104-106 is distinct from at least one other of the processing modules 104-106 (i.e., the processing modules 104-106 are heterogeneous). In other embodiments, the processing modules 104-106 are homogeneous. In some cases, as described further below with reference to FIG. 2, one or more of the processing modules 104-106 include one or more co-processors that perform tasks on behalf of the processing modules 104-106.

[0014] The memory controller 108 acts as an interface between a corresponding system memory 112 and other components of the processing system 100. Thus, data cached in a processing client's local cache hierarchy is typically manipulated as blocks of data, referred to as "cachelines," and addressed or otherwise located in the memory hierarchy using a physical address of the system memory 112. Cachelines are accessed from the system memory 112 by the memory controller 108 in response to an access request from the processing client, and the cachelines are installed or cached in one or more caches of the processing client. Similarly, when a cacheline containing modified data needs to be evicted from a processing client's local cache hierarchy and therefore updated in the system memory 112, the memory controller 108 manages this write-back process. Additionally, as described below with reference to FIGS. 2-5, in some cases, the PIM module 120 processes data on behalf of one or more of the processing modules 104-106. In some embodiments, the processing system 100 includes multiple system memories 112. In addition, in some embodiments, the system memory 112 includes a number of PIM modules 120 .

[0015] The I / O devices 114, 115 operate to transfer data to and from the processing system 100 using DMA access operations. For example, any of the I / O devices 114, 115 may include a network interface card (NIC) for connecting the node to a network to receive and transmit data, or a hard disk drive (HDD) or other mass storage device for non-volatile storage of relatively large amounts of data used by the processing modules 104-106, etc. In at least one embodiment, the I / O hub 110 manages the I / O devices 114, 115 and serves as an interface between the data fabric 102 and the I / O devices 114, 115. To illustrate, in some embodiments, the I / O hub 110 includes a Peripheral Component Interconnect Express (PCIe) root complex to operate as a PCIe interconnect between the I / O devices 114, 115 and the data fabric 102.

[0016] As a general operational overview, the data fabric 102 transports commands, data, requests, status communications, and other signaling between other components of the processing system 100 and between the processing system 100 and other nodes 141. One such subset of these transport operations is the storage of data provided by the I / O devices 114, 115 in the system memory 112 for use by one or more of the processing modules 104-106. In some cases, data stored in the system memory 112 is processed using the PIM module 120 in response to kernel instructions sent from one or more of the processing modules 104-106 via the data fabric 102, as will be further described with respect to FIGS. 2-5. The I / O agent 140 acts as a coherent agent for the I / O hub 110 and the I / O devices 114, 115. Additionally, in some embodiments, the transport layer 130 is coupled to corresponding transport layers or processing modules 104-16 of one or more other nodes 141 via one or more bridge components or coherent agents (not shown). In various embodiments, the data fabric 102 is compatible with one or more standardized interconnect specifications, such as the HyperTransport™ specification or the Infinity Fabric™ specification.

[0017] 2 is a block diagram of a PIM architecture 200 of the processing system 100, according to some embodiments. In other embodiments, the PIM architecture 200 corresponds to a different processing system that includes additional or fewer components relative to the processing system 100. In the illustrated embodiment, the PIM architecture includes at least the processing module 104 and the system memory 112. The processing module 104 includes a co-processing module 202. The system memory 112 includes a memory module 210. The memory module 210 includes a PCS module 212, a PIM kernel scheduler 214, a storage device 216, a storage map 218, and a PIM module 120. Although the illustrated embodiment shows a particular configuration of components, other combinations of components are contemplated in various embodiments. For example, in some embodiments, the system memory 112 includes only a single memory module 210. As another example, in some embodiments, the processing module 104 does not include a co-processing module 202. Additionally, in some embodiments, additional components, such as buffers or a data fabric 102, are contemplated but are not shown for ease of illustration.

[0018] The processing module 104 executes various programs. Data for at least some of these programs is stored in the system memory 112 (e.g., in the storage device 216). In some cases, a decision is made to execute at least some processing functions in the system memory 112 (e.g., using one or more of the PIM modules 120) rather than repeatedly sending data requests and responses between the processing module 104 and the system memory 112. Thus, the processing module 104, or possibly the co-processing module 202, sends kernel instructions 204 to the system memory 112. In the illustrated embodiment, the kernel instructions 204 correspond to multiple applications, different threads within a single application, or both, and are executed as separate PIM kernels in the system memory 112. In some embodiments, the processing module 104 also executes some kernel instructions (e.g., kernel instructions that are not part of the ISA of the PIM module 120, or kernel instructions that the PIM module 120 does not have the bandwidth to execute).

[0019] In the illustrated embodiment, the kernel instruction 204 includes one or more command indexes 206 and one or more bank local addresses 208. The command index 206 indicates a PIM command to be executed by a PIM module (e.g., PIM module 120-1). In some cases, the command index 206 indicates a respective PIM command to be executed via a respective instruction identifier. Additionally, as described below with reference to FIG. 3, in some cases, at least some of the command indexes 206 include virtual register addresses that indicate registers in the system memory 112 (e.g., in storage device 216-2) that are used as part of executing the PIM command. The bank local addresses 208 indicate one or more memory addresses in a storage device (e.g., storage device 216-1) of the system memory 112 to access when the PIM command performs a memory operation (e.g., a load operation). In some embodiments, the bank local addresses 208 are full memory addresses. In other embodiments, the bank local addresses 208 are modified or generated by the system memory 112 (e.g., by adding a base address of a memory array to an index of a data element). Because the PIM module 120 has a smaller ISA and fewer hardware resources, e.g., as compared to the processing modules 104-106, in some embodiments, a separate kernel instruction 204 is sent for each PIM command (e.g., one kernel instruction is sent to perform a load operation and a second kernel instruction is sent to perform an add operation). In some embodiments, multiple sets of kernel instructions 204 (e.g., 5 kernel instructions or 50 kernel instructions) for one application or multiple applications are sent together, and possibly multiple PIM commands are stored in respective entries of the PCS module 212 for processing by one or more of the PIM modules 120. However, in some embodiments, processing resources (e.g., storage space in the PCS module 212) are limited, and thus smaller groups of kernel instructions 204 are sent.In addition, as discussed above, in some cases, some kernel instructions are executed in the processing module 104. Further, in some cases, at least some PIM commands differ from corresponding instructions executed in the processing module 104 to perform similar operations. As a result, in some cases, the command index 206 is translated into a corresponding PIM command in the processing module 104. In other embodiments, the PIM kernel scheduler 214 translates the command indicated by the kernel instruction 204 into a corresponding PIM command.

[0020] System memory 112 includes one or more memory modules 210. In various embodiments, memory modules 210 perform operations for the same application or different applications. In some embodiments, memory modules 210 are dynamic random access memory (DRAM) devices. However, in other embodiments, memory modules 210 are other types of memory devices (e.g., static random access memory (SRAM) devices, synchronous DRAM (SDRAM) memory devices, flash memory devices, etc.). In some embodiments, memory modules 210 are the same type of memory devices. In other embodiments, memory modules 210 are different memory devices. In some embodiments, one or more of memory modules 210 include fewer or additional components. For example, in some embodiments, PIM kernel scheduler 214-1 corresponds to both memory modules 210-1 and 210-2, and memory module 210-2 does not include PIM kernel scheduler 214-2. In some embodiments, memory modules 210 include respective registers organized (e.g., physically or virtually) as a register bank in storage device 216. In other embodiments, the registers are part of one or more separate devices associated with one or more of memory modules 210.

[0021] The PCS module 212 stores the PIM commands and sends the PIM commands to one or more PIM modules 120 for execution. In some embodiments, one or more of the PCS modules 212 are virtualized, as further described in the example of Figure 4. For example, in some cases, the PCS module 212-2 stores sequences of PIM commands to be executed by the PIM module 120-2, where the PIM commands are part of multiple PIM kernels corresponding to multiple respective applications.

[0022] The PIM kernel scheduler 214 manages the virtualization of one or more PCS modules 212. For example, as described above, the command index 206 indicates the respective PIM command to be executed. In some embodiments, the PIM kernel scheduler 214-2 populates the PCS module 212-2, translating the command index 206 addressed to the PIM module 120-2 into a location within the PCS module 212-2, where each location stores a corresponding PIM command. Additionally, in some embodiments, one or more of the PIM kernel schedulers 214 maintain and populate a storage map 218-2, a virtual-to-physical mapping table that indicates the physical registers (e.g., of the storage device 216-2) in which data used by the PIM commands is stored. In some embodiments, one or more of the storage maps 218 are stored in a storage device that is physically close to (e.g., on the same chip as) the respective PIM module 120. In some embodiments, one or more of the storage maps 218 are separate from the memory modules 210 but are associated with one or more of the memory modules 210.

[0023] In some embodiments, to reduce PCS resource usage while submitting the PCS module 212, the PIM kernel scheduler 214 combines portions of at least two kernel instructions 204 corresponding to at least two PIM kernels. For example, as further described below with reference to FIG. 4, if two kernel instructions request "LOAD R0←[ADDR]" from the PIM module 120-1, the PIM kernel scheduler 214-1 submits only a single indication of "LOAD R0←[ADDR]" to the PCS module 212-1, thus saving storage space in the PCS module 212-1 and reducing the number of times the PCS module 212-1 is programmed. Additionally, in some embodiments, one or more of the PCS module 212, the PIM kernel scheduler 214, or both are shared among multiple PIM modules 120. Thus, in some embodiments, the PIM kernel scheduler detects that PIM modules 120-1 and 120-2 each execute "LOAD R0←[ADDR]" for a different PIM kernel and stores only a single indication of "LOAD R0←[ADDR]" in PCS module 212-1.

[0024] In various embodiments, the PIM kernel scheduler 214 further determines combinations of PIM kernels to schedule in the PIM module 120, possibly delaying execution of at least some PIM commands. These combinations are determined based on combination metrics indicative of various factors, such as commonality of respective kernel instructions among the PIM kernels, respective priorities of the PIM kernels, expected resource usage of the PIM kernels (e.g., PIM module usage, memory bandwidth usage, communication bandwidth, etc.), dependencies of the PIM kernels (e.g., as indicated in a dependency graph received from the processing module 104), etc. In some embodiments, the combinations are indicated by the processing module 104 (e.g., from a compiler). In other embodiments, the combinations are determined by the PIM kernel scheduler 214 at run time, such as by receiving a dependency graph of the PIM kernels. The order of PIM commands may also be determined based on various factors, such as the commonality of respective kernel instructions among the PIM kernels, the respective priorities of the PIM kernels, the expected resource usage of the PIM kernels, dependencies of the PIM kernels (e.g., as shown in a dependency graph received from the processing module 104), or the time at which the corresponding kernel instructions were received.

[0025] The PIM module 120 receives PIM commands from the PCS module 212 and processes those PIM commands, possibly using the storage device 216 and data stored in the registers of the storage device 216. In some embodiments, one or more of the PIM modules 120 are arithmetic logic units (ALUs). Additionally, in some embodiments, the storage device 216 is a register file organized as a physical or virtual register bank for the PIM module 120. In some cases, at least some registers of the register bank are assigned to a first kernel and at least some registers of the register bank are assigned to a second kernel. In response to a request from the processing module 104 (e.g., via kernel instructions 204), the PIM module 120 returns data to the processing module 104. Thus, the processing resources of the memory module 210 are virtualized, possibly such that the PIM module 120 processes multiple PIM kernels simultaneously.

[0026] In some embodiments, one or more storage maps 218 and portions of the PIM kernel scheduler 214 that correspond to the one or more storage maps 218 are stored in a scratchpad on the processing module 104. As a result, in these embodiments, a translation is performed on the bank local addresses 208 before the kernel instructions 204 are sent to the system memory 112.

[0027] FIG. 3 illustrates an example 300 virtual-to-physical kernel register mapping 310 according to at least some embodiments. As mentioned above, in some embodiments, the virtual-to-physical kernel register mapping 310 is performed by a PIM kernel scheduler 214, such as PIM kernel scheduler 214-1 of FIG. 2. In the illustrated example 300, a kernel 1 virtual register map 302 and a kernel 2 virtual register map 304 are received at the PIM kernel scheduler 214. The registers indicated by instructions in the kernel 1 virtual register map 302 are R0, R1, and R2. The PIM kernel scheduler 214 translates the received virtual register addresses into physical register addresses [0], [2], and [7], respectively, where the register addresses correspond to addresses in a storage device associated with the PIM kernel scheduler 214, such as a register in storage device 216-1. Similarly, the registers indicated by instructions in the kernel 2 virtual register map 304 are R0, R1, R2, and R3. The PIM kernel scheduler translates these received virtual register addresses into physical register addresses [3], [4], [8], [9], respectively. Thus, the PIM module can process kernel instructions that indicate a virtual index. In addition, in some cases, at least some registers are allocated to a first kernel and at least some registers are allocated to a second kernel.

[0028] FIG. 4 illustrates an example 400 of kernel instruction merging and mapping 420 according to at least some embodiments. As mentioned above, in some embodiments, merging and mapping 420 is performed by a PIM kernel scheduler, such as PIM kernel scheduler 214-2 of FIG. 2. In the illustrated example 400, each of kernel 1 instruction 402 and kernel 2 instruction 404 corresponds to a respective index. Before storing in a PCS module, such as PCS module 212-2, the instructions are merged to reduce storage utilization in the PCS module. Thus, the "LOAD R0←[ADDR]" of both kernel 1 instruction 402 and kernel 2 instruction 404 are merged to physical index 0 in kernel 1 instruction index 406 and kernel 2 instruction index 408. Additionally, the PIM kernel scheduler determines the order of instructions, resulting in PCS merged kernel content 410. As a result, kernel 1 instruction 402 and kernel 2 instruction 404 are processed simultaneously by a PIM module, such as PIM module 120-2, in the order shown. The bank-local addresses for each PIM kernel are generated by the host instruction and sent along with the respective PCS module index, so that in some cases, the same PCS module entry is used by multiple kernels but with different physical registers. Thus, the PIM module can process kernel instructions for multiple kernels simultaneously, reducing the resource consumption of the PCS module.

[0029] 5 is a flow diagram illustrating a method 500 for simultaneously processing kernel instructions for multiple kernels using a PIM concurrent processing system, in accordance with some embodiments. In some embodiments, method 500 is initiated by one or more processors in response to one or more instructions stored by a computer-readable storage medium.

[0030] At block 502, kernel instructions for multiple kernels to be processed simultaneously by the PIM ALU are received. For example, kernel instructions 204 corresponding to multiple kernels to be processed by PIM module 120-1 are received from processing module 104, where the ISA implemented by PIM module 120-1 has fewer instructions than the ISA implemented by processing module 104. At block 504, virtual register addresses of commands indicated by the kernel instructions are translated to physical register addresses. For example, PIM kernel scheduler 214-1 translates the virtual register addresses indicated by kernel instructions 204 to physical register addresses of registers in storage device 216-1.

[0031] At block 506, at least some of the PIM commands indicated by the kernel instructions are merged. For example, the PIM kernel scheduler 214-1 analyzes the PIM commands indicated by the kernel instructions 204 and merges at least some of the PIM commands based on the combination metric. At block 508, the converted and merged command is stored in a PCS module. For example, the converted and merged command is stored in the PCS module 212-1. At block 510, at least one merged command is retrieved and executed. For example, at least one PIM command corresponding to at least two PIM kernels is retrieved from the PCS module 212-1 and sent to the PIM module 120-1 for execution. Thus, a method for simultaneously processing kernel instructions for multiple kernels is illustrated.

[0032] In some embodiments, a computer readable storage medium includes any non-transitory storage medium or combination of non-transitory storage media that can be accessed by a computer system during use to provide instructions and / or data to the computer system. Such storage media may include, but are not limited to, optical media (e.g., compact disc (CD), digital versatile disc (DVD), Blu-ray disc), magnetic media (e.g., floppy disk, magnetic tape, or magnetic hard drive), volatile memory (e.g., Random Access Memory (RAM) or cache), non-volatile memory (e.g., Read-Only Memory (ROM) or flash memory), or Microelectromechanical System (MEMS) based storage media. In some embodiments, the computer readable storage medium is incorporated into the computer system (e.g., system RAM or ROM), fixedly attached to the computer system (e.g., a magnetic hard drive), removably attached to the computer system (e.g., an optical disk or Universal Serial Bus (USB)-based flash memory), or coupled to the computer system via a wired or wireless network (e.g., network accessible storage (NAS)).

[0033] In some embodiments, certain aspects of the techniques described above are implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored or otherwise tangibly embodied in a non-transitory computer-readable storage medium. The software may include instructions and specific data that, when executed by the one or more processors, operate the one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer-readable storage medium may include, for example, a magnetic or optical disk storage device, a solid-state storage device such as a flash memory, a cache, a random access memory (RAM), or other non-volatile memory device(s), etc. The executable instructions stored in the non-transitory computer-readable storage medium may be implemented as source code, assembly language code, object code, or other form of instructions that can be interpreted or otherwise executed by one or more processors.

[0034] In addition to the above, it should be noted that not all activities or elements described in the summary description are required, some of the specific activities or devices may not be required, one or more additional activities may be performed, and one or more additional elements may be included. Furthermore, the order in which the activities are listed is not necessarily the order in which they are performed. Also, the concepts have been described with reference to specific embodiments. However, those skilled in the art will appreciate that various changes and modifications can be made without departing from the scope of the invention as set forth in the claims. Thus, the specification and drawings should be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the invention.

[0035] Benefits, other advantages, and solutions to problems have been described above with respect to specific embodiments. However, the benefits, advantages, solutions to problems, and features by which any benefit, advantage, or solution may occur or be manifested are not to be construed as critical, essential, or essential features of any or all claims. Moreover, the specific embodiments described above are illustrative only, as the disclosed invention may be modified and practiced in different but similar manners apparent to those skilled in the art having the benefit of the teachings herein. No limitations are intended to the details of construction or design herein shown, other than as set forth in the appended claims. It is therefore apparent that the specific embodiments described above may be altered or modified, and all such variations are considered to be within the scope of the disclosed invention. Accordingly, the protection sought herein is set forth in the appended claims.

[0036] Within this disclosure, in some cases, different entities (which are variously referred to as "components," "units," "devices," etc.) are described or claimed as being "configured" to perform one or more tasks or operations. This clear phrase, an entity configured to [perform one or more tasks], is used herein to refer to a structure (i.e., a physical thing, such as an electronic circuit). More specifically, this clear phrase is used to indicate that the structure is adapted to perform one or more tasks during operation. A structure can be said to be "configured to" perform some task even if the structure is not currently being operated. A "memory device configured to store data" is intended to encompass, for example, an integrated circuit having a circuit that stores data during operation, even if the subject integrated circuit is not currently being used (e.g., no power source is connected to it). Thus, an entity described or listed as being "configured to" perform some task refers to something physical, such as a device, a circuit, a memory that stores executable program instructions to perform a task. This phrase is not used herein to refer to something intangible. Moreover, the term "configured to" is not intended to mean "configurable to." For example, an unprogrammed field programmable gate array is not considered to be "configured to" perform any particular function, but may be "configurable to" perform that function after programming. In addition, reciting in the appended claims that a structure is "configured to" perform one or more tasks is clearly not intended to be construed as having means-plus-function elements.

Claims

1. 1. A memory device comprising: a plurality of storage devices configured to store data in response to program requests received from an external processor; a processing-in-memory (PIM) circuit configured to process PIM commands in response to kernel instructions received from the external processor, wherein an instruction set architecture (ISA) implemented by the PIM circuit has fewer instructions than an ISA implemented by the external processor, and processing resources of the PIM circuit are configured to be virtualized such that the PIM circuit processes PIM commands from multiple PIM kernels simultaneously. Memory device.

2. a PIM command storage (PCS) circuit configured to store the PIM commands and provide the PIM commands to the PIM circuit, the PCS circuit configured to be virtualized. The memory device of claim 1 .

3. In the PCS circuit, further comprising a PIM kernel scheduler configured to store a single PIM command corresponding to a portion of at least two kernel instructions corresponding to each of the at least two PIM kernels. The memory device of claim 2.

4. the PIM kernel scheduler is configured to combine portions of the at least two kernel instructions based on a combination metric; The memory device of claim 3.

5. the combination metric is based on at least one of a commonality of respective kernel instructions among PIM kernels awaiting execution in the PIM module, a weighted combination of kernel instructions expected based on resource usage of corresponding PIM kernels, or a dependency graph of the PIM kernels received from the processing unit. The memory device of claim 4.

6. the plurality of storage devices further comprising a register file configured as a register bank; the PIM kernel scheduler is configured to translate virtual register addresses indicated by the kernel instructions into physical register addresses in the register file; The memory device of claim 3.

7. the plurality of storage devices further comprising a register file configured as a register bank; the processing resources of the PIM circuit include the register files, a first register file configured to be assigned to a first PIM kernel of the at least two PIM kernels, and a second register file configured to be assigned to a second PIM kernel of the at least two PIM kernels; The memory device of claim 3.

8. the PIM kernel scheduler is configured to schedule the PIM commands for execution on the PIM circuitry based on the kernel instructions from the external processor; The memory device of claim 3.

9. 1. A method comprising: receiving kernel instructions for a plurality of kernels from a processor at a processing-in-memory (PIM) arithmetic logic unit (ALU), wherein an instruction set architecture (ISA) implemented by the PIM ALU has fewer instructions than an ISA implemented by the processor; retrieving PIM ALU commands corresponding to said kernel instructions from a PIM command storage (PCS) circuit, at least one PIMALU command corresponding to at least two kernels; and executing the PIM ALU commands including storing data for at least two of the kernels in a register bank of a memory that includes the PIM ALU. method.

10. The kernel instruction is: individual command indexes, each of which indicates a PIM ALU command; at least one individual bank local address; 10. The method of claim 9.

11. and prior to retrieving the PIM ALU commands from the PCS circuit, in response to detecting that at least two of the PIM ALU commands corresponding to the kernel instruction request a same operation in the PIM ALU, storing indices of the at least two PIM ALU commands in a single entry of the PCS circuit.

10. The method of claim 9.

12. fetching the PIM ALU commands includes delaying execution of at least one of the PIM ALU commands based on a dependency of a corresponding PIM kernel.

10. The method of claim 9.

13. 1. A system comprising: A processor; a memory device configured to receive kernel instructions from the processor, the kernel instructions causing a processing-in-memory (PIM) circuit of the memory device to process PIM commands for multiple PIM kernels simultaneously, where processing PIM commands simultaneously includes storing data for at least some of the PIM kernels in a virtual register bank of the memory device, and an instruction set architecture (ISA) implemented by the PIM circuitry has fewer instructions than an ISA implemented by the processor. system.

14. the memory device comprises a PIM command storage (PCS) circuit configured to store the PIM commands and provide the PIM commands to the PIM circuit; the processor comprises a portion of a PIM kernel scheduler that translates application addresses into PCS addresses; The system of claim 13.

15. the memory device comprises a second PIM circuit configured to simultaneously process second PIM commands for a second plurality of PIM kernels; The system of claim 13.

16. the processor comprises a co-processor configured to transmit the kernel instructions to the memory device; The system of claim 13.

Images