In-memory processor and method of using in-memory processor output instructions

Abstract

A processor-in-memory (PIM) and a method for outputting instructions using the PIM are disclosed. The PIM includes an internal processor, memory, and registers configured to store instruction metadata. The memory is configured to store a lookup table generated by predicting future instructions. An external processor is configured to retrieve the instruction corresponding to the instruction metadata from the lookup table in response to a PIM application programming interface (API) call instruction that includes the instruction metadata. The instruction corresponding to the instruction metadata is then output based on the retrieval result.

Description

[0001] This application claims the benefit of Korean Patent Application No. 10-2020-0154790, filed on November 18, 2020, with the Korean Intellectual Property Office, the entire disclosure of which is incorporated herein by reference for all purposes. Technical Field

[0002] The following description relates to processing-in-memory (PIM), and more specifically, to techniques for outputting instructions from PIM. Background Technology

[0003] In the standard model known as the von Neumann architecture, data is stored in memory. Memory is separate from the processor, and data moves between them. In this architecture, latency is unavoidable due to frequent data movements. Although memory has been improved with increasing processor speed, these improvements have primarily focused on the ability to store more data in a smaller space, rather than on transfer rates. As the latency required to fetch data from memory to the processor increases, the processor's speed may be limited by the transfer rate at the bottleneck. Therefore, the von Neumann bottleneck, where throughput is limited by latency, can occur in standard computer architectures. To address this bottleneck, in-memory processing has been proposed as a structure for performing operations within a single-chip memory. Summary of the Invention

[0004] This summary is provided to introduce, in a simplified form, the selection of concepts that will be further described in the detailed embodiments below. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to help determine the scope of the claimed subject matter.

[0005] In one general aspect, a process-in-memory (PIM) includes: an internal processor, a memory, and a register configured to store instruction metadata, wherein the memory is configured to store a lookup table generated by predicting future instructions, and in response to a PIM application programming interface (API) call instruction including the instruction metadata, an instruction corresponding to the instruction metadata is retrieved from the lookup table by an external processor, and based on the retrieval result, the instruction corresponding to the instruction metadata is output by the external processor, and the internal processor is configured to use the output instruction.

[0006] The lookup table can be generated by an internal or external processor by predicting future instructions based on a log of past instructions.

[0007] If the instruction corresponding to the instruction metadata is retrieved, the retrieved instruction can be output by the external processor.

[0008] If the instruction corresponding to the instruction metadata is not found, the instruction corresponding to the instruction metadata can be generated and output by an external processor.

[0009] If the instruction corresponding to the instruction metadata is not found, the instruction corresponding to the instruction metadata can be stored in a lookup table by an external processor.

[0010] In another general aspect, PIM includes an internal processor, memory, and registers configured to store instruction metadata, wherein, in response to a PIM API call instruction including instruction metadata, an instruction corresponding to the instruction metadata is generated by an external processor, the generated instruction is stored in a lookup table by the external processor, the generated instruction is output by the external processor, and the internal processor is configured to use the output instruction.

[0011] In response to the next PIM API call instruction, the instruction corresponding to the next PIM API call instruction can be retrieved from the lookup table by the external processor, and based on the retrieval result, the instruction corresponding to the next PIM API call instruction can be output by the external processor.

[0012] In another general aspect, a method for outputting instructions from a PIM including an internal processor, memory, and registers includes: generating a lookup table by predicting future instructions and storing the lookup table in memory; storing instruction metadata in registers; retrieving an instruction corresponding to the instruction metadata from the lookup table in response to a PIM application programming interface (API) call instruction including the instruction metadata; outputting the instruction corresponding to the instruction metadata based on the retrieval result; and having the output instruction used by the internal processor.

[0013] The steps for generating a lookup table may include: generating the lookup table by an internal or external processor by predicting future instructions based on a log of past instructions.

[0014] The steps for outputting instructions may include: if an instruction corresponding to the instruction metadata is retrieved, outputting the retrieved instruction.

[0015] The steps for outputting instructions may include: if no instruction corresponding to the instruction metadata is found, generating an instruction corresponding to the instruction metadata and outputting the generated instruction.

[0016] The steps for outputting instructions may include: if no instruction corresponding to the instruction metadata is found, storing the instruction corresponding to the instruction metadata in a lookup table.

[0017] In a further general aspect, a method for outputting instructions from a PIM that includes an internal processor, memory, and registers includes: storing instruction metadata in a register; generating an instruction corresponding to the instruction metadata in response to a PIM API call instruction that includes the instruction metadata; storing the generated instruction in a lookup table; outputting the generated instruction; and having the output instruction used by the internal processor.

[0018] The method may further include: in response to a next PIM API call instruction, retrieving an instruction corresponding to the next PIM API call instruction from a lookup table, and outputting the instruction corresponding to the next PIM API call instruction based on the retrieval result.

[0019] The step of generating the lookup table can be performed by an internal processor or an external processor.

[0020] The steps of generating, storing, retrieving, and outputting can be executed by an external processor.

[0021] In yet another general aspect, a non-transitory computer-readable storage medium can store instructions that, when executed by a processor, cause the processor to perform the method.

[0022] Other features and aspects will become clear from the following detailed description, drawings, and claims. Attached Figure Description

[0023] Figure 1 An example of accessing memory via Process in Memory (PIM) is shown.

[0024] Figure 2 This shows an example of the overall configuration of PIM.

[0025] Figure 3 This shows an example of the overall operation of the instruction output method.

[0026] Figure 4 This shows an example of a command output method.

[0027] Figure 5 This shows an example of a command output method.

[0028] Figure 6 This shows an example of the overall operation of the instruction output method.

[0029] Figure 7 This shows an example of a command output method.

[0030] Throughout the accompanying drawings and detailed embodiments, unless otherwise described or provided, the same reference numerals will be understood to denote the same elements, features, and structures. The drawings may not be to scale, and for clarity, illustration, and convenience, the relative sizes, proportions, and depictions of elements in the drawings may be exaggerated. Detailed Implementation

[0031] The examples will be described in detail below with reference to the accompanying drawings. However, various changes and modifications can be made to the examples. Here, the examples are not to be construed as limited to the disclosure. The examples should be understood to include all changes, equivalents, and substitutions within the concept and technical scope of this disclosure.

[0032] The terminology used herein is for the purpose of describing particular examples only and should not be construed as limiting the scope of those examples. Unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. It will also be understood that the terms “comprising” and / or “including / containing” as used herein indicate the presence of the stated features, wholes, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or groups thereof.

[0033] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by a person of ordinary skill in the art to which the examples pertain. It will also be understood that, unless expressly defined herein, terms (such as those defined in general dictionaries) shall be interpreted as having a meaning consistent with their meaning in the context of the relevant field and shall not be interpreted in an idealized or overly formalized manner.

[0034] When describing the examples with reference to the accompanying drawings, the same reference numerals denote the same constituent elements, and repetitive descriptions associated with them will be omitted. In the description of the examples, descriptions of well-known related structures or functions will be omitted where such detailed descriptions would lead to a vague interpretation of this disclosure.

[0035] Furthermore, in the description of components, terms such as first, second, A, B, (a), (b), etc., may be used when describing components of this disclosure. These terms are used only to distinguish one component from another, and the nature, order, or sequence of the components is not limited by these terms. When a component is described as being "connected," "joined," or "attached" to another component, it should be understood that the one component may be directly connected or attached to the other component, and intermediate components may also be "connected," "joined," or "attached" to the component.

[0036] The same names can be used to describe components included in the examples above and components with common functions. Unless otherwise stated, the descriptions of the examples are applicable to the following examples, therefore, for the sake of brevity, repeated descriptions will be omitted.

[0037] Figure 1 This illustrates an example of accessing memory via processing-in-memory (PIM).

[0038] PIM stands for Processor-in-Memory, which refers to a semiconductor where the processor and memory are integrated on a single chip. The memory can include Random Access Memory (RAM). PIM can also be referred to as processor-in-memory, in-memory processing, in-memory processor, computing-in-memory, in-memory computer, etc.

[0039] Operations within the PIM require unique instructions. These instructions may be referred to as PIM instructions, PIM enable instructions, or memory commands.

[0040] PIM instructions used in PIM can be accessed via an external host. To access these instructions, the external host or the interface between the host and memory can be changed. However, using existing memory commands incurs the overhead of translating high-level instructions into memory commands. In systems that iteratively execute operations, PIM instructions need to be generated for each operation. For example, to replace the Long Short-Term Memory (LSTM) operations of a deep neural network with PIM instructions, a certain number of identical PIM instructions need to be generated, where this number equals (number of layers × number of steps).

[0041] PIM can pre-generate instructions. Based on the size and type of the operation, and hardware information, PIM pre-generates the desired instructions and caches them in a lookup table. Subsequently, PIM retrieves the instructions from the lookup table in response to application programming interface (API) call instructions.

[0042] Reference Figure 1User application 112 can receive user input 111. User application 112 can invoke PIM software stack 113. In operation 119, PIM software stack 113 can generate instructions corresponding to user input 111 to execute memory operations in PIM. When user application 112 invokes an operation supported by PIM, PIM software stack 113 can generate the corresponding PIM instruction. Subsequently, in operation 116, the desired operation can be executed through operating system 114 and hardware architecture 115.

[0043] The PIM software stack 113 can perform in-memory operations. PIM can pre-generate instructions corresponding to instruction metadata based on the metadata information and store the generated instructions in a lookup table. The instruction metadata information indicates information about the operation. Instruction metadata information may include the type of operation, the size of the operation, and hardware information.

[0044] In operation 117, PIM obtains instruction metadata. In operation 118, PIM refers to a lookup table storing instructions. In operation 119, PIM searches the lookup table and outputs the instruction corresponding to the instruction metadata, or it can generate an instruction not found in the lookup table. When the same operation is called, PIM searches the lookup table for the instruction; if it exists, it uses the instruction; otherwise, it generates a new instruction. This reduces the overhead of generating instructions for each operation call.

[0045] If instruction statistics can be obtained from user application 112, a lookup table can be pre-generated before runtime, thereby reducing runtime overhead. PIM can statistically analyze instruction log information. Based on the analysis results, PIM can predict future instructions and pre-store the predicted instructions in the lookup table. Subsequently, PIM can retrieve instructions from the lookup table in response to API call instructions.

[0046] Each time a new instruction is invoked, PIM can update the lookup table with that instruction. For example, PIM can store all new instructions in the lookup table. As another example, PIM can add or remove new instructions based on the number of times they are invoked.

[0047] PIM can be applied to compilers or algorithms of systems that use PIM. PIM can be used in neural network processing. PIM can be used in various application areas such as natural language processing (NLP), image recognition, speech recognition, and time series prediction using deep neural networks.

[0048] Figure 2 This shows an example of the overall configuration of PIM.

[0049] Reference Figure 2The PIM 200 generates a lookup table by pre-analyzing instructions before execution at runtime. The PIM 200 can pre-generate expected instructions based on the size and type of the operation and hardware information, and cache the generated instructions in the lookup table. For this purpose, the PIM 200 includes memory 210, an internal processor 220, and registers 230. Registers 230 can store instruction metadata. Memory 210 can store a lookup table 211 generated by predicting future instructions.

[0050] Host 240 (e.g., an external processor) may retrieve an instruction from a lookup table in response to a call instruction. Host 240, in response to a PIM API call instruction including instruction metadata, retrieves the instruction corresponding to the instruction metadata from the lookup table. Host 240 outputs the instruction corresponding to the instruction metadata based on the retrieval result. Internal processor 220 may use the output instruction.

[0051] The lookup table 211 can be generated by the internal processor 220 or the host 240 by predicting future instructions based on a log of past instructions. Statistical analysis can be performed on the log of past instructions, and the lookup table 211 can be generated based on the analysis results in order of high instruction call frequency. The lookup table 211 may include instructions predetermined based on specifications set in the user application.

[0052] When an instruction corresponding to the instruction metadata is retrieved from lookup table 211, the host 240 can output the retrieved instruction. The internal processor 220 can use the output instruction. Since only the operation of searching and accessing lookup table 211 is performed, the processing of calculating the access address of the corresponding instruction based on the instruction metadata can be omitted, and the overhead of each execution calculation can be reduced.

[0053] When no instruction corresponding to the instruction metadata is found in lookup table 211, host 240 can generate an instruction corresponding to the instruction metadata and output the generated instruction. Internal processor 220 can use the output instruction.

[0054] When no instruction corresponding to the instruction metadata is found, the host 240 can generate an instruction corresponding to the instruction metadata and store the generated instruction in lookup table 211. The host 240 can also generate instructions that were not found and use the generated instructions to update lookup table 211. Subsequently, when the same instruction is invoked, the overhead can be reduced by searching lookup table 211.

[0055] In another example, PIM 200 may store instructions generated during runtime in lookup table 211, rather than preparing lookup table 211 in advance. For this purpose, PIM 200 includes memory 210, an internal processor 220, and registers 230. Registers 230 may store instruction metadata.

[0056] Host 240 (e.g., an external processor) generates instructions corresponding to the instruction metadata in response to a PIM API call instruction that includes instruction metadata. Host 240 generates lookup table 211 to store the generated instructions. Host 240 outputs the generated instructions.

[0057] The host 240 stores the generated instructions in lookup table 211. In response to a next PIM API call instruction, the host 240 retrieves the instruction corresponding to that next PIM API call instruction from lookup table 211. The host 240 can output the instruction corresponding to the next PIM API call instruction based on the retrieval result. When the same instruction is called, searching lookup table 211 can reduce overhead. The internal processor 220 can use the output instruction.

[0058] Figure 3 This shows an example of the overall operation of the instruction output method.

[0059] Reference Figure 3 In operation 301, the external processor generates a lookup table by predicting future instructions and stores the lookup table in memory. The internal or external processor can generate a lookup table by predicting future instructions based on a log of past instructions. The lookup table can store instructions predetermined based on specifications set in the user application.

[0060] In operation 303, instruction metadata is stored in a register. Instruction metadata may include the type of operation, the size of the operation, and hardware information. Instruction metadata may be pre-stored in a register or received externally.

[0061] In operation 305, the external processor responds to a PIM API call instruction that includes instruction metadata information by retrieving the instruction corresponding to the instruction metadata information from a lookup table.

[0062] In operation 307, the external processor outputs an instruction corresponding to the instruction metadata based on the retrieval results. When an instruction corresponding to the instruction metadata is retrieved, the external processor can output the retrieved instruction. When no instruction corresponding to the instruction metadata is retrieved, the external processor can generate an instruction corresponding to the instruction metadata and output the generated instruction. The internal processor can use the output instruction in operation 309.

[0063] When no instruction corresponding to the instruction metadata is found, the external processor can store the corresponding instruction in a lookup table. Subsequently, whenever the same operation is invoked, the overhead of generating instructions can be reduced.

[0064] Figure 4 This shows an example of a command output method.

[0065] In Operation 401, PIM can predict future instructions. Future instructions can be predicted by either an internal or external processor based on a log of past instructions.

[0066] In operation 402, PIM can generate a lookup table based on the prediction results. PIM can pre-generate the expected instructions based on the size of the operation, the type of the operation, and hardware information, and cache the generated instructions in the lookup table.

[0067] In Operation 403, user applications can invoke the PIM API. When a user application invokes an operation supported by PIM, PIM outputs the corresponding instruction. To do this, PIM first searches the lookup table and calculates the access information for the instruction based on the instruction metadata only when necessary.

[0068] In operation 404, PIM can retrieve the instruction from the lookup table. PIM can retrieve the instruction corresponding to the instruction metadata and output the memory address where the instruction is stored. Whether to execute operation 405 depends on the search result.

[0069] If no instruction is found, PIM can generate the instruction in operation 405. PIM can calculate the memory address where the instruction is stored based on the instruction metadata.

[0070] In operation 406, the PIM can output instructions. When an instruction is retrieved from the lookup table, the PIM can output the retrieved instruction. When no instruction is retrieved from the lookup table, the PIM can output instructions using instruction access information calculated by itself based on instruction metadata.

[0071] Figure 5 This shows an example of a command output method.

[0072] In Operation 501, PIM can predict future instructions. Future instructions can be predicted by either an internal or external processor based on a log of past instructions.

[0073] In operation 502, PIM can generate a lookup table based on the prediction results. PIM can pre-generate the expected instructions based on the size of the operation, the type of the operation, and hardware information, and cache the generated instructions in the lookup table.

[0074] In Operation 503, user applications can invoke the PIM API. When a user application invokes an operation supported by PIM, PIM outputs the corresponding instruction. To do this, PIM first searches the lookup table and calculates the access information for the instruction based on the instruction metadata only when necessary.

[0075] In operation 504, the PIM can retrieve the instruction from the lookup table. The PIM can retrieve the instruction corresponding to the instruction metadata and output the memory address where the instruction is stored. Whether operation 505 is executed depends on the search result.

[0076] If no instruction is found, PIM can generate the instruction in operation 505. PIM can calculate the memory address where the instruction is stored based on the instruction metadata.

[0077] In operation 506, the PIM can output instructions. When an instruction is retrieved from the lookup table, the PIM can output the retrieved instruction. When no instruction is retrieved from the lookup table, the PIM can output instructions using instruction access information calculated by itself from the instruction metadata.

[0078] In operation 507, PIM can update the lookup table based on the newly generated instruction. Subsequently, when the same instruction is invoked, the overhead can be reduced by searching the lookup table.

[0079] Figure 6 This shows an example of the overall operation of the instruction output method.

[0080] PIM can store instructions generated during runtime in a lookup table instead of preparing the lookup table in advance.

[0081] In Operation 601, PIM can store instruction metadata in registers. Instruction metadata may include the type of operation, the size of the operation, and hardware information. Instruction metadata may be pre-stored in registers or received externally.

[0082] In operation 603, PIM can generate instructions corresponding to the instruction metadata in response to a PIM API call instruction that includes instruction metadata. PIM can calculate access information for the instructions based on the instruction metadata.

[0083] In operation 605, PIM can store the generated instructions in a lookup table stored in memory. PIM can update the lookup table based on newly generated instructions.

[0084] In operation 607, PIM can output the generated instructions.

[0085] In Operation 609, PIM can use the generated instructions.

[0086] Subsequently, when the same instruction is invoked, PIM can search the lookup table, thereby reducing overhead. PIM can retrieve the corresponding instruction from the lookup table in response to the next PIM API call instruction. PIM can then output the instruction corresponding to the next PIM API call instruction based on the search results.

[0087] Figure 7 This shows an example of a command output method.

[0088] In Operation 703, user applications can call the PIM API. When a user application calls an operation supported by PIM, PIM outputs the corresponding instruction. To do this, PIM first searches the lookup table and calculates the access information for the instruction based on the instruction metadata only when necessary.

[0089] In operation 704, the PIM can retrieve instructions from the lookup table. The PIM can retrieve the instruction corresponding to the instruction metadata and output the memory address where the instruction is stored. Whether operation 705 is executed depends on the search result.

[0090] If no instruction is found, PIM can generate the instruction in operation 705. PIM can calculate the memory address where the instruction is stored based on the instruction metadata.

[0091] In operation 707, PIM can update the lookup table based on newly generated instructions. PIM can store the generated instructions in the lookup table.

[0092] In operation 706, the PIM can output instructions. When an instruction is retrieved from the lookup table, the PIM can output the retrieved instruction. When no instruction is retrieved from the lookup table, the PIM can output instructions using instruction access information calculated by itself based on instruction metadata.

[0093] The methods described in the examples above can be recorded in a non-transitory computer-readable medium, which includes program instructions for implementing the various operations described in the examples. The medium may also include data files, data structures, etc., alone or in combination with the program instructions. The program instructions recorded on the medium may be program instructions specifically designed and constructed for the purposes of the examples, or they may be of types known and available to those skilled in the art of computer software. Examples of non-transitory computer-readable media include: magnetic media (such as hard disks, floppy disks, and magnetic tapes); optical media (such as CD-ROMs, DVDs, and / or Blu-ray discs); magneto-optical media (such as optical discs); and hardware devices specifically configured to store and execute program instructions (such as read-only memory (ROM), random access memory (RAM), flash memory (e.g., USB flash drives, memory cards, memory sticks, etc.)). Examples of program instructions include files containing both machine code generated by a compiler and high-level code that can be executed by a computer using an interpreter. The aforementioned devices may be configured as one or more software modules to perform the operations described in the examples above, and vice versa.

[0094] Software may include computer programs, code, instructions, or combinations thereof, to independently or uniformly instruct or configure a processing device to operate as desired. Software and data may be permanently or temporarily implemented in any type of machine, component, physical or virtual device, computer storage medium, or apparatus, or in propagated signal waves capable of providing instructions or data to or being interpreted by the processing device. Software may also be distributed across networked computer systems, enabling it to be stored and executed in a distributed manner. Software and data may be stored on one or more non-transitory computer-readable recording media.

[0095] Several examples have been described above. However, it should be understood that various modifications can be made to these examples. For example, suitable results may be achieved if the described techniques are performed in a different order and / or if the components in the described system, architecture, apparatus, or circuit are combined in a different manner and / or replaced or supplemented by other components or their equivalents.

[0096] Therefore, other embodiments are within the scope of the appended claims.

Claims

1. A processor in a memory, comprising: Internal processor; Memory; as well as Registers are configured to store instruction metadata. The memory is configured to store lookup tables generated by predicting future instructions. The application programming interface call instructions are processed in response to memory containing instruction metadata. The instructions corresponding to the instruction metadata are retrieved from a lookup table by an external processor. Based on the retrieval results, the instruction corresponding to the instruction metadata is output by the external processor, and The internal processor is configured to use output instructions.

2. The processor in the memory according to claim 1, wherein, The lookup table is generated by an internal or external processor by predicting future instructions based on a log of past instructions.

3. The processor in the memory according to claim 1, wherein, If the instruction corresponding to the instruction metadata is retrieved, the retrieved instruction is output by the external processor.

4. The processor in the memory according to claim 1, wherein, If the instruction corresponding to the instruction metadata is not found, the instruction corresponding to the instruction metadata is generated and output by an external processor.

5. The processor in the memory according to any one of claims 1 to 4, wherein, If the instruction corresponding to the instruction metadata is not found, the instruction corresponding to the instruction metadata is generated by an external processor and stored in a lookup table by the external processor.

6. A processor in a memory, comprising: Internal processor; Memory; as well as Registers are configured to store instruction metadata. Specifically, the application programming interface call instructions are processed in memory containing instruction metadata, and the instructions corresponding to the instruction metadata are generated by an external processor. The generated instructions are stored in a lookup table by an external processor, and the generated instructions are output by the external processor. The internal processor is configured to use output instructions.

7. The processor in the memory according to claim 6, wherein, In response to an application programming interface (API) call instruction being processed in the next memory location, the corresponding instruction is retrieved from a lookup table by an external processor. Based on the search results, the instruction corresponding to the application programming interface call instruction in the next memory is output by the external processor.

8. A method for outputting instructions using a processor in memory, the processor in memory including an internal processor, memory, and registers, the method comprising: The lookup table is generated by predicting future instructions and then stored in memory. Store instruction metadata in registers; In response to processing application programming interface call instructions in memory including instruction metadata, the instruction corresponding to the instruction metadata is retrieved from the lookup table. Output the instruction corresponding to the instruction metadata based on the search results; and The instructions output by the internal processor.

9. The method according to claim 8, wherein, The steps for generating a lookup table include: predicting future instructions based on a log of past instructions.

10. The method according to claim 9, wherein, The step of generating the lookup table is performed by the internal processor.

11. The method according to claim 9, wherein, The step of generating the lookup table is performed by an external processor.

12. The method according to any one of claims 8 to 11, wherein, The steps for outputting instructions include: if an instruction corresponding to the instruction metadata is retrieved, outputting the retrieved instruction.

13. The method according to any one of claims 8 to 11, wherein, The steps for outputting instructions include: if no instruction corresponding to the instruction metadata is found, generating an instruction corresponding to the instruction metadata and outputting the generated instruction.

14. The method according to any one of claims 8 to 11, wherein, The steps for outputting instructions include: if no instruction corresponding to the instruction metadata is found, generating an instruction corresponding to the instruction metadata and storing the instruction corresponding to the instruction metadata in a lookup table.

15. The method according to claim 8, wherein, The steps of generating, storing, retrieving, and outputting are executed by an external processor.

16. A non-transitory computer-readable storage medium storing instructions, which, when executed by a processor, cause the processor to perform the method according to any one of claims 8 to 15.

Images