Memory device and method including a loop instruction memory queue

Abstract

A memory device includes a memory bank including one or more bank arrays, a PIM circuit configured to perform an operation logic processing operation, and an instruction memory including a first instruction queue segment to an mth instruction queue segment configured in a circular instruction queue to store instructions provided by a host, wherein the instructions stored in the first instruction queue segment to the mth instruction queue segment are executed in response to an operation request from the host, and each new instruction provided by the host is updated on the fully executed instructions in the circular instruction queue.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Provisional Application No. 62 / 816,509, filed March 11, 2019, with the United States Patent and Trademark Office and Korean Patent Application No. 10-2020-0013303, filed February 4, 2020, with the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] Exemplary embodiments of the present invention relate to memory devices, and more specifically, to memory devices and methods comprising a circular instruction memory queue. Background Technology

[0004] The capacity and speed of semiconductor memory devices used in high-performance electronic systems are increasing daily. Memory devices can be used to store various information items, such as data or operation results for various types of arithmetic logic processing operations (such as neural network operations). Furthermore, in order to efficiently perform the increasing number of operations, a method is envisioned in which at least some operations are performed by a processor-in-memory (PIM) circuitry within the memory device.

[0005] For neural network functions, multiple operations can be performed in a memory device, and instructions for performing these operations can be stored. As an increasing number of instructions are stored in the memory device, area efficiency may decrease, and the cost of protecting the memory device may increase. Summary of the Invention

[0006] An exemplary embodiment of the present invention provides a memory device capable of improving the efficiency of the storage space for instructions within the memory device to perform an increased number of operations, and a corresponding method for operating the memory device.

[0007] According to an exemplary embodiment of the present invention, a memory device includes: a memory bank, an array of one or more memory banks having memory cells; a processor-in-memory (PIM) circuitry configured to perform arithmetic logic processing operations using at least one of data provided by a host or data read from the memory bank; and an instruction memory including a circular instruction memory queue having a first instruction queue segment to an m-th instruction queue segment used in a loop, wherein m is an integer not less than 2, the instruction memory being configured to store instructions provided by a host. Instructions stored in the first instruction queue segment to the m-th instruction queue segment are executed in response to an operation request from the host, and when the instruction memory stores instructions based on a circular queue technique, new instructions provided by the host are updated on fully executed instructions at any segment of the circular queue.

[0008] According to an example embodiment of the inventive concept, a memory device includes a memory bank including one or more bank arrays each including memory cells; a processor-in-memory (PIM) circuit configured to perform an operation logic processing operation by using at least one of data provided by a host or data read from the memory bank; and an instruction memory including a first instruction queue segment to an m-th instruction queue segment, where m is an integer not less than 2, the instruction memory configured to store instructions provided by the host; an index manager configured to generate an index corresponding to the instructions stored in the instruction memory; and an index order storage circuit configured to store index order information representing an execution order of the instructions from the host. When an operation request is provided by the host, the instructions stored in the instruction memory are executed in an order according to the index order information.

[0009] According to an example embodiment of the inventive concept, a method of operating a memory device includes loading some of a plurality of instructions for a neural network function onto a plurality of instruction queue segments of an instruction memory; a PIM circuit performing an operation logic processing operation by sequentially executing the loaded instructions; writing unexecuted other instructions of the plurality of instructions for the neural network function onto the instruction queue segments on the previously executed instructions; and the PIM circuit performing the operation logic processing operation by sequentially executing the unexecuted instructions. BRIEF DESCRIPTION OF DRAWINGS

[0010] The example embodiments of the inventive concept will become more fully understood from the detailed description given herein below, and the accompanying drawings, wherein:

[0011] Figure 1 is a schematic block diagram illustrating a data processing system including a memory device according to an example embodiment of the inventive concept;

[0012] Figure 2 is a schematic block diagram illustrating an embodiment example of an instruction memory of Figure 1

[0013] Figure 3 is a schematic block diagram illustrating an embodiment example of a memory device of Figure 1

[0014] Figure 4 is a flowchart illustrating a method of operating a memory device according to an example embodiment of the inventive concept;

[0015] Figure 5 is a partially schematic conceptual diagram illustrating an example of operation of a memory system according to an example embodiment of the inventive concept;

[0016] Figure 6 and​​Figure 7 is a part-schematic conceptual diagram illustrating an example of an instruction update operation according to example embodiments of the inventive concept;

[0017] Figure 8 is a part-schematic conceptual diagram illustrating an example of a signal related to an instruction update operation according to example embodiments of the inventive concept;

[0018] Figure 9 is a schematic block diagram illustrating an example of an implementation of a memory device according to example embodiments of the inventive concept;

[0019] Figure 10 is a part-schematic conceptual diagram illustrating an example of an instruction being executed based on an index order according to example embodiments of the inventive concept;

[0020] Figure 11 is a part-schematic conceptual diagram illustrating an example of an instruction store example and an index allocation example according to example embodiments of the inventive concept;

[0021] Figure 12 is a schematic block diagram illustrating an example of an implementation of a memory device according to example embodiments of the inventive concept;

[0022] Figure 13 is a schematic block diagram illustrating a data processing system including a memory device according to example embodiments of the inventive concept;

[0023] Figure 14 is a part-schematic perspective view illustrating an example in which a memory device includes a High Bandwidth Memory (HBM) according to example embodiments of the inventive concept; and

[0024] Figure 15 is a schematic block diagram illustrating a server system according to example embodiments of the inventive concept. DETAILED DESCRIPTION

[0025] In the following, example embodiments of the inventive concept will be described in detail with reference to the accompanying drawings.

[0026] Figure 1 a data processing system including a memory device according to example embodiments of the inventive concept is illustrated.

[0027] Reference Figure 1The memory system 10 can include a memory controller 100 and a memory device 200. The memory controller 100 can include a memory interface 110 and can control memory operations, such as recording and reading, by providing various signals to the memory device 200 via the memory interface 110. For example, the memory controller 100 can access data DATA of the memory device 200 by providing a command CMD and an address ADD to the memory device 200.

[0028] The memory controller 100 can access the memory device 200 according to a request from a host HOST, and the memory interface 110 can interface with the memory device 200. The memory controller 100 can communicate with the host HOST by using various protocols. According to an exemplary embodiment, the memory controller 100 can correspond to or be provided in a component of the host HOST. The host HOST, the memory controller 100, and the memory device 200 can form a data processing system. Accordingly, the memory system 10 can correspond to the data processing system or one or more components included in the data processing system.

[0029] The memory device 200 can include a memory bank 210, an instruction memory 220, arithmetic logic 230, and control logic 240. The memory bank 210 can include a plurality of bank arrays, such as, for example, a first bank array BANK1 to an Nth bank array BANKN, where N is an integer not less than 2, and each of the first bank array BANK1 to the Nth bank array BANKN can include a memory cell array including a plurality of memory cells. Further, the arithmetic logic 230 can include one or more processor-in-memory (PIM) circuits. In Figure 1 In the middle, an example is shown in which the arithmetic logic 230 includes a plurality of PIM circuits, such as, for example, a first PIM circuit 231_1 to an Nth PIM circuit 231_N, to correspond to the N bank arrays of the memory bank 210. However, the arithmetic logic 230 can include various numbers of PIM circuits. For example, when a plurality of PIM circuits are arranged to correspond to one bank or less than N bank arrays, the number of PIM circuits can be greater than the number of bank arrays.

[0030] When a PIM circuit is shared by two or more memory bank arrays, the number of PIM circuits can be less than the number of memory bank arrays. Other embodiments are also possible, such as one PIM circuit for the first two of every three memory bank arrays, and two PIM circuits for the third memory bank array. In addition, each of the PIM circuits can perform an operation by using at least one of data from the host HOST or information read from the memory bank 210. In the case of at least two PIM circuits per bank, one of the two PIM circuits can perform an operation by using data from the host HOST, and the other of the two PIM circuits can perform an operation by using information read from the memory bank 210.

[0031] The memory bank in which data access is to be performed can be selected by the address ADD from the memory controller 100, and a memory cell within the memory bank can be selected. In addition, the control logic 240 can include a command and address decoder 241, and the command and address decoder 241 can perform a decoding operation on the command CMD and / or the address ADD from the memory controller 100. The control logic 240 can perform an internal control operation on the memory device 200, so that a memory bank operation or another operation can be performed according to a decoding result. For example, the control logic 240 can output an internal command for performing an internal control operation.

[0032] The memory device 200 can include a dynamic random-access memory (DRAM), such as a double data rate synchronous dynamic random-access memory (DDR SDRAM), a low power double data rate (LPDDR) SDRAM, a graphics double data rate (GDDR) SDRAM, or a Rambus dynamic random-access memory (RDRAM). However, embodiments of the inventive concept are not limited thereto. For example, the memory device 200 can be implemented as a non-volatile memory, such as a flash memory, a magnetic RAM (MRAM), a ferroelectric RAM (FeRAM), a phase change RAM (PRAM), or a resistive RAM (ReRAM).

[0033] Furthermore, the memory device 200 may correspond to a semiconductor chip, or it may be a component corresponding to a channel in a memory device comprising multiple channels having independent interfaces. Alternatively, the memory device 200 may be a component corresponding to a memory module. Alternatively, a memory module may include multiple memory chips, and Figure 1 The memory device 200 can correspond to a memory chip mounted on a module board.

[0034] In the following, an example of storing instructions in instruction memory 220 and performing instruction processing operations therein, according to exemplary embodiments of the present invention, is described. Various types of instruction processing operations can be performed by memory device 200. For example, in relation to Artificial Intelligence (AI), at least some of the multiple operations for neural network functions can be performed by memory device 200. For example, a host can control memory device 200 via memory controller 100, such that at least some of the multiple operations can be performed by memory device 200. Furthermore, in the embodiments described below, memory controller 100 is shown as controlling memory device 200. However, embodiments of the present invention are not limited thereto. For example, memory controller 100 may correspond to a component provided in a host, wherein the host controls memory device 200.

[0035] The instruction memory 220 can store multiple instructions based on circular queue technology and can include multiple instruction queue segments. An instruction queue can include one or more storage circuits for storing information. According to an exemplary embodiment, an instruction queue segment can be defined as a unit for storing one instruction.

[0036] The memory controller 100 can send instructions (Inst) to the memory device 200 to perform arithmetic logic processing operations. For example, the memory controller 100 can send multiple instructions (Inst) to the memory device 200, allowing multiple arithmetic logic processing operations to be executed serially. For example, before executing the arithmetic logic processing operations, a mode for loading multiple instructions (Inst) into the instruction memory 220 can be executed, such as an instruction load mode, where multiple instructions (Inst) can be loaded into the circular instruction queue of the instruction memory 220.

[0037] exist Figure 1 The diagram shows that the instruction Inst and data DATA are transmitted via a separate transmission bus. However, embodiments of the inventive concept can be implemented differently. For example, the instruction Inst can be transmitted via the same transmission bus as the data DATA.

[0038] All instructions for a neural network function corresponding to all operations performed by the memory device 200 can be loaded onto the instruction memory 220 and executed. The instruction memory 220 can include a storage space having a size smaller than a size for storing all instructions. For example, some of all instructions can be stored in an instruction queue section of the instruction memory 220 from which stored instructions can be sequentially executed, and some instruction queue sections can store instructions that have been completely executed and are no longer needed. According to an exemplary embodiment of the inventive concept, other ones of all instructions can be sequentially updated on the instruction queue sections containing completely executed instructions.

[0039] Each of the first PIM circuit 231_1 to the N-th PIM circuit 231_N can include a processing element PE, and the processing element PE can perform an operation logic processing operation by using at least one of information read from a corresponding bank and / or data DATA provided by the memory controller 100. In Figure 1 In the drawings, one processing element PE is shown with respect to each of the PIM circuits. However, each of the PIM circuits can include a plurality of processing elements PE. The processing element PE can include various kinds of operators, for example, Single Instruction Multi-Data (SIMD) and Arithmetical and Logical Unit (ALU). Further, each of the first PIM circuit 231_1 to the N-th PIM circuit 231_N can include various components related to operation processing. For example, each of the first PIM circuit 231_1 to the N-th PIM circuit 231_N can further include a storage circuit such as a register for temporarily storing data for an operation logic processing operation or temporarily storing an operation processing result.

[0040] The memory device 200 can selectively perform a memory storage operation or an operation logic processing operation in response to a command CMD and an address ADD from the memory controller 100. For example, an additional command CMD for requesting the execution of an operation logic processing operation can be defined. In this case, when the command CMD defined for requesting the execution of an operation logic processing operation is received, an instruction is executed so that the operation logic processing operation can be performed. Alternatively, in an embodiment, the operation logic processing operation can be performed in response to a prescribed normal command CMD such as a data record / read. As an operation example, the value of the address ADD from the memory controller 100 can belong to a range for pointing to a region of a substantial memory cell array or another range, and an instruction can be executed based on the result of decoding the command CMD and / or the address ADD from the memory controller 100 so that the operation logic processing operation can be performed. In the following embodiments, it is assumed that the operation of executing an instruction and updating a new instruction is performed in response to a normal command CMD.

[0041] According to an exemplary embodiment, the memory controller 100 can provide a command CMD and an address ADD for storing or loading an instruction, and can provide a queue address ADD Q for pointing to a location of the instruction memory 220 with respect to an instruction load and update operation as the address ADD to the memory device 200. Further, the queue address ADD Q from the memory controller 100 can point to the instruction queue in which a completely executed instruction is stored when an instruction is updated to the instruction queue by a circular queue technique. The above-described instruction update operation can be performed together with an operation logic processing operation by a run-time technique so that, in response to receiving a command CMD and / or an address ADD for requesting the execution of an operation logic processing operation, an operation processing and an instruction update operation can be performed. Alternatively, according to various embodiments, the instruction update operation and the operation logic processing operation can be performed at different timings by separate commands.

[0042] According to the above-described embodiments of the inventive concept, since a storage space for storing an instruction for performing a large number of operation logic processing operations can be reduced, the efficiency of a region can be improved, and the expense for implementing a memory for storing an instruction can be reduced. Further, since the capacity of the instruction memory 220 can be reduced, a high-performance memory having a high access speed can be used. For example, a storage circuit such as a flip-flop or a latch having a higher access speed than a static random access memory (SRAM) can be used for the instruction memory 220. , FF ) or a latch having a higher access speed than a static random access memory (SRAM) can be used for the instruction memory 220.

[0043] In Figure 1In the example shown, the memory bank array, the instruction memory 220, and the arithmetic logic 230 are shown as separate components. However, embodiments of the inventive concept are not limited thereto. For example, each of the memory bank arrays can be defined to include various components as well as the array of memory cells. For example, the instruction queue can be defined to be included in each of the memory bank arrays, and at least some of the components included in the PIM circuit can be defined to be included in the memory bank arrays.

[0044] The memory system 10 or a data processing system including the memory system 10 can be implemented by a personal computer (PC), a data server, a cloud system, an artificial intelligence server, a network-attached storage (NAS), an Internet of Things (IoT) device, or a portable electronic device. Also, when the data processing system is the portable electronic device, the data processing system can be a laptop computer, a mobile phone, a smart phone, a tablet PC, a personal digital assistant (PDA), an enterprise digital assistant (EDA), a digital still camera, a digital camcorder, an audio device, a portable multimedia player (PMP), a personal navigation device (PND), an MP3 player, a handheld game console, an e-book, or a wearable device.

[0045] Figure 2 An embodiment example of the instruction memory is shown in Figure 1

[0046] Referring to Figure 1 and Figure 2 The instruction memory 220 can include instruction queue segments (such as, for example, a first instruction queue segment IQ 0 through an nth instruction queue segment IQ n-1) for storing a plurality of instructions by a circular queue technique. Each of the first instruction queue segment IQ 0 through the nth instruction queue segment IQ n-1 can include a storage circuit. In the example shown, the first instruction queue segment IQ 0 through the nth instruction queue segment IQ n-1 are shown as separate components. However, embodiments of the inventive concept are not limited thereto. For example, each of the first instruction queue segment IQ 0 through the nth instruction queue segment IQ n-1 can be defined to include various components as well as the storage circuit. For example, at least some of the components included in the PIM circuit can be defined to be included in the first instruction queue segment IQ 0 through the nth instruction queue segment IQ n-1. Figure 2 ​In some embodiments, an example in which latches or flip-flops are used is shown. However, embodiments of the inventive concept are not limited thereto, and various kinds of other storage circuits can be applied. Also, if any latches or flip-flops are used, one such latch or flip-flop can be used only on the first segment, any segment, multiple segments, or all segments. If there are multiple circular queues, a latch or flip-flop can be used only on one circular queue, several (e.g., every odd-numbered) circular queues, or all circular queues. In the case of multiple circular queues, one circular queue can correspond to each PIM, or one circular queue can correspond to every multiple of PIMs, such as, for example, one circular queue for every pair of PIMs. Alternatively, each PIM can have multiple circular queues. In yet another embodiment, some of the multiple circular queues can be dynamically reassigned to other PIMs as needed to maximize speed or efficiency, or to improve, for example, local temperature variations within the die.

[0047] As an example of operation, some of the multiple instructions for a neural network function can be stored in the first instruction queue segment IQ 0 through the nth instruction queue segment IQ n-1. For example, in an instruction load mode, the first instruction Inst 0 through the nth instruction Inst n-1 can be loaded onto the instruction memory 220. Then, in response to a command CMD and / or address ADD from the memory controller 100, the first instruction Inst 0 through the nth instruction Inst n-1 stored in the instruction memory 220 can be sequentially executed. In addition, other instructions of the multiple instructions can be updated and stored in the instruction queue in which the fully executed instructions are stored.

[0048] The instruction update operation can be performed at various timings. The operation of updating a new instruction to the instruction queue can also be performed when the operation logic processing operation is performed. For example, in the process of sequentially executing the first instruction Inst 0 through the nth instruction Inst n-1, the instruction update operation can start at a certain timing, and one instruction is executed, and one new instruction can be updated to the instruction queue. For example, when the next second instruction Inst 1 is executed after the first instruction Inst 0 is executed, the instruction update operation can start, and the (n+1)th instruction can be updated to the fully executed first instruction queue segment IQ 0. Also, when the third instruction Inst 2 through the nth instruction Inst n-1 are sequentially executed, new instructions can be sequentially updated to the second instruction queue segment IQ 1 through the (n-1)th instruction queue segment IQ n-2. Through the above circular queue method, all instructions included in the neural network function can be loaded onto the instruction memory 220 and executed.

[0049] According to an exemplary embodiment, the instruction update operation can start after a plurality of instructions are executed. As an operation example, when the instructions stored in the kth instruction queue (where k is an integer smaller than n) are executed after and the next instruction is being executed, the instruction update operation can start from the first instruction queue segment IQ 0. Alternatively, when the instructions of the last instruction queue of the instruction memory 220, such as, for example, the nth instruction queue segment IQ n-1, are executed, the instruction update operation can be performed. For example, when the instructions of the nth instruction queue segment IQ n-1 are executed, the instruction update operation can start from the first instruction queue segment IQ 0.

[0050] According to some embodiments, the instruction update operation can be performed at a timing different from the timing of executing the instructions. As an operation example, after the instructions of some of the first instruction queue segment IQ 0 to the nth instruction queue segment IQ n-1 are executed, based on the control from the memory controller 100, the operation of updating new instructions to at least some of the fully executed instruction queue segments can be performed. Then, the instructions of the other instruction queue segments of the first instruction queue segment IQ 0 to the nth instruction queue segment IQ n-1 are executed. Then, the operation of updating at least some of the fully executed instruction queue segments can be performed again. Through the above process, the operation of updating new instructions and the operation of executing instructions can be alternately or simultaneously performed. Thus, a plurality of instructions larger in size than the instruction memory can be stored in the instruction memory and executed.

[0051] In the above-described embodiments, the fact that the operation logic processing operation and the instruction update operation are performed together does not necessarily mean that the above operations are performed simultaneously. As an operation example, the memory device 200 can sequentially perform the operation logic processing operation and the instruction update operation, or can perform the operation logic processing operation and the instruction update operation in parallel in response to the command CMD and / or the address ADD from the memory controller 100. Alternatively, whenever the operation logic processing operation is performed, for the instruction update operation, at least one of the command CMD and the address ADD can be further provided to the memory device 200, or the instruction update operation can be performed in response to a separate command CMD and / or address ADD.

[0052] Figure 3 An embodiment example of the memory device 200 is shown Figure 1

[0053] Referring to Figures 1 to 3 , the memory device 200 can further include, in addition to Figure 1 ​The memory device 200 can further include a processing controller 232 and a program counter 233, and the processing controller 232 can control the overall operation related to the operation processing in the memory device 200. For example, the processing controller 232 can perform an operation of reading and decoding instructions stored in the instruction memory 220, and can control the first PIM circuit 231_1 to the Nth PIM circuit 231_N so that the operation logic processing operation corresponding to the read instruction can be performed.

[0054] The program counter 233 can point to an instruction to be read from the instruction memory 220 by performing a program count operation. For example, in response to receiving a command CMD and / or an address ADD for operation processing from the memory controller 100, an instruction in a position corresponding to a current count value of the program counter 233 is read, and the count value can be increased based on a count operation. Alternatively, according to some embodiments, in response to the command CMD and / or the address ADD for operation processing, the current count value of the program counter 233 is increased, and an instruction can be read from a position corresponding to the increased count value. Further, specifically, in the case where the host and / or the operation logic does not require one or more instructions to be executed in order, the instructions from a non-sequential section within the circular queue can be effectively executed out of order.

[0055] Figure 3 The processing controller 232 and the program counter 233 illustrated in FIG. 2 can be implemented in various forms. For example, the processing controller 232 and the program counter 233 can be provided in the operation logic 230. Further, in the case where the memory device 200 is implemented as a PIM device, the processing controller 232 and the program counter 233 can be provided in the PIM circuit. Figure 3 In FIG. 2, the instruction memory 220, the processing controller 232, and the program counter 233 are illustrated as being collectively arranged in the first PIM circuit 231_1 to the Nth PIM circuit 231_N. However, the memory device 200 can be implemented such that the above components can be separately arranged in the PIM circuits, respectively.

[0056] The instruction load operation and the instruction update operation can be performed based on the control of the control logic 240. For example, the control logic 240 can receive various information from the memory controller 100 through one or more buses. For example, the control logic 240 can receive instructions through a data bus DQ BUS, and can receive a command and / or an address through a command address bus CA BUS. The control logic 240 can control an operation of storing the instructions received through the data bus DQ BUS in the instruction memory 220 in the instruction load mode. Further, according to an exemplary embodiment of the present inventive concept, the instruction update operation can be performed by rewriting the instructions in the instruction memory 220 through a circular queue technique.

[0057] As an operation example, when an instruction update operation is performed together with an operation logic processing operation, the operation logic processing operation can be performed based on an instruction pointed by a program counter 233 by control of a processing controller 232. Also, a control logic 240 can receive an address (such as, for example, a queue address ADD_Q) indicating a position of an instruction queue segment to be updated and an instruction from a memory controller 100, and can update the instruction to an instruction memory 220 based on the received address by a circular queue technique.

[0058] Figure 4 A method of operating a memory device according to an exemplary embodiment of the inventive concept is illustrated.

[0059] Reference Figure 4 The memory device can include an instruction memory for storing instructions for operation processing, and the instruction memory can include a plurality of instruction queue segments in a circular queue technique. Also, a plurality of instructions for a neural network function are executed by the memory device such that an operation logic processing operation can be performed, and when one instruction is stored in one instruction queue, since the number of instruction queue segments of the instruction memory is less than the number of all instructions for the neural network function, some of the plurality of instructions can be loaded on the instruction queue in operation S11.

[0060] Then, the memory device can perform an operation logic processing operation based on control from the memory controller. For example, in operation S12, the memory device can sequentially execute the instructions loaded on the instruction memory based on a program count operation. After executing the instructions loaded on the instruction memory, the executed instructions do not need to be further used.

[0061] The memory device can perform an instruction update operation together with the operation logic processing operation or at a timing different from the timing at which the operation logic processing operation is performed. For example, in operation S13, the memory device can rewrite the executed instructions with new instructions (such as, for example, other instructions among all the above-described instructions) on the previously executed instructions in the circular queue based on control from the memory controller. Based on the number of all instructions related to the neural network function and the number of instruction queue segments provided in the instruction memory, the update operation can be performed on one instruction queue one or more times.

[0062] When the instructions are updated as described above, after all the initially loaded instructions are executed, in operation S14, the unexecuted or updated instructions can be executed from the circular instruction queue, and all the instructions related to the neural network function can be executed.

[0063] Figure 5 An operation example of a memory system according to an exemplary embodiment of the inventive concept is illustrated. In Figure 5In the middle, an example is shown in which the memory device communicates with a host HOST. The host HOST can include a memory controller or a memory control module that communicates with the memory device.

[0064] The memory device can operate in a plurality of modes with respect to the operation processing. Referring to Figure 5 In the instruction load mode, the host HOST can send a plurality of instructions required for the operation processing to the memory device, and the sent instructions can be stored in the instruction queue segment of the circular instruction queue in the memory device. For example, according to the above-described embodiment, only some of the plurality of instructions related to the neural network function can be initially stored in the instruction queue segment of the circular instruction queue.

[0065] The memory device can then operate in an instruction execution mode in which the host HOST can send operation requests, such as, for example, commands and / or addresses directed to the operation processing, to the memory device, and the memory device can sequentially execute the instructions loaded in the instruction load mode in response to the operation requests based on the program count. For example, various kinds of arithmetic logic processing operations such as addition, subtraction, and multiplication can be performed by the memory device, and by decoding the instructions, the arithmetic logic processing operations corresponding to the instructions can be performed. Further, according to the exemplary embodiment, operation results can be provided from the memory device to the host HOST. Further, the processed data can be stored in the memory bank.

[0066] The memory device can then operate in an instruction execution and update mode in which the host HOST can send new instructions to be updated to the memory device along with the operation requests, and the memory device can perform the arithmetic logic processing operations by executing the instructions corresponding to the operation requests. Further, the memory device can update the new instructions from the host HOST to the previously executed instruction queue segment. For example, one new instruction can be updated to the segment of the executed instruction each time one instruction is executed.

[0067] By the above-described circular queue technique, all the instructions included in the neural network function can be stored in and updated to the instruction memory. Thus, after all the instructions are stored in the instruction memory, the memory device can operate in the instruction execution mode. In the instruction execution mode, the host HOST can send operation requests to the memory device, and the memory device can sequentially execute the instructions stored in the instruction queue segment, and can send operation results to the host HOST and / or store them in the memory bank.

[0068] Although not shown in the above-described exemplary embodiment, the memory device can operate in an instruction load and update mode in which the host HOST can send a plurality of instructions to the memory device, and the memory device can sequentially execute the instructions and update the new instructions to the instruction queue segment of the circular instruction queue. Figure 5In particular, the host HOST can provide data to the memory device and the memory device can perform the arithmetic logic processing operation by using at least one of the data from the host HOST and the information stored in the memory bank when the arithmetic logic processing operation is performed.

[0069] Figure 6 and Figure 7 An exemplary instruction update operation according to an exemplary embodiment of the inventive concept is illustrated.

[0070] Referring to Figure 6 , the instruction memory can include first instruction queue segments IQ 0 through nth instruction queue segments IQ n-1, and in the instruction load mode, first instructions Inst 0 through nth instructions Inst n-1 can be stored in the first instruction queue segments IQ 0 through the nth instruction queue segments IQ n-1. Then, when the arithmetic logic processing operation is performed, the first instructions Inst 0 through the nth instructions Inst n-1 can be sequentially executed.

[0071] According to an exemplary embodiment, the instructions can be sequentially executed in the instruction execution mode, and an operation of updating new instructions can be performed at a prescribed timing. In Figure 6 , an example is illustrated in which the instruction update operation can start from the first instruction queue segment IQ 0 when the kth instruction Inst k-1 is executed, and the (n+1)th instruction Inst n is updated to the first instruction queue segment IQ 0. Then, when the (k+1)th instruction Inst k is executed, the (n+2)th instruction Inst n+1 can be updated to the second instruction queue segment IQ 1, and additional new instructions can be sequentially updated.

[0072] The instruction update operation illustrated in Figure 6 may be performed by the instruction memory based on a similar method as Figure 7 illustrated.

[0073] Referring to Figure 7 A, an example is illustrated in which, after the first instruction queue segments IQ 0 through the nth instruction queue segments IQ n-1 are loaded, the update operation is performed every time the instructions are executed. For example, after the instructions stored in the first instruction queue segment IQ 0 are executed, the operation of updating the first instruction queue segment IQ 0 is performed when the instructions stored in the second instruction queue segment IQ 1 are being executed, and the operation of updating the second instruction queue segment IQ 1 can be performed when the instructions stored in the third instruction queue segment IQ 2 are being executed. Further, before all the instructions related to the neural network function are updated by the above-described method, the executed instructions can be overwritten with new or unexecuted instructions.

[0074] ReferringFigure 7 B, the instruction update operation can start after a certain number of instructions are executed after the first instruction queue segment IQ 0 to the nth instruction queue segment IQ n-1 are completely loaded. For example, the update operation can start from the first instruction queue segment IQ 0 when the instructions stored in the fifth instruction queue segment IQ 4 are executed after the instructions stored in the first instruction queue IQ 0 to the fourth instruction queue IQ 3 are executed. Then, the instruction update operation can be sequentially performed every time the operation logic processing operation is sequentially performed, and new instructions can be rewritten before all instructions are updated.

[0075] Figure 7 C shows an example in which the operation logic processing operation and the instruction update operation are performed at different non-simultaneous timings.

[0076] As an operation example, the instruction update operation can start after a certain number of instructions are executed after the first instruction queue segment IQ 0 to the nth instruction queue segment IQ n-1 are completely loaded. For example, the update operation can start from the first instruction queue segment IQ 0 when the instructions stored in the fifth instruction queue segment IQ 4 are executed after the instructions stored in the first instruction queue IQ 0 to the fourth instruction queue IQ 3 are executed. Then, the instruction update operation can be sequentially performed every time the operation logic processing operation is sequentially performed, and new instructions can be rewritten before all instructions are updated. Figure 7 In C, an example is shown in which the instruction update operation starts after the instructions stored in the first instruction queue segment IQ 0 to the eighth instruction queue segment IQ 7 are executed, and a certain number (for example, four) of instructions are updated at a timing different from the timing at which the operation logic processing operation is performed. Then, the operation of executing and updating a prescribed number (for example, four) of instructions can be repeated until all instructions are loaded. However, embodiments of the present inventive concept are not limited thereto. The instruction update operation can start at various timings, and the operation of executing and updating instructions can be performed in various numbers of instructions. Alternatively, the instruction update operation can start at a dynamically variable timing, and the operation of executing and updating instructions can be performed in a dynamically variable number of instructions.

[0077] Figure 8 An example of a signal related to the instruction update operation according to an exemplary embodiment of the present inventive concept is shown. Figure 8 The address ADD shown in C can correspond to the queue segment address mentioned in the above-described embodiments. For convenience of description, the instructions stored in the instruction memory are denoted by numbers 1 to 14. The number 1 stored in the instruction memory can correspond to the first instruction Inst 0, and the number 8 can correspond to the eighth instruction Inst 7. Further, in Figure 8 In C, the instructions marked with solid lines in the instruction memory represent executed instructions, and the instructions marked with dotted lines represent newly updated instructions.

[0078] Referring to Figure 8, the memory device can generate an internal command Int.CMD by receiving a command CMD and / or an address ADD from the host HOST and decoding the received command CMD and / or address ADD. According to the generated internal command Int.CMD, an internal operation of the memory device can be controlled. For example, an operation logic processing operation PE and an update operation U can be performed. Further, in Figure 8 , it is assumed that a command for a normal recording operation W is used as the command CMD related to the operation processing, and the operation of the eight instruction queue segments IQ 0 to IQ 7 is initially shown.

[0079] The operation logic processing operation can be performed only according to the result of decoding the command CMD and / or the address ADD. Although not specifically shown in Figure 8 , when an operation request is defined based on a combination of the command CMD and the address ADD, the address ADD directed to the operation processing can be received by the memory device. For example, an address ADD for a prescribed range directed to the operation processing is defined, and the address ADD defining the range can be received in order to direct the operation processing.

[0080] When the instruction 4 stored in the fourth instruction queue segment IQ 3 is executed, the instruction update operation U&PE of the instruction 9 from the data bus DQ can be performed, and the memory device can perform the instruction update operation while the operation logic processing operation is performed based on the instruction stored in the fourth instruction queue segment IQ 3 according to the internal command Int.CMD. For example, an address directed to the first instruction queue segment IQ 0 to which the instruction is to be updated can be received from the host HOST. Accordingly, the ninth instruction can be updated to the first instruction queue.

[0081] According to exemplary embodiments, during the partial operation processing, the instruction update operation can not be immediately performed. In Figure 8 , an example is shown in which when the instruction stored in the fifth instruction queue segment IQ 4 is executed, the instruction update operation is not performed with respect to IQ 1. Then, when the command CMD and / or the address ADD to perform the operation logic processing operation and the instruction update operation are received, the instruction update operation can be sequentially performed with respect to the second instruction queue segment IQ 1 to the sixth instruction queue segment IQ 5.

[0082] Figure 9 An embodiment example of the memory device 300 according to exemplary embodiments of the inventive concept is shown. In Figure 9 , an example is shown in which with respect to the instruction loading and execution operation, an index corresponding to the instruction is generated, and the instruction is executed based on the index.

[0083] Reference is made to Figure 9The memory device 300 can include a control logic 310, a processing controller 320, an instruction memory 330, a program counter 340, a PIM circuit 350, and an index sequential storage circuit 360. For ease of description, although not specifically shown in Figure 9 the memory device 300 can further include memory banks having a plurality of bank arrays. Also, in Figure 9 the memory device 300, only one PIM circuit 350 is shown. However, a plurality of PIM circuits can be provided in the memory device 300 to correspond to the plurality of bank arrays.

[0084] According to the above-described embodiment, the instruction memory 330 can include a plurality of instruction queue segments for storing instructions based on a circular queue technique. Also, the instruction memory 330 can include a plurality of rows and a plurality of columns, and can be pointed to instruction storage locations by a queue address including a row address RA and a column address CA. For example, when it is assumed that each of the instruction queue segments of the instruction memory 330 stores one instruction, one row of the instruction memory 330 can include a plurality of instruction queue segments.

[0085] The memory device 300 can receive commands and addresses from the host HOST through a command address bus CA BUS, and can receive a queue address regarding an instruction load or update operation. Also, the memory device 300 can receive an instruction Inst from the host HOST. For example, the memory device 300 can receive the instruction Inst through a data bus DQ BUS. In addition, in the instruction execution and update mode, based on the control of the processing controller 320, the instruction can be read from the instruction memory 330 and executed, and based on the control of the control logic 310, the instruction Inst from the host HOST can be updated to the instruction memory 330.

[0086] The control logic 310 can include an index manager 311, and the index manager 311 can generate an index Index corresponding to the instruction Inst. According to an exemplary embodiment, the index Index can be assigned to an entry of an instruction queue segment of the instruction memory 330 in which the instruction is stored, and an operation of pointing to the executed instruction can be performed based on the index Index. For example, the index manager 311 can calculate the index Index corresponding to the instruction by using at least some of the bits included in the row address RA and the column address CA, and when storing the instruction in the instruction memory 330, the corresponding index Index can be stored together with the instruction. Alternatively, according to the embodiment shown in Figure 9 the index memory 331 is further provided in the memory device 300, and the index Index can be stored in the index memory 331.

[0087] The index order storage circuit 360 can store index order information provided from the host HOST. For example, after a plurality of instructions are loaded into the instruction memory 330 and the instruction loading operation is completed, in order to set an execution order of the instructions loaded on the instruction memory 330, the index order information can be provided from the host HOST to the memory device 300.

[0088] Then, when the arithmetic logic processing operation is performed, the memory device 300 can read and execute the instructions in the index order stored in the index order storage circuit 360. For example, every time the arithmetic logic processing operation is performed, the index Index stored in the index order storage circuit 360 is read, and the control logic 310 can provide the read index Index to the processing controller 320. The index Index corresponding to a plurality of instructions to be processed in series can be included in the index order information.

[0089] According to an exemplary embodiment, the index Index can be set to have various values. For example, in order to correspond to m instructions stored in m instruction queue segments included in the instruction memory 330, an index Index having one or more bits is generated, and indexes Index having different bit values can be stored in the instruction memory 330 to correspond to the instructions. The processing controller 320 can compare the index Index read from the index order storage circuit 360 with the indexes Index stored in the instruction memory 330, can read the instructions corresponding to the received index Index, and can control the arithmetic logic processing operation.

[0090] According to the above-described embodiment, a plurality of instructions for pointing to the same kind of operation do not need to be repeatedly stored in the instruction memory 330. For example, a plurality of indexes can be used for a single stored instruction within a loop queue to repeatedly execute the instruction before being overwritten. Accordingly, the size of the instruction memory 330 can be reduced. Further, without executing the instructions in the order stored in the instruction memory 330, based on the information stored in the index order storage circuit 360, the instructions stored at an arbitrary location can be executed as needed.

[0091] In the arithmetic logic processing operation according to the above-described embodiment, the access operation of the instruction memory 330 based on the counting operation of the program counter 340 can be selectively performed. For example, some of a plurality of operations of a neural network function can be performed based on the counting operation of the program counter 340, and other operations can be performed based on the information stored in the index order storage circuit 360. Alternatively, according to some embodiments, the program counter 340 can be deactivated when the instructions are executed based on the information stored in the index order storage circuit 360. Alternatively, the program counter 340 need not be provided in the memory device 300.

[0092] The instruction update operation according to the above-described embodiments can be applied to the embodiments shown in FIGS. 1 to 4. Figure 9 As an example of the operation, the instructions stored in the instruction memory 330 can be executed one or more times. After the instructions are completely executed a prescribed number of times or indexes, new instructions can be updated to the instruction queue in which the completely executed instructions have been stored. For example, the instructions of the instruction queue segments included in the instruction memory 330 can be sequentially executed, and some of the instructions can be executed multiple times according to the index order information. In this case, according to the above-described embodiments, the instruction update operation can be substantially sequentially performed on the plurality of instruction queue segments.

[0093] Figure 10 An example in which instructions are executed based on index orders according to exemplary embodiments of the inventive concept is shown. In Figure 10 for convenience of description, the first instructions Inst 0 to the fourth instructions Inst 3 stored in the first instruction queue segment to the fourth instruction queue segment are indicated by the numbers 0 to 3, and an example in which the first index 0 to the fourth index 3 are assigned to correspond to the first instructions Inst 0 to the fourth instructions Inst 3 is shown.

[0094] Referring to Figure 9 and Figure 10 , the first instructions Inst 0 to the fourth instructions Inst 3 can be loaded on the instruction memory, and the first index 0 to the fourth index 3 corresponding to the first instructions Inst 0 to the fourth instructions Inst 3 can be stored in the instruction memory. In addition, the first instructions Inst 0 to the fourth instructions Inst 3 can perform different functions. For example, the first function Func 0 to the fourth function Func 3 performed by the first instructions Inst 0 to the fourth instructions Inst 3 can correspond to different kinds of operations.

[0095] The memory controller or the host can provide the memory device with index order information related to instruction execution, and the index order information can be stored in an index order storage circuit in the memory device. The index order information can indicate the order of instructions to be executed for operation processing, and the same instruction (such as, for example, an instruction stored in one instruction queue segment) can be repeatedly executed multiple times when the same index is repeatedly stored. In Figure 10 , an example in which the index order is set to "0, 1, 1, 1, 1, 2, 3" is shown.

[0096] The memory device can sequentially perform a plurality of operation logic processing operations in response to commands and / or addresses from a memory controller, and each time an operation logic processing operation is performed, an index stored in an index sequential storage circuit can be referenced. For example, a first instruction Inst 0 stored in a first instruction queue can be executed according to a first index 0, and then a second instruction Inst 1 stored in a second instruction queue can be repeatedly executed four times according to a second index 1. Then, a third instruction Inst 2 stored in a third instruction queue and a fourth instruction Inst 3 stored in a fourth instruction queue can be sequentially executed according to a third index 2 and a fourth index 3.

[0097] According to the above-described embodiments, without repeatedly storing the same instruction for performing the same kind of operation in a plurality of instruction queue segments, instructions stored in instruction queues pointed by indexes previously stored in the memory device can be read and repeatedly executed. Accordingly, repeated operations can be performed with low memory usage.

[0098] Figure 11 An instruction storage example and an index allocation example according to an example embodiment of the inventive concept are illustrated.

[0099] Reference Figure 11 An example instruction memory can include a plurality of instruction queue segments, and the plurality of instruction queue segments can be addressed by row addresses RA of m rows and column addresses CA of four columns. For example, the plurality of instruction queue segments can be included in one row. In Figure 11 In the above-described embodiments, an example is illustrated in which when four instructions are stored in one row, 4m instructions Inst 0 to Inst 4m-1 are stored in a circular queue of the instruction memory including m rows and four columns, as illustrated. In alternative embodiments, the rows can each define one of a plurality of circular queues, the columns can each define one of a plurality of circular queues, or any combination thereof. For example, each circular queue can correspond to each PIM, or one circular queue corresponds to a plurality of PIMs or a plurality of circular queues correspond to one PIM.

[0100] According to the above-described embodiments, based on at least some of the bits included in the row addresses RA and the column addresses CA of the instruction memory, a physical location in which an instruction is stored can be determined. Accordingly, based on at least some of the bits of the row addresses RA and the column addresses CA, an index corresponding to the instruction can be calculated. For example, based on a queue address representing a storage location of a first instruction Inst 0, an index having a first value 0 is calculated, based on a queue address representing a storage location of a second instruction Inst 1, an index having a second value 1 is calculated, and an index corresponding to other instructions can be calculated. In alternative embodiments, the location can be virtual or remapped, rather than physical.

[0101] Although not specifically shown in Figures 9 to 11 , the host HOST can manage the index corresponding to the instruction by the same method as the method of the memory device based on the queue address to be provided to the memory device. Further, the host HOST can determine the kind or type of the plurality of operations serially performed with respect to the neural network function, can generate the index order information so that the instruction can be executed to correspond to the order of the plurality of operations, and can provide the generated index order information to the memory device.

[0102] Figure 12 Embodiment examples of a memory device 400 according to exemplary embodiments of the inventive concept are shown. In Figure 12 , an example is shown in which components related to operation processing are arranged to correspond to the memory bank array.

[0103] Referring to Figure 12 , the memory device 400 can include first to Nth memory bank arrays 410_1 to 410_N and first to Nth PIM circuits 430_1 to 430_N arranged to correspond to the first to Nth memory bank arrays 410_1 to 410_N. Each of the first to Nth PIM circuits 430_1 to 430_N can perform an operation logic processing operation by using at least one of information read from the corresponding memory bank or data provided by the host HOST.

[0104] According to the above-described embodiments, repetitive descriptions can be omitted. Here, the memory device 400 can include an instruction memory for storing instructions related to operation processing, and the instruction memory can be arranged to correspond to the first to Nth memory bank arrays 410_1 to 410_N. For example, first to Nth instruction memories 421_1 to 421_N can be provided in the memory device 400, and each of the first to Nth instruction memories 421_1 to 421_N can each include a circular queue having a plurality of instruction queue segments for storing instructions by a circular queue technique. Further, when the index in the above-described embodiments is applied, an index corresponding to the instructions stored in the first to Nth instruction memories 421_1 to 421_N can be further stored. In Figure 12 , first to Nth index memories 422_1 to 422_N arranged to correspond to the first to Nth memory bank arrays 410_1 to 410_N are shown. According to the current embodiment, the index memory and the instruction memory are shown to be separate from each other. However, the index can alternatively be stored in the corresponding circular instruction queue or their respective instruction queue segments.

[0105] Further, according to the above-described embodiments, a storage circuit for storing index order information from the host HOST can be further provided in the memory device 400, and the storage circuit for storing the index order information can also be arranged to correspond to the memory bank arrays. According to exemplary embodiments, the index orders can be set to be the same as or different from each other to correspond to the first memory bank array 410_1 to the Nth memory bank array 410_N. Accordingly, in the instruction memory arranged to correspond to different memory bank arrays, instructions stored in instruction queue segments at different locations can be executed.

[0106] Each of the first PIM circuit 430_1 to the Nth PIM circuit 430_N can include one or more processing elements PE for performing an operation logic processing operation, and instructions read from the first instruction memory 421_1 to the Nth instruction memory 421_N can be provided to the corresponding PIM circuit. Each of the first PIM circuit 430_1 to the Nth PIM circuit 430_N can also include an instruction decoder (Inst Dec) for decoding instructions, and an operation logic processing operation according to a kind of instruction decoding result can be performed.

[0107] According to Figure 12 As shown in the embodiments illustrated in the above-described embodiments, since the instruction memory is arranged to correspond to the memory bank arrays and / or the PIM circuits, the transmission path of instructions to the PIM circuits can be simplified and the instruction read speed can be improved. Further, since different instructions can be provided to the first PIM circuit 430_1 to the Nth PIM circuit 430_N, different kinds of operations can be processed in parallel. Further, according to the embodiments of the inventive concept, by providing the instruction memory using the circular queue technique, the size of the instruction memory corresponding to the PIM circuits can be reduced. Accordingly, even when each of the plurality of instruction memories is arranged to correspond to each of the plurality of memory bank arrays, respectively, the increase in the size of the plurality of instruction memories can be minimized.

[0108] Figure 13 A data processing system 500 including a memory device according to exemplary embodiments of the inventive concept is illustrated. As Figure 13 illustrated, the data processing system 500 can include an application processor (AP) 510 and a memory device 520, and the application processor 510 can include a memory control module 511 for communicating with the memory device 520.

[0109] The application processor 510 can perform Figure 1the function of the host HOST in the above-described embodiments. According to the above-described embodiments, in the instruction load mode, a plurality of instructions Inst can be provided to the memory device 520, and the memory device 520 can be controlled such that the instruction update operation is performed during the execution of the instructions. Further, according to the above-described embodiments, when the instructions are executed based on the index, the application processor 510 can generate index order information, and can provide the generated index order information to the memory device 520.

[0110] The application processor 510 can be implemented by a System on Chip (SoC) including a system bus. As a standard specification of the system bus, an Advanced RISC Machine (ARM) Advanced Microcontroller Bus Architecture (AMBA) protocol can be applied. The bus type of the AMBA protocol can be an Advanced High-performance Bus (AHB), an Advanced Peripheral Bus (APB), an Advanced eXtensible Interface (AXI), AXI4, or AXI Coherency Extensions (ACE). In addition to the above protocols, another type of protocol, etc., such as uNetwork of SONICs Limited, CoreConnect of IBM, or an Open Core Protocol of OCP-IP can be applied.

[0111] The memory control module 511 can perform the function of the memory controller in the above-described embodiments, and can control a memory bank operation or an operation logic processing operation in the memory device 520 by transmitting a command CMD and / or an address ADD to the memory device 520. According to the above-described embodiments, the memory device 520 can include a memory bank 521, an instruction memory 522, an operation logic 523, and a control logic 524, wherein the operation logic 523 can include one or more PIM circuits PIM_1 to PIM_N, and the control logic 524 can include a command and / or address decoder 524_1. Further, the instruction memory 522 can include a circular queue-based instruction queue segment. According to the above-described embodiments, after the execution of the previously stored instructions, an update operation can be performed, wherein a new instruction is written on the previously executed instructions in the instruction queue based on the control of the application processor 510.

[0112] Figure 14An example in which a memory device includes a high bandwidth memory (HBM) 600 according to exemplary embodiments of the inventive concept is shown.

[0113] The HBM 600 can have increased bandwidth by including multiple channels with independent interfaces. Referring to Figure 14 , the HBM 600 can include multiple dies, e.g., a buffer die (or logic die) 610 and one or more core dies 620 stacked on the buffer die 610. Referring to Figure 14 , an example in which first to fourth core dies are included in the HBM 600 is shown. However, the number of core dies 620 can vary.

[0114] Further, each of the core dies 620 can include one or more channels. In Figure 14 , an example in which each of the core dies 620 includes two channels is shown, such that the HBM 600 includes eight channels in total, CH1 to CH8. For example, the first core die can include a first channel CH1 and a third channel CH3, the second core die can include a second channel CH2 and a fourth channel CH4, the third core die can include a fifth channel CH5 and a seventh channel CH7, and the fourth core die can include a sixth channel CH6 and an eighth channel CH8.

[0115] The buffer die 610 can include an interface (IF) circuit 611 for communication with a host or a memory controller, and can receive commands and / or addresses and data from the host HOST through the interface circuit 611. The host HOST can transmit the commands and / or addresses and data through buses arranged to correspond to the channels. The buses can be divided by the channels, or some buses can be shared by two channels or multiple channels per die. The interface circuit 611 can transmit the commands and / or addresses and data to the channels requested by the host HOST to perform memory operations or arithmetic logic processing operations.

[0116] Each of the channels of the HBM 600 can include the PIM circuit 621 and the instruction memory 622 according to the above-described embodiments. Repetitive descriptions are omitted. The instruction memory 622 can include a plurality of instruction queue segments for storing instructions based on a circular queue technique.

[0117] Although not specifically shown in Figure 14 , when the index is applied, the channels of the HBM 600 can further store the index corresponding to the instruction in the instruction memory 622, and can additionally store index order information from the host HOST in the channels.

[0118] The buffer die 610 can further include a Through-Silicon-Via (TSV) region 612, a Physical (PHY) region 613, and a Direct Access (DA) region 614. Although not specifically shown in Figure 14 the processor for controlling the overall operation of the HBM 600, such as the control of routing of data, can also be included in the buffer die 610.

[0119] In the TSV region 612, TSVs for communication with the core die 620 are formed. Further, the physical PHY region 613 can include a plurality of input and output circuits for communication with an external host HOST. For example, the physical PHY region 613 can include one or more ports for communication with the host HOST.

[0120] The direct access region 614 can be arranged to directly communicate with an external tester through a conductive unit arranged on an outer surface of the HBM 600 in a mode of testing the HBM 600.

[0121] Figure 15 A server system 700 including a data processing system according to an embodiment of the present inventive concept is illustrated.

[0122] Referring to Figure 15 , the server system 700 can include a manager 710 and a plurality of servers 720_1 to 720_K. Each of the plurality of servers 720_1 to 720_K can correspond to the data processing system described in the above-described embodiments. The plurality of servers 720_1 to 720_K are connected to each other through a bus supporting a protocol such as, for example, PCI or PCIe. For example, the plurality of servers 720_1 to 720_K can communicate with each other through a P2P connection structure based on the control of the manager 710.

[0123] Referring to one server, such as, for example, the first server 720_1, the first server 720_1 can include a host HOST and one or more memory devices MEM according to the above-described embodiments, which can perform various kinds of arithmetic logic processing operations according to the functions of the server and the above-described embodiments and can store the processing results (wherein the repetitive descriptions are omitted). According to an embodiment, each of the memory devices MEM can include a memory device of the embodiments shown in Figures 1 to 14 . Thus, each of the memory devices MEM can include a circular queue-based instruction memory.

[0124] While example embodiments of the inventive concept have been particularly shown and described with reference to several drawings, it will be understood that various changes in form and details can be made therein without departing from the scope or spirit of the disclosure or the following claims.

Claims

1. A memory device comprising: a memory bank including at least one bank array of memory cells; a processor-in-memory (PIM) circuit connected to the memory bank and configured to perform an operation-logic processing operation by using at least one of data provided by a host or data read from the memory bank; and an instruction memory connected to the PIM circuit and including a circular instruction memory queue, wherein the circular instruction memory queue has a first instruction queue segment to an mth instruction queue segment cyclically configured to store instructions provided by the host, wherein the instructions stored in the first instruction queue segment to the mth instruction queue segment are executed in response to an operation request from the host, and new instructions provided by the host are updated on instructions that are completely executed in the circular instruction memory queue at an instruction queue segment where the instructions that are completely executed are executed, wherein, with respect to a plurality of commands related to one neural network function received from the host, the operation-logic processing operation and an operation for writing the new instructions are performed together based on a result of decoding some commands selected from the plurality of commands, wherein the operation-logic processing operation is selectively performed based on a result of decoding other commands selected from the plurality of commands.

2. The memory device of claim 1, wherein some of a plurality of instructions are stored in the first instruction queue segment to the mth instruction queue segment with respect to the one neural network function, and wherein other instructions of the plurality of instructions are sequentially updated on the completely executed instructions among the first instruction queue segment to the mth instruction queue segment.

3. The memory device of claim 1, further comprising: control logic configured to decode commands or addresses received from the host, control memory bank operations on the memory bank based on the decoded commands or addresses, or perform control operations that control the PIM circuit to perform the operation-logic processing operation.

4. The memory device of claim 1, wherein the memory bank includes a first bank array to an Nth bank array, and the PIM circuit includes substantially identical first PIM circuit to an Nth PIM circuit arranged corresponding to the first bank array to the Nth bank array, where N is an integer not less than 2, and wherein the instruction memory is arranged corresponding to each of the first PIM circuit to the Nth PIM circuit, and instructions read from the instruction memory are provided to the corresponding PIM circuit.

5. The memory device of claim 1, wherein the memory device operates in a load mode, an execution mode, and an execution and update mode, wherein, in the load mode, first instructions to mth instructions are stored in the first instruction queue segment to the mth instruction queue segment, wherein, in the execution mode, at least some of the first instructions to the mth instructions are sequentially executed in response to the operation request from the host, and wherein, in the execution and update mode, the first instructions to the mth instructions are sequentially executed in response to the operation request from the host, and the new instructions provided by the host are updated on the completely executed instructions in the circular instruction memory queue. wherein, in the execution and update mode, in response to the operation request from the host, new instructions are updated on previously executed instructions in the circular instruction memory queue while executing instructions stored in the instruction memory.

6. The memory device of claim 1, wherein, first through mth instructions are stored in the first through mth instruction queue segments, and wherein, after the first instruction is executed, the m+1th instruction is updated on the first instruction in the circular instruction memory queue while the second instruction is being executed.

7. The memory device of claim 1, wherein, first through mth instructions are stored in the first through mth instruction queue segments, and wherein, after the first through kth instructions are sequentially executed, an operation of updating new instructions starts from a first executed instruction in the circular instruction memory queue, where k is an integer smaller than m.

8. The memory device of claim 1, further comprising: an index memory connected to the instruction memory and configured to store indexes corresponding to instructions stored in the instruction memory; and an index order storage circuit connected to the instruction memory and configured to store index order information representing an execution order of instructions from the host, wherein, when the operation request is provided from the host, instructions stored in the instruction memory are executed in an order according to the index order information.

9. The memory device of claim 8, wherein, When multiple indexes pointing to the same instruction are included in the index order information, an instruction stored in one instruction queue segment is executed multiple times.

10. The memory device of claim 1, wherein, Each of the first through mth instruction queue segments includes at least one of a flip-flop or a latch.

11. A memory device, comprising: a memory bank including at least one bank array of memory cells; a processor-in-memory (PIM) circuit connected to the memory bank and configured to perform an arithmetic logic processing operation by using at least one of data provided by a host or data read from the memory bank; an instruction memory connected to the PIM circuit and including a circular instruction memory queue, wherein the circular instruction memory queue has first through mth instruction queue segments configured to store instructions provided by the host, where m is an integer not less than 2, wherein instructions stored in the first through mth instruction queue segments are executed in response to an operation request from the host, and new instructions provided by the host are updated on instructions that are completely executed in the circular instruction memory queue at an instruction queue segment where the completely executed instructions are executed when executed; an index manager connected to the instruction memory and configured to generate indexes corresponding to instructions stored in the instruction memory; and an index order storage circuit connected to the index manager and configured to store index order information representing an execution order of instructions from the host, wherein, when the operation request is provided by the host, the instructions stored in the instruction memory are executed in an execution order according to the stored index order information, wherein, with respect to a plurality of commands related to one neural network function received from the host, the operation logic processing operation and the operation for writing the new instruction are executed together based on a result of decoding some of the commands selected from the plurality of commands, wherein, the operation logic processing operation is selectively executed based on a result of decoding other commands selected from the plurality of commands.

12. The memory device of claim 11, wherein, When a plurality of indexes pointing to the same instruction are included in the index order information, the instructions stored in one instruction queue segment are executed the same number of times.

13. The memory device of claim 11, wherein, The index manager receives a row address and a column address representing a storage location of the instruction memory, and generates the index by using at least some of a plurality of bits of the row address and the column address.

14. The memory device of claim 11, further comprising: An index memory connected to the instruction memory and configured to store an index corresponding to an instruction stored in the instruction memory, wherein, the index manager stores the index generated to correspond to the instruction in the index memory.

15. A method of operating a memory device, wherein the memory device has: an instruction memory configured to store instructions related to an operation process; and a memory-in-processor PIM circuit configured to perform an operation logic processing operation corresponding to the instruction, the method comprising: loading some of a plurality of instructions for a neural network function into a plurality of instruction queue segments of a circular instruction queue of the instruction memory; the PIM circuit executing the operation logic processing operation by sequentially executing the loaded instructions; rewriting an instruction queue segment having a previously executed instruction with other instructions of the plurality of instructions for the neural network function; and the PIM circuit executing an operation logic processing operation by sequentially executing the rewriting with other instructions of the plurality of instructions, wherein, the instruction memory includes a first instruction queue segment to an mth instruction queue segment, where m is an integer not less than 2, wherein, the instructions stored in the first instruction queue segment to the mth instruction queue segment are executed in response to an operation request, and a new instruction is updated on a completely executed instruction at an instruction queue segment where the completely executed instruction is located when executed, wherein, with respect to a plurality of commands related to one neural network function, the operation logic processing operation and the operation for writing the new instruction are executed together based on a result of decoding some of the commands selected from the plurality of commands, wherein, the operation logic processing operation is selectively executed based on a result of decoding other commands selected from the plurality of commands.

16. The method of claim 15, wherein the memory device operates in a load mode, an execution mode, and an execution and update mode, wherein, in the load mode, the instructions are stored in the instruction memory, wherein, in the execution mode, the instructions stored in the instruction memory are sequentially executed in response to the operation request from the host, and wherein, in the execution and update mode, in response to the operation request from the host, a new instruction is updated on a previously executed instruction in an instruction queue segment while an instruction stored in the instruction memory is executed.

17. The method according to claim 15, wherein, the same first instruction to mth instruction are stored in the first instruction queue segment to mth instruction queue segment, and wherein, after the first instruction is executed, an m+1th instruction is updated on the first instruction in the loop instruction queue while a second instruction is being executed.

18. The method according to claim 15, wherein the same first instruction to mth instruction are stored in the first instruction queue segment to mth instruction queue segment, and wherein, after the first instruction to kth instruction are sequentially executed, an operation of updating a new instruction is started from the first instruction queue segment, where k is an integer smaller than m.

19. The method of claim 15, wherein, the operation of performing an operation process by executing one or more instructions and the operation of overwriting one or more new instructions on an instruction queue segment containing a previously executed instruction are alternately performed.

20. The method according to claim 15, further comprising: storing an index corresponding to the instruction; and receiving index order information representing an execution order of the instruction from a host, wherein, when the operation request is provided by the host, the instruction stored in the instruction memory is executed in an order according to the received index order information.

Images