Memory device that performs in-memory processing

Abstract

A memory device for performing in-memory processing is provided. The memory device includes: an in-memory operation unit that performs in-memory processing of operations pipelined into multiple pipeline stages; a memory bank that is allocated to the in-memory operation unit so that a group formed by n memory banks is allocated to each in-memory operation unit, and each memory bank performs an access operation of data requested by each of the multiple in-memory operation units while the pipelined operation is performed, wherein n is a natural number; and a memory die, in which the in-memory operation unit, the memory bank, and a command pad configured to receive a command signal from an external source are arranged. Each group formed by n memory banks includes a first memory bank having a first data transmission distance to the command pad and a second memory bank having a second data transmission distance to the command pad that is greater than the first data transmission distance.

Description

[0001] This application claims the benefit of Korean Patent Application No. 10-2020-0073733 filed on June 17, 2020, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein in its entirety for all purposes by reference. Technical Field

[0002] The following description relates to a memory device for performing in-memory processing by using an in-memory operation unit. Background Art

[0003] Applications such as processing of graphics algorithms, processing of neural networks, etc. are computationally intensive operations and require computing systems with a large amount of computing operations and memory capacity. Processor-in-memory (PIM) type memory devices that can perform some of the computing operations of the computing system via internal processing (or in-memory processing) of the memory device are being developed. In this way, the load of the computing operations of the computing system can be reduced via the internal processing of the memory device. However, when separate processing hardware for internal processing is added to the memory device, a method of efficiently performing interfacing and operation processing of the separate processing hardware may be required. Summary of the invention

[0004] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0005] In one general aspect, a memory device for performing in-memory processing includes: a plurality of in-memory operation units configured to perform in-memory processing pipelined into a plurality of pipeline stages; a plurality of memory banks assigned to the plurality of in-memory operation units so that a group formed by n memory banks is assigned to each of the plurality of in-memory operation units, each memory bank configured to perform an access operation of data requested by each of the plurality of in-memory operation units while the pipelined operation is performed, wherein n is a natural number; and a memory die, wherein the plurality of in-memory operation units, the plurality of memory banks, and a command pad configured to receive a command signal from an external source are arranged in the memory die. Each group formed by the n memory banks includes a first memory bank having a first data transmission distance to the command pad and a second memory bank having a second data transmission distance to the command pad, the second data transmission distance being greater than the first data transmission distance.

[0006] Each of the plurality of in-memory operation units may be configured to access a first memory bank of a corresponding group of n memory banks before accessing a second memory bank of a corresponding group of n memory banks to perform in-memory processing of pipelined operations.

[0007] The memory die may also include: a memory bank selector configured to: for each group formed by n memory banks, sequentially select a first memory bank and a second memory bank based on a first data transmission distance and a second data transmission distance; and a multiplexer configured to: provide data accessed from the first memory bank or the second memory bank selected by the memory bank selector to a corresponding in-memory operation unit.

[0008] The multiple memory storage bodies may include a first group of memory storage bodies arranged on a first side of the memory die and a second group of memory storage bodies arranged on the other side of the memory die, and the multiple in-memory operation units may include a first group of in-memory operation units and a second group of in-memory operation units, the first group of in-memory operation units are arranged adjacent to and connected to the first group of memory storage bodies in an area between the first group of memory storage bodies and the second group of memory storage bodies, and the second group of in-memory operation units are arranged adjacent to and connected to the second group of memory storage bodies in the area.

[0009] The command pad may be arranged between a first group of in-memory operation units and a second group of in-memory operation units in the region, and the first memory bank and the second memory bank of each group formed of n memory banks may be classified according to a first data transmission distance and a second data transmission distance within the region.

[0010] The first group of memory storage bodies can be classified into two groups, the two groups including a first group arranged closer to the command pad and a second group arranged farther from the command pad, and the first memory storage body of each group formed by n memory storage bodies is selected from the first group, and the second memory storage body of each group formed by n memory storage bodies is selected from the second group.

[0011] A first group formed by n memory storage bodies among the first group of memory storage bodies can be assigned to a first in-memory operation unit among the first group of in-memory operation units, and the first in-memory operation unit can process pipelined operations on data accessed from a first memory storage body of the first group formed by the n memory storage bodies before data is accessed from a second memory storage body of the first group formed by the n memory storage bodies.

[0012] The memory device may be a double data rate 4 dynamic random access memory (DDR4 DRAM) module, the first group of memory storage bodies may include 8 memory storage bodies, and the second group of memory storage bodies may include 8 memory storage bodies, and the first group of memory operation units may include 8 / n memory operation units, and the second group of memory operation units may include 8 / n memory operation units.

[0013] The first group of memory storage bodies may include a first memory storage body, a second memory storage body, a third memory storage body, a fourth memory storage body, a fifth memory storage body, a sixth memory storage body, a seventh memory storage body and an eighth memory storage body in order of a first direction, the first group of in-memory operation units may include a first in-memory operation unit, a second in-memory operation unit, a third in-memory operation unit and a fourth in-memory operation unit in order of the first direction, and the first memory storage body and the third memory storage body may be assigned to the first in-memory operation unit, the second memory storage body and the fourth memory storage body may be assigned to the second in-memory operation unit, the fifth memory storage body and the seventh memory storage body may be assigned to the third in-memory operation unit, and the sixth memory storage body and the eighth memory storage body may be assigned to the fourth in-memory operation unit.

[0014] The connections between the second group of memory banks and the second group of in-memory operation units may be symmetrical to the connections between the first group of memory banks and the first group of in-memory operation units.

[0015] Each of the plurality of in-memory operation units may include at least one pipeline register configured to buffer sub-operation results in each pipeline stage of the pipelined operation.

[0016] The pipelined operations may include multiply-accumulate (MAC) operations of the neural network.

[0017] In another general aspect, a memory device for performing in-memory processing includes: a plurality of in-memory operation units configured to perform in-memory processing pipelined into a plurality of pipeline stages; and a plurality of memory banks assigned to the plurality of in-memory operation units, such that a group formed by n memory banks is assigned to each of the plurality of in-memory operation units, each memory bank configured to perform an access operation of data requested by each of the plurality of in-memory operation units while the pipelined operation is performed, wherein n is a natural number. Each group formed by n memory banks may include: a first memory bank having a first physical distance for data transmission of a command signal provided from an external source and a second memory bank having a second physical distance for data transmission of the command signal, the second physical distance being greater than the first physical distance.

[0018] Each of the plurality of in-memory operation units may be configured to access a first memory bank of a corresponding group of n memory banks before a second memory bank of a corresponding group of n memory banks to perform in-memory processing of pipelined operations.

[0019] The memory device may also include: a memory bank selector configured to: for each group formed by n memory banks, sequentially select a first memory bank and a second memory bank based on a first physical distance and a second physical distance; and a multiplexer configured to: provide data accessed from the first memory bank or the second memory bank selected by the memory bank selector to a corresponding in-memory operation unit.

[0020] In another general aspect, a memory device includes: a plurality of in-memory operation units configured to perform in-memory processing of pipelined operations of a neural network; a plurality of memory banks, each memory bank being assigned to one of the plurality of in-memory operation units, such that a group formed of n memory banks is assigned to each of the plurality of in-memory operation units, each memory bank being configured to perform access operations of data requested by the assigned in-memory operation unit while the pipelined operations are performed, wherein n is an integer greater than or equal to 2; and a command pad configured to receive a command signal from an external source. Each of the plurality of in-memory operation units is configured to: access memory banks from a corresponding assigned group formed of n memory banks in an order determined based on a distance from each memory bank in the corresponding group formed of n memory banks to the command pad.

[0021] A distance from each memory bank in a corresponding group of n memory banks to the command pad may be a data transfer distance, and each of the plurality of in-memory operation units may be configured to access memory banks from the corresponding allocated group of n memory banks in ascending order of the data transfer distance.

[0022] The distance from each memory bank in the corresponding group of n memory banks to the command pad may be a physical distance, and each of the plurality of in-memory operation units may be configured to access memory banks from the corresponding allocated group of n memory banks in ascending order of the physical distance.

[0023] Other features and aspects will be apparent from the following detailed description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a block diagram of a memory system according to an example.

[0025] Figure 2A and Figure 2B is a diagram for describing an allocation ratio between a memory bank and an operation unit within a memory according to an example.

[0026] Figure 3 is a diagram for describing hardware connections between memory operation units and memory banks on a memory die according to an example.

[0027] Figure 4 is a diagram for describing a pipeline of operations according to an example.

[0028] Figure 5 is used to describe that according to an example, a pipeline stage is assigned to each memory bank for Figure 4 FIG. 1 is a diagram of in-memory processing of pipelined operations.

[0029] Figure 6 is a diagram for describing a delay difference that may occur when memory banks arranged at different locations are accessed (read / written).

[0030] Figure 7 is a diagram for describing an arrangement of operation units within a memory on a memory die according to an example.

[0031] Figure 8 is a diagram for describing a method by which a memory bank processing operation is mapped to an in-memory operation unit according to an example.

[0032] Fig. 9is a diagram for comparing a case where a far memory bank is accessed first and a case where a near memory bank is accessed first according to an example.

[0033] Fig.10 is a block diagram of a computing system including a memory device according to an example.

[0034] Throughout the drawings and detailed description, unless otherwise described or provided, the same figure reference numerals will be understood to refer to the same elements, features, and structures. The drawings may not be to scale, and the relative sizes, proportions, and depictions of the elements in the drawings may be exaggerated for clarity, illustration, and convenience. DETAILED DESCRIPTION

[0035] The following specific embodiments are provided to help the reader obtain a comprehensive understanding of the method, device and / or system described herein. However, after understanding the disclosure of the present application, various changes, modifications and equivalents of the method, device and / or system described herein will be clear. For example, the order of operations described herein is only an example, and is not limited to the order of operations set forth herein, but except for the operations that must occur in a specific order, the order of operations can be changed as will be clear after understanding the disclosure of the present application. In addition, for greater clarity and simplicity, the description of features known after understanding the disclosure of the present application can be omitted.

[0036] The features described herein can be implemented in different forms and should not be construed as being limited to the examples described herein. Rather, the examples described herein have been provided to illustrate only some of the many possible ways to implement the methods, devices, and / or systems described herein that will be clear after understanding the disclosure of the present application.

[0037] Throughout the specification, when a component is described as being "connected to" or "coupled to" another component, the component may be directly "connected to" or "coupled to" the other component, or there may be one or more other components intervening. Conversely, when an element is described as being "directly connected to" or "directly coupled to" another element, there may be no other elements intervening. Similarly, similar expressions (e.g., "between" versus "immediately between" and "adjacent to" versus "immediately adjacent to") should also be interpreted in the same manner. As used herein, the term "and / or" includes any one of the associated listed items and any combination of any two or more.

[0038] Although terms such as "first", "second", and "third" may be used herein to describe various members, components, regions, layers, or portions, these members, components, regions, layers, or portions should not be limited by these terms. Instead, these terms are only used to distinguish one member, component, region, layer, or portion from another member, component, region, layer, or portion. Therefore, without departing from the teachings of the examples described herein, the first member, first component, first region, first layer, or first portion referred to in the examples may also be referred to as the second member, second component, second region, second layer, or second portion.

[0039] The terms used herein are only used to describe various examples and are not intended to limit the disclosure. Unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. The terms "comprise", "include" and "have" indicate the presence of the described features, quantities, operations, components, elements and / or combinations thereof, but do not exclude the presence or addition of one or more other features, quantities, operations, components, elements and / or combinations thereof.

[0040] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those of ordinary skill in the art to which the present disclosure belongs and based on the understanding of the disclosure of the present application. Unless explicitly defined as such herein, terms (such as those defined in general dictionaries) will be interpreted as having a meaning consistent with their meaning in the context of the disclosure of the relevant art and the present application, and should not be interpreted in an idealized or overly formal way. The use of the term "may" for an example or embodiment herein (e.g., with respect to what an example or embodiment may include or implement) indicates that there is at least one example or embodiment that includes or implements such a feature, and all examples are not limited thereto.

[0041] Hereinafter, examples will be described in detail with reference to the accompanying drawings. However, examples may be implemented in many different forms and are not limited to those described herein.

[0042] Figure 1 is a block diagram of a memory system 1 according to an example.

[0043] Reference Figure 1 , the memory system 1 may include a memory device 10 and a memory controller 20. The memory device 10 may include a plurality of in-memory operation units 110 that perform in-memory processing and a plurality of memory banks 120 including a plurality of memory cells.

[0044] The memory device 10 may be implemented as a memory chip or a memory module. The memory controller 20 may be implemented as a host or part of the memory device 10, and the memory controller 20 may be arranged in a memory module. In other words, the implementation form may vary and is not limited to one. Although in Figure 1 The host is not shown in the figure, but is a functional block that performs a computing operation by controlling the memory system 1, and may correspond to a central processing unit (CPU), a digital signal processor (DSP), a graphics processing unit (GPU), or an application processor (AP).

[0045] The memory controller 20 may provide various signals to the memory device 10 to control all operations of the memory device 10. For example, the memory controller 20 may control memory access operations (such as reading and writing) of the memory device 10. Specifically, the memory controller 20 may provide a command CMD and an address ADDR to the memory device 10 to write data DATA on the memory device 10 or read data DATA from the memory device 10. In addition, the memory controller 20 may also provide a clock signal CLK to the memory device 10.

[0046] The command CMD may include an activation command for switching the memory bank 120 to an activation state to read or write data. In response to the activation command, the memory device 10 may activate a row (i.e., a word line) included in the memory bank 120. In addition, the command CMD may include a precharge command for switching the memory bank 120 from an activation state to a standby state after the reading or writing of data is completed. In addition, the command CMD may include a refresh command for controlling a refresh operation of the memory bank 120. However, the types of command CMD described herein are merely examples, and other types of command CMD may exist.

[0047] The memory controller 20 may provide various signals to the memory device 10 to control the in-memory processing operation of the in-memory operation unit 110. For example, the memory controller 20 may provide a signal indicating the in-memory processing operation of the in-memory operation unit 110 to the memory device 10 via a combination of a command CMD, an address ADDR, and / or a clock signal CLK.

[0048] The in-memory operation unit 110 may be implemented as a processing element (PE) for performing an operation process in the memory device 10. In other words, the in-memory operation unit 110 may perform an in-memory process (or internal process) in the memory device 10.

[0049] Specifically, the in-memory operation unit 110 may perform a data operation on the data DATA stored in the memory bank 120 and / or the data DATA received from the memory controller 20, and may store the operation result of the data DATA in the memory bank 120 or provide the operation result of the data DATA to the memory controller 20. Therefore, the in-memory operation unit 110 may also be referred to as a processor in memory (Processor In Memory, PIM) or a function in memory (Function In Memory, FIM).

[0050] The memory operation unit 110 may be an arithmetic logic unit (ALU) or a multiply-accumulate (MAC). For example, the memory operation unit 110 may perform logic operations (such as data inversion, data shift, data exchange, data comparison, and, and exclusive OR (XOR)), mathematical operations (such as addition and subtraction), and data operations.

[0051] The number of memory operation units 110 and the number of memory banks 120 included in the memory device 10 may vary. In addition, n memory banks 120 may be assigned to one memory operation unit 110, where n is a natural number. In one example, n may be an integer greater than or equal to 2.

[0052] For example, when the memory device 10 corresponds to a double data rate 4 dynamic random access memory (DDR4 DRAM) module, the number of memory banks 120 may be 16 and the number of in-memory operation units 110 may be 8, and the in-memory operation units 110 and the memory banks 120 may be mapped in a ratio of 1:2 (n=2). Hereinafter, for ease of description, it is assumed that two memory banks 120 are assigned to one in-memory operation unit 110 (n=2), but the configuration is not limited thereto.

[0053] Each memory bank 120 may include a plurality of memory cells. Specifically, the memory cells in the memory bank 120 may be at points where a plurality of word lines and a plurality of bit lines intersect each other. The memory bank 120 may store in-memory processing data. The in-memory processing data may include data to be operated by the in-memory operation unit 110 and / or data generated as a result of an operation performed by the in-memory operation unit 110.

[0054] The memory device 10 may include various types of memories, for example, may include a DRAM such as a double data rate synchronous DRAM (DDR SDRAM), a low power double data rate (LPDDR) SDRAM, a graphic double data rate (GDDR) SDRAM, or a rambus DRAM (RDRAM). However, the configuration is not limited thereto, and the memory device 10 may include a nonvolatile memory such as a flash memory, a magnetic RAM (MRAM), a ferroelectric RAM (FeRAM), a phase change RAM (PRAM), or a resistive RAM (ReRAM).

[0055] Figure 2A and Figure 2B is a diagram for describing an allocation ratio between a memory bank and an operation unit within a memory according to an example.

[0056] Figure 1 The memory device 10 may include a Figure 1 The in-memory operation unit 110 and Figure 1 The memory bank 120 of the memory die 100, Figure 2A and Figure 2B Some of a certain number of in-memory operation units 110 and some of a certain number of memory banks 120 disposed on the memory die 100 are shown.

[0057] like Figure 2A As shown in the figure, two memory storage bodies (first memory storage body 121 and second memory storage body 122) can be assigned to one in-memory operation unit (first operation unit 111), and two memory storage bodies (third memory storage body 123 and fourth memory storage body 124) can be assigned to one in-memory operation unit (second operation unit 112).

[0058] like Figure 2B As shown in FIG. 1 , four memory banks (first to fourth memory banks 121 to 124 ) may be allocated to one in-memory operation unit (first operation unit 111 ).

[0059] In other words, the memory banks 120 provided in the memory device 10 may share one memory operation unit in units of n memory banks, and each memory operation unit may perform data operations by accessing the corresponding n memory banks. Here, when the memory device 10 corresponds to a DDR4 DRAM module, n may be one of 2, 4, and 8, but is not limited thereto. As described above, in the example, for ease of description, n=2.

[0060] Figure 31 is a diagram for describing hardware connections between memory operation units and memory banks on the memory die 100 according to an example.

[0061] Reference Figure 3 , an example in which two memory banks (first memory bank 121 and second memory bank 122) are assigned to one memory operation unit (first operation unit 111) on the memory die 100 is shown. This example may be one of the examples of all memory operation units 110 and memory banks 120 provided on the memory die 100, and the remaining memory operation units 110 and memory banks 120 on the memory die 100 may be similar to Figure 3 to implement and operate with examples.

[0062] On the memory die 100, each in-memory operation unit (first operation unit 111) and memory banks (first memory bank 121 and second memory bank 122) are connected to each other. Here, the memory die 100 may include a bank selector 130 for selecting one of two memory banks (first memory bank 121 and second memory bank 122) assigned to the first operation unit 111, and a multiplexer (MUX) 140 for providing data accessed from the memory bank selected by the bank selector 130 to the first operation unit 111. Hardware components implemented on the memory die 100 may be connected to each other via a data bus 160.

[0063] The first operation unit 111 operates at an operation frequency (clock signal CLK (e.g., the operation frequency of the clock signal CLK)) to perform in-memory processing of the pipelined operation 310. Here, the pipelined operation 310 may be pipelined into multi-pipeline stages of sub-operation units that may be processed within a first operation cycle corresponding to the first operation frequency of the first operation unit 111. The first operation unit 111 may include at least one pipeline register 1111, and the at least one pipeline register 1111 is used to buffer the sub-operation results in each pipeline stage of the pipelined operation 310.

[0064] Each of the first memory bank 121 and the second memory bank 122 operates at an operation frequency (clock signal CLK) to perform an access operation of data requested from the first operation unit 111 while the pipelined operation 310 is performed.

[0065] The operating frequency (clock signal CLK) is based on the external source ( Figure 1The first operation unit 111, the memory banks (the first memory bank 121 and the second memory bank 122) and the bank selector 130 in the memory die 100 operate according to the operation frequency (clock signal CLK) according to the bank clock signal provided by the memory controller 20 or the host.

[0066] Figure 4 is a diagram for describing a pipeline of operations according to an example.

[0067] Pipelining is a technique for increasing the processing speed of an operation by dividing the process of performing the operation into stages and processing the stages in parallel (simultaneously).

[0068] Reference Figure 4 , a given operation may be pipelined into multiple pipeline stages (first pipeline stage STAGE 1 to third pipeline stage STAGE 3) including some sub-operations (e.g., first sub-operation to seventh sub-operation), and thus a pipelined operation 400 may be generated. Here, the given operation may be, for example, a MAC operation of a neural network, and each of the sub-operations (first sub-operation to seventh sub-operation) may be pipelined according to a series of individual operations (such as data acquisition, multiplication, addition, and data output).

[0069] The pipelined operation 400 includes multiple pipeline stages of seven sub-operations (first to seventh sub-operations). Specifically, the first pipeline stage STAGE 1 includes the first sub-operation and the second sub-operation, the second pipeline stage STAGE 2 includes the third sub-operation, the fourth sub-operation and the 5-1st sub-operation, and the third pipeline stage STAGE 3 includes the 5-2nd sub-operation, the sixth sub-operation and the seventh sub-operation. Here, the 5-1st sub-operation and the 5-2nd sub-operation are divided from the fifth sub-operation. In this way, one sub-operation is again divided into some sub-operations to classify the operations that can be processed within one cycle (e.g., operation cycle k (ns)) of the operation unit in the memory.

[0070] When a given operation is performed with an in-memory operation unit ( Figure 3 When the operation frequency (clock signal CLK) of the first operation unit 111 is pipelined in units of operation period k (ns), the operation units in the memory ( Figure 3The first operation unit 111) can process the pipelined operation 400 (three pipeline stages, i.e., the first pipeline stage STAGE 1 to the third pipeline stage STAGE 3) throughout a total of 3 cycles, and execute the pipeline for each cycle (operation cycle: k (ns)). Because the operation frequency (clock signal CLK) of each memory storage body (the first memory storage body 121 or the second memory storage body 122) is the same as the operation frequency (clock signal CLK) of the in-memory operation unit (the first operation unit 111), the in-memory operation unit (the first operation unit 111) can access one memory storage body (the first memory storage body 121 or the second memory storage body 122) within the operation cycle (k (ns)) of the memory storage body (the first memory storage body 121 or the second memory storage body 122).

[0071] Figure 5 is used to describe that according to an example, a pipeline stage is assigned to each memory bank for Figure 4 4 is a diagram of in-memory processing of pipelined operations 400 .

[0072] Reference Figure 5 , case 501 and case 502 are shown, in case 501 the in-memory operation unit ( Figure 3 The first operation unit 111) processes from Figure 3 The operation data accessed by the first memory bank 121 (which may be referred to as the operation #_ EVEN OP#_ EVEN ), in case 502 the in-memory operation unit (first operation unit 111) processes from Figure 3 The operation data accessed by the second memory bank 122 (which may be referred to as the operation #_ ODD OP#_ ODD In case 501 and case 502, the first memory bank 121 and the second memory bank 122 operate to respectively process the pipelined operation 400 (first pipeline stage STAGE 1 to third pipeline stage STAGE 3) requested by the in-memory operation unit (first operation unit 111).

[0073] The pipeline stage allocation 500 in each time line (cycle) will be described. Here, each cycle in the pipeline stage allocation 500 represents an operation cycle (k (ns)) corresponding to the operation frequency (clock signal CLK) described above.

[0074] In the Nth cycle, the first operation unit 111 accesses the first memory bank 121 to read the operation 1_ EVEN OP1_ EVEN Data and process operation 1_ EVENOP1_ EVEN The first pipeline stage STAGE 1.

[0075] In the (N+1)th cycle, the first operation unit 111 accesses the second memory bank 122 to read the operation 1_ ODD OP1_ ODD Data and process operation 1_ ODD OP1_ ODD In the (N+1)th cycle, the first operation unit 111 may perform pipeline processing operation 1_ EVEN OP1_ EVEN The second pipeline stage STAGE 2.

[0076] In the (N+2)th cycle, the first operation unit 111 accesses the first memory bank 121 to read and operate 1 _EVEN OP1 _EVEN Different another operation 2 _EVEN OP2 _EVEN The first operation unit 111 processes the operation 2 _EVEN OP1 _EVEN In the (N+2)th cycle, the first operation unit 111 may perform pipeline processing operation 1 based on the pipeline processing operation 1. _EVEN OP1 _EVEN The third pipeline stage STAGE 3 and operation 1 _ODD OP1 _ODD The second pipeline stage STAGE 2.

[0077] In other words, the first operation unit 111 may alternately access the first memory storage body 121 and the second memory storage body 122 in each cycle to read data of each operation, and may process the first pipeline stage STAGE 1 to the third pipeline stage STAGE 3 in parallel by using the data of the pipelined operation 400 read from each of the first memory storage body 121 and the second memory storage body 122 in each cycle.

[0078] Figure 6 is a diagram for describing a delay difference that may occur when memory banks arranged at different locations are accessed (read / written).

[0079] Reference Figure 6, a first group of memory banks BANK 1 to BANK 8 arranged on one side and a second group of memory banks BANK 9 to BANK 16 arranged on the other side may be arranged on the memory die 100. In addition, an IO interface including a command / input / output (IO) (CMD / IO) pad (or pad) 610 may be provided in an area 600 between the first group of memory banks BANK 1 to BANK 8 and the second group of memory banks BANK 9 to BANK 16 on the memory die 100. The CMD / IO pad 610 may also be referred to as a command pad. The IO interface may further include a global decoder for decoding a signal sent from an external source to the CMD / IO pad 610 and sending the decoded signal to the memory bank.

[0080] For example, a case in which a command signal is transmitted from an external source to the CMD / IO pad 610 and data of two memory banks BANK 1 and BANK 3 are accessed according to the command signal decoded by the global decoder will be described.

[0081] Reference Figure 6 , the data transmission distance from the CMD / IO pad 610 to the memory bank BANK 3 on the memory die 100 is shorter than the data transmission distance from the CMD / IO pad 610 to the memory bank BANK 1. Therefore, due to the difference in physical distance between the memory banks BANK 1 and BANK 3 for data access, a delay difference (e.g., about Δ2.5 ns) may occur in the command / data signal for accessing each of the memory banks BANK 1 and BANK 3.

[0082] In other words, due to the difference in physical distance for data transmission according to the locations of the memory banks BANK 1 to BANK 16 on the memory die 100, Figure 6 The delay difference described in .

[0083] Figure 7 1 is a diagram for describing the arrangement of operation units within a memory on a memory die 100 according to an example.

[0084] Reference Figure 7 , as referenced Figure 6 As described above, a first group of memory banks BANK 1 to BANK 8 arranged on one side and a second group of memory banks BANK 9 to BANK 16 arranged on the other side may be arranged on the memory die 100. In addition, an IO interface including a CMD / IO pad 710 may be arranged in an area 700 between the first group of memory banks BANK 1 to BANK 8 and the second group of memory banks BANK 9 to BANK 16 on the memory die 100.

[0085] A first group of in-memory operation units IMOU1 to IMOU4 arranged adjacent to and connected to the first group of memory storage bodies BANK 1 to BANK 8, and a second group of in-memory operation units IMOU5 to IMOU8 arranged adjacent to and connected to the second group of memory storage bodies BANK 9 to BANK 16 may be arranged in area 700.

[0086] The memory banks BANK1 to BANK16 may be allocated to the in-memory operation units IMOU1 to IMOU8 in units of n memory banks, where n=2.

[0087] Specifically, an example of a mapping relationship between the first group of memory banks BANK1 to BANK8 and the first group of in-memory operation units IMOU1 to IMOU4 will be described.

[0088] The first group of memory banks BANK 1 to BANK 8 may be classified based on the location of the CMD / IO pad 710 into memory banks BANK 3 to BANK 6 relatively close to the CMD / IO pad 710 and memory banks BANK 1, BANK 2, BANK 7, and BANK 8 relatively far from the CMD / IO pad 710. Each of the first group of in-memory operation units IMOU1 to IMOU4 may be mapped (allocated) and connected to one of the memory banks BANK 3 to BANK 6 relatively close to the CMD / IO pad 710 and one of the memory banks BANK 1, BANK 2, BANK 7, and BANK 8 relatively far from the CMD / IO pad 710.

[0089] For example, the first in-memory operation unit IMOU1 may be mapped to the far memory bank BANK 1 and the near memory bank BANK 3, and the second in-memory operation unit IMOU2 may be mapped to the far memory bank BANK 2 and the near memory bank BANK 4. Similarly, the third in-memory operation unit IMOU3 may be mapped to the far memory bank BANK 7 and the near memory bank BANK 5, and the fourth in-memory operation unit IMOU4 may be mapped to the far memory bank BANK 8 and the near memory bank BANK 6. In other words, each of the first group of in-memory operation units IMOU1 to IMOU4 may be arranged between a corresponding far memory bank and a corresponding near memory bank, wherein the distance from each in-memory operation unit to the corresponding far memory bank is equal to the distance from each in-memory operation unit to the corresponding near memory bank.

[0090] The second group of memory banks BANK 9 to BANK 16 and the second group of in-memory operation units IMOU5 to IMOU8 may be mapped in a similar manner as described above.

[0091] According to such an arrangement and mapping method of the in-memory operation units IMOU1 to IMOU8, since the data transmission distances (i.e., physical distances) of two memory banks mapped to one in-memory operation unit from the CMD / IO pad 710 are different from each other, the above-mentioned reference error may occur. Figure 6 In one example, each in-memory operation unit may access memory banks in the mapped memory banks in ascending order of data transfer distance (or physical distance) of each of the memory banks mapped thereto from the CMD / IO pad 710 .

[0092] Figure 8 is a diagram for describing a method by which a memory bank processing operation is mapped to an in-memory operation unit according to an example.

[0093] Reference Figure 8 , an example of processing operations of the first in-memory operation unit IMOU1 and the memory banks BANK 1 and BANK 3 mapped to the first in-memory operation unit IMOU1 will be described.

[0094] The far memory bank BANK 1 and the near memory bank BANK 3 are mapped to the first in-memory operation unit IMOU 1. Here, because the data transmission distances (i.e., physical distances) of the far memory bank BANK 1 and the near memory bank BANK 3 from the CMD / IO pad 710 are different from each other, there may be a delay difference regarding data access.

[0095] The first in-memory operation unit IMOU1 may perform in-memory processing faster according to a command signal sent from an external source to the CMD / IO pad 710 in the case 810 of performing data access to the near memory bank BANK 3 than in the case 820 of performing data access to the far memory bank BANK 1.

[0096] Therefore, the first in-memory operation unit IMOU1 processes the pipelined operation by first accessing the memory bank BANK 3 relatively close to the CMD / IO pad 710 among the mapped memory banks BANK 1 and BANK 3. Then, the first in-memory operation unit IMOU1 processes the pipelined operation by accessing the relatively far memory bank BANK 1.

[0097] According to this method, refer to Figure 5 The description of the first operation unit 111 processing the operation data accessed from the first memory bank 121 may correspond to the first in-memory operation unit IMOU1 processing the data accessed from the nearest memory bank BANK 3①. Similarly, referring to Figure 5 The description of the first operation unit 111 processing the operation data accessed from the second memory bank 122 may correspond to the first in-memory operation unit IMOU1 processing the data accessed from the remote memory bank BANK 1②.

[0098] The in-memory processing of pipelined operations can also be performed in the same manner as in reference to Figure 8 A similar approach is described in Figure 7 The remaining memory banks are performed between the memory operation units.

[0099] Fig. 9 is a diagram for comparing a case where a far memory bank is accessed first and a case where a near memory bank is accessed first according to an example.

[0100] Reference Fig. 9 , operating units within memory (e.g., Figure 8 The first in-memory operation unit IMOU1 first accesses a far memory bank (e.g. Figure 8 In the case 910 of the memory bank BANK 1), a total of 10 cycles may be required until the first MAC operation to the eighth MAC operation are performed. However, in the first memory operation unit IMOU1 first accesses the nearest memory bank (eg, Figure 8In the case 920 of the memory bank BANK 3), a total of 9 cycles may be required until the first MAC operation to the eighth MAC operation are executed. In other words, when the in-memory operation unit first accesses a memory bank that is relatively close to the CMD / IO pad among the memory banks mapped to the in-memory operation unit, the in-memory processing can be performed at a fast / faster speed. In other words, in the case 920 where the closest memory bank is accessed first, no operation cycle (NOP) can be reduced before a subsequent operation with data dependency (i.e., the first addition operation) is executed, and therefore, the in-memory processing can be completed at a fast / faster speed. According to Fig. 9 According to simulation results, when the in-memory operation unit first accesses the nearest memory bank, about 10% of the operation time (about 1 cycle = about 2.5ns) can be reduced.

[0101] Fig.10 1 is a block diagram of a computing system 1000 including a memory device according to an example. The memory device that performs the above-described in-memory processing may be installed on the computing system 1000 (such as a mobile device or a desktop computer) as a random access memory (RAM) 1020. One of the above-described embodiments may be applied to the memory device installed as the RAM 1020.

[0102] Specifically, a memory device that performs in-memory processing and can be installed as RAM 1020 includes multiple in-memory operation units and multiple memory storage bodies, the multiple in-memory operation units perform in-memory processing of pipelined operations, and the multiple memory storage bodies are allocated to each in-memory operation unit in units of n memory storage bodies and perform operations of accessing data requested by each in-memory operation unit while the pipelined operations are processed.

[0103] The memory device includes a memory die having in-memory operation units and memory banks arranged thereon, the plurality of memory banks may include a first group of memory banks arranged on one side of the memory die and a second group of memory banks arranged on the other side, the plurality of in-memory operation units may include a first group of memory operation units and a second group of memory operation units, the first group of memory operation units being arranged adjacent to and connected to the first group of memory banks in a region between the first group of memory banks and the second group of memory banks, the second group of memory operation units being arranged adjacent to and connected to the second group of memory banks in the region. Each in-memory operation unit may perform pipelined operations on accessed data from a relatively close-arranged memory bank before a relatively far-arranged memory bank on the memory die.

[0104] The memory die may include a bank selector that sequentially selects one of n memory banks assigned to a first in-memory operation unit based on a data transmission distance, and a multiplexer that provides data accessed through the memory bank selected by the bank selector to the first in-memory operation unit.

[0105] When the memory device mounted on the RAM 1020 is a DDR4 DRAM module, the plurality of memory banks include a first group of memory banks formed of 8 memory banks and a second group of memory banks formed of 8 memory banks, and the plurality of in-memory operation units may include a first group of in-memory operation units formed of 8 / n in-memory operation units and a second group of in-memory operation units formed of 8 / n in-memory operation units, where n is one of 2, 4, and 8. Here, the first group of memory banks may include a first memory bank to an eighth memory bank in order of a first direction (x-axis), and the first group of in-memory operation units may include a first memory bank to a fourth memory operation unit in order of a first direction (x-axis). When n=2, the first memory bank and the third memory bank may be assigned to the first memory operation unit, the second memory bank and the fourth memory bank may be assigned to the second memory operation unit, the fifth memory bank and the seventh memory bank may be assigned to the third memory operation unit, and the sixth memory bank and the eighth memory bank may be assigned to the fourth memory operation unit. The connections between the second group of memory banks and the second group of in-memory operation units may be symmetrical to the connections between the first group of memory banks and the first group of in-memory operation units.

[0106] The computing system 1000 includes a central processing unit (CPU) 1010, a RAM 1020, a user interface 1030, and a nonvolatile memory 1040 electrically connected to each other via a bus 1050. The nonvolatile memory 1040 may be a mass storage device such as a solid state drive (SSD) or a hard disk drive (HDD).

[0107] When the above-described memory device (or memory system) is applied to the computing system 1000 , the memory device included in the RAM 1020 may perform in-memory processing.

[0108] Although the present disclosure includes specific examples, it will be clear after understanding the disclosure of the present application that various changes in form and detail can be made in these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein will be considered to be merely descriptive and not for limiting purposes. The description of the features or aspects in each example will be considered to be applicable to similar features or aspects in other examples. If the described techniques are performed in a different order, and / or if the components in the described systems, architectures, devices, or circuits are combined in different ways, and / or replaced or supplemented by other components or their equivalents, suitable results can be achieved. Therefore, the scope of the disclosure is not limited by a specific embodiment, but by the claims and their equivalents, and all changes within the scope of the claims and their equivalents should be interpreted as included in the disclosure.

Claims

1. A memory device for performing in-memory processing, the memory device comprising: a plurality of in-memory operation units configured to perform in-memory processing of operations pipelined into a plurality of pipeline stages; a plurality of memory banks allocated to the plurality of in-memory operation units such that a group formed of n memory banks is allocated to each of the plurality of in-memory operation units, each memory bank being configured to perform an access operation of data requested by each of the plurality of in-memory operation units while a pipelined operation is performed, wherein n is an integer greater than or equal to 2; and a memory die, the plurality of in-memory operation units, the plurality of memory banks, and a command pad configured to receive a command signal from an external source being arranged in the memory die, wherein each group formed by n memory banks includes a first memory bank having a first data transmission distance to the command pad and a second memory bank having a second data transmission distance to the command pad, the second data transmission distance being greater than the first data transmission distance, In which, the distance from the first memory storage body of each group formed by n memory storage bodies to the in-memory operation unit to which each group formed by n memory storage bodies is assigned is equal to the distance from the second memory storage body of each group formed by n memory storage bodies to the in-memory operation unit to which each group formed by n memory storage bodies is assigned.

2. The memory device according to claim 1, wherein: Each of the plurality of in-memory operation units is configured to access a first memory bank of a corresponding group of n memory banks before accessing a second memory bank of a corresponding group of n memory banks to perform in-memory processing of pipelined operations.

3. The memory device according to claim 1, wherein: The memory die also includes: a bank selector configured to: for each group formed of n memory banks, sequentially select a first memory bank and a second memory bank based on a first data transmission distance and a second data transmission distance; and The multiplexer is configured to provide data accessed from the first memory bank or the second memory bank selected by the bank selector to the corresponding in-memory operation unit.

4. The memory device according to claim 1, wherein: The plurality of memory banks include a first set of memory banks arranged on a first side of the memory die and a second set of memory banks arranged on a second side of the memory die, and The multiple in-memory operation units include a first group of in-memory operation units and a second group of in-memory operation units, the first group of in-memory operation units are arranged adjacent to and connected to the first group of memory storage bodies in an area between the first group of memory storage bodies and the second group of memory storage bodies, and the second group of in-memory operation units are arranged adjacent to and connected to the second group of memory storage bodies in the area.

5. The memory device according to claim 4, wherein: The command pad is arranged between the first group of in-memory operation units and the second group of in-memory operation units in the region, and The first memory bank and the second memory bank of each group formed of n memory banks are classified according to the first data transmission distance and the second data transmission distance within the area.

6. The memory device according to claim 4, wherein: The first group of memory banks is classified into two sub-groups including a first sub-group arranged closer to the command pad and a second sub-group arranged farther from the command pad, and The first memory bank of each group of n memory banks is selected from the first subset and the second memory bank of each group of n memory banks is selected from the second subset.

7. The memory device according to claim 4, wherein: A first group of n memory banks among the first group of memory banks is assigned to a first in-memory operation unit among the first group of in-memory operation units, and The first in-memory operation unit processes pipelined operations on data accessed from a first memory bank of a first group formed of the n memory banks before data accessed from a second memory bank of the first group formed of the n memory banks.

8. The memory device according to claim 4, wherein: The memory device is a double data rate 4 dynamic random access memory module, The first group of memory banks includes 8 memory banks, and the second group of memory banks includes 8 memory banks, and The first group of in-memory operation units includes 8 / n in-memory operation units, and the second group of in-memory operation units includes 8 / n in-memory operation units.

9. The memory device according to claim 4, wherein: The first group of memory banks includes a first memory bank, a second memory bank, a third memory bank, a fourth memory bank, a fifth memory bank, a sixth memory bank, a seventh memory bank, and an eighth memory bank in order of a first direction, The first group of in-memory operation units includes a first in-memory operation unit, a second in-memory operation unit, a third in-memory operation unit, and a fourth in-memory operation unit in order of a first direction, and The first memory storage body and the third memory storage body are assigned to the first in-memory operation unit, the second memory storage body and the fourth memory storage body are assigned to the second in-memory operation unit, the fifth memory storage body and the seventh memory storage body are assigned to the third in-memory operation unit, and the sixth memory storage body and the eighth memory storage body are assigned to the fourth in-memory operation unit.

10. The memory device according to claim 4, wherein: The connection between the second group of memory banks and the second group of in-memory operation units is symmetrical to the connection between the first group of memory banks and the first group of in-memory operation units.

11. The memory device according to any one of claims 1 to 10, wherein: Each of the plurality of in-memory operation units includes at least one pipeline register configured to buffer sub-operation results in each pipeline stage of the pipelined operation.

12. The memory device according to any one of claims 1 to 10, wherein: The pipelined operations include the multiply-accumulate operations of the neural network.

13. A memory device for performing in-memory processing, the memory device comprising: a plurality of in-memory operation units configured to perform in-memory processing of operations pipelined into a plurality of pipeline stages; and a plurality of memory banks allocated to the plurality of in-memory operation units, such that a group formed by n memory banks is allocated to each of the plurality of in-memory operation units, each memory bank being configured to perform an access operation of data requested by each of the plurality of in-memory operation units while a pipelined operation is performed, wherein n is an integer greater than or equal to 2, wherein each group formed of n memory banks includes: a first memory bank having a first physical distance for data transmission of a command signal provided from an external source and a second memory bank having a second physical distance for data transmission of the command signal, the second physical distance being greater than the first physical distance, In which, the distance from the first memory storage body of each group formed by n memory storage bodies to the in-memory operation unit to which each group formed by n memory storage bodies is assigned is equal to the distance from the second memory storage body of each group formed by n memory storage bodies to the in-memory operation unit to which each group formed by n memory storage bodies is assigned.

14. The memory device of claim 13, wherein: Each of the plurality of in-memory operation units is configured to access a first memory bank of a corresponding group of n memory banks before a second memory bank of a corresponding group of n memory banks to perform in-memory processing of pipelined operations.

15. The memory device according to claim 13 or 14, further comprising: a memory bank selector configured to: for each group formed by n memory banks, sequentially select a first memory bank and a second memory bank based on a first physical distance and a second physical distance; and The multiplexer is configured to provide data accessed from the first memory bank or the second memory bank selected by the bank selector to the corresponding in-memory operation unit.

16. A memory device comprising: a plurality of in-memory operation units configured to perform in-memory processing of pipelined operations of the neural network; a plurality of memory banks, each memory bank being assigned to one of the plurality of in-memory operation units, such that a group formed of n memory banks is assigned to each of the plurality of in-memory operation units, each memory bank being configured to perform an access operation of data requested by the assigned in-memory operation unit while a pipelined operation is performed, wherein n is an integer greater than or equal to 2; and A command pad, configured to receive a command signal from an external source, wherein each of the plurality of in-memory operation units is configured to: access memory banks from a corresponding assigned group of n memory banks in an order determined based on a distance from each memory bank in the corresponding group of n memory banks to the command pad, wherein each group formed by n memory banks comprises a first memory bank having a first distance to the command pad and a second memory bank having a second distance to the command pad, the second distance being greater than the first distance, In which, the distance from the first memory storage body of each group formed by n memory storage bodies to the in-memory operation unit to which each group formed by n memory storage bodies is assigned is equal to the distance from the second memory storage body of each group formed by n memory storage bodies to the in-memory operation unit to which each group formed by n memory storage bodies is assigned.

17. The memory device of claim 16, wherein: A distance from each memory bank in a corresponding group of n memory banks to a command pad is a data transmission distance.

18. The memory device of claim 17, wherein: Each of the plurality of in-memory operation units is configured to access memory banks from a corresponding allocated group formed of n memory banks in ascending order of data transfer distance.

19. The memory device of claim 16, wherein: The distance from each memory bank in the corresponding group of n memory banks to the command pad is a physical distance.

20. The memory device of claim 19, wherein: Each of the plurality of in-memory operation units is configured to access memory banks from a corresponding allocated group of n memory banks in ascending order of physical distance.

Images