Memory device and operation method performed by same

Abstract

The invention discloses a memory device and an operation method performed by the same. The memory device includes: a first scalar register file storing a first input segment; a second scalar register file storing a second input segment; an arithmetic logic unit (ALU); and a control circuit. The control circuit is configured to perform a first operation between the first input segment and the first weight segment using the ALU based on a first operation command received from the host, and perform a second operation between the second input segment and the second weight segment using the ALU based on a second operation command received from the host.

Description

[0001] This application claims priority from Korean Patent Application No. 10-2023-0142326 filed on October 23, 2023, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein in its entirety for all purposes by reference. Technical Field

[0002] The following description relates to a memory device and operating methods performed thereby. Background Art

[0003] Efficient and high-performance neural network processing is important for devices such as computers, smart phones, tablets, and wearable devices. In some cases, when processing performance increases as the power consumption of the device decreases, a dedicated hardware accelerator for performing dedicated tasks may be implemented for the device. For example, various hardware accelerators may be interconnected to generate computational graphs for applications in natural language processing (NLP), language translation, and text generation. In such a configuration, a subsystem dedicated to accelerating NLP, language translation, and text generation may be established, the subsystem including a network of dedicated hardware accelerators. In these examples, a near memory accelerator may refer to an accelerator implemented by a hardware accelerator near the memory, and in-memory computing may refer to hardware acceleration implemented in the memory. Summary of the invention

[0004] According to an embodiment of the present disclosure, a memory device includes: a first scalar register file storing a first input fragment; a second scalar register file storing a second input fragment; an arithmetic logic unit (ALU); and a control circuit. The control circuit is configured to: based on a first operation command received from a host, use the ALU to perform a first operation between the first input fragment and a first weight fragment, and based on a second operation command received from the host, use the ALU to perform a second operation between the second input fragment and a second weight fragment.

[0005] According to an embodiment of the present disclosure, an operation method performed by a memory device includes: based on a first operation command received from a host, loading a first input fragment from a first scalar register file and loading a first weight fragment from a memory storage body; using an ALU to perform a first operation between the first input fragment and the first weight fragment; based on a second operation command received from the host, loading a second input fragment from a second scalar register file and loading a second weight fragment from the memory storage body; and using the ALU to perform a second operation between the second input fragment and the second weight fragment.

[0006] According to an embodiment of the present disclosure, a computing system includes: a host, configured to send operation commands indicating weight fragments of different memory rows in the same memory storage body without a time corresponding to a barrier; and a memory device, configured to load different input fragments corresponding to the operation commands from multiple scalar register files, and use an ALU to perform operations between the weight fragments and the input fragments.

[0007] According to an embodiment of the present disclosure, a method for performing a processing in memory (PIM) operation includes: generating a scalar register file index and a vector register file index; loading an input value from a scalar register file based on the scalar register file index; performing an operation on the input value and a weight value from a memory storage body; and storing a result of the operation in a vector register file based on the vector register file index. The scalar register file index indicates a scalar register file among a plurality of scalar register files, and the vector register file index indicates a vector register file among a plurality of vector register files.

[0008] Additional aspects of the embodiments will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] These and / or other aspects, features and advantages of the invention will become clear and more easily understood through the following description of the embodiments in conjunction with the accompanying drawings, in which: Figure 1 A computing system according to an embodiment is shown.

[0010] Figure 2 A process-in-memory (PIM) block of a memory device according to an embodiment is shown.

[0011] Figure 3 and Figure 4 An operation method in a memory device according to an embodiment is shown.

[0012] Figure 5 A general matrix-vector multiplication (GEMV) operation is shown processed in a computing system according to an embodiment.

[0013] Figure 6 and Figure 7 A portion of a GEMV operation processed by various memory banks of a memory device according to an embodiment is shown.

[0014] Figures 8 to 13 A portion corresponding to various operation commands of a GEMV operation according to an embodiment is shown.

[0015] Fig.14 and Fig.15The loading of input fragments stored in a scalar register file is shown according to an embodiment.

[0016] Fig.16 Illustrated is the selection of a vector register file according to an embodiment.

[0017] Figures 17 to 20 It is shown that a time corresponding to a barrier is reduced in a memory device using multiple scalar register files and multiple vector register files according to an embodiment.

[0018] Fig.21 A high bandwidth memory (HBM) device according to an embodiment is shown.

[0019] Fig. 22 An operation method in a memory device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0020] The following detailed structural or functional description is provided as an example. Various changes and modifications may be made to the examples. Therefore, the embodiments are not to be interpreted as limited to the disclosure, and should be understood to include all changes, equivalents and substitutes within the disclosed concept and technical scope.

[0021] Although terms such as first, second, etc. are used to describe various components, the components are not limited to these terms. These terms should only be used to distinguish one component from another component. For example, a first component may be referred to as a second component, or similarly, a second component may be referred to as a first component.

[0022] It should be noted that if a first component is described as being “connected,” “coupled,” or “engaged” to a second component, although the first component may be directly connected, coupled, or engaged to the second component, a third component may be “connected,” “coupled,” or “engaged” between the first and second components.

[0023] Unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. It will also be understood that when the terms "comprise" and / or "include" are used herein, it indicates the presence of stated features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0024] As used herein, each of "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "at least one of A, B or C" may include any one or all possible combinations of the items listed together in the corresponding phrase.

[0025] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art to which the present disclosure belongs. Unless otherwise defined herein, terms (such as those defined in commonly used dictionaries) should be interpreted as having a meaning that matches the contextual meaning in the relevant art and should not be interpreted as having an idealized or overly formal meaning.

[0026] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. When describing the embodiments with reference to the accompanying drawings, the same reference numerals denote the same components, and repeated descriptions thereof will be omitted.

[0027] Figure 1 A computing system according to an embodiment is shown.

[0028] The computing system 100 according to an embodiment may include a host 110 and a memory device 120 .

[0029] The host 110 is a management entity of a computer system (e.g., an electronic device) and may be implemented as a host processor or a server. The host processor may include, for example, a host central processing unit (CPU). For example, the host processor may include a processor core and a memory controller. The memory controller may control the memory device 120. The host processor may process data. For example, the processor core may be used to receive data from the memory device 120 through the memory controller. In some embodiments, the memory controller may send a command to the memory device 120. Examples of write commands and operation commands may be described as commands sent to the memory device 120. For example, a write command may be a command to instruct the memory device 120 to write a register file of input values ​​for a target operation (e.g., a multiply-accumulate (MAC) operation). The operation command may indicate a matrix product operation (such as a general matrix-vector multiplication (GEMV) operation). The operation involves processing a fragment of a partial weight matrix (or, alternatively, a weight vector) in conjunction with an input vector. In some embodiments, the processing in memory (PIM) block 121 of the memory device 120 may perform a product operation (multiplication operation) between the input value and the weight value and an addition operation of the partial operation results based on the operation command.

[0030] PIM refers to a block within a memory device that enables data processing directly within the memory architecture, thus reducing the need to move data between the memory and a separate processing unit. PIM blocks can be used to reduce power consumption and improve the efficiency of data processing. PIM blocks may include components such as control circuits, memory bank interfaces, ALUs (arithmetic logic units), and register files.

[0031] The ALU performs arithmetic and logic operations required for data processing. This includes basic arithmetic operations such as addition and subtraction and logic operations such as XOR, AND, and OR. For example, the ALU performs computational tasks within the system and operates under the control of a control device.

[0032] The operation command may be a dynamic random access memory (DRAM) command provided to the memory device 120 set in the PIM all-bank mode. A DRAM RD command of the DRAM command may correspond to the operation command, and a DRAM WR command of the DRAM command may correspond to a write command. Each operation command may include information indicating an internal address of the memory device 120 (e.g., address information).

[0033] The memory device 120 includes a designated memory area for data storage. The memory area can facilitate both reading data from the memory device 120 and writing data to the memory device 120. The memory area can be a physical area within a memory chip. In some embodiments, the memory area can be set in a memory die (or core die) of the memory device 120. The memory device 120 can process data from the memory area in cooperation with the host processor. For example, the memory device 120 can perform data operations or processing tasks based on commands or instructions received from the host processor. The memory device 120 can control the memory area in response to commands or instructions of the host processor. The memory device 120 can be separated from the host processor. For example, the host processor can supervise one or more operations and delegate specific operations that benefit from acceleration (such as PIM operations) to the memory device 120.

[0034] The memory device 120 may include a PIM block 121 and a memory (e.g., a plurality of memory banks 122). For example, the memory device 120 may use the data stored in the memory through the plurality of PIM blocks 121 to perform a target operation. The target operation may be, for example, a GEMV operation. For example, the input data and output data of the GEMV operation may be data in a vector form, and the weight data may be data in a matrix form. Since the weight data has a capacity (e.g., a memory size) greater than the cache size of the host 110, the weight data may be stored in the memory device 120 (e.g., a DRAM device). As will be described based on Figure 5 As described in more detail, the GEMV operation can be, for example, an operation between a weight matrix and an input vector (e.g., a matrix-vector product operation). For example, the input vector can be divided into a plurality of partial input vectors, and each partial input vector can include a plurality of input fragments (e.g., scalar values).

[0035] The memory can store data. One or more memory chips of the memory device 120 can be used to generate multiple memory banks 122. Each memory bank 122 in the multiple memory banks 122 can store a selection of values ​​(such as weight fragments). In some examples, the weight fragment is a portion of the full weight matrix and is utilized in the target operation. For example, the weight fragment can be data in the form of a vector. For example, the target operation can be decomposed into multiple partial operations, and each memory bank can store some data to be included in the partial operation. For example, the GEMV operation can be decomposed into multiple MAC operations. The memory device 120 and / or the host 110 can divide the weight matrix of the target GEMV operation into multiple partial weight matrices, and store the multiple partial weight matrices in multiple memory banks 122. Each memory bank may include multiple storage cells for storing values ​​in a memory array on a memory die of the memory device 120. Multiple storage cells may be arranged along row lines and column lines. A portion including storage cells arranged along any row line in a memory bank may be referred to as a memory row. A memory row may be a group of storage cells arranged along the same row line. Similarly, a memory column may be a group of memory cells arranged along the same column line.

[0036] The PIM block 121 may perform data operations or processing tasks using data stored in the memory according to the operation command. Each PIM block 121 may perform some operations among a plurality of partial operations included in the target operation (e.g., GEMV operation). For example, the PIM block 121 may access a memory bank disposed adjacent to the PIM block 121 itself among a plurality of memory banks 122 of the memory device 120. The PIM block 121 may obtain data of a portion of the accessible memory bank corresponding to the operation command. The PIM block 121 may perform partial operations using the obtained data. For example, the PIM block 121 may perform partial operations using a weight segment corresponding to the operation command in a partial weight matrix stored in a memory bank accessible to the PIM block 121. The weight segment may be data including weights in the form of a vector, for example. The PIM block 121 may obtain a weight segment and an input segment specified by the operation command, and then perform a multiplication operation, an addition operation, or both operations using the obtained data. In some embodiments, the input segment may be a scalar value.

[0037] Figure 1An example in which the PIM block 121 is allocated to a memory bank in a core die is shown, but the embodiment is not limited thereto. The PIM block 121 may be disposed adjacent to a plurality of memory banks 122 (e.g., two memory banks), and the PIM block 121 may access a plurality of memory banks. The PIM block 121 may determine the number of MAC units to be used for a partial operation based on the size of a weight fragment corresponding to one operation command (e.g., the number of weight values ​​included in the weight fragment). The PIM block 121 may use a combination of a plurality of arithmetic logic units (ALUs) (e.g., multipliers and adders) to form a determined number of MAC units. In some embodiments, the PIM block 121 may load a weight fragment from a memory bank and load an input fragment from a register file (e.g., a scalar register file (SRF)). The SRF is part of a memory architecture and may be used to hold data in scalar form. Data in scalar form has a single or singular data value. For example, the SRF may be used to store a scalar value or an input fragment. For example, the SRF may store different fragments of input data, wherein each scalar register file is capable of storing a plurality of such fragments. The PIM block 121 may perform data manipulation or processing tasks between the loaded weight fragments and the input fragments using the above-described MAC units.

[0038] The instructions may include instructions for executing operations of a host processor, a processor in a memory device 120 or various devices and / or operations of each component of the processor. For example, instructions (or programs) executable by a host processor may be stored in another memory device 120, but the embodiment is not limited thereto. In some embodiments, the instructions may include a barrier instruction (also referred to as a "fence instruction" or "barrier instruction"). The barrier instruction may be an instruction for executing an operation of disabling the host 110 or a resource during a time corresponding to the barrier (e.g., a barrier time). For example, a barrier instruction for a specified PIM block 121 may be an instruction for performing the following operations: holding, waiting, and / or limiting the sending of a command (e.g., a write command or an operation command) for a specified PIM block 121 from the host 110. In some embodiments, due to constraints such as the small size of a buffer (e.g., a row buffer) in each memory bank and the limited use of a specific memory address (including a row address), a fencing operation may be involved. The barrier operation consumes a long cycle, and a method and structure for reducing the barrier operation of the PIM block 121 will be described herein. For example, the host 110 may send an operation command indicating a weight segment of a different memory row in the same memory bank without a time corresponding to a barrier.

[0039] The memory device 120 (e.g., a PIM device) including the PIM block 121 can perform data operations or processing tasks to accelerate applications (e.g., machine learning and big data) that require large memory bandwidth. The memory device 120 can perform operations by accessing multiple memory banks 122 in parallel through multiple PIM blocks 121. The memory device 120 can perform operations using an internal memory bandwidth that is 4 to 8 times higher than the external memory bandwidth. Therefore, the memory device 120 can significantly reduce the execution time of applications with memory-intensive characteristics. In some embodiments, since the memory device 120 only moves data between the PIM block 121 and the memory bank 122, power consumption can also be reduced.

[0040] Figure 2 2 shows a PIM block of a memory device according to an embodiment. The PIM block 210 may include a control circuit 211, a memory interface 230, an ALU 217, and a register file. In some embodiments, the memory interface 230 may be provided outside the PIM block 210, but is not limited thereto. In some embodiments, the PIM block 210 may be provided adjacent to a corresponding memory bank 220 among a plurality of memory banks of the memory device 200. The PIM block 210 may access the adjacent memory bank 220 through the memory interface 230. The memory bank 220 may store weight segments. The weight segments may be data in the form of vectors.

[0041] The control circuit 211 may receive an operation command for a partial operation from a host. In some embodiments, the operation command may include address information (such as a row address and a column address where a value (e.g., a weight) is stored) of a specific location within the memory storage body 220 of the memory device 200. The row address information is information indicating a memory row address, and may include the number of a memory row corresponding to the location where the value in the memory array is stored (e.g., a row index). The column address information is information indicating a memory column address, and may include the number of a memory column corresponding to the location where the value in the memory array is stored (e.g., a column index). The control circuit 211 may load a value (e.g., a weight fragment) of a memory address (e.g., a DRAM address) indicated by the received operation command. In addition, based on the received operation command, the control circuit 211 may identify a register file (e.g., an SRF) storing input fragments in the PIM block 210, and identify a register file (e.g., a vector register file (VRF)) storing the result of the partial operation corresponding to the operation command. The VRF may be used to store vector values ​​or the result of the partial operation. For example, when a series of operation commands instruct the same VRF, the VRF accumulates the results of these operations (such as the results of arithmetic operations). According to some embodiments, the system may have multiple VRFs, each of which can store the results of these operations. The control circuit 211 may determine information (such as an index) identifying a specific register file among the register files that holds the values ​​for the partial operation. The determination is based on the memory address specified by the operation command, including both the memory row address and the memory column address.

[0042] In some embodiments, the control circuit 211 may include a buffer (eg, a trigger buffer) and Figures 14 to 16 The following will be based on Figures 14 to 16 An operation of selecting a register file to be included in a partial operation from an operation command is described.

[0043] The register file may be an element including a logic circuit (eg, a digital logic circuit) that implements a storage function. The register file may include a scalar register file 213 for storing scalar values ​​and a vector register file 215 for storing vector values.

[0044] The scalar register file 213 may store input fragments. For example, the first scalar register file SRF 0 may store a first input fragment. The second scalar register file SRF 1 may store a second input fragment. In some embodiments, the input fragment may be an input scalar value, and each scalar register file may store multiple input fragments. For example, an input vector included in a GEMV operation may include multiple partial input vectors, and each partial input vector may include multiple input fragments (e.g., input scalar values). Figure 2N scalar register files 213 are shown (eg, a first scalar register file SRF 0 to an Nth scalar register file SRF N−1). N may be an integer greater than or equal to 2.

[0045] The vector register file 215 may store the results of partial operations. For example, when multiple operation commands indicate the same vector register file among the vector register files 215, the result generated by adding the results of partial operations associated with a specific operation command (such as multiplication results) may be stored in the corresponding vector register file. Figure 2 P vector register files 215 are shown (eg, a first vector register file VRF 0 to a Pth vector register file VRF P-1). P may be an integer greater than or equal to 2.

[0046] The ALU 217 may perform arithmetic logic operations required in part of the operations under the control of the control device. For example, the ALU 217 may be a digital circuit that calculates arithmetic operations including addition and subtraction and logic operations including XOR, AND, and OR. However, the embodiment is not limited thereto, and the ALU 217 may also include a combination of digital circuits having functions of MAC operation and / or modulo operation.

[0047] According to an embodiment, a partial operation (e.g., a memory bank partial operation) of a GEMV operation corresponding to a memory bank 220 may be allocated to a PIM block 210 of a memory device 200. This is because the PIM block 210 can only access adjacent memory banks 220 in the memory device 200. However, the amount of data that can be loaded via a single operation command (i.e., the data size (such as a read size)) may have a limit. Specifically, the data size may be smaller than both the overall size of the memory bank 220 and the size of a single memory row. Due to such a data size limit, a memory bank partial operation (or an operation corresponding to a memory row) may be divided into a plurality of operation commands, and the control circuit 211 may perform a plurality of partial operations. For example, a host may send operation commands corresponding to weight segments for a portion of a GEMV operation to the memory device 200. The PIM block 210 may receive operation commands corresponding to a plurality of weight segments corresponding to the allocated memory bank partial operation. The PIM block 210 may perform partial operations corresponding to the received operation commands. For example, the control circuit 211 may perform a data operation or processing task between a first input segment and a first weight segment using the ALU 217 based on a first operation command received from the host. The control circuit 211 may perform a data operation or processing task between a second input segment and a second weight segment using the ALU 217 based on a second operation command received from the host. The first operation command and the second operation command are different operation commands and may include address information indicating a memory address (e.g., a DRAM address) at which different weight segments are stored.

[0048] In some embodiments, based on an operation command corresponding to any partial operation, the control circuit 211 may determine: a memory address where a weight fragment included in the corresponding partial operation is stored, a scalar register file storing an input fragment included in the corresponding partial operation, and a vector register file where a result of the corresponding partial operation (e.g., a partial operation result) is to be stored. Figure 3 and Figure 4 Describe the mapping relationship between these three pieces of information (eg, the memory address where the weights are stored, the index of the scalar register file, and the index of the vector register file).

[0049] In some embodiments, the memory device 200 may load different input fragments corresponding to the operation command from multiple scalar register files 213. The PIM block 210 may also obtain the weight fragment by accessing the portion of the memory storage body 220 corresponding to the operation command. The memory device 200 may use the ALU 217 to perform operations between the weight fragments and the input fragments respectively. For example, the memory device 200 may perform operations one-to-one between each weight fragment and the input fragment corresponding thereto. For example, the memory device 200 may perform a multiplication operation between the weight fragment and the corresponding input fragment for each operation command. The memory device 200 may accumulate (e.g., sum) the results of the multiplication operation in the corresponding vector register file.

[0050] Figure 3 and Figure 4 An operation method in a memory device according to an embodiment is shown.

[0051] When a request is sent from the host 420 to a memory device (e.g., a DRAM device), the order of the request may be changed by the host 420 and the memory controller. The memory device may perform a data operation or processing task based on the address of the request (e.g., a memory address), regardless of the order of the request received from the host 420. The memory device may use an address alignment mode (AAM, address aligned mode) method. In a memory device using AAM, the index of the scalar register file for input and the index of the vector register file for output may be specified based on a memory address map. The indexes of both the scalar register file and the vector register file are determined by the address information of the memory device, regardless of the order in which the request is made for the memory command (such as the DRAM request order). The memory address map may be a map of addresses in a memory device (e.g., a memory bank 430) where values ​​are stored. The mapping between the memory address of the memory address map, the index of the scalar register file, and the index of the vector register file may be predefined. The control circuit 411 may include a circuit (e.g., a logic circuit) configured to generate an index of the register file from the memory address information of the operation command according to the above mapping.

[0052] For example, any weight fragment in the weight matrix for GEMV operation can be stored at any memory address in the memory device. Any scalar register file and any vector register file can correspond to the corresponding memory address. In some embodiments, the control circuit 411 can be configured to generate the index of the corresponding scalar register file and the index of the corresponding vector register file from the corresponding memory address. It can be explained that the index of the scalar register file in which the input fragment will be stored and the index of the vector register file in which the result of the partial operation will be stored are pre-specified based on the memory address, where the weight fragment of the partial operation for GEMV operation is stored in the memory address. Therefore, the control circuit 411 can use the operation command to access the weight fragment, and identify the "index of the scalar register file in which the input fragment to be included in the corresponding partial operation is stored" and "the index of the vector register file in which the result of the partial operation will be stored" from the operation command.

[0053] The computing system according to an embodiment may unload the input values ​​to be included in the partial operation to the memory device before executing the operation. Before the operation command, the host 420 may send a write command "for writing a partial input vector as part of the input vector to the PIM block 410 of the memory device". The write command may include information indicating the index (e.g., SRF index) of the scalar register file. For example, in a general matrix-vector multiplication (GEMV) operation, the input fragment that needs to be multiplied with the corresponding weight fragment may be stored in the scalar register file. The scalar register file is mapped to a memory address aligned with the position of the corresponding weight fragment. Therefore, the SRF index of the write command can be predetermined based on the memory address information of the operation command. For example, the write command includes information directly indicating the SRF index, and may be different from the memory address (e.g., DRAM address) of the memory device. According to some embodiments, the write operation writes a partial input vector to the scalar register file at a time, and only the SRF index is required in the write operation, and the column index indicating the value of each scalar element may be unnecessary.

[0054] First, the PIM block 410 of the memory device may store a partial input vector in a corresponding scalar register file in response to a write command (eg, a WR command). Figure 4 An example is shown in which the control circuit 411 of the PIM block 410 receives two WR commands from the host 420. The control circuit 411 may receive a first write command and a second write command. The control circuit 411 may perform write operations WRPB0 and WRPB1 in response to the write commands. The control circuit 411 may write the first partial input vector to a first scalar register file SRF 0 corresponding to the first write command among a plurality of scalar register files in response to receiving a first write command for writing a first partial input vector of the input vector. The first partial input vector may be a vector having a plurality of input fragments including a first input fragment (e.g., an input scalar value). The control circuit 411 may write the second partial input vector to a second scalar register file SRF 1 corresponding to the second write command among a plurality of scalar register files in response to receiving a second write command for a second partial input vector of the input vector. The second partial input vector may be a vector having a plurality of input fragments including a second input fragment (e.g., an input scalar value).

[0055] The first scalar register file SRF 0 may store the first input fragment before the first operation command and the second operation command are received. The second scalar register file SRF 1 may store the second input fragment before the first operation command and the second operation command are received. Therefore, the PIM block 410 may store partial input vectors in the scalar register file in advance and prepare them for subsequent partial operations across multiple memory rows in the memory bank 430. According to some embodiments, the computing system stores partial input vectors in advance before the partial operation, and multiple partial input vectors may be stored in the scalar register file using only one barrier operation 429.

[0056] In operation 310, the memory device may load a first input fragment from a first scalar register file SRF 0 and a first weight fragment from a memory bank 430 based on a first operation command (e.g., an RD command) received from a host 420. In some embodiments, the control circuit 411 of the PIM block 410 may obtain the first weight fragment by accessing a memory row Row 0 of the memory bank 430 using the first operation command. The control circuit 411 may generate an index indicating a first scalar register file SRF 0 storing a first input fragment to be included in a corresponding partial operation from the first operation command. The control circuit 411 may select a first scalar register file SRF 0 corresponding to the generated index, and obtain an input fragment (e.g., an input scalar element) corresponding to the first operation command from the selected first scalar register file SRF 0.

[0057] In operation 320, the memory device may use the ALU to perform a data operation or processing task between the first input fragment and the first weight fragment. The control circuit 411 of the PIM block 410 may use the ALU to perform a multiplication operation between the first input fragment and the first weight fragment. Since the first input fragment is a value in scalar form and the first weight fragment is data in vector form, the multiplication result may be in vector form. The control circuit 411 performs element-by-element multiplication by multiplying the first input fragment by each weight in the first weight fragment. Therefore, the result of this partial operation (including the multiplication result) maintains the same vector form (e.g., the same vector dimension) as the first weight fragment.

[0058] The control circuit 411 may identify a first vector register file VRF 0 for storing a result of a partial operation derived from a first operation command. The control circuit 411 may generate an index (e.g., a VRF index) indicating one of a plurality of vector register files from the first operation command. The control circuit 411 may select a first vector register file VRF 0 corresponding to the generated VRF index. The control circuit 411 may store a result of an operation (e.g., a multiplication operation) between the first input fragment and the first weight fragment in the selected first vector register file VRF 0.

[0059] For example, the weight of one of the memory rows, Row 0, may be stored in the row buffer due to limitations (e.g., size limitations) of the row buffer. Therefore, in order to perform a data operation or a processing task using weights stored in different memory rows, after the operation in one memory row, Row 0, is completed, the buffer row may be initialized for the operation of another memory row. Figure 7 In some embodiments, in a comparative embodiment where the PIM unit uses a single vector register file, when the memory row changes, the operation results using the values ​​of different memory rows may not be distinguished and stored. In some embodiments, the memory device according to the embodiment may select a vector register file in which a partial operation result will be stored based on both the memory column address and the memory row address of the operation command. Therefore, even if there is no time corresponding to the barrier operation 429 between the above-mentioned operation 320 and the operation 330 to be described below, the memory device may store the results of the partial operation in the vector register file by dividing the results for each of the multiple operation commands.

[0060] In operation 330, the memory device may load the second input fragment from the second scalar register file SRF 1 and the second weight fragment from the memory bank 430 based on the second operation command (e.g., RD ​​command) received from the host 420. Similarly, the control circuit 411 of the PIM block 410 may obtain the second weight fragment by accessing the memory bank 430 using the second operation command. The control circuit 411 generates an index indicating the second scalar register file SRF1 as the storage location of the second input fragment. Then, the index is used to perform a partial operation based on the second operation command, thereby allowing the control circuit to exclusively access and utilize the second input fragment from SRF 1 for the operation. The control circuit 411 may select the second scalar register file SRF 1 corresponding to the generated index, and obtain the input fragment (e.g., input scalar element) corresponding to the second operation command from the selected second scalar register file SRF 1.

[0061] In operation 340, the memory device may use the ALU to perform a data operation or processing task between the second input segment and the second weight segment. The control circuit 411 of the PIM block 410 may use the ALU to perform a multiplication operation between the second input segment and the second weight segment. In some embodiments, the second input segment is in scalar form, and the second weight segment is in vector form, and the result of the multiplication may be in vector form. The control circuit 411 may perform element-by-element multiplication by multiplying the second input segment by each weight of the second weight segment. The result of the partial operation (e.g., the multiplication result) may have a vector form that is the same as the vector form (e.g., vector dimension) of the second weight segment.

[0062] The control circuit 411 may identify a vector register file in which a result of a partial operation according to the second operation command is to be stored. The control circuit 411 may generate an index (eg, a VRF index) indicating one of the plurality of vector register files from the second operation command. Figure 8 When showing an example of a VRF index corresponding to the first operation command, the following Fig. 9 Can be as Figure 8 An example of a VRF index corresponding to the second operation command is shown as shown in Figures 11 to 13 An example is shown in which the VRF index is different for each of the multiple operation commands. The control circuit 411 may select a vector register file corresponding to the generated VRF index. The control circuit 411 may store the result of the operation (such as multiplication) between the second input fragment and the second weight fragment in the selected vector register file. When the VRF index corresponding to the first operation command and the VRF index corresponding to the second operation command are the same, the control circuit 411 may add a partial operation result based on the second operation command to the first vector register file VRF 0 corresponding to the first operation command.

[0063] Figure 5 GEMV operations processed in a computing system according to an embodiment are shown.

[0064] The computing system according to the embodiment may perform GEMV operation as the target operation. Figure 5 As shown in , the GEMV operation can be a matrix product operation between a matrix and a vector. For example, the weight matrix 510 can be a matrix of M×K dimensions, the input vector 520 can be a vector of K×1 dimensions, and the output vector 530 can be a vector of M×1 dimensions. M and K are both integers of 1 or greater, where K represents the input dimension and M represents the output dimension. For example, here, the dimension can be represented as the number of elements.

[0065] Figure 6 and Figure 7Portions of a GEMV operation handled by various memory banks of a memory device are shown according to an embodiment.

[0066] In one example, M=2048, K=2048, and the memory size occupied by each element of the matrix and the vector is 2 bytes. In this example, the input vector 620 and the output vector 630 can be vectors of 2048×1 dimensions. The memory size occupied by the input vector 620 and the output vector 630 can be 4096 bytes. The weight matrix can be a matrix of 2048×2048 dimensions. In the following, the size and dimension of the vector and matrix can be represented based on the number of elements. For example, the read size read by a single operation command can be 32 bytes. Since each element value is represented by 2 bytes, 16 elements (or values) can be read by one operation command.

[0067] Figure 6 Shown based on Figure 5 The above weight matrix is ​​divided into partial weight matrices.

[0068] Each partial weight matrix 611 may be stored in a memory bank. Figure 6 As shown in , the weight matrix can be divided into partial weight matrices corresponding to the size of the memory storage body along the column direction of the matrix. Each partial weight matrix 611 can be a group of vectors (e.g., row vectors) along the row direction in the matrix. For example, the rows of the memory can be referred to as memory rows, the columns of the memory can be referred to as memory columns, the rows of the matrix can be referred to as matrix rows, and the columns of the matrix can be referred to as matrix columns. Although the partial weight matrix 611 is described as a group of row vectors of the matrix, the partial weight matrix in the embodiments of the present disclosure is not limited thereto, and the weight matrix can be divided in various ways based on the needs of the system.

[0069] Figure 6An example of using 16 memory banks (e.g., the first bank Bank 0 to the sixteenth bank Bank 15) to divide the weight matrix into 16 partial weight matrices is shown. The partial weight matrix 611 may be a matrix of M / 16×K=128×2048 dimensions. The partial weight matrix 611 may be multiplied by the input vector 620. The input vector 620 may be divided and stored in N scalar register files. Each scalar register file may store a partial input vector as a portion of the input vector 620. The result of the operation performed between the partial weight matrix 611 and the input vector 620 may be stored in a specific section of the various vector register files from the first vector register file VRF 0 to the Pth vector register file VRF P-1. The result of the operation between the partial weight matrix 611 and the input vector 620 may be a vector corresponding to a portion of the output vector 630. The result of the multiplication between the partial weight matrix 611 and the input vector 620 may be a vector having a dimension of M / 16×1=128×1. Each vector register file can store as much data as the read size (e.g., 32 bytes) corresponding to the operation command. Figure 6 In the example, since the multiplication result is a vector with M / 16×1 dimensions, eight vector register files (e.g., (M / 16) / 16=128 / 16=8) can be allocated to the memory banks. For example, the partial operation result corresponding to the first memory bank Bank 0 is stored in eight vector register files (e.g., the first vector register file VRF 0 to the eighth vector register file VRF 7).

[0070] Figure 7 Shown based on Figure 5 The above weight matrix is ​​divided into partial weight matrices.

[0071] According to an embodiment, memory bank 710 may include a plurality of memory rows. Figure 7L memory rows are included (e.g., Row 0, Row 1, Row 2, ..., Row L-1). L may be an integer greater than or equal to 2. The partial weight matrix corresponding to the memory bank 710 may be divided into partial matrices corresponding to the memory rows. In the partial weight matrix, the size or shape of the partial matrix responsible for the memory row 711 may vary based on the needs in the system, and the dimension of the partial matrix may be a multiple of the size corresponding to the operation command (e.g., 16 elements corresponding to 32 bytes) in the row direction or column direction. For example, the column dimension of the partial matrix may be the same as the size of the scalar register file 720. The size of the partial matrix may be the same as the size of the memory row 711 (or the size of the row buffer 719). When the memory row 711 can store 1024 elements (e.g., 2048 bytes) and the scalar register file 720 can store 16 element values ​​(e.g., the number of elements corresponding to 32 bytes), the partial matrix may be a matrix of 64×16 dimensions. This is because: Since the number of columns is 16, the number of rows needs to be 1024 / 16=64.

[0072] In an embodiment of the present disclosure, for better understanding, the memory banks 710 and memory rows 711 may be shown based on the position of the weight fragments in the weight matrix of the GEMV operation. For example, each memory bank 710 may store a partial weight matrix of the weight matrix of the target GEMV operation. The memory rows in each memory bank 710 may separately store the partial matrices forming the corresponding partial weight matrix. Each memory row 711 may be shown at a position corresponding to the partial matrix within the partial weight matrix. Therefore, the memory banks 710 and memory rows 711 may be shown differently from the actual physical arrangement or size in the memory device.

[0073] The memory bank 710 may include a buffer (eg, a read buffer) that holds data read in response to a command (eg, a DRAM command). In an embodiment of the present disclosure, an example in which the read buffer is a row buffer 719 will be described. Figure 7 A row buffer 719 is shown as an example of a buffer (e.g., a read buffer). The row buffer 719 may have a size corresponding to a single memory row 711. For example, the row buffer 719 is designed to store and save values ​​from memory cells belonging to a specific memory row (memory row 711) within the memory array. In some examples, the row buffer 719 may simultaneously hold values ​​read from memory cells of the same memory row (memory row 711). For example, when an operation command indicates a weight fragment having 16 elements and the memory row 711 stores 1024 elements, the PIM block may receive and process 64 operation commands to load the weight fragment of the first memory row 711 through the row buffer 719.

[0074] For example, in an embodiment of the present disclosure, an example of processing a memory bank 710 by dividing the memory bank 710 into memory rows is described, but the embodiment is not limited thereto. The memory bank 710 may be divided into memory columns according to the design, and a column buffer may be used to store and maintain the value of each memory column. In some embodiments, the storage cells of the memory bank 710 may be grouped into a plurality of memory groups based on the size of the read buffer. There is no limitation on the size of the read buffer or the grouping method of the memory bank. Therefore, unless described to the contrary, the description of the memory row 711 of the memory device in the embodiment of the present disclosure may also be applied to a memory column or a memory group grouped by another method.

[0075] Figures 8 to 13 A portion corresponding to various operation commands of a GEMV operation according to an embodiment is shown.

[0076] Figure 8 Display any operation command instructions Figure 7 Example of a weight fragment in a memory row of .

[0077] According to an embodiment, the control circuit may obtain a weight fragment (e.g., a first weight fragment 811) stored in the first memory row Row 0 by accessing the first memory row Row 0 of the memory storage body based on the first operation command. As shown in the figure, the first memory row Row 0 may be divided into units of the read size of the operation command. The first memory row Row 0 may store a partial matrix of 64×16 dimensions, and the partial matrix may be divided into 64 weight fragments. Each weight fragment may be a vector of 16×1 dimensions. For example, for intuitive understanding, all weight fragments are shown in the row buffer 810. The control circuit may perform data operations or processing tasks according to the first operation command, regardless of whether other weight fragments in the first memory row Row 0 are loaded. For example, once the control circuit receives the corresponding weight fragment from the first memory row Row 0 through the row buffer 810 by requesting the first operation command from the memory storage body, the control circuit may immediately perform a partial operation corresponding to the first operation command.

[0078] The control circuitry may use the first operation command to select the first scalar register file SRF 0 and load a first input fragment 821 (eg, a scalar element value) from the selected first scalar register file SRF 0 .

[0079] The control circuit may generate the first partial operation result 880 by performing a product operation between the first input segment 821 and the weight value of the first weight segment 811 using the ALU. The MAC unit 870 may be formed by a combination of ALUs. Figure 8An example is shown in which 16 MAC units (e.g., MAC0, MAC1 to MAC15) are formed. Each MAC unit may multiply the first input fragment 821 by the corresponding weight value in the first weight fragment 811. The first partial operation result 880 may be a vector of 16×1 dimensions. The control circuit may select the first vector register file VRF 0 from the plurality of vector register files 890 based on the first operation command. The control circuit may store the first partial operation result 880 in the selected first vector register file VRF 0. For example, when there is a value already stored in the first vector register file VRF 0, the control circuit may use the MAC unit 870 to sum the values ​​of the respective elements. For example, the control circuit may update the value stored in the first vector register file VRF 0 by summing the first partial operation result 880 as a vector of 16×1 dimensions with the vector of 16×1 dimensions stored in the first vector register file VRF 0.

[0080] In the following Fig. 9 In order to describe more clearly, the operation of the second memory row Row 2 is described, but the order of description has nothing to do with the operation order of the memory device. Figure 8 , Fig.10 , Fig.11 and Fig.12 As described in , the memory device may perform a data operation or processing task on the first memory row Row 0. When the operation on the first memory row Row 0 is completed, the memory device may similarly perform a data operation or processing task on the next memory row Row 1. Then, the memory device may perform a data operation or processing task on the second memory row Row 2. However, in the embodiments of the present disclosure, the memory device is not limited to processing operations in the order of memory rows. For example, the following Fig. 20 The memory device is shown to perform operations in the order of Row 0, Row 1, Row 2, and Row 3, but the memory device may perform operations in the order of Row 0, Row 2, Row 1, and Row 3. Depending on the design, the memory device may perform operations while changing the memory rows in a different order (e.g., randomly), or perform operations of other memory rows before completing operations of any memory row.

[0081] Fig. 9 The second operation command indicates Figure 8 An example of a weight fragment of another memory row that is adjacent to a memory row included in the operation.

[0082] The control circuit may obtain the weight fragment (e.g., the second weight fragment 911) stored in the second memory row Row 2 of the memory storage body by accessing the second memory row Row 2 based on the second operation command (e.g., through the row buffer 910). The second operation command may indicate the first vector register file VRF 0 that is the same as the first vector register file VRF 0 of the first operation command. However, the second operation command may indicate a second scalar register file SRF 1 that is different from the first scalar register file SRF 0 corresponding to the first operation command. In some embodiments, since a partial input vector (e.g., the second partial input vector) is also stored in the second scalar register file SRF 1 before the operation, the control circuit may obtain the second input fragment 921 from the second scalar register file SRF 1 in response to the second operation command without a time corresponding to the barrier. Only the values ​​included in the operation (e.g., the weight fragment and the input fragment in the scalar register file) are different, and the operation processing may be the same as Figure 8 Same or similar.

[0083] The control circuit may use the MAC unit 970 to perform data operations or processing tasks between the second input segment 921 and the second weight segment 911. The second partial operation result 980 may be generated by performing a product operation between the second input segment 921 and the weight value of the second weight segment 911.

[0084] According to an embodiment, when the vector register file identified based on the first operation command is the same as the vector register file identified based on the second operation command among the plurality of vector register files, the control circuit may update the value stored in the identified vector register file based on the “partial operation result 880 based on the first operation command and the partial operation result 980 based on the second operation command”. For example, the control circuit may first store the partial operation result 880 in the vector register file, then load the partial operation result 880 from the vector register file, and store the “result of merging the loaded partial operation result 880 and the partial operation result 980” in the identified vector register file (e.g., the first vector register file VRF 0). The process of merging the partial operation result 980 generally involves adding the newly calculated partial operation result 980 to the existing partial result already stored in the vector register file (such as performing vector summation). For example, when the first operation command and the second operation command indicate the same vector register file (e.g., the first vector register file VRF 0), the control circuit may store the result of merging the first partial operation result and the second partial operation result in the same vector register file. The first partial operation result, the second partial operation result, and the sum result thereof may have the same vector form (eg, a vector of 16×1 dimension).

[0085] Fig.10The third operation command indicates that Figure 8 Example of operating on the same vector register file in a memory row.

[0086] The third operation command may indicate a weight segment 1011 different from the weight segment 811 indicated by the first operation command in the memory row to which the weight segment 811 corresponding to the first operation command belongs. The control circuit may obtain the weight segment 1011 different from the weight segment 811 corresponding to the first operation command from the same memory row, and obtain the input segment 1021 different from the input segment 821 of the first operation command from the first scalar register file SRF 0. The processing may be based on the instruction of the third operation command. In the process based on Figure 8 and Fig. 9 In some embodiments, the partial operation result 1080 corresponding to the third operation command may be generated by the MAC unit 1070 performing the operation in the same manner.

[0087] When the first operation command and the third operation command indicate the same vector register file (for example, the first vector register file VRF 0), the control circuit may merge the partial operation result corresponding to the first operation command and the partial operation result corresponding to the third operation command into the same vector register file. Fig.10 In some embodiments, the merging of partial operation results may be performed by summing vectors of the same form. Each MAC unit may sum the corresponding element values.

[0088] Fig.11 and Fig.12 The fourth operation command indicates that Figure 8 Example of operating on different vector register files in memory rows.

[0089] The fourth operation command may indicate weight segments 1111 and 1211 different from the weight segment 811 indicated by the first operation command in the memory row to which the weight segment 811 corresponding to the first operation command belongs. Based on the fourth operation command, the control circuit may obtain input segments 1121 and 1221. The control circuit may obtain corresponding weight segments 1111 and 1211 based on the fourth operation command. The MAC units 1170 and 1270 may be used in the same manner as described above. Figure 8 and Fig. 9 The operation is performed in the same manner as described to generate partial operation results 1180 and 1280 corresponding to the fourth operation command.

[0090] In an example where the first operation command and the fourth operation command point to different vector register files, the control circuitry may be used to independently handle each operation. Specifically, the control circuitry may merge a portion of the operation result from the first operation command and a portion of the operation result from the fourth operation command into their respective vector register files. For example, the separate merge process ensures that the result corresponding to each command is accurately and differently merged into its designated vector register file. For example, in Fig.11 In the example shown in , the fourth operation command may indicate a different vector register file VRF 1. The control circuit may merge (eg, vector sum) the partial operation results 1180 in the different vector register file VRF 1.

[0091] Fig.12 Shown based on Fig.11 The fourth operation command is different from another fourth operation command. For example, Fig.12 In the example shown in , the fourth operation command may point to another vector register file VRF 3. The control circuit may merge the partial operation result 1280 into another vector register file VRF 3.

[0092] The processing of operation commands for weight fragments in a memory row can be based on Figure 8 and Figures 10 to 12 Describe the method to perform.

[0093] Fig.13 The fifth operation command indicates Figure 8 An example of a memory row weight fragment including different memory row weights in an operation.

[0094] The control circuit of the PIM block can also be based on Figures 8 to 12 The same or similar manner as described above performs data operations or processing tasks indicating operation commands of different memory rows. Fig. 9 The second operation command described indicates a scalar register file different from the scalar register file of the first operation command, however, Fig.13 The fifth operation command shown in may indicate the same scalar register file as that of the first operation command (eg, the first scalar register file SRF0).

[0095] Based on the fifth operation command, the control circuit may load the input fragment 1321 from the first scalar register file SRF 0 and load the weight fragment 1311 from the memory row Row 1. The control circuit may use the MAC unit 1370 to generate a partial operation result 1380 by performing an operation (such as a product operation) between the input fragment 1321 and the weight fragment 1311. The control circuit may sum the partial operation results 1380 and store the summed result in the vector register file VRF 4 indicated by the fifth operation command. For example, by Figures 8 to 12 The set of vector register files (e.g., VRF 0 to VRF 3) used by the described operation commands may be different from the set of vector register files used for Fig.13 A set of vector register files (e.g., VRF 4 to VRF 7) in memory row Row 1 of FIG.

[0096] Fig.14 and Fig.15 The loading of input fragments stored in a scalar register file is shown according to an embodiment.

[0097] The control circuit may include a circuit configured to generate an index indicating "a scalar register file storing input values ​​corresponding to the corresponding operation command and a vector register file in which a partial operation result will be stored" from the address information (e.g., row address information and column address information) of the operation command. For each memory address of the memory array, the index of the scalar register file in which the value to be included in the operation will be stored can be mapped to the index of the vector register file in which the operation result will be stored. The host, the memory device, or both may determine a specific scalar register file for storing input values ​​intended for the operation. In some cases, the host, the memory device, or both identify the storage location of the weight value (such as a specific memory address). These determinations are based on a predefined mapping between the memory address and the index of the register file. In the following, a structure for loading stored input values ​​according to a mapping relationship predetermined during design and manufacturing will be described.

[0098] For example, Fig.14 An example in which the control circuit 1400 includes an SRF index generator 1410 is shown.

[0099] The control circuit 1400 may select a first scalar register file among the plurality of scalar register files 1430 based on the memory row information of the first operation command. In some embodiments, the memory row information may include a row address of a memory row indicating that the corresponding weight fragment is stored from the memory address information. The control circuit 1400 may select an input scalar element among the plurality of input scalar elements of the first scalar register file as the first input fragment based on the memory column information of the first operation command. The memory column information may include a column address of a specific memory column in which the specified weight fragment is stored within the memory address information. The control circuit 1400 may include, for example, an SRF index generator 1410.

[0100] The SRF index generator 1410 may generate an index for identifying a scalar register file to be used for the received operation command among the plurality of scalar register files 1430. The SRF index generator 1410 may include an address check unit 1411, an SRF column index generator 1413, and an SRF selector 1412. The address check unit 1411, the SRF column index generator 1413, and the SRF selector 1412 are hardware logics and may be implemented as digital logic elements and / or circuits.

[0101] The address checking unit 1411 may receive an operation command (e.g., a DRAM command) sent from a host. The address checking unit 1411 may extract memory address information from the operation command. The address checking unit 1411 may include digital logic that refers to a bit value in a bit range (e.g., a specific bit position) indicated by the operation command. The address checking unit 1411 may send row address information among the extracted memory address information to the SRF selector 1412. However, the embodiment is not limited thereto. The address checking unit 1411 may send at least one of the row address information and the column address information to the SRF selector 1412. The address checking unit 1411 may send column address information among the extracted memory address information to the SRF column index generator 1413. However, the embodiment is not limited thereto. The address checking unit 1411 may send at least one of the row address information and the column address information to the SRF column index generator 1413.

[0102] The SRF selector 1412 may select a scalar register file to be used for the operation command based on the row address information and the number of scalar register files 1430 included in the PIM block. For example, the SRF selector 1412 may be implemented as a digital operation logic including a modulo operation unit and an ALU. In some examples, the modulo operation unit may be part of an arithmetic logic unit (ALU), or may be a separate functional unit. The SRF selector 1412 may calculate a remainder by performing a modulo operation on a memory row index extracted from the row address information. The SRF selector 1412 may determine an SRF index based on the calculated remainder. The SRF selector 1412 may generate an SRF selection signal corresponding to the SRF index. The SRF selector 1412 may select one of the plurality of scalar register files 1430 by sending the SRF selection signal to a selection device (e.g., a multiplexer (MUX)). However, the embodiment is not limited thereto, and the SRF selector 1412 may select a scalar register file based on at least one of the row address information and the column address information.

[0103] The SRF column index generator 1413 may select a specific data element to be utilized in the operation defined by the operation command from the corresponding scalar register file. The selection is based on the column address information and takes into account the total number of data elements included in the scalar register file. The elements stored in the scalar register file are input fragments and may have scalar element values. The SRF column index generator 1413 may be implemented as a digital operation logic including a modulus operation unit and an ALU. The SRF column index generator 1413 may calculate a remainder by performing a modulus operation on a memory column index extracted from the column address information. The SRF column index generator 1413 may determine a scalar element index based on the calculated remainder. The SRF column index generator 1413 may generate an SRF column index signal corresponding to the SRF index. The SRF column index generator 1413 may select a scalar element value among multiple scalar element values ​​by sending the SRF column index signal to a selection device (e.g., a demultiplexer (DEMUX)). However, embodiments are not limited thereto, and the SRF column index generator 1413 may select a scalar element value based on at least one of row address information and column address information.

[0104] The scalar element values ​​of the scalar register files selected as described above may be provided in common to the plurality of ALUs 1470 (e.g., as input In). Each of the plurality of scalar register files 1430 may include a decoder 1431. The decoder 1431 may output a scalar element value corresponding to the SRF column index signal. The scalar element value output by the decoder 1431 may be provided to the ALU 1470. The control circuit 1400 may perform a partial operation by providing a selected one scalar element value to a plurality of MAC units. The MAC unit may be a unit composed of each or a combination of the ALUs 1470.

[0105] In some embodiments, the configuration of the SRF index generator is not limited to the configuration described above. For example, Fig.15 The SRF index generator 1510 shown in FIG. 1 may be configured to provide an SRF column index signal to all scalar register files.

[0106] Fig.16 Illustrated is the selection of a vector register file according to an embodiment.

[0107] The control circuit 1600 may select one of the plurality of vector register files 1650 based on the memory row information and the memory column information of the first operation command. The control circuit 1600 may merge the partial operation result into the selected vector register file based on the first operation command. The control circuit 1600 may include a VRF index generator 1610 for selecting a vector register file.

[0108] The VRF index generator 1610 may generate an index (e.g., a VRF index) indicating a vector register file in which a result of a partial operation corresponding to an operation command is to be stored. The VRF index generator 1610 may include an address check unit 1611 and a VRF selection unit 1612. The address check unit 1611 and the VRF selection unit 1612 are hardware logics and may be implemented as digital logic elements, circuits, or both.

[0109] The address check unit 1611 may extract address information (e.g., row address (e.g., DRAM row information) and column address (e.g., DRAM column information)) of a memory (e.g., DRAM) from an operation command (e.g., DRAM command). The address check unit 1611 may include digital logic that refers to a bit value in a bit range (e.g., a specific bit position) indicated by the operation command. The address check unit 1611 may send the extracted address information to the VRF selection unit 1612.

[0110] The VRF selection unit 1612 may select a vector register file to be used for the operation based on a memory row address, a memory column address, and the number of the plurality of vector register files 1650. The VRF selection unit 1612 may be implemented as a digital operation logic including a modulo operation unit and an ALU. The VRF selection unit 1612 may generate an index (e.g., a VRF index) indicating a vector register file to be included in the operation. The VRF selection unit 1612 may generate a VRF selection signal corresponding to the VRF index. The VRF selection unit 1612 may send the VRF selection signal to the plurality of vector register files 1650. The decoder 1651 may select the vector register file indicated by the VRF selection signal. The control device may merge (e.g., vector sum) a partial operation result 1690 into the selected vector register file.

[0111] Figures 17 to 20 It is shown that a time corresponding to a barrier is reduced in a memory device using multiple scalar register files and multiple vector register files according to an embodiment.

[0112] Fig.17 A barrier operation according to a comparative example is shown.

[0113] The AAM provided in some embodiments has a narrow range for preventing reordering of memory requests. In some embodiments, operations may be performed on only a single memory row through the AAM, and each time the operation of each memory row is completed, the host 1720 needs to perform a barrier operation. For example, 64 operation commands (e.g., 64 RD commands (64 RD)) may be used to perform operations on weight fragments of one memory row. In some embodiments, barrier operations may be required in three cases.

[0114] The first barrier operation 1701 may occur between the write operation (WRPB) of the partial input vector and the PIM operation. In some embodiments, when the order of the memory request of the host 1720 to the memory device cannot be ensured, an operation error may occur due to the use of a value different from the input value used for the operation. In some embodiments, in order to ensure that the PIM operation is performed after the partial input vectors 1711 (e.g., the first partial input vector) and 1712 (e.g., the second partial input vector) are stored in the scalar register file, the host 1720 may perform the first barrier operation 1701 after the write command of each of the partial input vectors 1711 and 1712.

[0115] When the memory row accessed by the PIM block 1710 is changed, the second barrier operation 1702 may occur. In some embodiments, since the row address information of the memory row is not used, it is not possible to ensure whether the weight value stored in the row buffer is the value of the memory row before or after the change. Therefore, in some embodiments, the host 1720 may perform the second barrier operation 1702 when the memory row is changed.

[0116] The third barrier operation 1703 may occur after the operation of the PIM block 1710 is completed and before the new partial input vector 1712 is stored in the scalar register file. In some embodiments, the PIM block 1710 may receive an operation command using the second partial input vector before a sufficient time has passed from the operation according to the operation command using the first partial input vector. In this example, it cannot be ensured whether the operation of the PIM block 1710 uses the first partial input vector or the second partial input vector. Therefore, in some embodiments, the host 1720 may perform the third barrier operation 1703 before writing the new input vector.

[0117] In some embodiments, barrier operations 1701, 1702, and 1703 may be operations that cause the host 1720 to wait for a sufficient time before and after writing and / or reading each value in order to ensure the integrity of the PIM operation. Although barrier operations 1701, 1702, and 1703 may vary depending on the type of the host 1720, barrier operations 1701, 1702, and 1703 may take about hundreds of cycles. The host 1720 may perform the barrier operation. The memory device may not receive additional commands during the time corresponding to the barrier. Since the time for executing 64 operation commands is approximately 256 cycles, the overhead caused by the barrier operation should be reduced.

[0118] Fig.18 Operations performed when a PIM block includes multiple scalar register files in one embodiment are shown.

[0119] The control circuit of the PIM block 1810 may receive from the host 1820 a first write command for writing a first partial input vector 1811 of an input vector to a first scalar register file and a second write command for writing a second partial input vector 1812 to a second scalar register file. For example, before executing a target operation or a fragment of a target operation, the host 1820 may send several write commands to the PIM block 1810, each command being about a different partial input vector. Therefore, the PIM block 1810 may perform the first write operation and the second write operation without a time corresponding to a barrier between a first write operation based on the first write command and a second write operation based on the second write command. This is because the PIM block 1810 according to an embodiment includes a plurality of scalar register files and may store and maintain a plurality of partial input vectors. Therefore, the integrity of the PIM operation may be ensured using only one first barrier operation 1801 for a plurality of partial input vectors 1811 and 1812.

[0120] For example, in Fig.18 In the example shown in FIG. 1 , the first partial input vector 1811 can be used for the operation of the first memory row Row 0 and the second memory row Row 1. The second partial input vector 1812 can be used for the operation of the third memory row Row 2 and the fourth memory row Row 3. Fig.17 In the example shown in FIG. 1 , a first barrier operation 1701 is required after the operation of the second memory row Row 1 is completed and before the operation of the third memory row Row 2 begins. In some embodiments, Fig.18 In the example shown in FIG. 1 , since the second partial input vector 1812 is pre-stored in the scalar register file, the first barrier operation 1801 may be unnecessary after the third barrier operation 1803. Therefore, the host 1820 and the PIM block 1810 according to the embodiment may reduce the use of the first barrier operation 1801. For example, as shown below based on Fig.19 As described, the second barrier operation 1802 may be removed.

[0121] Fig.19 Operations performed when a PIM block includes multiple vector register files in one embodiment are shown.

[0122] The control circuitry of PIM block 1910 may perform operations using the same partial input vector and weight segments of different memory rows in the same memory bank without a barrier-corresponding time between multiple operations. For example, a barrier-corresponding time may not occur between operations using first partial input vector 1911. Similarly, a barrier-corresponding time may not occur between operations using second partial input vector 1912. Fig.19, in the PIM block 1910, both the memory row and the memory column are considered when processing an operation command. This method results in identifying a specific weight fragment from the memory row used for the operation even when the memory row changes. Therefore, when the operation of the second memory row Row 1 is performed after the operation of the first memory row Row 0, the PIM block 1910 can immediately perform the operation without a barrier. The PIM block 1910 can distinguish the operation result of the first memory row Row 0 from the operation result of the second memory row Row 1, and store the operation results in different vector register files.

[0123] Therefore, the host 1920 may perform only the first barrier operation 1901 and the third barrier operation 1903 , and may not perform the second barrier operation.

[0124] Fig. 20 An example is shown that includes both multiple vector register files and multiple scalar register files.

[0125] The control circuitry may perform operations using different partial input vectors 2011 and 2012 and weight fragments of different memory rows in the same memory bank without the time corresponding to the barrier. Fig. 20 A first memory row Row 0 , a second memory row Row 1 , a third memory row Row 2 , and a fourth memory row Row 3 belonging to the same memory bank are shown.

[0126] In one embodiment, both the host 2020 and the PIM block 2010 may perform a first barrier operation 2001. The first barrier operation 2001 occurs when a partial input vector is written to a scalar register file before executing a target operation so that the partial input vector is completely stored in the scalar register file. After the operation on the memory line corresponding to the target operation is completed, the host 2020 may perform a third barrier operation 2003. The remaining barrier operations may be removed.

[0127] The PIM block 2010 may process an operation command corresponding to one memory row and immediately process an operation command corresponding to the next memory row. This is because the operation result of each memory row may be stored separately in a plurality of vector register files. Therefore, the scope of preventing the reordering of memory commands (such as operation commands) within the address alignment mode (AAM) may be expanded to cover a plurality of memory rows. The host 2020 and the PIM block 2010 may perform partial operations on several memory rows in the same memory bank without the time corresponding to the second barrier. When the partial operations on the plurality of memory rows are completed, the PIM block 2010 may send the results of the partial operations stored in the plurality of vector register files to the host 2020. The host 2020 may aggregate the partial results to generate an overall operation result.

[0128] The PIM block 2010 according to an embodiment may perform operations on a plurality of memory rows and then merge partial operation results. The merged operation result may still be a part of the overall operation result. The PIM block 2010 may send the merged operation result to the host 2020. The host 2020 may collect the operation results (e.g., partial operation results) received from the PIM block 2010. The host 2020 may aggregate the collected operation results to generate an overall operation result.

[0129] Fig.21 A high bandwidth memory (HBM) device according to an embodiment is shown.

[0130] The memory device 2110 according to an embodiment may be implemented as an HBM device. For example, the memory device 2110 may include a memory die 2111 and a buffer die 2112 (eg, stacked on a board). The memory device 2110 may be disposed adjacent to a host 2120 including a memory controller (MC). Figures 1 to 20 In some embodiments, the memory device 2110 may perform partial operations between corresponding weight fragments and input fragments with minimal barriers in response to an operation command received from the host 2120 .

[0131] However, the memory device 2110 is not limited to the above. In addition to the HBM device, the memory device 2110 may also be implemented as a low power double data rate (LPDDR) device, a graphic double data rate (GDDR) device, or a double data rate (DDR) device. The computing system 2100 includes a host 2120 and a memory device 2110. The computing system 2100 may perform machine learning tasks. The machine learning tasks may include aspects of artificial intelligence (AI) and deep neural networks (DNN). For example, the computing system 2100 may include an artificial neural network (ANN). However, the embodiment is not limited thereto, and the computing system 2100 may execute a camera application. The computing system 2100 may also be implemented as a wearable device.

[0132] Fig. 22 An operation method in a memory device according to an embodiment of the present disclosure is shown. The operation method includes generating a scalar register file index and a vector register file index, loading input values, performing a calculation operation, and storing a result in a vector register file.

[0133] In operation 2210, the system generates a scalar register file index and a vector register file index, wherein the scalar register file index indicates a scalar register file among a plurality of scalar register files, and the vector register file index indicates a vector register file among a plurality of vector register files. The generation of these indexes can be achieved by various techniques. For example, a dynamic indexing method can be used to allocate indexes based on current processing requirements or data characteristics. This dynamic generation can promote optimal resource allocation, thereby improving the efficiency and accuracy of processing operations.

[0134] For example, after processing starts, the system generates a scalar register file index and a vector register file index. For example, in the case of large-scale matrix calculations, the index can be generated to give priority to speed and data throughput. For example, in the case of more complex but less data, the index can be optimized for accuracy and data integrity. The scalar register file index specifies a specific scalar register file among several available scalar register files, each of which is customized to store data fragments of a specific type. The vector register file index identifies and selects a vector register file from a group of vector register files. The selected vector register file will be used for result storage.

[0135] In operation 2220, the system loads an input value from a scalar register file based on a scalar register file index. After generating the index, the system loads the input value from the selected scalar register file. The loading process may vary depending on the input value. For example, the loading of the input value may be adjusted based on the type and size of the input value.

[0136] For example, operation 2200 may involve a loading method that is customized for the characteristics of the input value. For example, when processing smaller or simpler data fragments, a direct sequential loading method may be utilized to ensure fast and efficient acquisition of data. For example, for larger or complex structured input values, the system may employ more advanced techniques (such as concurrent loading or buffered loading). These loading methods may be effective in optimizing the processing of data-intensive tasks, minimizing processing bottlenecks, and enhancing overall system performance.

[0137] In operation 2230, the system performs operations on the input values ​​and weight values ​​from the memory bank. During this process, the system performs computational operations on the input values ​​and weight values ​​using methods ranging from basic arithmetic to complex functions (such as matrix operations) appropriate to the specific type and requirements of the data involved.

[0138] For example, in operation 2230, the system performs a computational process involving input values ​​and corresponding weight values ​​from a specified memory bank. The process can range from simple arithmetic calculations to more complex processes such as matrix multiplication or data transformation algorithms. The method employed can be based on the input and weight values, allowing the system to adapt its computational methods to different data types and processing requirements.

[0139] In operation 2240, the system stores the result of the operation in the vector register file based on the vector register file index. The system stores the output of the calculation operation in the specified vector register file as indicated by the previously determined vector register file index. This process can be used to maintain the integrity and accessibility of the calculated data for subsequent use or analysis. The result data from the calculation operation is stored in the vector register file indicated by the vector register file index.

[0140] In some examples, the storage process may be modified based on the data to be stored and the system requirements. The storage step may combine different techniques based on the needs of the system. For example, methods such as data compression or storage in a specific format may be used for efficient space usage or to meet specific data processing requirements. For example, in the case where the result data is large or complex, the system may use storage techniques such as hierarchical storage management or utilize a buffer memory to ensure seamless transfer of data into the vector register file.

[0141] The embodiments described herein may be implemented using hardware components, software components, and / or combinations thereof. The processing device may be implemented using one or more general or special computers (such as, for example, a processor, a controller, an ALU, a digital signal processor (DSP), a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of responding to and executing instructions in a defined manner). The processing device may run an operating system (OS) and one or more software applications running on the OS. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. In the embodiments of the present disclosure, the description of the processing device is used as a singular; however, those skilled in the art will appreciate that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, the processing device may include multiple processors, or a single processor and a single controller. In some embodiments, different processing configurations (such as parallel processors) are feasible.

[0142] Software may include computer programs, code segments, instructions, or some combination thereof, to independently or collectively instruct or configure a processing device to operate in accordance with the instructions. Software and data may be stored in any type of machine, component, physical or virtual device, or computer storage medium or device that can provide instructions or data to a processing device or be interpreted by a processing device. Software may also be distributed on networked computer systems so that the software is stored and executed in a distributed manner. Software and data may be stored by one or more non-transitory computer-readable recording media.

[0143] The method according to the above-described embodiment may be recorded in a non-transitory computer-readable medium, which includes program instructions for implementing various operations of the above-described embodiment. The medium may also include data files, data structures, etc., alone or in combination with program instructions. The program instructions recorded on the medium may be program instructions specially designed and constructed for the purpose of the embodiment, or the program instructions recorded on the medium may be of a type known and available to technicians in the field of computer software. Examples of non-transitory computer-readable media include magnetic media (such as hard disks, floppy disks, and tapes); optical media (such as CD-ROM disks, DVDs, and / or Blu-ray disks); magneto-optical media (such as optical disks); and hardware devices specifically configured to store and execute program instructions (such as read-only memory (ROM), random access memory (RAM), flash memory (e.g., USB flash drive, memory card, memory stick, etc.), etc.). Examples of program instructions include machine code such as generated by a compiler and files containing high-level code that can be executed by a computer using an interpreter.

[0144] The aforementioned hardware devices may be configured to function as one or more software modules in order to perform the operations of the aforementioned embodiments, and vice versa.

[0145] In some embodiments, although the embodiments have been described based on limited drawings, those skilled in the art may apply various technical modifications and changes based on them. For example, if the described techniques are performed in a different order and / or if the components in the described systems, architectures, devices, or circuits are combined in a different way and / or replaced or supplemented by other components or their equivalents, suitable results may be achieved. Therefore, other implementations are within the scope of the appended claims.

Claims

1. A memory device, comprising: a first scalar register file storing a first input fragment; a second scalar register file storing a second input fragment; Arithmetic logic unit; as well as The control circuit is configured to: based on a first operation command received from the host, use the arithmetic logic unit to perform a first operation between a first input fragment and a first weight fragment, and based on a second operation command received from the host, use the arithmetic logic unit to perform a second operation between a second input fragment and a second weight fragment.

2. The memory device according to claim 1, wherein: Before the first operation command and the second operation command are received, the first scalar register file stores the first input fragment, and The second scalar register file stores a second input fragment before the first operation command and the second operation command are received.

3. The memory device according to claim 1, wherein: The control circuit is configured to: receiving a first write command and a second write command from a host, the first write command being used to write a first portion of an input vector to the first scalar register file, and the second write command being used to write a second portion of an input vector to the second scalar register file, and The first write operation and the second write operation are performed without a time corresponding to a barrier between the first write operation based on the first write command and the second write operation based on the second write command.

4. The memory device according to claim 1, wherein: The control circuit is configured to: In response to receiving a first write command, writing a first portion of an input vector to the first scalar register file among a plurality of scalar register files; and In response to receiving a second write command, writing a second portion of the input vector to the second scalar register file among the plurality of scalar register files, The first portion of the input vector includes a plurality of input segments including a first input segment, and the second portion of the input vector includes a plurality of input segments including a second input segment.

5. The memory device according to claim 1, wherein: The control circuit is configured to: obtaining a weight fragment stored in a first memory row by accessing a first memory row of a memory bank based on a first operation command; and The weight fragment stored in the second memory row is obtained by accessing the second memory row of the memory bank based on the second operation command.

6. The memory device according to claim 1, wherein: The control circuit is configured to: determining that a first vector register file identified based on the first operation command and a second vector register file identified based on the second operation command among the plurality of vector register files are the same; and In response to the determination, a value stored in the first vector register file is updated based on a first partial operation result of the first operation command and a second partial operation result of the second operation command.

7. The memory device according to claim 1, wherein: The third operation command indicates that a third weight segment in the same memory row as the first weight segment is different from the first weight segment, and The control circuit is configured to: in response to the first operation command and the third operation command indicating the same vector register file, merge a partial operation result corresponding to the first operation command and a partial operation result corresponding to the third operation command into the same vector register file.

8. The memory device according to claim 7, wherein: The control circuit is configured to obtain a third weight segment and a third input segment different from the first input segment based on a third operation command.

9. The memory device according to claim 1, wherein: The fourth operation command indicates that a fourth weight segment in the same memory row as the first weight segment is different from the first weight segment, and The control circuit is configured to: in response to the first operation command and the fourth operation command indicating different vector register files, merge the partial operation result corresponding to the first operation command and the partial operation result corresponding to the fourth operation command into different vector register files respectively.

10. The memory device according to claim 9, wherein: The control circuit is configured to obtain a fourth input segment based on a fourth operation command.

11. The memory device according to claim 1, wherein: The control circuit is configured to: Selecting the first scalar register file among a plurality of scalar register files based on the memory row information of the first operation command; and Based on the memory column information of the first operation command, a first input scalar element among a plurality of input scalar elements of the first scalar register file is selected as a first input fragment.

12. The memory device according to claim 1, wherein: The control circuit is configured to: Selecting a first vector register file among a plurality of vector register files based on the memory row information and the memory column information of the first operation command; and Part of the operation results of the first operation command are merged into the first vector register file.

13. The memory device according to claim 1, wherein: The control circuit is configured to perform a plurality of operations using the same partial input vector and different weight fragments of different memory rows in the same memory bank without a time corresponding to a barrier between the plurality of operations.

14. The memory device according to claim 1, wherein: The control circuit is configured to perform the plurality of operations using different partial input vectors and different weight fragments of different memory rows in the same memory bank without a time corresponding to a barrier between the plurality of operations.

15. The memory device according to claim 1, wherein: The control circuit is configured to: generating a first partial operation result by performing a product operation between a first input segment and a weight value of a first weight segment using the arithmetic logic unit; generating a second partial operation result by performing a product operation between the second input segment and the weight value of the second weight segment using the arithmetic logic unit; determining that the first operation command and the second operation command indicate the same vector register file; and In response to the determination, a result obtained by summing the first partial operation result and the second partial operation result is stored in the same vector register file.

16. The memory device of claim 15, wherein: The first partial operation result, the second partial operation result, and the summation result have the same vector form.

17. The memory device according to any one of claims 1 to 16, further comprising: a plurality of memory banks configured to respectively store weight fragments; as well as An in-memory processing block has the first scalar register file, the second scalar register file, the arithmetic logic unit and the control circuit, wherein the in-memory processing block is arranged adjacent to a corresponding memory bank among the plurality of memory banks.

18. The memory device according to any one of claims 1 to 16, wherein: The memory device is implemented as one of a high bandwidth memory device, a double data rate device, a low power double data rate device, and a graphics double data rate device.

19. An operation method performed by a memory device, the operation method comprising: Based on a first operation command received from a host, loading a first input segment from a first scalar register file and loading a first weight segment from a memory bank; performing a first operation between the first input segment and the first weight segment using an arithmetic logic unit; Based on a second operation command received from the host, loading a second input segment from a second scalar register file and loading a second weight segment from a memory bank; and A second operation between the second input segment and the second weight segment is performed using the arithmetic logic unit.

20. A computing system comprising: a host configured to send an operation command indicating a weight segment of a different memory row in the same memory bank without a time corresponding to the barrier; as well as The memory device is configured to load different input fragments corresponding to the operation command from a plurality of scalar register files, and perform operations between weight fragments and input fragments using an arithmetic logic unit.

21. A method for performing an in-memory processing operation, the method comprising: generating a scalar register file index and a vector register file index, wherein the scalar register file index indicates a scalar register file among a plurality of scalar register files, and the vector register file index indicates a vector register file among a plurality of vector register files; loading an input value from the scalar register file based on a scalar register file index; performing operations on input values ​​and weight values ​​from a memory bank; and A result of the operation is stored in the vector register file based on a vector register file index.

Images