Processing unit in memory for performing matrix-vector multiplication

Abstract

The present disclosure relates to a processing unit in a memory for performing a matrix-vector multiplication operation. The processing unit (PU) of a memory device may receive a matrix of data values and a vector of data values stored in a bank. The PU may perform a first plurality of multiplications on first data values of the vector using a first plurality of data values of a first column of the matrix. The first plurality of multiplications may be performed by a plurality of multiply-accumulate MAC units. Each of the first plurality of multiplications may be performed by a different MAC unit of the plurality of MAC units. The PU may perform a second plurality of multiplications on a second data value of the vector using a second plurality of data values of a second column of the matrix. Each of the second plurality of multiplications may be performed by a different MAC unit of the plurality of MAC units.

Description

Technical Field

[0001] The present disclosure relates generally to memory, and more particularly, to apparatus and methods associated with performing matrix-vector multiplication operations using processing units in memory. Background Art

[0002] Memory devices are typically provided as internal semiconductor integrated circuits in computers or other electronic devices. There are many different types of memory, including volatile and non-volatile memory. Volatile memory may require power to maintain its data and includes random access memory (RAM), dynamic random access memory (DRAM), and synchronous dynamic random access memory (SDRAM), among others. Non-volatile memory can provide permanent data by retaining stored data when not powered, and may include NAND flash memory, NOR flash memory, read-only memory (ROM), electrically erasable programmable ROM (EEPROM), erasable programmable ROM (EPROM), and resistance variable memory, such as phase change random access memory (PCRAM), resistive random access memory (RRAM), and magnetoresistive random access memory (MRAM), among others.

[0003] Memory is also used as volatile and nonvolatile data storage devices for a wide range of electronic applications. Nonvolatile memory can be used, for example, in personal computers, portable memory sticks, digital cameras, cellular phones, portable music players such as MP3 players, movie players, and other electronic devices. Memory cells can be arranged in an array, where the array is used in a memory device. Summary of the Invention

[0004] On the one hand, the present disclosure provides an apparatus for performing matrix-vector multiplication operations using a processing unit in a memory, comprising: a memory cell storage body; a processing unit coupled to the memory cell storage body and configured to: receive a matrix of data values ​​stored in the storage body; receive a vector of data values ​​stored in the storage body; perform a first plurality of multiplication operations on a first data value of the vector using a first plurality of data values ​​of a first column of the matrix, wherein the first plurality of multiplication operations are performed by a plurality of multiplication accumulate (MAC) units, and wherein each of the first plurality of multiplication operations is performed by a different MAC unit among the plurality of MAC units; and perform a second plurality of multiplication operations on a second data value of the vector using a second plurality of data values ​​of a second column of the matrix, wherein each of the second plurality of multiplication operations is performed by a different MAC unit among the plurality of MAC units.

[0005] On the other hand, the present disclosure provides a method for performing matrix-vector multiplication operations using a processing unit in a memory, comprising: receiving a first column of a matrix of data values ​​stored in a memory bank of the memory device at a processing unit of the memory device, wherein the processing unit is coupled to the memory bank; receiving a vector of data values ​​at the processing unit; storing the vector in a first plurality of registers; performing a first plurality of multiplication operations on first data values ​​of the vector using the first plurality of data values ​​of the column, wherein the first plurality of multiplication operations are performed by a plurality of multiplication accumulate (MAC) units, and wherein each of the first plurality of multiplication operations is performed by a different MAC unit among the plurality of MAC units; and receiving a second column of the matrix at the processing unit; performing a second plurality of multiplication operations on second data values ​​of the vector using the second plurality of data values ​​of the second column, wherein each of the second plurality of multiplication operations is performed by a different MAC unit among the plurality of MAC units; and storing outputs of the first plurality of multiplication operations and the second plurality of multiplication operations in the memory bank.

[0006] On the other hand, the present disclosure provides an apparatus for performing matrix-vector multiplication operations using a processing unit in a memory, comprising: a first bank of memory cells; a second bank of memory cells; a processing unit comprising a plurality of multiply-accumulate (MAC) units, the processing unit being coupled to the first bank and the second bank; wherein the plurality of MAC units are configured to: receive a first plurality of data values ​​of a first column of a data value matrix stored in the first bank, wherein each MAC unit from the plurality of MAC units receives a different data value of the first plurality of data values ​​from the first column; receive a first data value of a data value vector stored in the second bank, wherein each MAC unit from the plurality of MAC units receives the first data value of the vector; and multiply the first plurality of data values ​​of the vector by using the first plurality of data values ​​of the first column. The method comprises the steps of: performing a first plurality of multiplication operations on the data values, wherein each of the first plurality of multiplication operations is performed by a different MAC unit among the plurality of MAC units; receiving a second plurality of data values ​​for a second column of the matrix stored in the first memory bank, wherein each MAC unit among the plurality of MAC units receives a different data value from the second plurality of data values ​​of the second column; receiving a second data value of the vector, wherein each MAC unit among the plurality of MAC units receives the second data value of the vector; performing a second plurality of multiplication operations using the second data value of the vector and the second plurality of data values ​​of the second column, wherein each of the second plurality of multiplication operations is performed by a different MAC unit among the plurality of MAC units; and storing outputs of the first plurality of multiplication operations and the second plurality of multiplication operations in the second memory bank. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 is a block diagram of an apparatus in the form of a computing system including a memory device according to several embodiments of the present disclosure.

[0008] Figure 2 is a block diagram of a processing unit according to several embodiments of the present disclosure.

[0009] Figure 3 is a block diagram of a memory unit bank and a processing unit according to several embodiments of the present disclosure.

[0010] Figure 4 is a block diagram of multiple memory unit banks and processing units according to several embodiments of the present disclosure.

[0011] Figure 5 An example flow chart illustrating a method of performing matrix-vector multiplication operations using processing units in memory according to several embodiments of the present disclosure.

[0012] Figure 6 An example machine is illustrated of a computer system within which a set of instructions, for causing the machine to perform any one or more of the methodologies discussed herein, may be executed. DETAILED DESCRIPTION

[0013] The present disclosure includes apparatus and methods related to performing matrix-vector multiplication operations using a processing unit in a memory. The processing unit may receive a matrix of data values ​​stored in a bank of memory cells of a memory device. The processing unit may receive a vector of data values ​​stored in the bank. The processing unit may perform a first plurality of multiplication operations on first data values ​​of the vector using a first plurality of data values ​​of a first column of the matrix. The first plurality of multiplication operations may be performed by a plurality of multiply-accumulate (MAC) units. Each of the first plurality of multiplication operations may be performed by a different MAC unit from the plurality of MAC units. The processing unit may perform a second plurality of multiplication operations on second data values ​​of the vector using a second plurality of data values ​​of a second column of the matrix. Each of the second plurality of multiplication operations may be performed by a different MAC unit from the plurality of MAC units.

[0014] In previous memory methods, matrix and vector multiplication operations can be performed by multiplying each row of the matrix by a vector. For example, the first data value of the first row of the matrix can be multiplied by the first data value of the vector, and the second data value of the first row of the matrix can be multiplied by the second data value of the vector, and so on, until each of the data values ​​of the first row of the matrix is ​​multiplied by a different data value of the vector. Each result of the multiplication operation performed on the data values ​​of a row of the matrix and the data values ​​of the vector can be accumulated to produce a result vector. For example, the result of the multiplication operation performed on the first data value of the first row of the matrix using the first data value of the vector can be accumulated with the result of the multiplication operation performed on the second data value of the first row of the matrix using the second data value of the vector. The accumulated result can represent the first data value of the result vector.

[0015] However, in this approach, when the accumulator circuitry generates different result values, each data value of the result vector needs to be temporarily stored. This temporary storage can increase the cost of manufacturing the memory, can increase the size of the memory die, and / or can delay the output of the result vector. Output can be delayed because additional operations are required to combine the different data values ​​to generate the result vector.

[0016] As used herein, a matrix is ​​a grouping of data values ​​organized into rows and columns, where each data value has an order within the row and column. For example, the first data value of a matrix may be the first data value in the first row and the first data value in the first column. A vector is a plurality of data values ​​organized into a single column.

[0017] To address these and other deficiencies of previous approaches, embodiments of the present disclosure allow data values ​​of a column of a matrix to be multiplied with data values ​​of a vector to perform a matrix-vector multiplication operation. Each of the data values ​​of a single column can be multiplied with a single data value of the vector using multiple multiply-accumulate (MAC) units. For example, each of the data values ​​of a first matrix column can be multiplied with the same data value of the vector using a different MAC unit. The first MAC unit can be used to multiply the first data value of the first matrix column with the first data value of the vector and accumulate the results. The second MAC unit can also multiply the second data value of the first matrix column with the first data value of the vector. The first MAC unit can also be used to multiply the first data value of the second matrix column with the second data value of the vector and accumulate the result with a previous value stored (e.g., accumulated) in the first MAC unit. The second MAC unit can multiply the second data value of the second matrix column with the second data value of the vector and accumulate the result with a previous value stored in the first MAC unit.

[0018] Multiplying the data values ​​of a column of a matrix with the data values ​​of a vector has the advantage of not requiring temporary values ​​to be stored outside the MAC unit to generate the result vector. Instead, the final data value of the result vector can be retained in the MAC unit, which eliminates the need for temporary registers, thereby reducing the cost and / or size of the memory. Retaining the final data value of the result vector in the MAC unit also has the advantage of allowing the result vector to be read without performing additional operations on the data values ​​stored in the MAC unit, which eliminates the delay required to read the result vector (e.g., reduces the amount of time). Eliminating the delay for reading the result vector allows the matrix and vector to be multiplied in the same amount of time as the time required to read a single column of the memory address of the matrix and vector (e.g., 256 prefetches) from memory. Eliminating the delay for reading the result vector also allows the result vector to be generated and provided to the input / output (I / O) lines using the same data path as used to read data from the memory array (e.g., memory bank).

[0019] As used herein, "a number" of something may refer to one or more of such things. For example, a number of memory devices may refer to one or more memory devices. A "plurality" of something is intended to be two or more. Additionally, designators such as "N," as used herein (particularly with respect to reference numerals in the drawings), indicate that a number of the particular feature so indicated may be included in several embodiments of the present disclosure.

[0020] The figures herein follow a numbering convention in which the first digit or digits correspond to the figure number and the remaining digits identify the elements or components in the diagram. Similar elements or components between different figures can be identified by using similar digits. As will be appreciated, the elements shown in the various embodiments herein can be added, exchanged, and / or eliminated to provide several additional embodiments of the present disclosure. In addition, the proportions and relative scales of the elements provided in the figures are intended to illustrate various embodiments of the present disclosure and are not used in a limiting sense.

[0021] Figure 1 is a block diagram of an apparatus in the form of a computing system 100 including a memory device 120, according to several embodiments of the present disclosure. As used herein, memory device 120, memory array 130, and / or host 110 may also be individually referred to as a "apparatus."

[0022] In this example, system 100 includes a host 110 coupled to a memory device 120 via an interface 156. Computing system 100 may be a personal laptop computer, a desktop computer, a digital camera, a mobile phone, a memory card reader, or a device with Internet of Things (IoT) capabilities, among various other types of systems. Host 110 may include a number of processing resources (e.g., one or more processors, microprocessors, or some other type of control circuitry) capable of accessing memory 120. System 100 may include separate integrated circuits, or both host 110 and memory device 120 may be on the same integrated circuit. For example, host 110 may be a system controller for a memory system that includes multiple memory devices 120, where system controller 110 provides access to the respective memory devices 120 through another processing resource (e.g., a central processing unit (CPU)).

[0023] exist Figure 1 In the example shown in FIG, host 110 is responsible for executing an operating system (OS) and / or various applications that may be loaded thereto (e.g., from memory device 120 via controller 140). Host 110 may provide access commands and / or secure mode initialization commands to the memory device via interface 156.

[0024] For clarity, system 100 has been simplified to focus on features that are particularly relevant to the present disclosure. For example, memory array 130 may be a DRAM array, an SRAM array, an STT RAM array, a PCRAM array, a TRAM array, an RRAM array, a NAND flash array, and / or a NOR flash array. Array 130 may include memory cells arranged in rows coupled by access lines (which may be referred to herein as word lines or select lines) and columns coupled by sense lines (which may be referred to herein as digit lines or data lines). Although Figure 1 A single array 130 is shown in FIG, but embodiments are not limited thereto. For example, memory device 120 may include several arrays 130 (eg, several banks of DRAM cells).

[0025] Memory device 120 includes address circuitry 142 for latching address signals provided via interface 156. For example, the interface may include a physical interface employing a suitable protocol (e.g., a data bus, an address bus, and a command bus, or a combined data / address / command bus). This protocol may be custom or proprietary, or interface 156 may employ a standardized protocol such as Peripheral Component Interconnect Express (PCIe), Gen-Z, CCIX, or the like. Address signals are received and decoded by row decoder 146 and column decoder 152 to access memory array 130. Data can be read from memory array 130 by sensing voltage and / or current changes on sense lines using sense circuitry 150. For example, sense circuitry 150 may include sense amplifiers that can read and latch a page (e.g., a row) of data from memory array 130. I / O circuitry 144 may be used for bidirectional data communication with host 110 via interface 156. Read / write circuitry 148 is used to write data to or read data from memory array 130. As examples, circuitry 148 may include various drivers, latch circuitry, and the like.

[0026] Controller 140 decodes signals provided by host 110. These signals may include chip enable signals, write enable signals, and address latch signals, which are used to control operations performed on memory array 130, including data read operations, data write operations, and data erase operations. In various embodiments, controller 140 is responsible for executing instructions from host 110. Controller 140 may include a state machine, a sequencer, and / or some other type of control circuitry, which may be implemented in hardware, firmware, or software, or any combination thereof.

[0027] In various examples, controller 140 may receive signals provided by host 110, including signals requesting that processing unit (PU) 102 perform an operation. For example, controller 140 may provide a signal to PU 102 requesting that a matrix-vector multiplication operation be performed. Controller 140 may receive the signal from host 110 and may cause a matrix of data values ​​and a vector of data values ​​to be sensed (e.g., read) from memory array 130 and provided to PU 102. As used herein, PU 102 may include hardware, firmware, and / or software for performing operations using data provided by memory array 130. For example, PU 102 may perform a multiplication operation according to embodiments of the present disclosure. PU 102 may multiply a matrix of data values ​​and a vector of data values. As used herein, a data value is a number that can be used to perform an operation, such as a multiplication operation.

[0028] In various examples, PU 102 can utilize I / O lines 103 to receive a matrix of data values ​​and a vector of data values ​​and output (e.g., provide) a result vector of data values ​​(e.g., the result of a multiplication operation). The result vector of data values ​​can be stored back in memory array 130 and / or provided to host 110. Utilizing the same I / O lines 103 to read data from memory array 130, provide data to PU 102, and / or provide data from PU 102 can allow PU 102 to be added to memory device 120 without substantially adding to the die area of ​​memory device 120. For example, PU 102 can be added to memory device 120 by increasing the die size of memory device 120 by 1% to 3% compared to a solution that does not include memory device 120. This is a 1% to 3% increase in die size compared to a solution where PU 102 is added to memory device 120 so that PU 102 does not receive and / or provide data via I / O lines 103.

[0029] In various examples, PU 102 may receive columns of matrix and vector data values ​​from memory array 130 to perform a matrix-vector multiplication operation. The matrix data values ​​may be stored in memory array 130 so that the data values ​​organized into columns can be read rather than reading rows of memory array 130.

[0030] In various examples, controller 140 may cause data values ​​received from host 110 to be organized and stored in memory array 130 such that columns of a matrix are stored in memory cells coupled to the same word line. Providing columns of data values ​​to PU 102 allows PU 102 to perform operations on the columns of data values ​​such that the results of matrix-vector multiplication operations are stored in the accumulators of the MAC units of PU 102 without having to perform additional operations to combine the results into result vectors. Utilizing I / O lines 103 to provide the result vectors of the matrix-vector multiplication operations and store the result vectors in the accumulators of the MAC units of PU 102 allows the result vectors to be generated and provided to I / O lines 103 in the same amount of time as it takes to read a single column of a memory address of a matrix and / or vector from memory array 130 (e.g., a 256 prefetch).

[0031] Figure 2 2 is a block diagram of a PU 202 according to several embodiments of the present disclosure. PU 202 is coupled to I / O lines 203. PU 202 includes registers 239, a MAC unit 243, control logic 225, and output logic 224. PU 202 may receive a data strobe signal 227, a control signal 226, and an input signal.

[0032] The input signal may provide matrix and / or vector data values ​​(e.g., data values ​​234, 234). The matrix and / or vector data values ​​may be provided sequentially. For example, a vector data value (e.g., data value 235) may be stored in register 239. The matrix data value (e.g., data value 234) may be provided directly to MAC unit 243 or may be stored in a different register (not shown) before being provided to MAC unit 243. Figure 2 The instance of does not contain registers for storing the data values ​​of the matrix. Figure 3 and 4 An instance of contains registers for storing the data values ​​of the matrix.

[0033] exist Figure 2 In the example of , the width of the input data bus can be 256 bits. In this example where the vector to be operated on comprises 8 bits, 32 8-bit vectors can be provided in a single 256-bit data block. The data values ​​of the matrix can also be provided to the PU in the form of 256-bit blocks. Each of the vector and matrix data values ​​can comprise 8 bits. Register 239 (shift register) can provide each of the copied data values ​​to fill the 256 bits provided from register 239 to MAC unit 243. For example, the first data value (V0) can be copied 32 times to generate 256 bits. Each of the MAC units 243 can receive the same 8 bits (V0) from the 256 bits.

[0034] MAC unit 243 may receive data values ​​from register 239 and matrix data values ​​from I / O line 203. MAC unit 243 may include multiplication circuitry 221, adder circuitry 222, and register 223. MAC unit 243 may utilize multiplication circuitry 221, adder circuitry 222, and register 223 to multiply and accumulate the vector data values ​​and the matrix data values. Output logic 224 may be controlled to output an output vector. The output vector may be provided to I / O line 203.

[0035] Data strobe 227 may be used to provide timing signals to latch data values ​​in register 239 and perform operations of MAC unit 243. Data strobe 227 may also be used to determine when to forward output vectors to I / O lines 203.

[0036] Control signals provided over control bus 226 may provide information necessary to perform a number of operations to control logic 225. For example, control signals may be used to indicate to register 239 that a data value should be copied and / or shifted within register 239. The control signals may cause control logic 225 to indicate to output logic 224 when to forward an output vector. Data strobe 227 and / or control signals may be provided by the control circuitry of the memory device.

[0037] The control signals may be used to load register 239, forward (eg, read and / or load) an output vector, and provide the data value to MAC unit 243. The control signals may also be used to indicate that register 239 should shift data.

[0038] Figure 3 3 is a block diagram of a bank of memory cells 330 and a PU 302 according to several embodiments of the present disclosure. The bank 330 includes memory cells configured to store a matrix 334 of data values, a vector 335 of data values, and a result (e.g., output) vector 336 of data values. Although a single bank 330 is shown as storing the matrix 334, vector 335, and / or result vector 336, the matrix 334, vector 335, and / or result vector 336 may be stored in different banks. The PU 302 includes a matrix register 333, a vector register 339, and a plurality of MAC units 332 (e.g., MAC units 332-0, 332-1, 332-2, 332-3).

[0039] PU 302 may be configured to perform a matrix-vector multiplication operation. PU 302 may perform a matrix-vector multiplication operation on vector 335 using matrix 334. The matrix-vector multiplication operation may include multiple iterations. The result of each iteration may be stored in MAC unit 332. For example, the result of each iteration may be stored in an accumulator of MAC unit 332. The result of each iteration of the matrix-vector multiplication operation may be accumulated with the previous result of the previous iteration of the matrix-vector multiplication operation.

[0040] After each of the iterations of the matrix-vector multiplication operation, the MAC unit 332 may store and / or provide a result vector 336. The result vector 336 may be the output of the matrix-vector multiplication operation.

[0041] As used herein, a matrix 334 may include a plurality of columns 337-0, 337-1, 337-2, 337-3 of data values ​​(e.g., M00, M10, M20, M30, M01, M11, M21, M31, M02, M12, M22, M32, M03, M13, M23, M33). The columns 337-0, 337-1, 337-2, 337-3 of the matrix 334 may be referred to as columns 337.

[0042] The vector 335 may include a plurality of data values ​​338-0, 338-1, 338-2, 338-3, and the result vector 336 may include a plurality of data values ​​338-4, 338-5, 338-6, 338-7.

[0043] In various examples, matrix 334 may be stored in memory bank 330 such that columns 337 are provided together to PU 302 rather than providing rows of matrix 334 together to PU 302. For example, matrix 334 may be stored in memory bank 330 such that a prefetch of data read from memory bank 330 may include the columns of the matrix. The columns of matrix 334 included in a prefetch of data read from memory bank 330 may be provided to PU 302. For example, a first prefetch may include column 337-0 of the matrix, a second prefetch may include column 337-1, a third prefetch may include column 337-2, and a fourth prefetch may include column 337-3. Vector 335 may also be provided as a prefetch of data read from memory bank 330.

[0044] The matrix-vector multiplication operation is defined as Y0=M00×V0+M01×V1+M02×V2+M03×V3, Y1=M10×V0+M11×V1+M12×V2+M13×V3, Y2=M20×V0+M21×V1+M22×V2+M23×V3, and Y3=M30×V0+M31×V1+M32×V2+M33×V3, where the result vector 336 includes data values ​​338-4, 338-5, 338-6, 338-7 (e.g., Y0, Y1, Y2, Y3). Each of the data values ​​338-4, 338-5, 338-6, 338-7 (e.g., Y0, Y1, Y2, Y3) can be calculated by a different MAC unit from the MAC unit 332. For example, data value 338-4 (e.g., Y0) may be calculated by MAC unit 332-0, data value 338-5 (e.g., Y1) may be calculated by MAC unit 332-1, data value 338-6 (e.g., Y2) may be calculated by MAC unit 332-2, and data value 338-7 (e.g., Y3) may be calculated by MAC unit 332-3.

[0045] In various examples, vector 335 may be read from memory bank 330 and stored in vector register 339. The host may provide a command to the memory device to cause the memory device to read vector 335. PU 302 may intercept vector 335 and may store vector 335 in vector register 339 in response to receiving the command from the host. For example, data value 338-0 of vector 335 may be stored in register 339-0, data value 338-1 of vector 335 may be stored in register 339-1, data value 338-2 of vector 335 may be stored in register 339-2, and data value 338-3 of vector 335 may be stored in register 339-3.

[0046] Matrix 334 may be read from memory bank 330 and stored in matrix register 333. A host may provide a command to the memory device to cause the memory device to read matrix 334. For each iteration of the matrix-vector multiplication operation, a different column of matrix 334 may be stored in matrix register 333. For example, column 337-0 of matrix 334 may be stored in matrix register 333 in a first iteration of the matrix-vector multiplication operation, column 337-1 of matrix 334 may be stored in matrix register 333 in a second iteration of the matrix-vector multiplication operation, column 337-2 of matrix 334 may be stored in matrix register 333 in a third iteration of the matrix-vector multiplication operation, and column 337-3 of matrix 334 may be stored in matrix register 333 in a fourth iteration of the matrix-vector multiplication operation.

[0047] In the first iteration of the matrix-vector multiplication operation, column 337-0 and vector 335 may be stored in matrix register 333 and vector register 339, respectively. A first data value 338-0 of vector 335 may be stored in register 339-0. First data value 338-0 may be provided to each of MAC units 332. For example, different instances of first data value 338-0 may be provided to MAC unit 332-0, MAC unit 332-1, MAC unit 332-2, and MAC unit 332-3. Different data values ​​from column 337-0 may be provided to each of MAC units 332. For example, data value M00 may be provided to MAC unit 332-0, data value M10 may be provided to MAC unit 332-1, data value M20 may be provided to MAC unit 332-2, and data value M30 may be provided to MAC unit 332-3.

[0048] The MAC unit 332 may multiply data values ​​received from the matrix register 333 and the vector register 339. For example, the MAC unit 332-0 may multiply the data value M00 and the data value 338-0, the MAC unit 332-1 may multiply the data value M10 and the data value 338-0, the MAC unit 332-2 may multiply the data value M20 and the data value 338-0, and the MAC unit 332-3 may multiply the data value M30 and the data value 338-0.

[0049] The results of the multiplication operations may be stored in the accumulators of the MAC unit 332. For example, the result of the multiplication operation performed on the data value M00 using the data value 338-0 may be stored in the accumulator of the MAC unit 332-0. The result of the multiplication operation performed on the data value M10 using the data value 338-0 may be stored in the accumulator of the MAC unit 332-1. The result of the multiplication operation performed on the data value M20 using the data value 338-0 may be stored in the accumulator of the MAC unit 332-2. The result of the multiplication operation performed on the data value M30 using the data value 338-0 may be stored in the accumulator of the MAC unit 332-3.

[0050] Once column 337-0 of matrix 334 has been multiplied by data value 338-0 of vector 335, the data values ​​stored in register 339 may be rotated. For example, data value 338-1 may be stored in register 339-0, data value 338-2 may be stored in register 339-1, data value 338-3 may be stored in register 339-2, and data value 338-0 may be stored in register 339-3. The rotation of the data values ​​of the vector in register 339, the completion of the multiplication operation performed by MAC unit 332, and / or the accumulation of the results of the multiplication operation performed by MAC unit 332 may conclude the first iteration of the matrix-vector multiplication operation.

[0051] In the second iteration of the matrix-vector multiplication operation, column 337-1 of data values ​​may be stored in matrix register 333. Vector 335 was previously stored and rotated in the previous iteration (e.g., the first iteration) of the matrix-vector multiplication operation. A second data value 338-1 of vector 335 may be stored in register 339-0. Second data value 338-1 may be provided to each of MAC units 332. Different data values ​​from column 337-1 may be provided to each of MAC units 332. For example, data value M01 may be provided to MAC unit 332-0, data value M11 may be provided to MAC unit 332-1, data value M21 may be provided to MAC unit 332-2, and data value M31 may be provided to MAC unit 332-3.

[0052] The MAC unit 332 may multiply the data values ​​received from the matrix register 333 and the vector register 339. For example, the MAC unit 332-0 may multiply the data value M01 and the data value 338-1, the MAC unit 332-1 may multiply the data value M11 and the data value 338-1, the MAC unit 332-2 may multiply the data value M21 and the data value 338-1, and the MAC unit 332-3 may multiply the data value M31 and the data value 338-1.

[0053] The results of the multiplication operations may be accumulated with previous results stored in the accumulators of the MAC unit 332. For example, the result of the multiplication operation performed on the data value M01 using the data value 338-1 may be accumulated with the data value stored in the accumulator of the MAC unit 332-0. The result of the multiplication operation performed on the data value M11 using the data value 338-1 may be accumulated with the data value stored in the accumulator of the MAC unit 332-1. The result of the multiplication operation performed on the data value M21 using the data value 338-1 may be accumulated with the data value stored in the accumulator of the MAC unit 332-2. The result of the multiplication operation performed on the data value M31 using the data value 338-1 may be accumulated with the data value stored in the accumulator of the MAC unit 332-3.

[0054] Once column 337-1 of matrix 334 has been multiplied by data value 338-1 of vector 335, the data values ​​stored in register 339 may be rotated. For example, data value 338-2 may be stored in register 339-0, data value 338-3 may be stored in register 339-1, data value 338-0 may be stored in register 339-2, and data value 338-1 may be stored in register 339-3. The rotation of the data values ​​in register 339, the completion of the multiplication operation performed by MAC unit 332, and / or the accumulation of the results of the multiplication operation performed by MAC unit 332 may conclude the second iteration of the matrix-vector multiplication operation.

[0055] In the third iteration of the matrix-vector multiplication operation, column 337-2 of data values ​​may be stored in matrix register 333. Vector 335 was previously stored and rotated in a previous iteration (e.g., the second iteration) of the matrix-vector multiplication operation. A third data value 338-2 of vector 335 may be stored in register 339-0. Third data value 338-2 may be provided to each of MAC units 332. Different data values ​​from column 337-2 may be provided to each of MAC units 332. For example, data value M02 may be provided to MAC unit 332-0, data value M12 may be provided to MAC unit 332-1, data value M22 may be provided to MAC unit 332-2, and data value M32 may be provided to MAC unit 332-3.

[0056] The MAC unit 332 may multiply the data values ​​received from the matrix register 333 and the vector register 339. For example, the MAC unit 332-0 may multiply the data value M02 and the data value 338-2, the MAC unit 332-1 may multiply the data value M12 and the data value 338-2, the MAC unit 332-2 may multiply the data value M22 and the data value 338-2, and the MAC unit 332-3 may multiply the data value M32 and the data value 338-2.

[0057] The results of the multiplication operations may be accumulated with previous results stored in the accumulators of the MAC units 332. For example, the result of the multiplication operation performed on the data value M02 using the data value 338-2 may be accumulated with the data value stored in the accumulator of the MAC unit 332-0. The result of the multiplication operation performed on the data value M12 using the data value 338-2 may be accumulated with the data value stored in the accumulator of the MAC unit 332-1. The result of the multiplication operation performed on the data value M22 using the data value 338-2 may be accumulated with the data value stored in the accumulator of the MAC unit 332-2. The result of the multiplication operation performed on the data value M32 using the data value 338-2 may be accumulated with the data value stored in the accumulator of the MAC unit 332-3.

[0058] Once column 337-2 of matrix 334 has been multiplied by data value 338-2 of vector 335, the data values ​​stored in register 339 may be rotated. For example, data value 338-3 may be stored in register 339-0, data value 338-0 may be stored in register 339-1, data value 338-1 may be stored in register 339-2, and data value 338-2 may be stored in register 339-3. The rotation of the data values ​​in register 339, the completion of the multiplication operation performed by MAC unit 332, and / or the accumulation of the results of the multiplication operation performed by MAC unit 332 may conclude the third iteration of the matrix-vector multiplication operation.

[0059] In the fourth iteration of the matrix-vector multiplication operation, column 337-3 of data values ​​may be stored in matrix register 333. Vector 335 was previously stored and rotated in a previous iteration of the matrix-vector multiplication operation (e.g., the third iteration). A fourth data value 338-3 of vector 335 may be stored in register 339-0. Fourth data value 338-3 may be provided to each of MAC units 332. Different data values ​​from column 337-3 may be provided to each of MAC units 332. For example, data value M03 may be provided to MAC unit 332-0, data value M13 may be provided to MAC unit 332-1, data value M23 may be provided to MAC unit 332-2, and data value M33 may be provided to MAC unit 332-3.

[0060] The MAC unit 332 may multiply the data values ​​received from the matrix register 333 and the vector register 339. For example, the MAC unit 332-0 may multiply the data value M03 and the data value 338-3, the MAC unit 332-1 may multiply the data value M13 and the data value 338-3, the MAC unit 332-2 may multiply the data value M23 and the data value 338-3, and the MAC unit 332-3 may multiply the data value M33 and the data value 338-3.

[0061] The results of the multiplication operations may be accumulated with the previous results stored in the accumulators of the MAC units 332. For example, the result of the multiplication operation performed on the data value M03 using the data value 338-3 may be accumulated with the data value stored in the accumulator of the MAC unit 332-0. The result of the multiplication operation performed on the data value M13 using the data value 338-3 may be accumulated with the data value stored in the accumulator of the MAC unit 332-1. The result of the multiplication operation performed on the data value M23 using the data value 338-3 may be accumulated with the data value stored in the accumulator of the MAC unit 332-2. The result of the multiplication operation performed on the data value M33 using the data value 338-3 may be accumulated with the data value stored in the accumulator of the MAC unit 332-3.

[0062] Once column 337-3 of matrix 334 has been multiplied by data value 338-3 of vector 335, the data value stored in register 339 may or may not be rotated. Completion of the multiplication operation performed by MAC unit 332 and / or accumulation of the results of the multiplication operation performed by MAC unit 332 may conclude the fourth iteration of the matrix-vector multiplication operation.

[0063] The accumulated results stored in the MAC units 332 are result vectors 336. For example, the accumulated result stored in MAC unit 332-0 is data value 338-4 (e.g., Y0). The accumulated result stored in MAC unit 332-1 is data value 338-5 (e.g., Y1). The accumulated result stored in MAC unit 332-2 is data value 338-6 (e.g., Y2). The accumulated result stored in MAC unit 332-3 is data value 338-7 (e.g., Y3).

[0064] The accumulated results (e.g., Y0, Y1, Y2, Y3) may be read from the MAC unit 332 and / or provided to an I / O line. The accumulated results may be referred to as a result vector 336. The result vector 336 may be provided to an I / O line of the memory device in the same amount of time as a single column of memory addresses (e.g., a 256 prefetch) used to read the matrix 334 and / or vector 335 from the memory bank 330.

[0065] Although the examples described herein multiply a 4x4 matrix 334 by a 1x4 vector 335 to produce a 1x4 result vector, the embodiments described herein may be implemented on matrices and vectors having different sizes and / or dimensions. Figure 4 Provides an example that performs matrix-vector multiplication on a 4x8 matrix and a 1x4 vector.

[0066] Figure 4FIG. 4 is a block diagram of a plurality of memory unit banks 430 - 1 , 430 - 2 and a PU 402 according to several embodiments of the present disclosure. The PU 402 includes a matrix register 433 , a vector register 439 , and a MAC unit 432 .

[0067] In various examples, matrix 434 and vector 435 may be stored in different memory banks. For example, matrix 434 is stored in memory bank 430-1 (eg, memory bank A) and vector 435 is stored in memory bank 430-2 (eg, memory bank B). Figure 3 Compared with the matrix 334, Figure 4 438-7 in a first number of iterations and may be used to generate data values ​​438-8, 438-9, 438-10, 438-11 in a second number of iterations. The data values ​​438-4, 438-5, 438-6, 438-7 may be stored in the MAC unit 432 during a first number of iterations, while the data values ​​438-8, 438-9, 438-10, 438-11 may be stored in the MAC unit 432 during a second number of iterations. The data values ​​438-4, 438-5, 438-6, 438-7 and the data values ​​438-8, 438-9, 438-10, 438-11 may be stored in the same number of registers before being provided to the I / O lines as the result vector 436.

[0068] like Figure 4 , matrix 434 can be stored in memory bank 430-1 using a column-major memory layout. As used herein, a column-major memory layout refers to a matrix layout in which the columns of the matrix are stored in a memory bank so that they can be read from the memory bank together. For example, the columns of matrix 434 are stored in memory bank 430-1 so that the columns or portions 437-0, 437-1, 437-2, 437-3, 437-4, 437-5, 437-6, 437-7 of the columns can be read from memory bank 430-1 together. Portions 437-0, 437-4 comprise the first column of matrix 434, portions 437-1, 437-5 comprise the second column of matrix 434, portions 437-2, 437-6 comprise the third column of matrix 434, and portions 437-3, 437-7 comprise the fourth column of matrix 434.

[0069] Each of portions 437-0, 437-1, 437-2, 437-3, 437-4, 437-5, 437-6, and 437-7 (referred to as portions 437) may include a number of data values. For example, each of portions 437 includes four data values. Portion 437-0 includes data values ​​M00, M10, M20, and M30. Portion 437-1 includes data values ​​M01, M11, M21, and M31. Portion 437-2 includes data values ​​M02, M12, M22, and M32. Portion 437-3 includes data values ​​M03, M13, M23, and M33. Portion 437-4 includes data values ​​M40, M50, M60, and M70. Portion 437-5 includes data values ​​M41, M51, M61, and M71. Portion 437-6 includes data values ​​M42, M52, M62, and M72. Portion 437-7 includes data values ​​M43, M53, M63, and M73.

[0070] The matrix-vector multiplication operation performed by PU 402 includes 8 iterations. Each iteration of the matrix-vector multiplication operation corresponds to a different one of portions 437. In each iteration of the matrix-vector multiplication operation, a different portion of the matrix may be stored in matrix register 433. Vector 435 may be read from memory bank 430-2 and stored once in vector register 439. Data values ​​438-0, 438-1, 438-2, 438-3 may be read after each iteration of the matrix-vector multiplication operation in a manner similar to Figure 3 The rotation is performed in the manner described in the example.

[0071] Figure 4The matrix-vector multiplication operation of a 4x8 matrix multiplied by a 1x4 vector is defined as Y0 = M00 × v0 + M01 × V1 + M02 × V2 + M03 × V3, Y1 = M10 × V0 + M11 × V1 + M12 × V2 + M13 × V3, Y2 = M20 × V0 + M21 × V1 + M22 × V2 + M23 × V3, Y3 = M30 × v0 + M31 × V1 + M32 × V2 + M33 × V3, Y4 = M40 × V0 + M41 × V1 + M42 × V2 + M43 × V3, and Y5 = M50 × V0 + M51 × V1 + M52 × V2 + M53 × V3. Each of Y0, Y1, Y2, Y3, Y4, Y5, Y6, Y7 may be calculated by MAC unit 432. For example, Y0 and Y4 may be calculated by MAC unit 432-0, Y1 and Y5 may be calculated by MAC unit 432-1, Y2 and Y6 may be calculated by MAC unit 432-2, and Y3 and Y7 may be calculated by MAC unit 432-3. Although a particular MAC unit 432 is described as calculating a particular data value for result vector 436, any of the MAC units 432 may be used to calculate any of the data values ​​for result vector 436.

[0072] In various examples, vector 435 may be read from memory bank 430-2 and stored in vector register 439. The host may provide a command to the memory device to cause the memory device to read vector 435. PU 402 may intercept vector 435 and may store vector 435 in vector register 439 in response to receiving the command from the host. For example, data value 438-0 of vector 435 may be stored in register 439-0, data value 438-1 of vector 435 may be stored in register 439-1, data value 438-2 of vector 435 may be stored in register 439-2, and data value 438-3 of vector 435 may be stored in register 439-3.

[0073] Matrix 434 may be read from memory bank 430-1 and stored in matrix register 433. A host may provide a command to the memory device to cause the memory device to read matrix 434. For each iteration of the matrix-vector multiplication operation, a different portion 437 of a column of matrix 434 may be stored in matrix register 433. For example, portion 437-0 of a first column of matrix 434 may be stored in matrix register 433 in a first iteration of the matrix-vector multiplication operation, portion 437-1 of a second column of matrix 434 may be stored in matrix register 433 in a second iteration of the matrix-vector multiplication operation, portion 437-2 of a third column of matrix 434 may be stored in matrix register 433 in a third iteration of the matrix-vector multiplication operation, and portion 437-3 of a fourth column of matrix 434 may be stored in matrix register 433 in a fourth iteration of the matrix-vector multiplication operation. Portion 437-4 of the first column of matrix 434 may be stored in matrix register 433 during the fifth iteration of the matrix-vector multiplication operation, portion 437-5 of the second column of matrix 434 may be stored in matrix register 433 during the sixth iteration of the matrix-vector multiplication operation, portion 437-6 of the third column of matrix 434 may be stored in matrix register 433 during the seventh iteration of the matrix-vector multiplication operation, and portion 437-7 of the fourth column of matrix 434 may be stored in matrix register 433 during the eighth iteration of the matrix-vector multiplication operation.

[0074] In a first iteration of the matrix-vector multiplication operation, portion 437-0 of the first column and vector 435 may be stored in matrix register 433 and vector register 439, respectively. A first data value 438-0 of vector 435 may be stored in register 439-0. First data value 438-0 may be provided to each of MAC units 432. For example, different instances of first data value 438-0 may be provided to MAC unit 432-0, MAC unit 432-1, MAC unit 432-2, and MAC unit 432-3. Different data values ​​from portion 437-0 of the first column may be provided to each of MAC units 432. For example, data value M00 may be provided to MAC unit 432-0, data value M10 may be provided to MAC unit 432-1, data value M20 may be provided to MAC unit 432-2, and data value M30 may be provided to MAC unit 432-3.

[0075] The MAC unit 432 may multiply data values ​​received from the matrix register 433 and the vector register 439. For example, the MAC unit 432-0 may multiply the data value M00 and the data value 438-0, the MAC unit 432-1 may multiply the data value M10 and the data value 438-0, the MAC unit 432-2 may multiply the data value M20 and the data value 438-0, and the MAC unit 432-3 may multiply the data value M30 and the data value 438-0.

[0076] The results of the multiplication operations may be stored in the accumulators of the MAC unit 432. For example, the result of the multiplication operation performed on the data value M00 using the data value 438-0 may be stored in the accumulator of the MAC unit 432-0. The result of the multiplication operation performed on the data value M10 using the data value 438-0 may be stored in the accumulator of the MAC unit 432-1. The result of the multiplication operation performed on the data value M20 using the data value 438-0 may be stored in the accumulator of the MAC unit 432-2. The result of the multiplication operation performed on the data value M30 using the data value 438-0 may be stored in the accumulator of the MAC unit 432-3.

[0077] Once the portion 437-0 of the first column of the matrix 434 has been multiplied by the data value 438-0 of the vector 435, the data value stored in the register 439 may be combined similarly to Figure 3 The rotation is performed in the manner described. For example, data value 438-1 may be stored in register 439-0. The rotation of register 439, the completion of the multiplication operation performed by MAC unit 432, and / or the accumulation of the results of the multiplication operation performed by MAC unit 432 may conclude the first iteration of the matrix-vector multiplication operation.

[0078] In the second iteration of the matrix-vector multiplication operation, portion 437-1 of the second column data value may be stored in matrix register 433. Vector 435 was previously stored and rotated in the previous iteration (e.g., the first iteration) of the matrix-vector multiplication operation. Second data value 438-1 of vector 435 may be stored in register 439-0. Second data value 438-1 may be provided to each of MAC units 432. A different data value from portion 437-1 of the second column may be provided to each of MAC units 432. For example, data value M01 may be provided to MAC unit 432-0, data value M11 may be provided to MAC unit 432-1, data value M21 may be provided to MAC unit 432-2, and data value M31 may be provided to MAC unit 432-3.

[0079] The MAC unit 432 may multiply the data values ​​received from the matrix register 433 and the vector register 439. For example, the MAC unit 432-0 may multiply the data value M01 and the data value 438-1, the MAC unit 432-1 may multiply the data value M11 and the data value 438-1, the MAC unit 432-2 may multiply the data value M21 and the data value 438-1, and the MAC unit 432-3 may multiply the data value M31 and the data value 438-1.

[0080] The results of the multiplication operations may be accumulated with previous results stored in the accumulators of the MAC unit 432. For example, the result of the multiplication operation performed on the data value M01 using the data value 438-1 may be accumulated with the data value stored in the accumulator of the MAC unit 432-0. The result of the multiplication operation performed on the data value M11 using the data value 438-1 may be accumulated with the data value stored in the accumulator of the MAC unit 432-1. The result of the multiplication operation performed on the data value M21 using the data value 438-1 may be accumulated with the data value stored in the accumulator of the MAC unit 432-2. The result of the multiplication operation performed on the data value M31 using the data value 438-1 may be accumulated with the data value stored in the accumulator of the MAC unit 432-3.

[0081] Once portion 437-1 of the second column of matrix 434 has been multiplied by data value 438-1 of vector 435, the data value stored in register 439 may be rotated. For example, data value 438-2 may be stored in register 439-0. The rotation of register 439, the completion of the multiplication operation performed by MAC unit 432, and / or the accumulation of the result of the multiplication operation performed by MAC unit 432 may conclude the second iteration of the matrix-vector multiplication operation.

[0082] In the third iteration of the matrix-vector multiplication operation, portion 437-2 of the third column data value may be stored in matrix register 433. Vector 435 was previously stored and rotated in the previous iteration of the matrix-vector multiplication operation (e.g., the second iteration). A third data value 438-2 of vector 435 may be stored in register 439-0. Third data value 438-2 may be provided to each of MAC units 432. A different data value from portion 437-2 of the third column may be provided to each of MAC units 432. For example, data value M02 may be provided to MAC unit 432-0, data value M12 may be provided to MAC unit 432-1, data value M22 may be provided to MAC unit 432-2, and data value M32 may be provided to MAC unit 432-3.

[0083] The MAC unit 432 may multiply the data values ​​received from the matrix register 433 and the vector register 439. For example, the MAC unit 432-0 may multiply the data value M02 and the data value 438-2, the MAC unit 432-1 may multiply the data value M12 and the data value 438-2, the MAC unit 432-2 may multiply the data value M22 and the data value 438-2, and the MAC unit 432-3 may multiply the data value M32 and the data value 438-2.

[0084] The results of the multiplication operations may be accumulated with previous results stored in the accumulators of the MAC unit 432. For example, the result of the multiplication operation performed on the data value M02 using the data value 438-2 may be accumulated with the data value stored in the accumulator of the MAC unit 432-0. The result of the multiplication operation performed on the data value M12 using the data value 438-2 may be accumulated with the data value stored in the accumulator of the MAC unit 432-1. The result of the multiplication operation performed on the data value M22 using the data value 438-2 may be accumulated with the data value stored in the accumulator of the MAC unit 432-2. The result of the multiplication operation performed on the data value M32 using the data value 438-2 may be accumulated with the data value stored in the accumulator of the MAC unit 432-3.

[0085] Once portion 437-2 of the third column of matrix 434 has been multiplied by data value 438-2 of vector 435, the data values ​​stored in register 439 may be rotated. For example, data value 438-3 may be stored in register 439-0. The rotation of register 439, the completion of the multiplication operation performed by MAC unit 432, and / or the accumulation of the results of the multiplication operation performed by MAC unit 432 may conclude the third iteration of the matrix-vector multiplication operation.

[0086] In the fourth iteration of the matrix-vector multiplication operation, portion 437-3 of the fourth column data value may be stored in matrix register 433. Vector 435 was previously stored and rotated in a previous iteration of the matrix-vector multiplication operation (e.g., the third iteration). A fourth data value 438-3 of vector 435 may be stored in register 439-0. Fourth data value 438-3 may be provided to each of MAC units 432. A different data value from portion 437-3 of the fourth column may be provided to each of MAC units 432. For example, data value M03 may be provided to MAC unit 432-0, data value M13 may be provided to MAC unit 432-1, data value M23 may be provided to MAC unit 432-2, and data value M33 may be provided to MAC unit 432-3.

[0087] The MAC unit 432 may multiply the data values ​​received from the matrix register 433 and the vector register 439. For example, the MAC unit 432-0 may multiply the data value M03 and the data value 438-3, the MAC unit 432-1 may multiply the data value M13 and the data value 438-3, the MAC unit 432-2 may multiply the data value M23 and the data value 438-3, and the MAC unit 432-3 may multiply the data value M33 and the data value 438-3.

[0088] The results of the multiplication operations may be accumulated with previous results stored in the accumulators of the MAC unit 432. For example, the result of the multiplication operation performed on the data value M03 using the data value 438-3 may be accumulated with the data value stored in the accumulator of the MAC unit 432-0. The result of the multiplication operation performed on the data value M13 using the data value 438-3 may be accumulated with the data value stored in the accumulator of the MAC unit 432-1. The result of the multiplication operation performed on the data value M23 using the data value 438-3 may be accumulated with the data value stored in the accumulator of the MAC unit 432-2. The result of the multiplication operation performed on the data value M33 using the data value 438-3 may be accumulated with the data value stored in the accumulator of the MAC unit 432-3.

[0089] Once portion 437-3 of the fourth column of matrix 434 has been multiplied by data value 438-3 of vector 435, the data value stored in register 439 may be rotated. For example, data value 438-0 may be stored in register 439-0. The rotation of register 439, the completion of the multiplication operation performed by MAC unit 432, and / or the accumulation of the result of the multiplication operation performed by MAC unit 432 may conclude the fourth iteration of the matrix-vector multiplication operation.

[0090] The data values ​​stored in the MAC unit 432 may be data values ​​438-4, 438-5, 438-6, 438-7 of the result vector 436. The data values ​​438-4, 438-5, 438-6, 438-7 may be provided to the I / O lines, while the data values ​​438-8, 438-9, 438-10, 438-11 may be provided to the I / O lines at different times. It may be desirable to provide the entire result vector 436 simultaneously, so that the data values ​​438-4, 438-5, 438-6, 438-7 may be read from the MAC unit 432 and stored in different memories (e.g., registers) not shown until the data values ​​438-8, 438-9, 438-10, 438-11 are generated in the fifth through eighth iterations of the matrix-vector multiplication operation. Given the different data values ​​(eg, data values ​​438 - 8 , 438 - 9 , 438 - 10 , 438 - 11 ) that the MAC unit 432 will use to generate the result vector 436 , the MAC unit 432 may be reset after the fourth iteration.

[0091] In the fifth iteration of the matrix-vector multiplication operation, portion 437-4 of the first column may be stored in matrix register 433. Vector 435 was previously stored and rotated in a previous iteration of the matrix-vector multiplication operation (e.g., the fourth iteration). A first data value 438-0 of vector 435 may be stored in register 439-0. First data value 438-0 may be provided to each of MAC units 432. A different data value from portion 437-0 of the first column may be provided to each of MAC units 432. For example, data value M40 may be provided to MAC unit 432-0, data value M50 may be provided to MAC unit 432-1, data value M60 may be provided to MAC unit 432-2, and data value M70 may be provided to MAC unit 432-3.

[0092] The MAC unit 432 may multiply the data values ​​received from the matrix register 433 and the vector register 439. For example, the MAC unit 432-0 may multiply the data value M40 and the data value 438-0, the MAC unit 432-1 may multiply the data value M50 and the data value 438-0, the MAC unit 432-2 may multiply the data value M60 and the data value 438-0, and the MAC unit 432-3 may multiply the data value M70 and the data value 438-0.

[0093] The results of the multiplication operations may be stored in the accumulators of the MAC unit 432. For example, the result of the multiplication operation performed on the data value M40 using the data value 438-0 may be stored in the accumulator of the MAC unit 432-0. The result of the multiplication operation performed on the data value M50 using the data value 438-0 may be stored in the accumulator of the MAC unit 432-1. The result of the multiplication operation performed on the data value M60 using the data value 438-0 may be stored in the accumulator of the MAC unit 432-2. The result of the multiplication operation performed on the data value M70 using the data value 438-0 may be stored in the accumulator of the MAC unit 432-3.

[0094] Once portion 437-4 of the first column of matrix 434 has been multiplied by data value 438-0 of vector 435, the data value stored in register 439 may be rotated. For example, data value 438-1 may be stored in register 439-0. The rotation of register 439, the completion of the multiplication operation performed by MAC unit 432, and / or the accumulation of the result of the multiplication operation performed by MAC unit 432 may conclude the fifth iteration of the matrix-vector multiplication operation.

[0095] In the sixth iteration of the matrix-vector multiplication operation, portion 437-5 of the second column data value may be stored in matrix register 433. Vector 435 was previously stored and rotated in a previous iteration of the matrix-vector multiplication operation (e.g., the fifth iteration). Second data value 438-1 of vector 435 may be stored in register 439-0. Second data value 438-1 may be provided to each of MAC units 432. A different data value from portion 437-5 of the second column may be provided to each of MAC units 432. For example, data value M41 is provided to MAC unit 432-0, data value M51 is provided to MAC unit 432-1, data value M61 is provided to MAC unit 432-2, and data value M71 is provided to MAC unit 432-3.

[0096] The MAC unit 432 may multiply the data values ​​received from the matrix register 433 and the vector register 439. For example, the MAC unit 432-0 may multiply the data value M41 and the data value 438-1, the MAC unit 432-1 may multiply the data value M51 and the data value 438-1, the MAC unit 432-2 may multiply the data value M61 and the data value 438-1, and the MAC unit 432-3 may multiply the data value M71 and the data value 438-1.

[0097] The results of the multiplication operations may be accumulated with previous results stored in the accumulators of the MAC unit 432. For example, the result of the multiplication operation performed on the data value M41 using the data value 438-1 may be accumulated with the data value stored in the accumulator of the MAC unit 432-0. The result of the multiplication operation performed on the data value M51 using the data value 438-1 may be accumulated with the data value stored in the accumulator of the MAC unit 432-1. The result of the multiplication operation performed on the data value M61 using the data value 438-1 may be accumulated with the data value stored in the accumulator of the MAC unit 432-2. The result of the multiplication operation performed on the data value M71 using the data value 438-1 may be accumulated with the data value stored in the accumulator of the MAC unit 432-3.

[0098] Once portion 437-5 of the second column of matrix 434 has been multiplied by data value 438-1 of vector 435, the data value stored in register 439 may be rotated. For example, data value 438-2 may be stored in register 439-0. The rotation of register 439, the completion of the multiplication operation performed by MAC unit 432, and / or the accumulation of the result of the multiplication operation performed by MAC unit 432 may conclude the sixth iteration of the matrix-vector multiplication operation.

[0099] In the seventh iteration of the matrix-vector multiplication operation, portion 437-6 of the third column data value may be stored in matrix register 433. Vector 435 was previously stored and rotated in a previous iteration of the matrix-vector multiplication operation (e.g., the sixth iteration). A third data value 438-2 of vector 435 may be stored in register 439-0. Third data value 438-2 may be provided to each of MAC units 432. A different data value from portion 437-6 of the third column may be provided to each of MAC units 432. For example, data value M42 may be provided to MAC unit 432-0, data value M52 may be provided to MAC unit 432-1, data value M62 may be provided to MAC unit 432-2, and data value M72 may be provided to MAC unit 432-3.

[0100] The MAC unit 432 may multiply the data values ​​received from the matrix register 433 and the vector register 439. For example, the MAC unit 432-0 may multiply the data value M42 and the data value 438-2, the MAC unit 432-1 may multiply the data value M52 and the data value 438-2, the MAC unit 432-2 may multiply the data value M62 and the data value 438-2, and the MAC unit 432-3 may multiply the data value M72 and the data value 438-2.

[0101] The results of the multiplication operations may be accumulated with previous results stored in the accumulators of the MAC unit 432. For example, the result of the multiplication operation performed on the data value M42 using the data value 438-2 may be accumulated with the data value stored in the accumulator of the MAC unit 432-0. The result of the multiplication operation performed on the data value M52 using the data value 438-2 may be accumulated with the data value stored in the accumulator of the MAC unit 432-1. The result of the multiplication operation performed on the data value M62 using the data value 438-2 may be accumulated with the data value stored in the accumulator of the MAC unit 432-2. The result of the multiplication operation performed on the data value M72 using the data value 438-2 may be accumulated with the data value stored in the accumulator of the MAC unit 432-3.

[0102] Once portion 437-6 of the third column of matrix 434 has been multiplied by data value 438-2 of vector 435, the data values ​​stored in register 439 may be rotated. For example, data value 438-3 may be stored in register 439-0. The rotation of register 439, the completion of the multiplication operation performed by MAC unit 432, and / or the accumulation of the results of the multiplication operation performed by MAC unit 432 may conclude the seventh iteration of the matrix-vector multiplication operation.

[0103] In the eighth iteration of the matrix-vector multiplication operation, portion 437-7 of the fourth column data value may be stored in matrix register 433. Vector 435 was previously stored and rotated in the previous iteration of the matrix-vector multiplication operation (e.g., the seventh iteration). A fourth data value 438-3 of vector 435 may be stored in register 439-0. Fourth data value 438-3 may be provided to each of MAC units 432. A different data value from portion 437-7 of the fourth column may be provided to each of MAC units 432. For example, data value M43 is provided to MAC unit 432-0, data value M53 is provided to MAC unit 432-1, data value M63 is provided to MAC unit 432-2, and data value M73 is provided to MAC unit 432-3.

[0104] The MAC unit 432 may multiply the data values ​​received from the matrix register 433 and the vector register 439. For example, the MAC unit 432-0 may multiply the data value M43 and the data value 438-3, the MAC unit 432-1 may multiply the data value M53 and the data value 438-3, the MAC unit 432-2 may multiply the data value M63 and the data value 438-3, and the MAC unit 432-3 may multiply the data value M73 and the data value 438-3.

[0105] The results of the multiplication operations may be accumulated with previous results stored in the accumulators of the MAC units 432. For example, the result of the multiplication operation performed on the data value M43 using the data value 438-3 may be accumulated with the data value stored in the accumulator of the MAC unit 432-0. The result of the multiplication operation performed on the data value M53 using the data value 438-3 may be accumulated with the data value stored in the accumulator of the MAC unit 432-1. The result of the multiplication operation performed on the data value M63 using the data value 438-3 may be accumulated with the data value stored in the accumulator of the MAC unit 432-2. The result of the multiplication operation performed on the data value M73 using the data value 438-3 may be accumulated with the data value stored in the accumulator of the MAC unit 432-3.

[0106] Once portion 437-7 of the fourth column of matrix 434 has been multiplied by data value 438-3 of vector 435, the matrix-vector multiplication operation may be complete. The accumulated data values ​​(e.g., data values ​​438-8, 438-9, 438-10, 438-11) in MAC unit 432 may be read and stored along with data values ​​438-4, 438-5, 438-6, 438-7. Once stored together, data values ​​438-4, 438-5, 438-6, 438-7, 438-8, 438-9, 438-10, 438-11 may be read and provided to an I / O line or may be stored back into one or both of memory banks 430-1, 430-2.

[0107] although Figure 4 The example described in includes a 4x8 matrix, but the matrix can have different dimensions. For example, matrix 434 can include more than eight rows. The examples described herein can be applied to matrices of any dimension.

[0108] Figure 5 An example flow chart illustrating a method 580 for performing a matrix-vector multiplication operation using a processing unit in memory according to several embodiments of the present disclosure is shown. The method may be executed by a memory device of a computing system.

[0109] At 581, a memory device (e.g., Figure 1 The PU of the memory device 120 may receive data stored in a memory bank (e.g., Figure 3 or Figure 4 330 or 430-1) in a matrix of data values ​​(e.g., Figure 3 or Figure 4 334 or 434), wherein the processing unit is coupled to a bank of a memory device. At 582, the PU may receive a vector of data values ​​(e.g., Figure 3 or Figure 4 At 583, the PU may store the vector in a first plurality of registers (e.g., Figure 3 or Figure 4 339 or 439) and stores the first column in a second plurality of registers (e.g., Figure 3 or Figure 4 At 584, the PU may perform a first plurality of multiplication operations on the first data values ​​of the vector using the first plurality of data values ​​of the column, wherein the first plurality of multiplication operations are performed by a plurality of MAC units and wherein the first plurality of multiplication operations are performed by different MAC units of the plurality of MAC units.

[0110] At 585, the PU may receive a second column of the matrix. At 586, the PU may perform a second plurality of multiplication operations on the second data values ​​of the vector using a second plurality of data values ​​of the second column, wherein each of the second plurality of multiplication operations may be performed by a different MAC unit from the plurality of MAC units. At 587, the PU may store the outputs of the first plurality of multiplication operations and the second plurality of multiplication operations in a memory bank.

[0111] The PU may receive the vector from a bank of a memory device or a different bank of a memory device (e.g., the vector may be included in the same bank as the matrix, or a different bank than the matrix). In various examples, the PU may receive the vector from a bank of a different memory device.

[0112] In various examples, the PU may accumulate the results of the first plurality of multiplication operations and the results of the second plurality of multiplication operations. The results may be accumulated in a MAC unit of the PU.

[0113] The PU may provide the outputs of the first plurality of multiplication operations and the second plurality of multiplication operations from the plurality of MAC units to the I / O lines of the memory device. The outputs of the first plurality of multiplication operations and the second plurality of multiplication operations may be provided from the plurality of MAC units to the I / O lines without performing additional operations on the outputs. For example, the outputs may be provided without combining the outputs with different data values. The outputs may also be provided without performing additional multiplication operations and / or summation operations on the outputs. The output vector may be provided to the I / O lines in the same amount of time that a single column of memory addresses of matrices and / or vectors (e.g., a 256 prefetch) would be provided to the I / O lines.

[0114] In various examples, a PU of a memory device may receive a matrix of data values ​​stored in a memory bank. The PU may receive a vector of data values ​​stored in the memory bank. The PU may perform a first plurality of multiplication operations on first data values ​​of the vector using a first plurality of data values ​​of a first column of the matrix, wherein the first plurality of multiplication operations are performed by a plurality of MAC units, and wherein each of the first plurality of multiplication operations is performed by a different MAC unit from the plurality of MAC units. The PU may perform a second plurality of multiplication operations on second data values ​​of the vector using a second plurality of data values ​​of a second column of the matrix, wherein each of the second plurality of multiplication operations is performed by a different MAC unit from the plurality of MAC units.

[0115] The PU may store a first plurality of data values ​​for a first column and a second plurality of data values ​​for a second column in a first plurality of registers and store a first data value and a second data value for a vector in a second plurality of registers. The PU may send the first plurality of data values ​​from the first plurality of registers to a plurality of MAC units to perform a first plurality of multiplication operations. The PU may send the first data value from the second plurality of registers to a plurality of MAC units to perform the first plurality of multiplication operations.

[0116] The PU may rotate the first and second data values ​​of the vector. The PU may send the second plurality of data values ​​from the first plurality of registers to the plurality of MAC units to perform a second plurality of multiplication operations. The PU may send the second data values ​​from the second plurality of registers to the plurality of MAC units to perform a second plurality of multiplication operations.

[0117] In various examples, a memory bank may store data values ​​of a matrix in a column-major memory layout. A memory bank may store data values ​​in a column-major memory layout rather than in a row-major memory layout. A memory bank may provide data values ​​of a vector to a processing unit in a column-major memory layout.

[0118] In various examples, a MAC unit of the PU may receive a first plurality of data values ​​of a first column of a matrix of data values ​​stored in a first memory bank, wherein each MAC unit from the plurality of MAC units receives a data value different from the first plurality of data values ​​of the first column. The MAC unit may also receive a first data value of a vector of data values ​​stored in a second memory bank, wherein each MAC unit from the plurality of MAC units receives the first data value of the vector.

[0119] The MAC unit may perform a first plurality of multiplication operations on the first data values ​​of the vector using the first plurality of data values ​​of the first column.Each of the first plurality of multiplication operations may be performed by a different MAC unit of the plurality of MAC units.

[0120] The MAC unit may receive a second plurality of data values ​​from a second column of the matrix stored in the first memory bank, wherein each MAC unit in the plurality of MAC units receives a different data value from the second plurality of data values ​​in the second column. The MAC unit may also receive a second data value from the vector, wherein each MAC unit in the plurality of MAC units receives the second data value from the vector.

[0121] The MAC unit may perform a second plurality of multiplication operations using the second data value of the vector and the second plurality of data values ​​of the second column. Each of the second plurality of multiplication operations may be performed by a different MAC unit from the plurality of MAC units. The MAC unit may store outputs of the first plurality of multiplication operations and the second plurality of multiplication operations in a second memory bank.

[0122] Each of the MAC units may multiply the first data value with a different data value from a first plurality of data values ​​from a first column and may store the first plurality of results in accumulators of the plurality of MAC units. Each of the plurality of MAC units may multiply the second data value with a different data value from a second plurality of data values ​​from a second column and may add the second plurality of results to the accumulators of the plurality of MAC units.

[0123] In various examples, an address sequencer can be implemented in a memory device to generate addresses for moving data from the memory device's banks to MAC units. For artificial intelligence (AI) applications, the processing performed by the memory device is a matrix multiply-accumulate (MAC) operation. Performing the processing in the MAC unit allows for the use of serial data rather than random data. When performing MAC operations in a memory device at the bank level, an entire row of the array can be consumed after it is opened, which can reduce the time it takes to perform the operation because opening and closing a row of the memory array takes longer than reading data from that row.

[0124] However, the bank control of the PU of the memory device described herein may allow normal commands of the memory device and / or memory array. Normal commands may include activate row commands, read commands, and / or write commands, as well as other types of commands. The memory device may receive a specific (e.g., special) read command to redirect bank prefetch data (e.g., 256 bits) away from the I / O lines (e.g., DQ) and into the local MAC unit.

[0125] After a row of the memory array has been activated, a read command and column address can be issued within each column-to-column timing constraint (tCCD) of 5ns. For example, there may be 64 column addresses for a row, with a row of 16K bits, 256 bits prefetched, and 64 read commands issued after a row is activated. Each read command contains a command and address supplied off-chip. Knowing that data can be read in the same sequential order, circuitry can be added to the memory device that autonomously increments the address at the memory bank, eliminating the need to load addresses from off-chip. Generating addresses locally saves power and speeds up the reading of data into the MAC unit compared to receiving addresses off-chip.

[0126] In various examples, in a DRAM architecture, reading a matrix of data values ​​from a memory array may be faster when the matrix is ​​read along rows rather than along columns. Circuitry may be implemented in a memory device and / or a bank of memory cells that allows the matrix to be read along rows or along columns. For example, the circuitry may be configured to read the matrix along rows based on a first mode or along columns based on a second mode. Being able to read the matrix along rows or columns increases the flexibility of performing operations using rows or columns of the matrix.

[0127] In various examples, the PU can use the available internal bandwidth of the memory device. For example, rows in each of the memory banks (e.g., 16 memory banks) of the memory device can be activated simultaneously to provide data to the PU. In previous methods, the rows in each of the memory banks of the memory device were not activated simultaneously. After activation, a read command can be executed simultaneously for each memory bank. The read command can obtain the corresponding address from the read command address pins. Since each read command obtains the address from the read command address pins, each memory bank can receive the same column address for the read command. It can be beneficial to initiate read commands at different column addresses in different memory banks of the memory device. Column address offset registers can be implemented in each memory bank of the memory device to cause the incoming address to be offset at each memory bank of the memory device.

[0128] In various examples, the MAC unit can obtain data (e.g., a 256-bit prefetch) every tCCD (e.g., every 5 ns). A MAC unit can be implemented for each bank of the memory device. The MAC unit can process 256 bits as input. However, the MAC unit can perform several operations faster than 5 ns. The die size of the memory device can be reduced by reducing the MAC unit to handle 128-bit data instead of 256-bit data. The 256-bit data provided by the bank every 5 ns can be provided to the MAC unit in two 128-bit portions. Each 128-bit portion can be processed in 25 ns (through MAC serialization). Reducing the size of the MAC unit can reduce the cost of the memory device.

[0129] In several examples, having a MAC unit at each bank of a memory device allows data in the MAC unit to be read from or written to the bank. However, writing to the bank may not be necessary. Artificial intelligence (AI) algorithms may utilize multiplication / accumulation operations. Accumulated data is stored in the MAC unit and utilizes very little space. The ability to write to the bank from the MAC unit may be eliminated. Eliminating the ability to write to the bank from the MAC unit saves die size and simplifies operation. Removing the ability to write to the bank from the MAC unit also provides the benefit of making data concurrency a less significant issue. If the data in the MAC unit needs to be stored, the CPU (external controller) can read the data from the MAC unit and then store the data to the bank.

[0130] In previous approaches, a standard memory array (e.g., a bank) allows for one data path (e.g., a 256-bit prefetch data path) per tCCD (e.g., 5 ns). This is a physical (e.g., architectural) choice to meet DRAM datasheet specifications. Increasing the number of bits prefetched from a bank at a time does not benefit a standard memory array (e.g., DRAM). However, for a bank-local processing unit (e.g., a PU / MAC unit as described herein), increasing this data path would allow for higher bandwidth. For example, this local processing unit could double this path with a small increase in die size. The number of DSAs (data sense amplifiers) could be doubled and the global data lines (GIOs) could be segmented to accommodate the increased data path between the bank and the MAC unit.

[0131] Figure 6 An example machine illustrating a computer system 690 within which a set of instructions for causing the machine to perform any one or more of the methodologies discussed herein may be executed. In some embodiments, the computer system 690 may correspond to a host system (e.g., Figure 1 110 ) that includes, is coupled to, or utilizes a memory system (e.g., Figure 1 memory device 120) or may be used to execute a PU (e.g., Figure 1 In some embodiments, the machine may be connected (e.g., using a network) to other machines. The machine may operate in the capacity of a server or a client user machine in a client-server network environment, as a peer machine in a peer-to-peer (or distributed) network environment, or as a server or a client user machine in a cloud computing infrastructure or environment.

[0132] The machine may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a network appliance, a server, a network router, a switch or a bridge, or any machine capable of executing (sequentially or otherwise) a set of instructions that specify actions to be taken by the machine. Further, while a single machine is described, the term "machine" shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

[0133] The example computer system 690 includes a processing device 691, a main memory 693 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) (e.g., synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM)), etc.), a static memory 697 (e.g., flash memory, static random access memory (SRAM)), etc.), and a data storage system 698, which communicate with each other via a bus 696.

[0134] Processing device 691 represents one or more general-purpose processing devices, such as a microprocessor, a central processing unit, or the like. More specifically, the processing device may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor implementing another instruction set, or multiple processors implementing a combination of instruction sets. Processing device 691 may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), a network processor, or the like. Processing device 691 is configured to execute instructions 692 for performing the operations and steps discussed herein. Computer system 690 may further include a network interface device 694 for communicating over a network 695.

[0135] The data storage system 698 may include a machine-readable storage medium 699 (also referred to as a computer-readable medium) on which is stored one or more sets of instructions 692 or software embodying any one or more of the methodologies or functionality described herein. The instructions 692 may also reside, completely or at least partially, within the main memory 693 and / or within the processing device 691 during execution thereof by the computer system 690, the main memory 693 and the processing device 691 also constituting machine-readable storage media.

[0136] In one embodiment, instructions 692 include implementing instructions corresponding to Figure 1 The machine-readable storage medium 699 is shown as a single medium in the example embodiment, but the term "machine-readable storage medium" should be taken to include a single medium or multiple media that store one or more sets of instructions. The term "machine-readable storage medium" should also be taken to include any medium that can store or encode a set of instructions for execution by a machine and cause the machine to perform any one or more of the methodologies of the present disclosure. Thus, the term "machine-readable storage medium" should be taken to include, but not be limited to, solid-state memory, optical media, and magnetic media.

[0137] Although specific embodiments have been illustrated and described herein, it will be understood by those skilled in the art that arrangements calculated to achieve the same results may be substituted for the specific embodiments shown. The present disclosure is intended to cover adaptations or variations of the various embodiments of the present disclosure. It should be understood that the above description has been made in an illustrative and non-restrictive manner. After reviewing the above description, those skilled in the art will understand the combination of the above embodiments and other embodiments not explicitly described herein. The scope of the various embodiments of the present disclosure includes other applications in which the above structures and methods are used. Therefore, the scope of the various embodiments of the present disclosure should be determined with reference to the appended claims and the full scope of equivalents to which such claims are entitled.

[0138] In the foregoing detailed description, various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the disclosed embodiments of the disclosure necessarily utilize more features than are expressly recited in each claim. Rather, as the appended claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Therefore, the appended claims are hereby incorporated into the detailed description, with each claim standing on its own as a separate embodiment.

Claims

1. An apparatus for performing a matrix-vector multiplication operation using a processing unit in a memory, comprising: Memory cell banks (130, 330, 430-1, 430-2); a processing unit (102, 302, 402) coupled to the bank of memory cells and configured to: receiving a matrix of data values ​​stored in the memory bank (334, 434); receiving a vector of data values ​​stored in the memory bank (335, 435); performing a first plurality of multiplication operations on first data values ​​of the vector using a first plurality of data values ​​of a first column (337-0, 337-1, 337-2, 337-3, 437-0, 437-1, 437-2, 437-3, 437-4, 437-5, 437-6, 437-7, 437-9) of the matrix, wherein the first plurality of multiplication operations are performed by a plurality of multiply-accumulate MAC units (243), and wherein each of the first plurality of multiplication operations is performed by a different MAC unit among the plurality of MAC units; and performing a second plurality of multiplication operations on second data values ​​of the vector using a second plurality of data values ​​of a second column (337-0, 337-1, 337-2, 337-3, 437-0, 437-1, 437-2, 437-3, 437-4, 437-5, 437-6, 437-7, 437-9) of the matrix, wherein each of the second plurality of multiplication operations is performed by a different MAC unit (243) among the plurality of MAC units.

2. The apparatus of claim 1 , wherein the processing unit is further configured to store the first plurality of data values ​​of the first column and the second plurality of data values ​​of the second column in a first plurality of registers and to store the first data value and the second data value of the vector in a first plurality of registers.

3. The apparatus of claim 2, wherein the processing unit is further configured to send the first plurality of data values ​​from the first plurality of registers to the plurality of MAC units to perform the first plurality of multiplication operations.

4. The apparatus of claim 2, wherein the processing unit is further configured to send the first data values ​​from a second plurality of registers to the plurality of MAC units to perform the first plurality of multiplication operations.

5. The apparatus of claim 4, wherein the processing unit is further configured to rotate the first data value and the second data value of the vector.

6. The apparatus of claim 5, wherein the processing unit is further configured to send the second plurality of data values ​​from the first plurality of registers to the plurality of MAC units to perform the second plurality of multiplication operations.

7. The apparatus of claim 4, wherein the processing unit is further configured to send the second data values ​​from the second plurality of registers to the plurality of MAC units to perform the second plurality of multiplication operations.

8. An apparatus according to any one of claims 1 to 7, wherein the memory bank is configured to store the data values ​​of the matrix in a column-major memory layout and wherein the memory bank is configured to provide the data values ​​of the vector to the processing unit in the column-major memory layout.

9. A method for performing a matrix-vector multiplication operation using a processing unit in a memory, comprising: receiving, at a processing unit (102, 302, 402) of a memory device (120), a first column (337-0, 337-1, 337-2, 337-3, 437-0, 437-1, 437-2, 437-3, 437-4, 437-5, 437-6, 437-7, 437-9) of a matrix of data values ​​(334, 434) stored in a memory bank (130, 330, 430-1, 430-2) of the memory device, wherein the processing unit is coupled to the memory bank; receiving a data value vector (335, 435) at the processing unit; storing the vector in a first plurality of registers (239); performing a first plurality of multiplication operations on a first data value of the vector using a first plurality of data values ​​of the column, wherein the first plurality of multiplication operations are performed by a plurality of multiply-accumulate MAC units (243), and wherein each of the first plurality of multiplication operations is performed by a different MAC unit (243) among the plurality of MAC units; and receiving, at the processing unit, a second column (337-0, 337-1, 337-2, 337-3, 437-0, 437-1, 437-2, 437-3, 437-4, 437-5, 437-6, 437-7, 437-9) of the matrix; performing a second plurality of multiplication operations on second data values ​​of the vector using a second plurality of data values ​​of the second column, wherein each of the second plurality of multiplication operations is performed by a different MAC unit (243) among the plurality of MAC units; and Outputs of the first and second plurality of multiplication operations are stored in the memory bank.

10. The method of claim 9, further comprising receiving the vector from the memory bank of the memory device or a different memory bank (130, 330, 430-1, 430-2) of the memory device.

11. The method of claim 9, further comprising accumulating results of the first plurality of multiplication operations and results of the second plurality of multiplication operations in the plurality of MAC units.

12. The method of any one of claims 9 to 11, further comprising providing the outputs of the first and second plurality of multiplication operations from the plurality of MAC units to input / output (I / O) lines (103, 203) of the memory device without performing additional operations on the outputs.

13. The method of claim 12, wherein providing the outputs of the first and second plurality of multiplication operations from the plurality of MAC units to the I / O lines without performing additional operations on the outputs comprises providing the outputs without combining the outputs with different data values.

14. The method of any one of claims 9-11, further comprising providing the output to the I / O line in the same amount of time that a single column of memory addresses of the matrix would be provided to the I / O line.

15. An apparatus for performing a matrix-vector multiplication operation using a processing unit in a memory, comprising: a first bank (130, 330, 430-1, 430-2) of memory cells; a second bank (130, 330, 430-1, 430-2) of memory cells; a processing unit (102, 302, 402) comprising a plurality of multiply-accumulate (MAC) units (243), the processing unit being coupled to the first memory bank and the second memory bank; The plurality of MAC units are configured to: Receive a first column (337-0, 337-1, 337-2, a first plurality of data values ​​from said first column, wherein each MAC unit from said plurality of MAC units receives a different data value from said first plurality of data values; receiving a first data value of a vector of data values ​​(335, 435) stored in the second memory bank, wherein each MAC unit from the plurality of MAC units receives the first data value of the vector; performing a first plurality of multiplication operations on first data values ​​of the vector using the first plurality of data values ​​of the first column, wherein each of the first plurality of multiplication operations is performed by a different MAC unit (243) among the plurality of MAC units; receiving a second plurality of data values ​​for a second column (337-0, 337-1, 337-2, 337-3, 437-0, 437-1, 437-2, 437-3, 437-4, 437-5, 437-6, 437-7, 437-9) of the matrix stored in the first memory bank, wherein each MAC unit of the plurality of MAC units receives a different data value from the second plurality of data values ​​in the second column; receiving a second data value of the vector, wherein each MAC unit of the plurality of MAC units receives the second data value of the vector; performing a second plurality of multiplication operations using the second data value of the vector and the second plurality of data values ​​of the second column, wherein each of the second plurality of multiplication operations is performed by a different MAC unit (243) among the plurality of MAC units; and Outputs of the first plurality of multiplication operations and the second plurality of multiplication operations are stored in the second memory bank.

16. The apparatus of claim 15, wherein each of the plurality of MAC units multiplies the first data value with a different data value from the first plurality of data values ​​of the first column and stores a first plurality of results in an accumulator (223) of the plurality of MAC units.

17. The apparatus of claim 15, wherein each of the plurality of MAC units multiplies the second data value with a different data value from the second plurality of data values ​​of the second column and adds a second plurality of results to the accumulators of the plurality of MAC units.

Images