Methods for performing in-memory processing operations and related memory devices and systems

Abstract

Methods and related memory devices and systems for performing in-memory processing operations are described. Bit strings (e.g., vectors) may be extracted and processed in logic of a memory device without involving separate processing units. Operations (e.g., arithmetic operations) may be performed on numbers stored in a bit-parallel fashion during a single clock cycle sequence. Accordingly, the arithmetic may be performed in a single pass because two or more bit strings are extracted and have no intermediate storage of the number. Vectors may be extracted (e.g., identified, transmitted, received) from one or more bit lines. Register of the memory array may be used to write (e.g., store or temporarily store) results or auxiliary bits (e.g., carry bits or carry flags) that facilitate arithmetic operations. Circuitry proximate, proximate, or underneath the memory array may employ XOR or AND (or other) logic to extract, organize, or compute the data.

Description

[0001] Information about divisional applications

[0002] This application is a divisional application. The parent application is an invention patent application with an application date of August 24, 2020, application number 202080061754.1, and invention name “Method for performing in-memory processing operations and related memory devices and systems”.

[0003] Priority claim

[0004] This application is a national phase project of international patent application PCT / US2020 / 070450 filed on August 24, 2020, which designates the People's Republic of China and was published in English as international patent publication WO2021 / 046568A1 on March 11, 2021, which claims the benefits of U.S. patent application No. 16 / 841,222 filed on April 6, 2020 and U.S. patent application No. 62 / 896,228 filed on September 5, 2019 under Article 8 of the Patent Cooperation Treaty. Technical Field

[0005] Embodiments of the present disclosure relate to in-memory processing, and more particularly, to in-memory processing of fused multiply-add operations. More particularly, some embodiments relate to methods for performing fused multiply-add operations in a memory device with in-memory processing capabilities, and related memory devices, memory systems, and electronic systems. Background Art

[0006] Memory devices are typically provided as internal semiconductor-based integrated circuits in a computer or other electronic system. There are many different types of memory, including volatile and non-volatile memory. Volatile memory, which can retrieve stored information after a power cycle, can include flash memory, including NAND or NOR flash memory, among others. Volatile memory may require power to maintain its data (e.g., host data, error data, etc.) and includes random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), synchronous dynamic random access memory (SDRAM), content addressable memory (CAM), and thyristor random access memory (TRAM), among others.

[0007] Electronic systems often include a number of processing resources (e.g., one or more processors) that can retrieve and execute instructions and store the results of the executed instructions to appropriate locations. A processor may include, for example, a number of functional units, such as arithmetic logic unit (ALU) circuitry, floating point unit (FPU) circuitry, and combinational logic blocks, that can be used to execute instructions by performing operations on data (e.g., one or more operands). As used herein, operations may include, for example, Boolean operations such as AND, OR, NOT, NAND, NOR, and XOR, and / or other operations such as inversion, shifting, arithmetic, statistics, and many other possible operations. For example, a functional unit circuitry may be used to perform arithmetic operations such as addition, subtraction, multiplication, and division on operands through a number of logical operations.

[0008] Several components in an electronic system may involve providing instructions to a functional unit circuit for execution. The instructions may be executed, for example, by a processing resource such as a controller and / or a host processor. Data (e.g., data on which instructions may be executed) may be stored in a memory array accessible by the functional unit circuit system. The instructions and / or data may be retrieved from the memory array and sorted and / or buffered before the functional unit circuit begins executing the instructions on the data. Summary of the invention

[0009] One or more embodiments of the present disclosure include a method of operating a memory system. For example, the method may include loading a first number of bits into the circuitry of a memory device, each bit of the first number of bits having a first state or a second state. The method may also include loading a second number of groups of bits into the circuitry, each bit of the second number of groups of bits having a first state or a second state. Additionally, the method may include multiplying each bit group of the second number of groups of bits by each bit of the first number of bits to produce a plurality of scaled rows. Additionally, the method may include summing the plurality of scaled rows along associated bit positions to produce an output row.

[0010] According to another embodiment, a method of operating a memory system including a memory device may include loading a first number of bit groups of a memory array into a circuit system via a plurality of bit lines of a memory array of the memory device. The method may also include loading a plurality of operands into the circuit system. Additionally, the method may include multiplying each bit in each bit group of the first number of bit groups by a first bit of the operand to generate a first scaled row. And, the method may include generating a plurality of bits of a row of an output matrix based at least in part on the first scaled row.

[0011] In yet another embodiment, a method may include loading a group of rows of a memory array into a circuit system via a plurality of bit lines of a memory array of a memory device. The method may further include loading a group of operands into the circuit system. The method may also include multiply-accumulating each row in the group of rows by each associated operand in the group of operands to produce a first row of an output matrix.

[0012] Some embodiments of the present disclosure include a memory system. The memory system may include at least one memory array and logic coupled to the at least one memory array. The logic may be configured to receive an input including a plurality of bits. The logic may also be configured to receive a row of a memory array of the at least one memory array, the row including a bit group. In addition, the logic may be configured to multiply each bit in each bit group by each bit of the input to generate a plurality of scaled row groups, wherein each scaled row in each scaled row group in the scaled row groups includes a plurality of columns, and each scaled row in each scaled row group in the scaled row groups is shifted by at least one column position relative to an adjacent scaled row in the associated scaled row group. In addition, the logic may be configured to sum scaled rows across columns within each scaled row group to generate an output row.

[0013] Additional embodiments of the present disclosure include an electronic system. The electronic system may include at least one input device, at least one output device, and at least one processor device operably coupled to the input device and the output device. The electronic system may also include at least one memory device operably coupled to the at least one processor device. The at least one memory device may include at least one memory cell array and logic coupled to the at least one memory array. The logic may be configured to receive a first operand and a row of a memory array of the at least one memory array. The logic may also be configured to cause a plurality of bits of the row of the memory array to be multiplied, shifted, and accumulated by the bits of the operand to produce a first scaled row. The logic may also be configured to receive at least one additional operand. In addition, the logic may be configured to cause a plurality of bits of the row to be multiplied, shifted, and accumulated by the bits of at least one additional operand to produce at least one additional scaled row. And, the logic may be configured to generate a row of an output matrix based on the first scaled row and the at least one additional scaled row. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a block diagram of a system including a memory device according to various embodiments of the present disclosure.

[0015] Figure 2A Describes a multi-instance matrix.

[0016] Figure 2B Depicts an example binary multiplication with an accumulate operation.

[0017] Figure 3 A plurality of example bit vectors generated according to various embodiments of the present disclosure, intermediate matrices generated by multiplication of the bit vectors, and output bit vectors are illustrated.

[0018] Figure 4 A fused multiply-add unit including a sequencer and a sense amplifier array according to one or more embodiments of the present disclosure is described.

[0019] Figure 5A and 5B Various scaling lines and output lines generated via one or more embodiments of the present disclosure are described.

[0020] Figure 6 is a flowchart of an example method of operating a memory system according to various embodiments of the present disclosure.

[0021] Figure 7 is a simplified block diagram of an example memory system implemented in accordance with one or more embodiments described herein.

[0022] Figure 8 is a simplified block diagram of an example electronic system implemented in accordance with one or more embodiments described herein. DETAILED DESCRIPTION

[0023] Fused-multiply-add (FMA) and matrix multiplication operations are used in many applications, such as in machine learning, image processing, artificial intelligence, system modeling (e.g., power systems, mechanical systems, etc.), and many other applications. The logic of these operations can be relatively simple (e.g., multiplication and accumulation). However, conventional computer-based calculations can involve processor- and memory-intensive operations, including transferring large amounts of data between computing cores and memory arrays.

[0024] In many cases, the processing resources (e.g., processors and associated functional unit circuitry) may be external to the memory array, and access data via a bus between the processing resources and the memory array to execute a set of instructions. Processing performance may be improved in a processor-in-memory (PIM) device, where the processor may be implemented inside and / or near the memory (e.g., directly on the same chip as the memory array). A PIM device may save time and / or power by reducing and eliminating external communications.

[0025] Various embodiments of the present disclosure relate to processing in memory (PIM) operations, and more specifically, to performing PIM fused multiply-add (FMA) operations, wherein data to be used in a calculation can be reused for multiple calculations. In other words, data can be read (loaded) and used in multiple calculations (e.g., across multiple bit lines) instead of loading data each time the data is to be used in a calculation. More specifically, for example, according to some embodiments, an operand can be loaded into a sequencer of a memory system including a memory device. In addition, multiple bit groups of one or more rows of a memory array of a memory device can be loaded (e.g., via multiple bit lines of a memory array) into a sense amplifier array of the memory device. And, each bit in the multiple bit groups of one or more rows can be multiplied by an operand to generate a scaled row. The multiplication process can be repeated for one or more additional operands to generate one or more additional scaled rows. In addition, rows of an output matrix can be generated based on multiple scaled rows. Therefore, and as described more fully herein, an operand can be reused across multiple bits (e.g., all bits of a row of a matrix) to generate one or more scaled rows. Thus, the number of memory loads and memory stores may be reduced compared to conventional systems and methods, which may reduce latency and increase the efficiency of PIM operations.

[0026] As used herein, a device with processing-in-memory (PIM) capability refers to a memory device that is capable of performing logical and arithmetic operations on data stored in an array of memory cells using processing resources internal to the memory device (e.g., without transferring the data to external processing resources such as a host processor). For example, a device with PIM capability may include a memory array coupled to a sensing circuit system, wherein the sensing circuit system includes sensing components that can be operated as 1-bit processing elements (e.g., to perform parallel processing on a per-column basis). In addition to logical operations performed "in-memory", which may be referred to as "bit-vector operations", a device with PIM capability may also perform memory operations. As an example, a device with PIM capability may include a dynamic random access memory (DRAM) array with memory operations, wherein the memory operations include memory access operations (e.g., reads (e.g., loads) and writes (e.g., stores)), as well as other operations that do not involve operations on data. For example, depending on the type of program being executed (e.g., by a host), a PIM-capable device may operate a DRAM array as a "normal" DRAM array and / or as a PIM DRAM array, which may include both memory operations and bit-vector operations. For example, bit-vector operations may include logical operations, such as Boolean operations (e.g., AND, OR, XOR, etc.), and transfer operations, such as shifting data values ​​in the array and, for example, inverting data values. Arithmetic operations performed by a PIM device may be implemented by logical operations. For example, single-bit addition may be implemented by a set of operations: 0+0={0-sum, 0-carry}, 0+1={1-sum, 0-carry}, 1+0={1-sum, 0-carry}, 1+1={0-sum, 1-carry}. Thus, the sum may be an XOR operation, and the carry may be an AND operation.

[0027] As used herein, PIM operations may refer to various operations associated with performing in-memory processing using a PIM-capable device. A hierarchy of operations may be used to define PIM operations. For example, a first (e.g., lowest) level in the hierarchy of operations may include bit-vector operations (e.g., basic logical operations, which may be referred to as "primitive" operations). A next (e.g., middle) level in the hierarchy may include compound operations, which include multiple bit-vector operations. For example, compound operations may include mathematical operations such as addition, multiplication, etc., which may include multiple logical ANDs, ORs, XORs, shifts, etc. A third (e.g., highest) level in the hierarchy may include control flow operations (e.g., loops, branches, etc.) associated with executing a program, the execution of which involves performing processing using a PIM-capable device.

[0028] As described in greater detail herein, PIM operations may be performed by various components within a system that includes a PIM-capable device. For example, a first PIM control component (e.g., control logic, which may be referred to as a "scalar unit") that may be located on a host may perform control flow operations and provide complex operations to a second PIM control component (e.g., a sequencer), which may also be located on the host or on a PIM-capable device. In various embodiments, the second control component may provide low-level bit-vector operations to a PIM control component (e.g., bit-vector timing circuitry) located on the PIM-capable device, which may perform the bit-vector operations within a memory and pass the results back to the host. As further described herein, an interface for communicating PIM operations between a PIM-capable device and a host may include a channel, which may include a bus separate from a typical memory interface (e.g., a DDR interface) used to communicate commands, addresses, and / or data. Additionally, in various embodiments, providing a PIM control component on the host may provide benefits such as allowing PIM programs to use virtual addressing (eg, by resolving virtual addresses on the host since PIM-capable devices may only operate on physical addresses).

[0029] The techniques described herein may also be applicable to processing in storage devices (e.g., NAND or NOR flash memory, 3D XPoint TM PIM may also be referred to as computation in memory or computation in storage. In other words, the present disclosure encompasses processing or computation in a device (e.g., memory or storage) that is typically associated with storing or organizing data, which in some architectures is different from a CPU, GPU, GPGPU, FPGA, ASIC, etc.

[0030] Figure 1 1 is a block diagram of a system 100 including a memory device 120 for multiple embodiments of the present disclosure. The memory device 120, which may also be referred to herein as a "PIM-capable device" or a "PIM-capable memory device," may include any suitable memory device. For example, the memory device 120 may include volatile memory (e.g., RAM, DRAM, etc.) and / or non-volatile memory (e.g., flash memory, a cross-point memory device such as a 3D cross-point memory device, etc.). The memory device 120 may include a memory array 130 (i.e., including memory cells) coupled to a sensing circuit system, as described in more detail below. According to some embodiments, the memory device 120 may include a plurality of memory arrays 130 organized in groups, levels, stacks, plaintext, or some other formation.

[0031] The system 100 further includes a host 111 coupled to the memory device 120. The host 111 may include a host system, such as a personal laptop computer, a desktop computer, a digital camera, a smart phone, or a memory card reader, as well as various other types of hosts. The host 111 may include a system motherboard and / or a backplane, and may include multiple processing resources (such as one or more processors, microprocessors, or some other type of control circuitry). The system 100 may include separate integrated circuits, or both the host 111 and the memory device 120 may be part of the same integrated circuit (e.g., on the same chip). The system 100 may include, for example, a server system and / or a high performance computing (HPC) system and / or a portion thereof.

[0032] The host 111 may include various components, including PIM control components (e.g., control logic 131, sequencer 132), channel controller 143, and memory management unit (MMU) controller 134. The control logic 131 may be configured to execute control flow commands associated with executing a PIM program and provide compound commands to the sequencer 132. The control logic 131 may be or may include a RISC-type controller configured to generate and issue an extensible compound operation PIM command set to the sequencer 132, the extensible compound operation PIM command set including commands other than DDR commands. In some embodiments, the control logic 131 may be configured to issue a compound operation command to cause a bit vector operation to be performed on the memory device 120. In some embodiments, the compound operation command may be transmitted from the control logic 131 to the memory device 120 (e.g., via the sequencer 132 and the channel 157).

[0033] In some embodiments, control logic 131 may decode microcode instructions into function calls associated with performing bit-vector operations, which may be microcode function calls, implemented by sequencer 132. The microcode function calls may be operations that sequencer 132 receives and / or executes to cause memory device 120 to perform a particular bit-vector operation using sensing circuitry, such as sensing circuitry 150.

[0034] like Figure 1, the sequencer 132, the control logic 131, and the MMU controller 134 are located on the host 111, which can allow the control logic 131 and / or the MMU controller 134 to perform memory accesses using virtual addresses and perform virtual to physical address translations before transmitting instructions to the memory device 120. In some embodiments, the sequencer 132 and / or the control logic 131 are implemented in the memory device 120. In such cases, the amount of communication between the host 111 and the memory device 120 is reduced, which can improve the performance of the PIM device and / or reduce the power consumption of the PIM device. In this case, the virtual to physical address translation can still be performed by the MMU controller 134. However, for the simplicity of this operation and if the data used in the operation is pre-allocated in a fixed pattern, then this translation may be unimportant. For example, the pre-allocated data in the physical address space may be a simple contiguous piece of "pinned" memory, where the virtual address of the beginning of the piece can be calculated by a simple offset to the physical address location of the beginning of the piece of memory storing the data. Alternatively, a simple fixed formula or pattern or lookup table may be used for the conversion. For example, the pattern may be part of the microcode executed by the control logic 131 and / or the sequencer 132.

[0035] As used herein, a "bit vector" may refer to a physically contiguous number of bits, whether physically contiguous in a row (e.g., oriented horizontally) or a column (e.g., oriented vertically). A "PIM-capable device" may be configured to perform bit vector operations, such as logical operations and / or transfer operations, on multiple adjacent portions (e.g., "chunks") of a virtual address space. For example, a chunk of a virtual address space may have a bit length of 256 bits. A chunk may or may not be sequentially contiguous with other chunks in the virtual address space.

[0036] Can reside on host 111 (such as Figure 1 The MMU controller 134 (shown in FIG. 1 ) may be responsible for performing translation of virtual memory addresses (e.g., addresses associated with the host 111) to physical addresses (e.g., addresses associated with the memory device 120). The MMU controller 134 may also perform memory protection operations, cache control, and / or bus arbitration operations.

[0037] Timing circuit 133 may provide timing to coordinate the execution of logic operations and is responsible for providing timing to the array, e.g. Figure 1150. In various embodiments, the controller 140 and / or the timing management circuit system 135 may generate status information that may be transmitted to or from the host 111, for example, via the channel 157. The channel 157 may be independent of (e.g., separate from) a double data rate (DDR) memory interface (e.g., control bus 154) that may be used to transmit (e.g., transfer) DDR commands between the host 111 and the memory device 120. That is, in some embodiments, the channel 157 may be used to transmit commands that cause the execution of bit-vector operations from the host 111 to the memory device 120, while the control bus 154 may be used to transmit DRAM commands from the host 111 to the memory device 120. In some embodiments, the DRAM commands transmitted via the control bus 154 may be commands for controlling the operation of a DRAM, for example, a DDR1 SDRAM, a DDR2 SDRAM, a DDR3 SDRAM, and / or a DDR4 SDRAM. In other embodiments, memory device 120 may signal host 111 that it is ready for operation via bus 157. In some embodiments, bus 157 is combined with control bus 154 and / or data bus 156. Additionally, in some embodiments, host 111 may include multiple memory devices 120 with multiple channels and / or control buses.

[0038] In some embodiments, the sequencer 132 may include a very large instruction word (VLIW) type controller configured to operate on logic operation commands, and the control logic 131 may be configured to issue the logic operation command to the sequencer 132 in response to a signal from a processing resource (e.g., a GPU or GPUs) (not shown) of the host 111. For example, the sequencer 132 may be configured to order multiple logic operations so that a compound operation command can be issued by the sequencer 132. In some embodiments, the control logic 131 may be configured to generate a VLIW as a bit vector operation command. The VLIW may include a microcode instruction. The sequencer 132 may be or may include a VLIW type controller configured to decode the VLIW into multiple separate microcode instructions. For example, the sequencer 132 may decode the VLIW into an instruction to cause the execution of a compound operation (e.g., addition, multiplication, etc.). In some embodiments, the compound operation command may provide an entry point into a sequence of VLIW instructions to cause the execution of such a compound operation. Sequencer 132 may be coupled to memory device 120 and may communicate commands to coordinate bit-vector operations to memory device 120 via channel 157. Microcode instructions may be executed sequentially and / or in parallel by sequencer 132 itself and / or by other components in memory device 120 (e.g., bit-vector operation timing circuitry 139, timing circuitry 133, timing management circuitry 135, and / or sensing circuitry 150).

[0039] The memory array 130 may include, for example, a DRAM array, an SRAM array, an STT RAM array, a PCRAM array, a TRAM array, an RRAM array, a NAND flash array, and / or a NOR flash array. The memory array 130 may include memory cells arranged in rows coupled by access lines (which may be referred to herein as word lines or select lines) and columns coupled by sense lines (which may be referred to herein as data lines, digit lines, or bit lines). Although in Figure 1 A single memory array is shown in FIG. 1 , but embodiments are not limited thereto. For example, memory device 120 may include multiple memory arrays 130 (eg, several groups of DRAM cells, NAND flash memory cells, etc.).

[0040] The memory device 120 includes an address circuit system 142 to latch address signals of data provided by the I / O circuit system 144 via a bus 156 (e.g., a data / address bus). Status and / or exception information may be provided from the controller 140 on the memory device 120 to the channel controller 143 via an interface (e.g., a high-speed interface (HSI)) including a channel 157. The address signal is received by the address circuit system 142 and decoded by the row decoder 146 and the column decoder 152 to access the memory array 130. Data may be read from the memory array 130 by sensing the state of the memory cells on the digit lines using the sensing circuit system 150. Depending on the memory cell technology, the state of the memory cells may be sensed as, for example, voltage and / or current changes, magnetic state changes, resistivity, and quantum states, etc. The sensing circuit system 150 may read and latch a page (e.g., row) of data from the memory array 130. The I / O circuit system 144 may be used for bidirectional data communication with the host 111 via the bus 156. The write circuit system 148 may be used to write data to the memory array 130. In some embodiments, write circuitry 148 is combined with sense circuitry 150. In other embodiments, sense circuitry 150 may be part of column decoder 152 and / or row decoder 146. In some embodiments, bus 154 may function as both a control and address bus for DRAM control and addressing (e.g., according to the DDR protocol in which bus 154 functions as a unidirectional data bus). Although Figure 1 Although shown as separate buses in FIG. 1 , bus 154 and bus 156 may not be separate buses in some embodiments.

[0041] Controller 140, such as a memory controller, may decode signals from host 111 provided by control bus 154. These signals may include chip enable signals, write enable signals, and address latch signals, which are used to control DRAM operations performed on memory array 130, including data read, data write, and data erase operations. In various embodiments, controller 140 may be responsible for executing instructions from host 111 and sequencing accesses to memory array 130. Controller 140 may include a state machine, a sequencer, or some other type of controller, and include hardware and / or firmware (such as microcode instructions) in the form of an application specific integrated circuit (ASIC). In multiple embodiments, controller 140 may include bit vector operation timing circuitry 139. Controller 140 may control, for example, sensing circuitry 150. For example, controller 140 may control the generation of clock signals and the application of clock signals to calculate components associated with performing bit vector operations.

[0042] like Figure 1, bit-vector operation timing circuitry 139 may include timing circuitry 133 and timing management circuitry 135. Timing circuitry 133 may include a FIFO buffer to provide timing coordination with sensing circuitry 150 associated with memory array 130 of memory cells. In some embodiments, timing circuitry 133 may include a state machine such as an atomic state machine.

[0043] The timing management circuitry 135 may be configured to coordinate the timing of logic operations (e.g., a sequence of logic operations) associated with bit-vector operations performed using a row address strobe (RAS) / column address strobe (CAS) component 136 associated with the memory array 130. The RAS component 136 may be configured to send signals (e.g., RAS / CAS signals) to the memory array 130 and / or receive signals (e.g., RAS / CAS signals) from the memory array 130 to identify and / or select row and / or column addresses of the memory array 130. In some embodiments, the memory device 120 may be configured to perform DRAM operations, such as memory array access requests, which may be issued by the host 111 via the bus 154. In some embodiments, the timing management circuitry 135 may be configured to execute instructions to control the timing of execution of bit-vector operations.

[0044] In one or more embodiments, portions of controller 140 (e.g., bit-vector operation timing circuitry 139, timing circuitry 133, and / or timing management circuitry 135) may include a reduced instruction set computer (RISC) type controller that operates on 32 and / or 64 bit length instructions. In various embodiments, timing management circuitry 135 may be responsible for executing instructions received from timing circuitry 133 such that bit-vector operations involving data values ​​associated with sensing circuitry 150 are performed.

[0045] As further described below, in various embodiments, the sensing circuitry 150 may include a plurality of sensing components, which may each include a sense amplifier and a computation component. The computation component may be used as an accumulator, and the sensing circuitry 150 may be used to perform bit vector operations (e.g., on data associated with complementary digit lines). In various embodiments, the sensing circuitry 150 may be used to perform bit vector operations using data stored in the memory array 130 as input, and / or to store the results of the operations back to the memory array 130 without transferring the data via digit line address accesses (e.g., without activating column decode signals). For example, various operations (e.g., bit vector operations) may be performed using the sensing circuitry 150 and within the sensing circuitry 150, rather than being performed by (or associated with) processing resources external to the sensing circuitry 150 (e.g., by processing resources associated with the host 111 and / or other processing circuitry (e.g., ALU circuitry) located on the memory device 120 (e.g., on the controller 140 or elsewhere). In various embodiments, the sensing circuit system 150 (e.g., multiple sensing components) may be used to perform bit-vector operations in a single instruction multiple data (SIMD) manner, where the sensing components act as 1-bit processing elements on a per-column basis. In other embodiments, adjacent sensing components may exchange data bits between each other, thereby generating calculations based on multiple data sources. In other embodiments, the sensing components may generate different calculations depending on their location within the sensing circuit system 150, thereby providing calculations in a VLIW manner. In embodiments where the sensing circuit system 150 performs bit-vector operations, the sensing circuit system 150 may act as and / or be referred to as a "processor in memory." As described more fully below, in some embodiments, the sensing circuit system 150 may include a sequencer (e.g., similar to the sequencer 132).

[0046] In various conventional approaches, for example, data associated with the operands may be read from memory via the sense circuitry and provided to the external ALU circuitry via I / O lines (e.g., via local I / O lines and / or global I / O lines). The external ALU circuitry may include multiple registers and will perform bit-vector operations using the operands, and the results may be transferred back to the array via the I / O lines. In other embodiments, the sense circuitry 150 is configured to perform bit-vector operations on data stored in the memory array 130 and store the results back to the memory array 130 without enabling the I / O lines (e.g., local I / O lines) coupled to the sense circuitry 150.

[0047] In various embodiments, circuitry external to memory array 130 and sensing circuitry 150 may not be required to perform operations because sensing circuitry 150 may perform appropriate bit-vector operations without using external processing resources. Thus, sensing circuitry 150 may be used to supplement and / or replace such external processing resources (or at least the bandwidth consumption of such external processing resources) to at least some extent. However, in various embodiments, sensing circuitry 150 may be used to perform logic operations (e.g., execute instructions) in addition to logic operations performed by external processing resources (e.g., host 111). For example, host 111 and / or sensing circuitry 150 may be limited to performing only certain logic operations and / or a specific number of logic operations.

[0048] Enabling an I / O line may include enabling (e.g., turning on) a transistor having a gate coupled to a decode signal (e.g., a column decode signal) and a source / drain coupled to the I / O line. However, embodiments are not limited to using a sense circuitry (e.g., sense circuitry 150) to perform a logic operation without enabling the column decode lines of the array. Whether or not a local I / O line is used in association with performing a logic operation via sense circuitry 150, the local I / O line may be enabled to transfer the result to a suitable location other than back to array 130 (e.g., to an external register).

[0049] Various embodiments of the present disclosure relate to fused multiply-add (FMA) operations, and more particularly, to performing process-in-memory (PIM) FMA operations. According to various embodiments, data to be used in one or more calculations may be loaded (e.g., into a sequencer) and reused across multiple bits (e.g., loaded to a sense amplifier array via multiple bit lines). In other words, various embodiments may involve reusing sequencer logic across multiple bits associated with multiple bit lines. Thus, the number of memory loads and memory stores may be reduced, and thus the efficiency of the memory device may be improved and / or the latency associated with the memory device may be reduced, compared to conventional methods and systems.

[0050] For at least these reasons, as described more fully herein, various embodiments of the present disclosure provide technical solutions to one or more problems resulting from technology that may not reasonably be performed by an individual, and various embodiments disclosed herein arise from computer technology in order to overcome the problems and / or challenges described above. In addition, at least some embodiments disclosed herein may improve computer-related technology by allowing computer performance of functions that were previously not performable by a computer.

[0051] For matrix multiplication, both the central processing unit (CPU) and the graphics processing unit (GPU) can decompose the input matrix into tiles, and the tiles can be distributed to the processor cores. For each tile of the output matrix, the tile of the input matrix is ​​extracted exactly once, which achieves O(N) computational intensity (i.e., the processing will increase linearly and proportionally to the size of the input data set). The size of each tile of the output matrix can be selected to match the capacity of the L1 level cache or registers of the target processor.

[0052] For the GPU, tiles are selected so that the tiles form a hierarchical structure of threadblock tiles, warp tiles, and thread tiles. This tile structure creates paths for data to move from global memory to shared memory (i.e., matrices to threadblock tiles), from shared memory to register files (i.e., threadblock tiles to warp tiles), and from register files to GPU cores for computation (i.e., warp tiles to thread tiles).

[0053] In a PIM, there is no memory hierarchy other than a memory array (e.g., a DRAM array) and one or more registers (e.g., within a sense amplifier (SA) array). However, as described more fully herein, data may be reused in PIM FMA operations according to various embodiments.

[0054] Figure 2A A matrix A including a plurality of rows M and a plurality of columns K and a matrix B including a plurality of rows K and a plurality of columns N are described. Figure 2A Further illustrating a matrix C including M rows and N columns. As described more fully below, the data of matrix C may be generated based on operations performed using the data of matrix A and the data of matrix B.

[0055] According to various embodiments, to multiply matrices (e.g., matrix A and matrix B), each row of matrix A (e.g., of the first memory array) may be scanned (e.g., applying a dot product operation) across all columns in matrix B (e.g., of the second memory array) to calculate each row in the output matrix C. During the scan (e.g., dot product scan), each number in the input row of matrix A may be reused across all columns of matrix B. Thus, according to various embodiments, each bit in a column of a matrix (e.g., matrix A and / or matrix B) may be read only once, for example, by a PIM vector engine (e.g., a fused multiply-add (FMA) unit).

[0056] More specifically, according to various embodiments, each element a_(i,k) of the matrix A (e.g., of the first memory array) may be multiplied with a row b_(k) of the matrix B (e.g., of the second memory array) to produce a new temporary row scaled by the single element a_(i,k). In other words, each element a_(i,k) of the matrix A may be loaded into the FMA unit as a common operand along with the row b_(k) of the matrix B to be multiplied by the common operand. The result of the multiplication, i.e., the scaled row (e.g., the scaled row b'_(k)=b_(k)*a_(i,k)), may be stored (e.g., in a local register of the FMA unit or a designated portion of the memory array 130). In addition, a different element a_(i,k+1) of the matrix A may be loaded into the FMA unit as a common operand along with the row b_(k+1) of the matrix B to be multiplied by the common operand. As a result of the multiplication, a scaled row (e.g., scaled row b'_(k+1)=b_(k+1)*a_(i,k+1)) can be calculated and added (or accumulated) to the pre-calculated row b'_(k). In other words, the scaled rows b'_(k) (e.g., of the intermediate matrix) can be summed along columns and across rows to produce rows c_(i), which can be stored back in a memory array (e.g., a second memory array) as a complete row of the output matrix C. In other words, the rows of the intermediate matrix b'_(k) can be merged and accumulated across rows and along columns to produce a cumulative sum (a row), which can be stored back in a memory array (e.g., a second memory array) (e.g., as row c_(i)). These actions can be repeated for each row of matrix A and / or matrix B to produce other rows of matrix C.

[0057] According to some embodiments, bits from a row (e.g., row b_(k) of matrix B) can be co-assigned online along word lines of a memory array in a bit-parallel manner. In other words, each bit of a row (e.g., each bit of row b_(k) of matrix B) can be loaded (e.g., into an FMA unit) via a set of bit lines of the memory array. Thus, all bits from the row can be accessed simultaneously. In addition, as described above, a number (e.g., bit a_(i, k) of matrix A) can be applied (e.g., via a sequencer) as a common operand to the entire loaded row (e.g., row b_(k) of matrix B). In other words, numbers (e.g., bits a_(i, k) of matrix A) may be multiplied and accumulated by rows (e.g., row b_(k) of matrix B) (e.g., in a memory array on sense amplifier logic). According to various embodiments, a bit-parallel FMA unit may receive bits from a row (e.g., row b_(k) of matrix B), wherein the bit-parallel FMA unit may be based on sequencing the bits of numbers in the row (e.g., row b_(k) of matrix B) as specified by the bits of a common operand (e.g., bits a_(i, k) of matrix A). In some embodiments, the operand may include microcode for controlling a sequencer.

[0058] As described above, in at least some embodiments, the accumulation may be performed locally on the bit lines of the memory array, and thus the multiplication process may not require a large amount of hardware. Figure 2A In the example shown in , the FMA process may require, for example, M×K memory loads (each of N+1 elements) and M memory stores (each of N elements).

[0059] In addition, according to some embodiments, the matrix B may be split into tiles, and the matrix B may be replicated (e.g., to increase parallelism). Also, the inputs (i.e., input vectors, such as rows of the matrix A) may be distributed to tiles, and the inputs may be spread among the FMA array (e.g., sense amplifier array) (e.g., to increase computational concurrency and reduce communication distance). Furthermore, according to some embodiments, partial result accumulation may be performed in logarithmic time (e.g., to reduce communication cost).

[0060] For example, Figure 2B Depict an example of a binary multiplication involving an accumulation operation that is performed in logarithmic time, such as Figure 2BAs shown in FIG. 2 , non-zero rows of intermediate matrix 200 in block 202 are used in a first computation step (e.g., in block 204), where parallel computation is performed to produce the result shown in block 206. Additionally, in a second computation step (e.g., in block 206), the result of the first computation step is computed as the final result (i.e., shown in block 208). Thus, the complexity of the computation is logarithmic (i.e., log_w(bs), where w is the width of the FMA unit (i.e., how many operands the FMA unit can receive (e.g., 2 operands in this example), b is the number of bits (e.g., the bit width), and s is the number of skips (i.e., zero rows generated by multiplier bits having a value of 0 can be skipped)).

[0061] According to various embodiments, various operations (e.g., matrix multiplication) may be further parallelized according to one or more of the following options: 1) across rows a_(i) of matrix A, which may require replication of matrix B; 2) across columns a_(k) of matrix A, which may require large shared XOR and carry or intermediate storage devices on the memory device to perform the accumulation; and 3) across tiles in both columns and rows of matrix A, which may impose some intermediate requirements between options 1 and 2. In some of these embodiments, additional communication and / or integration of results (i.e., results generated from the split matrix across the memory array) may be required (e.g., depending on where the results are stored). Furthermore, in some embodiments, the results (which may be partial dot products) may need to be summed together to produce one or more rows in matrix C. Because the parallel summation may include data reduction operations, and may be completed in record time, in some embodiments, communication may occur locally (e.g., on the same chip), and thus fewer higher-level partial results (e.g., from different chips) may be summed together.

[0062] As described more fully below, in at least some embodiments, during an FMA operation, an intermediate matrix M may be generated via a shifted copy of bits in one vector conditioned on bits in another vector. Figure 3 , an example bit vector 'a' (i.e., 1 1 0 1) may be loaded (e.g., into a sequencer of an FMA unit), and an example bit vector 'b' (i.e., 1 0 01) may be loaded (e.g., into a sense amplifier of an FMA). Additionally, as described more fully below, an intermediate matrix M may be generated via shift replication of bits of vector 'b' conditioned on the bits in vector 'a'.

[0063] More specifically, the first bit of vector 'a' (which is '1') may be multiplied by the bit of vector 'b' to produce the first (i.e., top) row of matrix M (i.e., 1 0 0 1). Additionally, the second bit of vector 'a' (which is '0') may be multiplied by the bit of vector 'b' to produce the second row of matrix M (i.e., 0 0 0 0), which is shifted left by one column space relative to the top row of matrix M. Furthermore, the third bit of vector 'a' (which is '1') may be multiplied by the bit of vector 'b' to produce the third row of matrix M (i.e., 1 0 0 1), which is shifted left by one column space relative to the second row of matrix M. Additionally, the fourth bit of vector 'a' (which is '1') may be multiplied by the bit of vector 'b' to produce the fourth (i.e., bottom) row of matrix M (i.e., 1 0 0 1), which is shifted left by one column space relative to the third row of matrix M.

[0064] According to various embodiments, the bits of vector 'a' may be used one at a time. For each bit of vector 'a' (i.e., for each cycle), the value of vector 'b' is read, shifted, and added or not added to a total (e.g., a running total). For example, during a first cycle, when a=1 (i.e., the first bit of a), "1 0 0 1" is added to the total, during a second cycle, when a=0 (i.e., the second bit of a), "0 0 0 0" is added to the total (i.e., skipped), during a third cycle, when a=1 (i.e., the third bit of a), a shifted version of b is added to the total, and during a fourth cycle, when a=1 (i.e., the fourth bit of a), another shifted version of b is added to the total.

[0065] In some embodiments, for example, the sequencer may perform a shift copy and store the intermediate matrix M into a scratch pad area of ​​an interfaced memory array (e.g., a memory array containing vector 'b'). In some embodiments, the scratch pad area may be a portion of the memory array (e.g., a specifically designated portion of the memory array located close to the sense amplifier (e.g., to reduce digital line RC parasitics for faster data transfer to and from the scratch pad)). In some embodiments, the scratch pad area may include high-speed memory cells. After the matrix M is generated and stored, the matrix M may be read back into the memory array. During read, the sense amplifier may perform an XOR and carry operation using local accumulation. For example, the XOR and carry operation using local accumulation may be performed based on a bitwise carry adder, which may include minimal logic (e.g., 65 gate delays for a 32b adder). This operation may essentially merge the matrix M into the multiplication result R. In other words, the columns of the matrix M can be summed to produce R (ie, 1 1 1 0 1 0 1), which in this example is the row c_(i), such as Figure 2A As shown in . Figure 3The row Cin shown in may indicate a carry operation associated with summing the columns of the matrix M.

[0066] Figure 4 An FMA unit 400 is depicted in accordance with various embodiments of the present disclosure. The FMA unit 400, which may also be referred to herein as a "bit-parallel FMA unit," includes a sequencer 402 and an array 404, which may include an array of sense amplifiers (e.g., Figure 1 Array 404 may include multiple local FMA modules (e.g., FMA j, FMA j+1, etc.). Figure 4 The intermediate matrices M are further described, wherein each intermediate matrix M is associated with a FMA module.

[0067] According to some embodiments, the sequencer 402 may be configured to store data in a memory device (e.g., Figure 1 120) locally, and more specifically, in some embodiments, the sequencer 402 may be a sensing circuit system (e.g., Figure 1 In these embodiments, sequencer 402 may be configured to receive data from memory array 406 (e.g., via an array of bit lines 407). In other embodiments, sequencer 402 may be external to the memory device. For example, Figure 1 The sequencer 132 of the embodiment may include a sequencer 402. In these embodiments, the sequencer 402 may be configured to drive one or more memory arrays (e.g., at the same time or at different times). In addition, in any embodiment, the sequencer 402 may be configured to receive data and / or control signals from control logic, which may be part of the memory device (e.g., an external processor) or external to the memory device. Although the FMA unit 400 is depicted as including the sequencer 402, the present disclosure is not limited thereto, and in some embodiments, the sequencer 402 may be external to the FMA unit 400. In other embodiments, multiple sequencers 402 may be distributed among the FMA units 404.

[0068] FMA unit 400 may be configured to receive data via an array of bit lines 408 / 410 (also referred to herein as data lines (DL)). More specifically, the data from matrix A (see Figure 2A ) is loaded into sequencer 402 (via bit line 407), and the data from matrix B (see Figure 2A) into array 404 (via bit lines 408 / 410). Still more specifically, one or more bits from a row of matrix A may be loaded into sequencer 402, and a row of matrix B (e.g., multiple groups of bits), which may be split into multiple portions (e.g., tiles) 409_0, 409_1, etc., may be loaded into array 404, wherein each portion of the row of matrix B (e.g., 32 bits of the row) may be loaded into a local FMA module (e.g., FMAj, FMAj+1, etc.) of array 404. More specifically, a first portion (also referred to herein as a "group") of a row of matrix B (e.g., a 32-bit value) may be loaded into FMA j of array 404, and a second portion of the row of matrix B (e.g., a 32-bit value) may be loaded into FMAj+1 of array 404, and so on, such that the entire row of matrix B is loaded into array 404. Although FMA unit 400 is depicted as including sequencer 402 , the present disclosure is not limited in this regard, and in some embodiments, sequencer 402 may be external to FMA unit 400 .

[0069] refer to Figure 2A , 4 , 5A, and 5B, the intended operation of FMA unit 400 will be described. According to some embodiments, during a first cycle, a first bit group (e.g., 32-bit value) of a first row of an input matrix (e.g., matrix A) (also referred to herein as an "input bit vector" or "multiplier operand") may be loaded into sequencer 402, and at least a portion of a first row of another matrix (e.g., matrix B) (e.g., including a plurality of bit groups "multiplicand operand") may be loaded into array 404. More specifically, in some embodiments, for example, each 32-bit value of the first row may be loaded into each respective local FMA module of array 404. In other embodiments, as described more fully below, individual bits may be loaded into sequencer 402 in a sequential manner.

[0070] Continuing with this example, the first bit of the input bit vector in sequencer 402 may be multiplied by each bit of a row (eg, comprising a plurality of bit groups) loaded into array 404 to produce a scaled row b'(1) (see Figure 5A). Subsequently, during a second cycle, the second bit of the input bit vector in the sequencer 402 may be multiplied by each bit of the row in the array 404 to produce another scaled row b'(2). According to various embodiments, the scaled row b'(2) may be shifted by one column position (e.g., to the left in the intermediate matrix) relative to the scaled row b'(1). Additionally, during a third cycle, the third bit of the input bit vector in the sequencer 402 may be multiplied by each bit of the row in the array 404 to produce another scaled row b'(3). According to various embodiments, the scaled row b'(3) may be shifted by one column position (e.g., to the left in the intermediate matrix) relative to the scaled row b'(2).

[0071] This process may continue until the last bit of the input bit vector in the sequencer 402 is multiplied by the row of the matrix B loaded into the array 404. In other words, this process may continue until the last bit of the input bit vector in the sequencer 402 may be multiplied by each bit of the row loaded into the array 404 to produce the last scaled row b'(i).

[0072] Additionally, all values ​​of the scaled row b' in the column (i.e., along the digit line) may be accumulated (e.g., an accumulator (e.g., a digital accumulator) may maintain a running value along the digit line) to produce row c[i] of matrix C. More specifically, all values ​​of bb0 are summed to produce the first bit in row c[i], all values ​​of bb1 are summed to produce the second bit in row c[i], and all values ​​of bb2 ( Figure 4 32 in row c[i]). In addition, with each summation, a carry value may be generated and propagated from the least significant bit to the most significant bit (e.g., if the least significant bit is bb0, then the carry value is generated as a result of summing the value of bb0 to produce the first bit in row c[i], propagated to the value of bb1 and used to sum the value of bb1 to produce the second bit in row c[i] and its carry value, and so on, until the 32nd bit in row c[i]). As will be appreciated, after multiplying and accumulating all bit groups of the first row of the input matrix (e.g., matrix A) in the manner described herein, row c[i] includes the dot product of a row of matrix A and all columns of matrix B. In other words, the first bit group (e.g., 32-bit value) in row c[i] is the dot product of the first row of matrix A and the first column of matrix B, the second bit group in row c[i] of matrix C is the dot product of the first row of matrix A and the second column of matrix B, and so on.

[0073] Continuing with this example, the second bit group (e.g., 32-bit value) of the first row of the input matrix (i.e., matrix A) (also referred to herein as an "input bit vector" or "multiplier operand") may be loaded into sequencer 402, and the second row of matrix B (e.g., including a plurality of bit groups "multiplicand operands") may be loaded into array 404. Additionally, the first bit of the input bit vector in sequencer 402 may be multiplied by each bit of the row loaded in array 404 to produce a scaled row b" (1) (see Figure 5B ). Subsequently, the second bit of the input bit vector in the sequencer 402 may be multiplied by each bit of the row in the array 404 to produce another scaled row b"(2). According to various embodiments, the scaled row b"(2) may be shifted by one column position (e.g., to the left in the intermediate matrix) relative to the scaled row b"(1). Additionally, the third bit of the input bit vector in the sequencer 402 may be multiplied by each bit of the row in the array 404 to produce another scaled row b"(3). According to various embodiments, the scaled row b"(3) may be shifted by one column position (e.g., to the left in the intermediate matrix) relative to the scaled row b"(2).

[0074] This process may continue until the last bit of the input bit vector in sequencer 402 is multiplied by the row of matrix B loaded into array 404. In other words, this process may continue until the last bit of the input bit vector in sequencer 402 may be multiplied by each bit of the row loaded into array 404 to produce the last scaled row b"(i).

[0075] Additionally, all values ​​of the scaled row b″ in the columns (i.e., along the digit line) may be accumulated (e.g., an accumulator may maintain a running value along the digit line) to produce row c[i] of matrix C. More specifically, all values ​​of bb0 are summed to accumulate to the first bit in the pre-instantiated row c[i] in the manner described above, all values ​​of bb1 are summed to accumulate to the second bit in row c[i], and all values ​​of bb2 ( Figure 432 in row c[i]). In addition, with each summation, a carry value may be generated and propagated from the least significant bit to the most significant bit (e.g., if the least significant bit is bb0, then the carry value is generated as a result of accumulating the value of bb0 to the first bit in row c[i], propagated to the value of bb1 and used to accumulate the value of bb1 to the second bit in row c[i] and its carry value, and so on, until the 32nd bit in row c[i]). As will be appreciated, after multiplying and accumulating all bit groups of the first row of the input matrix (e.g., matrix A) in the manner described herein, row c[i] includes the dot product of a row of matrix A and all columns of matrix B. In other words, the first group of bits (e.g., a 32-bit value) in row c[i] is the dot product of the first row of matrix A and the first column of matrix B, the second group of bits in row c[i] of matrix C is the dot product of the first row of matrix A and the second column of matrix B, etc. The process described above with reference to Figures 2, 4, 5A, and 5B may be repeated for all other rows of matrix A and matrix B to generate bits for all columns of matrix C.

[0076] In some embodiments, it may not be necessary to generate the entire scaled matrix (i.e., a scaled version of the matrix B). More specifically, when the intermediate matrices (e.g., the intermediate matrix M consisting of rows b'(1), b'(2), etc.) are read and merged, the results can be accumulated and summed with the next intermediate matrix (i.e., the intermediate matrix generated by multiplying the next one in the sequence a_(i,k+1) by the row b_(k+1)). In addition, in some embodiments, it may not be necessary to generate and store the intermediate matrices. In fact, in these embodiments, calculations can be used inside the FMA module instead of operations so that the rows of the intermediate matrices are accumulated immediately. In addition, according to some embodiments, shift copy with XOR and carry (e.g., via 32 shifts and serial XOR and carry with each shift) can be performed within the array (e.g., in real time).

[0077] It should be noted that, according to some embodiments, the bits of each multiplier operand may be loaded into the sequencer 402 sequentially (e.g., loading bits as needed), rather than loading entire groups of bits of a row of bits of the input matrix (i.e., matrix A) into the sequencer 402. More specifically, in some embodiments, a single bit (e.g., a first bit of an operand) may be loaded into the sequencer 402, and the single bit may be multiplied by the row loaded into the array 404. Additionally, another bit (e.g., a second bit of an operand) may be loaded into the sequencer 402 during or after a multiplication. More specifically, for example, after an operation (e.g., a multiplication) is completed using the first bit of the operand, the second bit of the operand may be loaded (e.g., replacing the first operand) and made available by the sequencer 402 for a subsequent operation. Similarly, for example, after an operation (e.g., a multiplication) is completed using the second bit of the operand, the third of the operands may be loaded (e.g., replacing the second bit of the operand) and made available by the sequencer 402 for another operation. Loading the bits or operands sequentially into sequencer 402 can reduce the required die space without sacrificing performance.

[0078] As a non-limiting example, a single operation (i.e., multiply-accumulate a_(i, k) by row b_(k), wherein an intermediate matrix is ​​generated, stored, and read back) may include the following latency (in nanoseconds): 10+10 (i.e., reading numbers a_(i, k) and b_(k) simultaneously + 32×10 (i.e., generating and storing the matrix M). Additionally, to read back and merge-accumulate (including bitwise carry calculations) the matrix M and the pre-accumulated results may include the following latency: 10 (i.e., to read back the pre-accumulated results) + 10 (i.e., to read back the first row of the matrix M) + 65 (i.e., to merge on the bitwise carry adder). =The first row of the accumulation matrix M is combined with the accumulation result) + 65 × 31 (i.e., to merge the other rows of the accumulation matrix M with the accumulation result on the carry-by-bit adder) + 10 (i.e., to store the new accumulation result). In this example, the latency is equal to 2,450 nanoseconds. This example assumes a 32-bit number, and for a 32b carry-by-bit adder, the memory timing TRCD (i.e., row address to column address delay) and TRP (i.e., row precharge time) are equal to 10ns, 1ns gate delay, and 65 gate delays. It should be noted that these latency values ​​are provided as a reference, and the values ​​may vary (e.g., changed, scaled, optimized).

[0079] Using the same assumptions, in another non-limiting example (i.e., multiply-accumulate a_(i,k) by row b_(k), where no intermediate matrix is ​​generated), a single operation may include the following latency (in nanoseconds): 10+10 (i.e., to read numbers a_(i,k) and b_(k) simultaneously)+10 (i.e., to read back the pre-accumulated result)+65×32 (i.e., to merge-accumulate in place by shift-copy with XOR and carry on a bitwise carry adder)+10 (i.e., to store the new accumulated result). In this example, the latency is equal to 2,120 nanoseconds. It should be noted that these latency values ​​are provided as a reference, and the values ​​may vary (e.g., changed, scaled, optimized).

[0080] Figure 6 6 is a flowchart of an example method 600 of operating a memory system according to various embodiments of the present disclosure. The method 600 may be arranged according to at least one embodiment described in the present disclosure. In some embodiments, the method 600 may be performed by an apparatus or system, such as Figure 1 System 100, Figure 4 FMA unit 400, Figure 7 The system 700, and / or Figure 8 800, or another device or system. Although illustrated as discrete blocks, various blocks may be divided into additional blocks, combined into fewer blocks, or eliminated, depending on the desired implementation. It should be noted that various blocks may be executed simultaneously with other blocks, and / or some blocks may be operated in a pipelined manner.

[0081] Method 600 may begin at block 602, where an input vector may be loaded into a sequencer, and method 600 may proceed to block 604. For example, an input vector, which may include one or more bits, may be loaded into a sequencer within a memory device (e.g., within or proximate to a sensing circuitry) or external to a memory device (e.g., within an external controller). More specifically, for example, an input vector, which may include an input matrix (e.g., Figure 2A One or more bits of a row of a matrix A) may be loaded into Figure 4 In the sequencer 402.

[0082] At block 604, a row of a memory array of a memory device may be loaded into a sense amplifier array of the memory device, and method 600 may proceed to block 606. For example, the row, which may include a plurality of bits (e.g., a plurality of groups of bits), may be loaded into Figure 4 In one or more FMA modules (e.g., FMA j, FMA j+1, etc.). In addition, for example, the bit lines (e.g., Figure 4 bit lines 408 / 410) to load each bit of the row.

[0083] At block 606, each bit group of the row may be multiplied by the bits of the input vector to produce a scaled row comprising a plurality of columns, and method 600 may proceed to block 608. For example, the bits of the input vector, which may be across an array of sense amplifiers (e.g., Figure 4 Each bit of the row loaded in the sense amplifier array may be multiplied to produce a scaled row. For example, a scaled row may include Figure 5A The scaling row b'(1) described in .

[0084] At block 608, each bit group of the row may be multiplied by at least one additional bit of the input vector to produce at least one additional scaled row comprising a plurality of columns, and method 600 may proceed to block 610. For example, a second bit of the input vector, which may be across a sense amplifier array (e.g., Figure 4 404) can be shared, and each bit of the row loaded in the sense amplifier array can be multiplied to produce a second scaled row. In addition, for example, the third bit of the input vector, which can be shared across the sense amplifier array (e.g., Figure 4 404) can be shared by each bit of the row loaded in the sense amplifier array to generate a third scaled row. For example, the second scaled row may include Figure 5A The scaling row b'(2) described in , and the third scaling row may include Figure 5A The scaling row b'(3) described in .

[0085] At block 610, at least one additional scaled row may be shifted by one column position relative to the scaled row, and method 600 may proceed to block 612. For example, Figure 5A Each scaled row is shifted by one column position relative to the adjacent row. For example, in the case of a FMA unit (e.g., Figure 4 During or after performing an FMA operation to generate a scaled row by an FMA array 404 of the FMA unit, the FMA unit may shift the scaled row relative to a previously generated scaled row (eg, one column position to the left).

[0086] At block 612, the scaled row and at least one additional scaled row may be summed along the columns to produce a partial result that contributes to a row of the output matrix. Figure 4 The FMA array 404 of FIG. 404 may be configured to scale the row and at least one additional scaled row (eg, row b'(1), row b'(2), row b'(3), and row b'(i); see Figure 5A ) are summed to produce the rows that contribute to the output matrix (e.g., Figure 2A Generating an entire row of the output matrix may include multiple iterations of this process for each bit of each multiplier operand and the associated row of the memory array.

[0087] Modifications, additions, or omissions may be made to method 600 without departing from the scope of the present disclosure. For example, the operations of method 600 may be implemented in a different order. Furthermore, the outlined operations and actions are provided only as examples, and some of the operations and actions may be optional, combined into fewer operations and actions, or expanded into additional operations and actions without departing from the essence of the disclosed embodiments. For example, in various embodiments, another input vector may be loaded into the sequencer, and / or another row may be loaded into the sense amplifier array. Additionally, in some embodiments, the intermediate matrix (e.g., Figure 3 An intermediate matrix M) may be generated and / or stored (eg, in a scratch pad area interfacing with the sense amplifier array).

[0088] A memory system is also disclosed. According to various embodiments, the memory system may include a memory device (e.g., Figure 1 A memory device 120) includes one or more memory cell arrays. Figure 7 7 is a simplified block diagram of a memory system 700 implemented according to one or more embodiments described herein. The memory system 700 (which may include, for example, a semiconductor device) includes a memory array 702 and an FMA unit 704 (which may include Figure 4 4. Memory array 704 may include a plurality of memory groups and may include a plurality of memory cells.

[0089] The FMA unit 704 may be operably coupled to the memory array 702 to read or write any or all memory cells within the memory array 702. The FMA unit 704 may be configured to perform one or more embodiments disclosed herein. For example, in some embodiments, the FMA unit 704, which may include a sequencer and a sense amplifier array, may be configured to perform one or more FMA operations in accordance with various embodiments disclosed herein.

[0090] A system is also disclosed. According to various embodiments, a system may include a memory device including a plurality of memory banks, each memory bank having an array of memory cells. Each memory cell may include an access transistor and a storage element operably coupled to the access transistor.

[0091] Figure 8800 is a simplified block diagram of an electronic system 800 implemented according to one or more embodiments described herein. The electronic system 800 includes at least one input device 802, which may include, for example, a keyboard, a mouse, or a touch screen. The electronic system 800 further includes at least one output device 804, such as a monitor, a touch screen, or a speaker. The input device 802 and the output device 804 may not necessarily be separated from each other. The electronic system 800 further includes a storage device 806. The input device 802, the output device 804, and the storage device 806 may be coupled to a processor 808. The electronic system 800 further includes a memory device 810 coupled to the processor 808. The processor 808 may include Figure 7 The memory device 810 of the memory system 700 may include an array of memory cells. The electronic system 800 may include, for example, a computing, processing, industrial, or consumer product. By way of example and not limitation, the system 800 may include a personal computer or computer hardware component, a server or other networking hardware component, a database engine, an intrusion prevention system, a handheld device, a tablet computer, an electronic notebook computer, a camera, a phone, a music player, a wireless device, a display, a chipset, a game, a vehicle, or other known systems.

[0092] According to various embodiments disclosed herein, and in contrast to some conventional devices, systems, and methods, data used in one or more calculations (e.g., bits across rows of a matrix) may be reused. As a result, the number of reads from and / or writes to one or more memory arrays may be reduced, which may increase the efficiency and / or speed of PIM operations and / or reduce latency associated with PIM operations.

[0093] One or more embodiments of the present disclosure include a method of operating a memory system. For example, the method may include loading a first number of bits into the circuitry of a memory device, each bit of the first number of bits having a first state or a second state. The method may also include loading a second number of groups of bits into the circuitry, each bit of the second number of groups of bits having a first state or a second state. Additionally, the method may include multiplying each bit group of the second number of groups of bits by each bit of the first number of bits to produce a plurality of scaled rows. Additionally, the method may include summing the plurality of scaled rows along associated bit positions to produce an output row.

[0094] According to another embodiment, a method of operating a memory system including a memory device may include loading a first number of bit groups of a memory array into a circuit system via a plurality of bit lines of a memory array of the memory device. The method may also include loading a plurality of operands into the circuit system. Additionally, the method may include multiplying each bit in each bit group of the first number of bit groups by a first bit of the operand to generate a first scaled row. And, the method may include generating a plurality of bits of a row of an output matrix based at least in part on the first scaled row.

[0095] In yet another embodiment, a method may include loading a group of rows of a memory array into a circuit system via a plurality of bit lines of a memory array of a memory device. The method may further include loading a group of operands into the circuit system. The method may also include multiply-accumulating each row in the group of rows by each associated operand in the group of operands to produce a first row of an output matrix.

[0096] Some embodiments of the present disclosure include a memory system. The memory system may include at least one memory array and logic coupled to the at least one memory array. The logic may be configured to receive an input including a plurality of bits. The logic may also be configured to receive a row of a memory array of the at least one memory array, the row including a bit group. In addition, the logic may be configured to multiply each bit in each bit group by each bit of the input to generate a plurality of scaled row groups, wherein each scaled row in each scaled row group in the scaled row groups includes a plurality of columns, and each scaled row in each scaled row group in the scaled row groups is shifted by at least one column position relative to an adjacent scaled row in the associated scaled row group. In addition, the logic may be configured to sum scaled rows across columns within each scaled row group to generate an output row.

[0097] Additional embodiments of the present disclosure include an electronic system. The electronic system may include at least one input device, at least one output device, and at least one processor device operably coupled to the input device and the output device. The electronic system may also include at least one memory device operably coupled to the at least one processor device. The at least one memory device may include at least one memory cell array and logic coupled to the at least one memory array. The logic may be configured to receive a first operand and a row of a memory array of the at least one memory array. The logic may also be configured to cause a plurality of bits of the row of the memory array to be multiplied, shifted, and accumulated by the bits of the operand to produce a first scaled row. The logic may also be configured to receive at least one additional operand. In addition, the logic may be configured to cause a plurality of bits of the row to be multiplied, shifted, and accumulated by the bits of at least one additional operand to produce at least one additional scaled row. And, the logic may be configured to generate a row of an output matrix based on the first scaled row and the at least one additional scaled row.

[0098] According to convention, the various features illustrated in the drawings may not be drawn to scale. The descriptions presented in the present disclosure are not intended to be actual views of any particular device (e.g., apparatus, system, etc.) or method, but are only idealized representations for describing various embodiments of the present disclosure. Therefore, for the sake of clarity, the sizes of various features may be arbitrarily enlarged or reduced. In addition, some figures may be simplified for the sake of clarity. Therefore, the drawings may not depict all components of a given device (e.g., device) or all operations of a particular method.

[0099] As used herein, the term "device" or "memory device" may include a device having a memory, but is not limited to a device having only a memory. For example, a device or memory device may include a memory, a processor, and / or other components or functions. For example, a device or memory device may include a system on a chip (SOC). In some embodiments, the computing methods described herein may be applicable to a storage device, such as a solid-state drive. Therefore, the term "memory device" used herein may include a storage device.

[0100] The terms used herein, and especially in the following claims (e.g., the bodies of the following claims), are generally intended to be “open” terms (e.g., the term “including” should be interpreted as “including, but not limited to,” the term “having” should be interpreted as “having at least,” the term “includes” should be interpreted as “including, but not limited to,” etc.).

[0101] Additionally, if a specific number of introduced claim recitations is intended, such intent will be explicitly recited in the claim, and in the absence of such a recitation, such intent is absent. For example, to aid understanding, the appended claims may contain the use of the introductory phrases "at least one" and "one or more" to introduce claim recitations. However, the use of such phrases should not be interpreted as implying that the introduction of a claim recitation by the indefinite article "a / an" limits any particular claim containing such introduced claim recitation to embodiments containing only one such recitation, even when the same claim contains the introductory phrases "one or more" or "at least one" and an indefinite article such as "a" (e.g., "a" should be interpreted as meaning "at least one" or "one or more"); the same applies to the use of definite articles to introduce claim recitations. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0102] Furthermore, even if a specific number of an introduced claim recitation is explicitly recited, it should be understood that such recitation should generally be interpreted to refer to at least the number of the recitations (e.g., a simple recitation of "two recitations" without other modifiers means at least two recitations, or two or more recitations). Moreover, in those instances where a convention similar to "at least one of A, B, and C, etc." or "one or more of A, B, and C, etc." is used, generally such constructions are intended to include only A, only B, only C, A and B together, A and C together, B and C together, or A, B, and C together, etc. For example, use of the term "and / or" is intended to be interpreted in this manner.

[0103] In addition, as used herein, including in the claims, "or" used in a list of items (e.g., a list of items beginning with a phrase such as "at least one of" or "one or more of") indicates an inclusive list, so that, for example, a list of at least one of A, B, or C means A or B or C or AB or AC or BC or ABC (i.e., A and B and C). Also, as used herein, the phrase "based on" should not be interpreted as a reference to a closed set of conditions. For example, an exemplary step described as "based on condition A" may be based on both condition A and condition B without departing from the scope of the present disclosure. In other words, as used herein, the phrase "based on" should be interpreted in the same manner as the phrase "based at least in part on."

[0104] In addition, the use of the terms "first", "second", "third", etc. is not necessarily used in this article to mean a specific order or number of elements. In general, the terms "first", "second", "third", etc. are used as universal identifiers to distinguish different elements. In the absence of the expression that the terms "first", "second", "third", etc. mean a specific order, these terms should not be understood to mean a specific order. In addition, in the absence of the expression that the terms "first", "second", "third", etc. mean a specific number of elements, these terms should not be understood to mean a specific number of elements.

[0105] The embodiments of the present disclosure described above and shown in the accompanying drawings do not limit the scope of the present disclosure, which is within the scope of the attached claims and their legal equivalents. Any equivalent embodiments are within the scope of the present disclosure. In fact, in addition to the contents of the alternative applicable combinations of the described elements shown and described herein, various modifications of the present disclosure will become apparent to those skilled in the art from the description. Such modifications and embodiments also fall within the scope of the attached claims and equivalents.

Claims

1. A system comprising: Logic configured to: multiplying each of the plurality of bits from the array by a first bit of an input vector from the sequencer to produce bits for a first row; multiplying each of the plurality of bits by one or more additional bits of the input vector to generate one or more additional rows of bits; and An output row of bits is generated based at least on the first row of bits and the one or more additional rows of bits.

2. The system of claim 1, the logic comprising one or more fused multiply-add (FMA) units to generate the one or more additional rows of bits.

3. The system of claim 1 , wherein the logic comprises: The sequencer is configured to receive the input vector; and A sense amplifier is configured to receive the plurality of bits in a bit-parallel manner.

4. The system of claim 1, wherein the logic is further configured to: multiplying each bit of a second plurality of bits from the array by a first bit of a second input vector from the sequencer to produce a third row of bits; multiplying each of the second plurality of bits by one or more additional bits of the second input vector to produce at least one additional row of bits; generating an additional output row of bits based at least on the third row of bits and the at least one additional row of bits; and The output rows and the additional output rows are summed along columns to produce rows of an output matrix.

5. The system of claim 1, further comprising a memory device comprising the logic, wherein the memory device comprises the sequencer.

6. The system of claim 1, further comprising a memory device including the logic, wherein the sequencer is external to the memory device.

7. The system of claim 1, wherein the logic comprises a plurality of sense amplifier groups, each sense amplifier group of the plurality of sense amplifier groups configured to receive a tile associated with a portion of the plurality of bits.

8. The system of claim 1, wherein the logic is configured to sum bits of the first row and bits of the one or more additional rows along associated bit positions.

9. The system of claim 1, further comprising: at least one input device; at least one output device; at least one processor device coupled to said input device and said output device; and At least one memory device is coupled to the at least one processor device and includes the logic.

10. A method comprising: multiplying each of the plurality of bits from the array by a first bit of the operand from the sequencer to produce bits for a first row; multiplying each of the plurality of bits by one or more additional bits of the operand to produce one or more additional rows of bits; and An output row is generated based at least on the bits of the first row and the bits of the one or more additional rows.

11. The method according to claim 10, further comprising: multiplying each bit of a second plurality of bits from the array by a second bit from the operand to produce a third row of bits; multiplying each of the second plurality of bits by at least one additional bit of the operand to produce at least one additional row of bits; and Another output row is generated based at least on the bits of the third row and the bits of the at least one additional row.

12. The method of claim 10, further comprising loading a plurality of bits of the operand from a second array into the sequencer.

13. The method of claim 10, further comprising loading the plurality of bits from the array into a second array.

14. A memory system comprising: at least one memory array; and logic coupled to the at least one memory array and configured to: multiplying each bit in the bit group by each bit of the input to generate a plurality of rows, each row in the plurality of rows comprising a plurality of columns, and each row in each of the plurality of rows being shifted by at least one column position relative to an adjacent row within the plurality of rows; and An output row is generated based on the plurality of rows.

15. The memory system of claim 14, wherein the logic is configured to sum bits across columns within the plurality of rows to produce the output rows.

16. The memory system of claim 14, the logic further configured to: multiplying each bit in the second group of bits by each bit of the input to generate a plurality of additional rows, each row of the plurality of additional rows comprising a plurality of additional columns, and each row of each of the plurality of additional rows being shifted by at least one column position relative to an adjacent row within the plurality of additional rows; and Additional output rows are generated based on the plurality of additional rows.

17. The memory system of claim 16, wherein the logic is configured to sum bits across columns within the plurality of additional rows to produce the additional output rows.

18. The memory system of claim 14, the logic further configured to receive the group of bits from a memory array of the at least one memory array, wherein each bit in the group of bits is received substantially simultaneously at the logic.

19. The memory system of claim 14, wherein the logic comprises a sequencer, the sequencer including the input.

20. The memory system of claim 14, wherein the logic comprises sense amplifiers configured to receive the groups of bits in a bit-parallel manner.

Images