In-memory computer

Abstract

A multi-processor computer system comprises an in-memory comprising a compute engine and storage elements and a processor external to the in-memory computer. The processor has a memory address space for writing data into at least some of the storage elements and for reading data from at least some of the storage elements. The compute engine is operable to process data stored in a storage element in response to a read operation or a write operation on a storage element by the processor. The compute engine can comprise bit-serial multipliers and bits of a number can be stored in sequential addresses in the storage elements. A dot-product between numbers in vectors can be computed by serially accessing, multiplying, counting, and accumulating bits in the numbers.

Description

[0001] IN-MEMORY COMPUTER

[0002] TECHNICAL FIELD

[0003] The present disclosure relates generally to distributed digital memory and computing element architectures, devices, and methods that facilitate matrix multiplication.

[0004] BACKGROUND

[0005] Tensor calculations are important operations in many mathematical computations. For example, linear algebra can employ matrix multiplication to solve systems of linear equations such as differential equations. Such mathematical computations are applied, for example, in pattern matching, artificial intelligence, analytic geometry, engineering, physics, natural sciences, computer science, computer animation, and economics.

[0006] Matrix multiplication is typically performed in digital computers executing stored programs. The programs describe the operations to be performed and hardware in the computer, for example digital multipliers and adders, perform the operations. The data (matrices) operated upon are stored in digital memories, for example static random access memory (SRAM) or dynamic random access memory (DRAM) accessed through a memory- and-address bus. The number of bits retrieved at a time in parallel with a single access or read command is limited by the bus bit width and corresponds to the number of bits in the memory enabled by an address provided to the memory. Thus, access to data stored in memory is an intrinsic limitation in conventional digital computing systems. In some computing systems, specially designed hardware can accelerate the rate of computation.

[0007] In general, calculations requiring large matrices and high data rates can take longer to solve and use more power than desired. There is a need therefore, for computing logic and memory architectures that can perform matrix multiplication at higher data rates and with less power.

[0008] SUMMARY

[0009] Embodiments of the present disclosure can provide, inter alia, in-memory computers having architectures suitable for performing matrix multiplication with improved efficiency at faster rates with less power. The in-memory computers can provide massively parallel processing of large numbers of values, for example performing many matrix multiplication operations, such as dot products, at a same time. According to embodiments of the present disclosure, a multi-processor computer system can comprise an in-memory computer and a processor. The in-memory computer can comprise a compute engine and storage elements for storing data. The compute engine can be operable to process data stored in the storage elements of the in-memory computer. The processor can be external to the in-memory computer and connected to the in-memory computer. The processor can have a memory address space for writing data into at least some of the storage elements and for reading data from at least some of the storage elements. In embodiments, the compute engine can be operable to process data stored in a storage element (i) in response to a read operation on a storage element by the processor, (ii) in response to a write operation on a storage element by the processor, or (iii) both (i) and (ii). The in-memory computer can be operable to return the processed data to the processor in response to the read operation or in response to the write operation. In some embodiments, storage elements (e.g., memory in the in-memory computer) can be divided into two or more groups, each group responsive to a different address provided at a same time (e.g., responsive to addresses provided on separate address lines) so that data from the two or more groups can be accessed at a same time in parallel. In some embodiments, storage elements (e.g., memory in the in-memory computer) is responsive to only one address (e.g., has only one set of address lines).

[0010] In some embodiments, the read or write operation can provide an RW address in the memory address space, the storage element can store the data at the RW address, and the compute engine can be operable to process the stored data. In some embodiments, the compute engine can be operable to store the processed data in the storage element at the RW address. In some embodiments, the compute engine can be operable to store the processed data in the storage element at a process address different from the RW address. In some embodiments, the read or write operation can provide an RW address in the memory address space, the storage element can store data at a different address in the memory address space from the RW address, and the compute engine can be operable to process data stored in the storage element at the different address. In some embodiments, the compute engine can be operable to store the processed data at the RW address. In some embodiments, the compute engine can be operable to store the processed data at a process address in the memory address space different from the RW address. In some embodiments, the compute engine can be operable to process data in different ways depending on one of multiple different RW addresses.

[0011] According to embodiments of the present disclosure, the in-memory computer can comprise a memory-mode controller, the processor can be connected to the memory-mode controller with any combination of a function line, one or more address lines, and one or more data lines, and the memory-mode controller can be connected to the storage elements and the compute engine. In some embodiments, the memory-mode controller can be responsive to the RW address to control the compute engine to process data stored in the storage element. In some embodiments, the memory-mode controller can comprise an address analysis circuit for analyzing the RW address and controlling the compute engine in response to the address analysis. In some embodiments, the memory-mode controller can be responsive to the function line to control the compute engine to process data stored in the storage element. In some embodiments, the memory-mode controller can comprise a function analysis circuit for analyzing the function line and controlling the compute engine in response to the function-line analysis. In some embodiments, the memory-mode controller can be operable to (i) read processed data from a storage element (22) or (ii) write processed data to a storage element, or (iii) both (i) and (ii).

[0012] According to embodiments of the present disclosure, at least one of the storage elements can be mapped to multiple addresses in a memory space of the processor. In some embodiments, no storage element is mapped to multiple addresses in a memory space of the processor. In some embodiments, the memory-mode controller can be operable to map a write address to a write storage element in response to a write operation from the processor and map a read address to a read storage element different from the write storage element in response to a read operation to the write address from the processor. In some embodiments, the memory-mode controller can be operable to map a write address to a write storage element in response to a write operation from the processor and map a read address to a read storage element different from the write storage element in response to a read operation and the function line. In some embodiments, the RW address is mapped to at least three different storage elements and the compute engine processes data differently depending on the function line.

[0013] According to embodiments of the present disclosure, a multi-processor computer system can comprise multiple compute engines, wherein each compute engine is operable to multiply a bit of each number in two one-dimensional arrays of numbers at a same time. Some embodiments can comprise a number of compute engines sufficient to multiply a bit of each number in two two-dimensional arrays of numbers at a same time. In some embodiments, (i) at least some processing instructions are stored in storage elements within the in-memory- computer address range, (ii) at least some data is stored in storage elements within the in- memory-computer address range, or (iii) both (i) and (ii).

[0014] According to embodiments of the present disclosure, a multi-processor computer system can comprise an in-memory computer comprising a compute engine and storage elements for storing data, the compute engine operable to read, write, and process data stored in the storage elements of the in-memory computer and a processor external to the inmemory computer connected to the in-memory computer, the processor having a memory address space for writing data into at least some of the storage elements and for reading data from at least some of the storage elements. In some embodiments, the compute engine can be operable to process data stored in a storage element in response to a read or write operation on the storage element by the processor and at least one of the storage elements is mapped to multiple addresses in a memory space of the processor. According to embodiments of the present disclosure, a multi-processor computer system can comprise an in-memory computer and a processor. The in-memory computer can comprise a memory-mode controller, compute engine, and storage elements for storing data. The compute engine can be operable to read, write, and process data stored in the storage elements of the in-memory computer. The processor can be external to the in-memory computer and connected to the in-memory computer. The processor can have a memory address space for writing data into at least some of the storage elements and for reading data from at least some of the storage elements. In some embodiments, the processor can be connected to the memory-mode controller with any combination of a function line, one or more address lines, and one or more data lines. The memory-mode controller can be connected to the storage elements and the compute engine.

[0015] According to embodiments of the present disclosure, a dot-product calculator can comprise a controller operable to sequentially provide memory addresses, a memory operable to receive the memory addresses from the controller and store at least one vector X and a vector or matrix Y. Each vector can have numbers, each number can have bits, and a bit of each of the numbers for vector X and a bit of each of the numbers for vector Y can be accessible in parallel at a same time. In some embodiments, bit multipliers such as AND gates can be connected to the memory such that each bit multiplier inputs a bit of a number of vector X and a bit of a number of vector Y at a same time in response to the memory address provided by the controller and outputs the product of the input bits. A population counter can be operable to count the number of non-zero bits output from the bit multipliers. A serial accumulator can sequentially accumulate the count, each count and bit having a place value or position corresponding to the same address. In some embodiments, the same bit of each of the numbers for both of the vectors X and Y is stored at a same address of the memory addresses provided by the controller. In some embodiments, the memory addresses in the sequence are numerically adjacent. In some embodiments, the population counter is analog, (ii) the serial accumulator is analog, or (iii) both (i) and (ii). In some embodiments, the population counter is digital, (ii) the serial accumulator is digital, or (iii) both (i) and (ii).

[0016] In some embodiments, the memory can be operable to store multiple pairs of vectors X and Y and the dot-product calculator can comprise bit multipliers (such as AND gates) for each of the pairs connected to the memory. Each bit multiplier (e.g., each AND gate) can be operable to input a bit of a number of vector X in a pair and a bit of a number of vector Y in the same pair at a same time and output the product of the input bits. A population counter for each pair can be operable to count the number of non-zero bits output from the bit multipliers (e.g., AND gates) in the pair. A serial accumulator for each population counter can be operable to sequentially accumulate all or at least a portion of the count from the population counter, each count and bit having a place value (e.g., place position) corresponding to the same address. In some embodiments, the multiple pairs of vectors X can form a first two- dimensional matrix of numbers and the multiple pairs of vectors Y can form a second two- dimensional matrix of numbers and the dot-product calculator can be operable to calculate an array of scalar values corresponding to a dot product between the first two-dimensional matrix and the second two-dimensional matrix.

[0017] According to embodiments of the present disclosure, a dot-product calculator can comprise a controller operable to sequentially provide memory addresses and a memory operable to receive the memory addresses from the controller and store a first two- dimensional array (a first matrix) of numbers and a second two-dimensional array (a second matrix) of numbers. A bit of each of the numbers in the first matrix and a bit of each of the numbers in the second matrix can be accessible in parallel at a same time. A bit multiplier (e.g., an AND gate) for each of the numbers in the first matrix or in the second matrix can be connected to the memory and can be operable to input a bit from a number in the first matrix and input a bit from a number in the second matrix in parallel at a same time and output the product of the input bits. A population counter for each one-dimensional array of numbers in the first matrix or in the second matrix can be operable to count the number of non-zero bits output from the bit multipliers (e.g., AND gates) receiving the one-dimensional array of numbers. A serial accumulator for each population counter can be operable to sequentially accumulate all or at least a portion of the count of the population counter. Each count and bit can have a place value corresponding to the same address.

[0018] According to embodiments of the present disclosure, a dot-product calculator can comprise a controller operable to sequentially provide memory addresses, a memory operable to receive the memory addresses from the controller and store a first n-dimensional array (a first matrix) of numbers and a second m-dimensional array (a second matrix) of numbers, wherein the same bit of each of the numbers in the first matrix and each of the numbers in the second matrix is accessible in parallel at a same time, and a compute engine connected to the memory operable to compute an array of scalar numbers corresponding to a dot product between the first matrix and the second matrix. In some embodiments, n or m is two, three, four, five, six, seven, eight, nine, ten, eleven, or twelve. In some embodiments, the storage elements comprise bit cells and the compute engine comprises a capacitor associated with pairs of bit cells or comprises a capacitor associated with pairs of groups of multiple bit cells. In some embodiments, the storage elements comprise bit cells and the compute engine comprises binary logic associated with pairs of bit cells or comprises binary logic associated with pairs of groups of multiple bit cells.

[0019] According to embodiments of the present disclosure, a multi-processor computer system can comprise an in-memory computer and a processor. The in-memory computer can comprise a memory-mode controller, compute engine, and storage elements for storing data. The compute engine can be operable to read, write, and process data stored in the storage elements of the in-memory computer. The processor can be external to the in-memory computer and connected to the in-memory computer. The processor can have a memory address space for writing data into at least some of the storage elements and for reading data from at least some of the storage elements. The processor can be connected to the memorymode controller and the memory-mode controller can be connected to the storage elements and the compute engine. The memory-mode controller can receive memory addresses from the processor and map the memory addresses from a first memory address to a second memory address different from the first memory address. In some embodiments, the first memory address can be a virtual address. In some embodiments the first memory address can be a physical memory address. In some embodiments, the second memory address can be a physical memory address.

[0020] According to embodiments of the present disclosure, a multi-processor computer system can comprise an in-memory computer comprising a compute engine and a processor. The in-memory computer can comprise storage elements for storing multi-bit numbers. The compute engine can be operable to process the multi-bit numbers (e.g., data) stored in the storage elements of the in-memory computer. The processor can be external to the inmemory computer and connected to the in-memory computer. The processor can have a memory address space for writing the multi-bit numbers into at least some of the storage elements and for reading data (e.g., multi-bit numbers) from at least some of the storage elements. In some embodiments, the different bits of the multi-bit numbers are stored in the storage elements in different storage elements at different storage element addresses, for example so that a read operation to a storage element can only retrieve one bit of any of the multi-bit numbers, e.g., one bit of each of the multi-bit numbers, for example the one bit having the same place or position value. Thus, each bit of a multi-bit number can be stored at a different storage element address from any other bit of the multi-bit number. In some embodiments, only some but not all of the bits of a multi-bit number can be stored in a storage element at a different storage element address from any other bit of the multi-bit number. For example some of the bits in a multi-bit number can be stored in different memory banks (comprising storage elements) separately responsive to a common storage element address at the same time.

[0021] According to embodiments of the present disclosure, a multi-processor computer system can comprise an in-memory computer comprising a compute engine, storage elements for storing data, and a processor. The compute engine can be operable to process data stored in the storage elements of the in-memory computer. The processor can be external to the in-memory computer connected to the in-memory computer and can have a memory address space for writing data into at least some of the storage elements and for reading data from at least some of the storage elements. In some embodiments, the compute engine can be operable to (i) first process data stored in a storage element in response to a read or write on a storage element at a first address by the processor and (ii) second process data stored in a storage element in response to a read or write on the storage element at a second address by the processor, wherein the first address is different from the second address and the first process is different from the second process.

[0022] According to embodiments of the present disclosure, a method of multiplying a number A times a number B, A and B having bits each in a bit place can comprise (i) sequentially multiplying a bit of A times a bit of B to provide a bit product for each combination (e.g., pair) of bits in A and B having a common bit place and adding the bit products, (ii) shifting the added bit products by the product place, (iii) accumulating the shifted added bit products, and (iv) repeating steps (i)-(iii) for each bit place of A and B to provide a product of A and B.

[0023] According to embodiments of the present disclosure, a method of calculating a dot product between a vector X having numbers A and a vector Y having numbers B, where A and B each have bits in a bit place, can comprise (i) multiplying a bit of A times a bit of B to provide a bit product for a combination (e.g., a pair) of bits in A and B having a common bit place for each A and B in vectors X and Y, (ii) counting the bit products, (iii) accumulating the count, (iv) repeating steps (i) - (iii) for each combination of bits having a common bit place, (v) shifting the accumulated count by the bit place, (vi) adding the shifted accumulated count, and repeating steps (i)-(vi) for each bit place of a product of A and B to provide a dot product of A and B that is the added shifted accumulated count.

[0024] According to embodiments of the present disclosure, a multi-processor computer system can comprise an in-memory computer comprising a compute engine and storage elements for storing multi-bit numbers. In some embodiments, at least some of (e.g., all of) the different bits of the multi-bit numbers are stored in the storage elements at different storage element addresses. In some embodiments, the in-memory computer is operable to provide one or more storage element addresses to the storage elements and retrieve bits of different multi-bit numbers having a common bit place from the storage elements. In some embodiments, the compute engine comprises bit multipliers to multiply bits of different multi-bit numbers having a common bit place, providing a product for each pair of multiplied bits. In some embodiments, the compute engine comprises a population counter for counting the number of non-zero products of the pairs of multiplied bits. In some embodiments, at least some of the different bits of the multi-bit numbers are stored in the storage elements in different simultaneously accessible storage elements (e.g., in different memory banks responsive to different memory addresses). Those bits can be simultaneously multiplied and counted using separate bit multipliers and population counters, thereby reducing the time, number of cycles, and energy required to compute a dot product of the multi-bit numbers. Simultaneously addressable and separately accessible compute memories or storage elements can be separate memory banks. As used herein different memory banks simultaneously responding to a common memory address can be considered to have different memory addresses because they are different memory banks responsive to different and separate enable, power, or access signals that can be considered to be part of a memory address.

[0025] Embodiments of the present disclosure provide fast, efficient, and low-power digital computing circuitry suitable for matrix multiplication, for example as is commonly found in pattern matching, machine learning, and artificial intelligence applications.

[0026] BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The foregoing and other objects, aspects, features, and advantages of the present disclosure will become more apparent and better understood by referring to the following description taken in conjunction with the accompanying drawings, in which:

[0028] Fig. 1 is a schematic block diagram of an in-memory computer system according to illustrative embodiments of the present disclosure;

[0029] Fig. 2 is a memory allocation diagram according to illustrative embodiments of the present disclosure;

[0030] Fig. 3A is a flow diagram of a write operation for the in-memory computer system according to illustrative embodiments of the present disclosure;

[0031] Fig. 3B is a flow diagram of a read operation for the in-memory computer system according to illustrative embodiments of the present disclosure;

[0032] Fig. 4A is a flow diagram of a read operation with processing for the in-memory computer system according to illustrative embodiments of the present disclosure;

[0033] Fig. 4B is a flow diagram of a write operation with processing for the in-memory computer system according to illustrative embodiments of the present disclosure;

[0034] Fig. 5A is a flow diagram of a write operation to a first address followed by a read operation from a second address different from the first address with processing of data in the first address for the in-memory computer system according to illustrative embodiments of the present disclosure;

[0035] Fig. 5B is a flow diagram of a write operation to a first address followed by a read operation from any one of multiple second addresses different from the first address with processing of data in the first address dependent on the second address for the in-memory computer system according to illustrative embodiments of the present disclosure;

[0036] Figs. 6A, 6B, and 7 are hardware block diagrams of an in-memory computer according to illustrative embodiments of the present disclosure; Fig. 8 is a hardware block diagram of an in-memory computer having multidimensional arrays of computing elements according to illustrative embodiments of the present disclosure;

[0037] Fig. 9 is a schematic diagram of an in-memory computer with a serial accumulator according to illustrative embodiments of the present disclosure;

[0038] Fig. 10 is a schematic diagram of a two-dimensional array of memory elements and a one-dimensional array of compute elements according to illustrative embodiments of the present disclosure;

[0039] Figs. 11 A and 11 B illustrate rows and columns of bits in a binary multiplication useful in understanding embodiments of the present disclosure;

[0040] Fig. 12A is pseudo-code calculation of a row-wise dot-product computation with bitserial accumulation according to illustrative embodiments of the present disclosure;

[0041] Fig. 12B is a flow diagram of a column-wise dot-product computation with bit-serial accumulation according to illustrative embodiments of the present disclosure;

[0042] Fig. 13A is a schematic block diagram of a controller, compute memory, and compute engine with digital processing according to illustrative embodiments of the present disclosure;

[0043] Fig. 13B is a schematic block diagram of a controller, compute memory, and compute engine with analog processing according to illustrative embodiments of the present disclosure;

[0044] Fig. 14A is a schematic diagram illustrating digital bit multiplication and serial accumulation according to illustrative embodiments of the present disclosure;

[0045] Fig. 14B is a schematic diagram illustrating analog bit multiplication and serial accumulation according to illustrative embodiments of the present disclosure;

[0046] Fig. 15A is a hardware block diagram of a bit-multiplier compute element comprising a transistor according to illustrative embodiments of the present disclosure;

[0047] Fig. 15B is a hardware block diagram of a bit-multiplier compute element comprising pairs of transistors according to illustrative embodiments of the present disclosure; and

[0048] Fig. 16 is a logic diagram of the first stage of a population counter according to illustrative embodiments of the present disclosure.

[0049] The features and advantages of the present disclosure will become more apparent from the detailed description set forth below when taken in conjunction with the drawings, in which like reference characters identify corresponding elements throughout. In the drawings, like reference numbers generally indicate identical, functionally similar, and / or structurally similar elements. The figures are not necessarily drawn to scale. DETAILED DESCRIPTION OF CERTAIN EMBODIMENTS

[0050] In some embodiments of the present disclosure, a multi-processor computer system comprises an in-memory computer comprising a memory-mode controller, a compute memory, and a compute engine. The compute memory can be operable to store data and is accessible by the memory-mode controller and the compute engine. The compute engine can be operable to read, write, and process data stored in the compute memory. The compute memory can comprise one or more memory elements. Each memory element can store one or more bits, for example a bit, a byte, a word, a long word, or a single value of any number of bits. The compute engine can comprise one or more compute elements. Each compute element can comprise one or more processing elements, for example a bit multiplier. As used herein, the term compute engine can refer to one or more computing elements or collectively to multiple computing elements.

[0051] Embodiments of the present disclosure can comprise a processor external to the inmemory computer connected to and operable to write data to and read data from at least some portions of the compute memory by sending function commands and addresses to the memory-mode controller. Function commands can include read and write commands. Optionally, in some embodiments the processor sends data to the memory-mode controller or reads data from the memory-mode controller. In some embodiments, the processor sends data directly to the compute memory or to the compute engine or reads data directly from the compute memory or from the compute engine.

[0052] According to embodiments of the present disclosure and as illustrated in Fig. 1 , a multi-processor computer system 80 can generally comprise an in-memory computer 10 and a processor 70 external to in-memory computer 10 connected to in-memory computer 10. Inmemory computer 10 can comprise a memory-mode controller 12 (or MMC 12), compute engine 30, and compute memory 20 (e.g., a memory accessed with an address) for storing data. Compute engine 30 can be operable to read, write, and process data stored in compute memory 20 of in-memory computer 10. Processor 70 can have a memory address space for writing data into at least some of compute memory 20 and for reading data from at least some of compute memory 20. Memory-mode controller 12 can operate as an interface between processor 70 and compute memory 20, can operate as an interface between processor 70 and compute engine 30, or can operate as an interface between processor 70 and both compute memory 20 and compute engine 30, as shown in Fig. 1. In embodiments, address and function lines (e.g., read or write lines) of processor 70 are connected to memory-mode controller 12. Thus, memory-mode controller 12 can be operable to read processed data from a memory element 22 of compute memory 20 or write processed data to a memory element 22 of compute memory 20, or both, for example in response to processor 70. Memory elements 22 are also bit cells 22 for storing a bit or multiple bits and a storage element 22 for storing data. Similarly, memory-mode controller 12 can be operable to read processed data from a data store (e.g., registers) in compute engine 30 or write processed data to a data store (e.g., registers) in compute engine 30, or both. Furthermore, memory-mode controller 12 can be operable to transform, map, or convert addresses received from or written to processor 70 in the processor 70 memory address space accessible by in-memory computer 10. A conversion or transformation of a memory address from or to processor 70 by memory-mode controller 12 can be a memory mapping operation and the converted or transformed addresses are memory-mapped addresses.

[0053] Signals transmitted to and received from processor 70 can be communicated over a bus 13 (e.g., busbar) connecting processor 70 to memory-mode controller 12. Memory-mode controller 12 can be operable to intercept any combination of read signals, write signals, other function signals (collectively function signals 13A), address signals 13B, and data signals 13C sent to compute memory 20 or compute engine 30 from processor 70 and convert them to signals suitable for embodiments of the present disclosure. Similarly, memory-mode controller 12 can be operable to intercept any combination of read signals, write signals, other function signals (collectively function signals 13A), address signals 13B, and data signals 13C sent from compute memory 20 or compute engine 30 to processor 70 and convert them to signals suitable for embodiments of the present disclosure. Bus 13 can comprise wires or lines (e.g., electrical or optical conductors) carrying or transmitting signals and the wires, lines, or signals have a common label herein to avoid confusion, e.g., a wire or line and a signal transmitted on the wire or line can have the same label and reference. Thus, processor 70 can be connected to memory-mode controller 12 with any combination of a function line 13A (including read or write control lines), one or more address lines 13B, and one or more data lines 13C and memory-mode controller 12 can be connected to compute memory 20 and compute engine 30 with any combination of the same lines, or other lines, necessary to control or interact with compute memory 20 and compute engine 30.

[0054] In some embodiments, memory-mode controller 12 (also referred to as MMC 12 herein) is operable to map (e.g., modify, convert, or transform an address to reference a different physical storage or memory element 22) read or write addresses provided by processor 70 to access memory element 22 locations (addresses) to other, different memory element 22 locations (addresses) in compute memory 20, for example memory element 22 address locations in which compute engine 30 stores (writes) processed data. In some embodiments, compute engine 30 stores processed data in a local memory or register separate from compute memory 20 or accesses data from a local memory or register separate from compute memory 20 and MMC 12 maps a read or write address from processor 70 to the local memory or register. Mapping an address means converting, modifying, or transforming an address indicating one physical memory location to another, different physical memory location. By mapping a read or write address from processor 70 to a memory element 22 (or other memory or register device) with MMC 12, interface and control for storing, processing, and accessing processed data by processor 70 is greatly simplified and provides a simple, well-understood, or standardized interface to in-memory computer 10 from processor 70, reducing and simplifying system interface complexity, hardware, and software for processor 70.

[0055] Thus, in some embodiments of the present disclosure, a multi-processor computer system 80 can comprise an in-memory computer 10 comprising a compute engine 30 and compute memory 20 for storing data and a processor 70 external to in-memory computer 10 connected to in-memory computer 10. Processor 70 can have a memory address space for writing data into at least some of compute memory 20 and for reading data from at least some of compute memory 20. Compute engine 30 is operable to read, write, and process data stored in compute memory 20 of in-memory computer 10 and is operable to process data stored in a memory element 22, for example in response to a read operation (e.g., a read command or read function) on a memory element 22 by processor 70 or in response to a write operation (e.g., a write command or write function) on a memory element 22 from or by processor 70, or both. The processed data can be stored in compute memory 20, in local storage in compute engine 30, transmitted to MMC 12, or provided directly to processor 70 in response to the processor 70 operation. Data lines 13C as used by processor 70 can transmit binary data stored on bit lines 24 (BL 24) and intended for writing to or reading from compute memory 20 at addresses communicated on word lines 26 (WL 26). Thus, data lines 13C can comprise or be bit lines 24 and address lines 13B can comprise or be word lines 26. In some embodiments, a write operation causes compute memory 20 to receive data and compute engine 30 to operate upon the written data to process the data and store the processed data values in compute memory 20. In some embodiments, a read operation causes compute engine 30 to access data in compute memory 20, operate upon the accessed data to process the data, and return the processed data values to MMC 12 or directly to processor 70.

[0056] Processor 70 can be any computational device external to in-memory computer 10, for example a stored-program machine or state machine (e.g., a digital or analog computer, processor, or controller). In-memory computer 10 can comprise any of the elements described above or logic circuits or other computational or data processing circuits or systems, and compute memory 20 can be any data storage structure as described above or memory devices such as registers, or flip-flops or addressable memory devices such as SRAMs or DRAMs. Memory-mode controller 12 (MMC 12) can similarly comprise or be a circuit, such as a logic circuit or digital, analog, or mixed-signal circuit, for example implemented in an integrated circuit, such as a CMOS circuit comprising electronic devices such as transistors, diodes, capacitors, and resistors. MMC 12 can comprise one or more packaged integrated circuits or can be incorporated into other components of in-memory computer 10, e.g., packaged with compute memory 20 or compute engine 30.

[0057] In some embodiments, in operation, a write request (e.g., a write control signal) from processor 70 with data values (e.g., on data lines 13C) and a write address (e.g., on address lines 13B) is intercepted by MMC 12. In some embodiments, only the write address (e.g., transmitted on address lines 13B) is intercepted by MMC 12 and data values are transmitted on data lines 13C directly to compute memory 20 or compute engine 30. In other embodiments, both the write address (e.g., transmitted on address lines 13B) and data values (e.g., transmitted on data lines 13C) are intercepted by MMC 12. In some embodiments, the write address is not modified by MMC 12, and the data values are written to compute memory 20 at the write address. In some embodiments, the write address is modified (e.g., converted, mapped, or transformed) by MMC 12 and the data values are written to compute memory 20 or compute engine 30 at the different, modified (e.g., memory mapped) address. In some embodiments, the write operation is then complete. However, in some other embodiments, the write control signal is intercepted by MMC 12 and triggers a control signal (e.g., a control signal F) for compute engine 30, e.g., after the data values are written or independently of the data value writing, to process the written data values or other stored data values that are not part of the write request. The processed data can be stored in one or more compute memory 20 or in a memory, register, or flip-flops in compute engine 30, e.g., memory element 22.

[0058] In some embodiments, in operation, a read request (e.g., a read control signal) from processor 70 and a read address (e.g., on address lines 13B) is intercepted by MMC 12. In some embodiments, only the read address (e.g., transmitted on address lines 13B) is intercepted by MMC 12 and any data values returned in response to the read request can be transmitted on data lines 13C directly from compute memory 20 or compute engine 30. In other embodiments, both the read address (e.g., transmitted on address lines 13B) and data values (e.g., transmitted from compute memory 20 or compute engine 30) are intercepted by MMC 12. In some embodiments, the read address is not modified by MMC 12, and the data values are read from compute memory 20 at the read address. In some embodiments, the read address is modified (e.g., converted, mapped, or transformed) by MMC 12, and the data values are read from compute memory 20 or compute engine 30 at the different, modified (e.g., memory mapped) address, for example using a lookup table defining the memory mappings and possible functional operations. In some embodiments, the read operation is then complete. However, in some other embodiments, the read control signal is intercepted by MMC 12 and triggers a control signal (e.g., a control signal F) for compute engine 30 to process data values stored in compute memory 20 or compute engine 30. The processed data can be stored in a memory, register, or flip-flops in compute engine 30 or in one or more compute memory 20. In either case, in embodiments in-memory computer 10 can be operable to return the processed data to processor 70, e.g., on data lines 13C in response to the read operation (or even in response to a write operation).

[0059] As used herein, the address of a read or write operation, for example as provided by processor 70, can be called an “RW address” to distinguish it from a converted (modified, transformed, or mapped) address different from the RW address provided by processor 70. The RW address can be an address in the memory space of processor 70. A read operation or command by processor 70 with an RW address can be operable to read data, e.g., processed data, from a memory element 22 or other memory or register mapped to the RW address by MMC 12 and provide the read data to processor 70, e.g., on data lines 13C. A write operation or command by processor 70 with an RW address can be operable to write data from processor 70, e.g., from data lines 13C, to a memory element 22 or other memory or register mapped to the RW address by MMC 12.

[0060] In some embodiments, MMC 12 can map read and write addresses as illustrated in Fig. 2, but MMC 12 is not limited to such a specific embodiment; as will be appreciated by those knowledgeable in computing and digital design; other embodiments providing similar functionality are possible. As shown in Fig. 2, a memory space of processor 70 can comprise multiple portions or banks. Compute memory 20 can have a logical address at an arbitrary bank start address chosen to accommodate processor 70 and any other memories associated with processor 70. A first portion of compute memory 20 (labeled memory address space beginning at “bank start address” in Fig. 2) and the addresses associated with the first portion can be interpreted by MMC 12 as a normal SRAM access, so that any data read or written from an RW address having a value within the memory address space is read or written as a normal memory. In such embodiments, MMC 12 does not need to modify, convert, map, or transform the RW address and compute memory 20 in in-memory computer 10 is programmed (e.g., with firmware or address logic) to respond to the RW address with a read or write memory operation.

[0061] In contrast, RW addresses in compute memory 20 address space (beginning at “map start address” in Fig. 2) can be modified, mapped, converted, or transformed so that data values other than those stored in compute memory 20 at the RW address are accessed and processed in response to the read operation. For example, a data value can be written into compute memory 20 address 100i6 (all addresses are presumed to be in hexadecimal notation). A subsequent read operation at compute memory 20 address 200 can access, process, and return the data value at 100. MMC 12 analyzes the RW address 200 and modifies it to address 100 to access the data and perform the operation. Alternatively, a data value can be written into address 200 and a subsequent read operation on address 200 can trigger a processing operation by compute engine 30 on the data value written into address 200 of compute memory 20 and return the processed value. Optionally, the processed value can be written into compute memory 20.

[0062] In some embodiments and as illustrated in Fig. 2, the in-memory computer 10 (e.g., IMC 10) address space can map multiple copies of data stored in either the memory address space or IMC address space, e.g., the IMC address space can be subdivided as shown with the dashed lines in Fig. 2. Thus, a data value can be written into the memory address space, for example at address 100. If a subsequent read operation from address 200 (e.g., in the IMC address space) is performed by processor 70, compute engine 30 can perform a first process (e.g., function, operation, or algorithm) on the data at address 100 in response to a control signal from MMC 12. If a subsequent read operation from address 300 (e.g., in the IMC address space) is performed by processor 70, compute engine 30 can perform a second different process on the data at address 100. If a subsequent read operation from address 400 (e.g., in the IMC address space) is performed by processor 70, compute engine 30 can perform a third different process on the data at address 100. Thus, a read operation can instigate different functional operations on data stored in compute memory 20 depending on the address provided by the read operation. In some embodiments, rather than using different read addresses, MMC 12 can analyze function values (e.g., provided by processor 70 on function lines 13A) to determine the operation desired and control compute engine 30 to perform the indicated operation.

[0063] Compute memory 20, and more generally any memory, registers, or flip-flops in inmemory computer 10, can be logically mapped to a memory space of processor 70 using logic circuits and stored variables, simplifying the logical interface between processor 70 and inmemory computer 10. According to embodiments of the present disclosure, a processor 70 can write one or more data values to in-memory computer 10 and then read processed data values from in-memory computer 10 without requiring other operations by processor 70 and, for example, using conventional memory (e.g., static random access memory (SRAM) or dynamic RAM (DRAM)) read and write memory control signals, thereby simplifying a hardware, software, or logical interface between processor 70 and in-memory computer 10 and also improving performance by reducing the number of data write or read (access) operations in the system.

[0064] In embodiments of the present disclosure, the operation of compute engine 30 can be responsive to one of multiple different addresses, either write addresses or read addresses. For example, a first function can be performed on data stored in compute memory 20 if data is read using a first address and a second function different from the first function can be performed if the data is read using a second address different from the first address. MMC 12 can control compute engine 30 to process data differently in response to the different read addresses and also control the memory mapping so that data is returned from a same data storage location in memory element 22 or compute engine 30 or, in other embodiments, returns the processed data from a different physical memory. Similarly, a first function can be performed on data stored in compute memory 20 if data is written using a first address and a second function different from the first function can be performed if the data is written using a second address different from the first address. MMC 12 can control compute engine 30 to process data differently in response to the different write addresses and also control the memory mapping so that data is written to a same data storage location in memory element 22 or compute engine 30 or, in other embodiments, stores the written data to a different memory. Thus, compute engine 30 is operable to process data in different ways depending on one of multiple different RW addresses.

[0065] Memory-mode controller 12 can comprise an analysis circuit 19 for analyzing the RW address and controlling compute engine 30 in response to the address analysis. Analysis circuit 19 can comprise logic circuits, can be programmable, or can comprise one or more state machines. In some embodiments of the present disclosure, at least one memory element 22 of compute memory 20 is mapped to multiple addresses in a memory space of processor 70 and memory-mode controller 12 is operable to map a write address to a write memory element 22 of compute memory 20 in response to a write operation from processor 70 and map a read address to a read memory element 22 different from write memory element 22 in response to a read operation to the write address from processor 70. Thus, in embodiments the same RW address received from processor 70 can be mapped to one or more different physical compute memory 20 by MMC 12 depending on the function associated with the address by processor 70.

[0066] In some embodiments, different functions can be provided to MMC 12. For example, a single bit can encode a read or write signal or operation, but multiple function lines 13A can provide additional function signals to MMC 12. The additional function lines 13A can specify different operations done in conjunction with a read or write operation by compute engine 30. MMC 12 can input and analyze a function signal 13A provided by processor 70 and control compute engine 30 in response to the analyzed function. Function signal 13A can also encode a read or write operation. Thus, in some embodiments in-memory computer 10 comprises a memory-mode controller 12, processor 70 is connected to memory-mode controller 12 with any combination of one or more function lines 13A, one or more address lines 13B, and one or more data lines 13C, memory-mode controller 12 is connected to compute memory 20 and compute engine 30, and compute engine 30 is operable to process data in different ways (e.g., using different functions, algorithms, or mathematical operations) depending on function signals 13A provided to MMC 12 from processor 70.

[0067] In some such embodiments, memory-mode controller 12 can be responsive to function line 13A to control compute engine 30 to process data stored in memory element 22. In embodiments, memory-mode controller 12 is operable to map a write address to a write memory element 22 in response to a write operation from processor 70 and map a read address to a read memory element 22 different from write memory element 22 in response to a read operation and function line 13A. Memory-mode controller 12 can comprises an analysis circuit 19 for analyzing function signal 13A and controlling compute engine 30 in response to the function-line analysis. In some embodiments, the RW address is mapped to at least two or three different compute memory 20 and compute engine 30 processes data differently depending on function signal 13A, e.g., in response to control signals from MMC 12. Analysis circuit 19 can analyze both or either function signals 13A and address signals 13B. Use of a function line or lines can increase the variety of functions performed by compute engine 30 and hence the utility of in-memory computer 10.

[0068] The flow diagrams of Figs. 3A-5B illustrate various methods of the present disclosure performed by in-memory computer 10. In these Figures, each vertical column refers to a function or operation performed by the device or circuit labelled at the top of the column. Fig. 3A illustrates a simple write operation without an associated compute engine 30 operation (without processing data). As shown in Fig. 3A, processor 70 issues a write command (e.g., on function lines 13A and optionally with an additional function indicated on function lines 13A) with an RW address (e.g., on address lines 13B) and data values (e.g., on data lines 13C) in step 100. MMC 12 receives the issued write command, analyzes the write command (e.g., in conjunction with any other function signals 13A) in step 110 to determine that no operation is to be performed by compute engine 30, and passes the write command to compute memory 20. Compute memory 20 stores the data values at the RW address in step 120. This process can be operatively similar to a conventional write command provided by a processor 70 to an SRAM or DRAM or any other useful memory compatible with read or write commands provided by MMC 12, processor 70, or compute engine 30.

[0069] As shown in Fig. 3B, processor 70 issues a read command (e.g., on function lines 13A and optionally with an additional function indicated on function lines 13A) with an RW address (e.g., on address lines 13B) in step 105. MMC 12 receives the issued read command and analyzes the read command (e.g., in conjunction with any other function signals 13A) in step 110 to determine that no operation is to be performed by compute engine 30 and passes the read command to compute memory 20. Compute memory 20 reads the data values at the RW address in step 125. The read data value is returned (e.g., on data lines 13C) in step 150 (step 150 can functionally be the same step as read step 125). This process can be operatively similar to a conventional read command provided by a processor 70 to an SRAM or DRAM.

[0070] According to embodiments of the present disclosure, a data read operation can be performed as illustrated in Fig. 4A. Processor 70 can issue a read command (e.g., on function lines 13A and optionally with an additional function indicated with on function lines 13A) with an RW address (e.g., on address lines 13B) in step 105. MMC 12 receives the issued read command, analyzes the read command and RW address (e.g., in conjunction with any other function signals 13A, if present) in step 115 to determine that an operation is to be performed by compute engine 30 on data stored in compute memory 20 (stored at either RW address or a converted (memory mapped) address), and passes the read command to compute memory 20 and to compute engine 30. Compute memory 20 reads the data values at the address determined by MMC 12 in step 115 and provides the data values to compute engine 30 in step 125. Compute engine 30 processes the data values in step 130 and returns the processed data to processor 70 in step 150. Optionally, the processed data is stored in compute memory 20 in step 140. Thus, a simple read operation (step 105) by processor 70 returns processed data in step 150 without any further operations by processor 70, providing a simple interface to processor 70.

[0071] According to embodiments of the present disclosure, a data write operation can be performed in step 100 as illustrated in Fig. 4B. Then, steps 115, 125, and 130 proceed as described with respect to Fig. 4A. Processed data can be returned to processor 70 in step 150, optionally storing the processed data in compute memory 20 in step 140. This can preserve data in compute memory 20 for further operations in subsequent processing steps.

[0072] Fig. 5A illustrates methods of the present disclosure in which processed data is stored at a different physical location in compute memory 20 from the original, source or input data. As shown in Fig. 5A, data is first written into RW address A in step 100 by processor 70, the RW address is analyzed in step 110 by MMC 12 in response to the write command, and the data is stored into compute memory 20 in step 120. Subsequently, in step 125 processor 70 reads back the written data using the same RW address A, MMC 12 receives the RW address and read command, analyzes the RW address in step 115, and controls compute memory 20 to access the stored data and compute engine 30 to process the stored data in step 130. In this method, processed data is stored in address B different from RW address A in step 140 and returned to processor 70 in step 150. Thus, effectively a read operation by processor 70 on RW address A returns processed data stored at address B. The processing and storage location details in in-memory computer 10 are invisible to processor 70, simplifying the interface between processor 70 and in-memory computer 10, reducing the number of processor 70 operations, and improving performance.

[0073] Fig. 5B illustrates methods of the present disclosure in which a read operation to an address can specify an operation on data stored at that, or at another, address. As shown in Fig. 5B, data is first written into RW address A in step 100 by processor 70, the RW address is analyzed in step 110 by MMC 12 in response to the write command, and the data is stored into compute memory 20 in step 120. Subsequently, in step 125 processor 70 reads back the written data using an RW address different from A, MMC 12 receives the RW address and read command, analyzes the RW address in step 115, and controls compute memory 20 to access the stored data and compute engine 30 to process the stored data in step 430 according to an operation selected depending on the address provided by MMC 12. For example, if RW address is B, operation X is performed, if RW address is C, operation Y is performed, if RW address is D, operation Z is performed. The processed data can be stored in any suitable storage location (e.g., physical locations A, B, C, or D) and returned to processor 70 in step 150. In some embodiments, the processed data can be returned directly to processor 70 and MMC 12 need not actually perform an operation, particularly if data lines 13C do not pass through MMC 12. Thus, effectively a read operation by processor 70 with a selected address performs a selected operation on the data. The processing and storage location details in in-memory computer 10 are invisible to processor 70, simplifying the interface between processor 70 and in-memory computer 10, reducing the number of processor 70 operations, and improving performance.

[0074] In general, MMC 12 can control compute engine 30 to perform one or a variety of operations depending on signals provided on function lines 13A. Processor 70 can provide an address A on address lines 13B, data D on data lines 13C, and a function F on function lines 13A to MMC 12. In some embodiments, function F is a read R or a write W command, in other embodiments, function F can encode processing operations. MMC 12 receives A, optionally D, and F and analyzes at least address A and function(s) F. If a write operation F is indicated (e.g., F=W), the data D is written to compute memory 20. If a read operation F is indicated (e.g., F=R), the data D is read from compute memory 20. If a function operation is indicated, either by a function line 13A or by an address specification, compute engine 30 accesses stored data and processes the data according to control signals (instructions) provided by MMC 12 in response to the specification. The control signals provided by MMC 12 can depend on function F, on address A, or on a combination of both function F and address A. After compute engine 30 completes processing the data, it is optionally stored in compute memory 20 and returned to processor 70, or directly returned to processor 70, for example on data lines 13C.

[0075] In some embodiments of the present disclosure, a default operation is performed by compute engine 30 in response to a read or write request from processor 70 and no additional functional information is needed or provided (e.g., on function lines 13A).

[0076] In some embodiments, a single read (or write) operation can perform an entire vector or matrix operation. For example, multiple data vectors can be written into the memory address space by processor 70. A single read operation from one of the addresses at which the data vector is stored (or another address location) can be detected by MMC 12, and an entire vector operation (e.g., a dot product) can be performed by compute engine 30 on the multiple data vectors to produce a single value that is returned to processor 70. The operation performed can depend on the address of the read operation, for example IMC address space or vector space.

[0077] The output space indicated in Fig. 2 can provide additional storage in compute memory 20 accessible to processor 70 or provide readable storage for processed data (e.g., can be an extension of the memory address space). Configuration and operational values specifying functional operations and data types (e.g., data format and vector specification) can be stored in the config (configuration) and status space. A read or write operation on data stored in the compute trigger space can also trigger a compute engine 30 data processing function. These locations can also specify an interaction between compute engine 30 processing and processor 70 data access, for example specifying data polling, a blocking or non-blocking read, or an interrupt. These functions can be specified by the format of a bus (e.g., an APB, AHB, or AXI bus, or any other suitable bus) transmitting data, addresses, and functions, e.g., bus 13.

[0078] Processor 70 and in-memory computer 10 can operate together invisibly to a user by providing a programming language with a compiler that automatically controls the operation of processor 70 and in-memory computer 10. MMC 12, compute memory 20, and compute engine 30 operations can be controlled by compiled object or machine code. Such a compiler can manage access to the memory banks of compute memory 20 and functional operation of compute engine 30 for processor 70 so that the processor 70 program essentially controls inmemory computer 10. In some embodiments, no compiler is necessary and processor 70 hardware signals (e.g., read, write, or functional signals 13) can be interpreted by in-memory computer 10 (e.g., by MMC 12) to perform a desired operation.

[0079] In some embodiments of the present disclosure, compute engine 30 processing is very fast and processor 70 is not delayed in receiving processed data any more than can be the case with an SRAM. In other embodiments, processor 70 can experience some delay. In such embodiments, multi-processor computer system 80 can comprise a read or write handshake (an interactive control signal, e.g., as specified by bus 13 format and control signals) to indicate to processor 70 that processed data is ready and the read operation can be completed.

[0080] According to embodiments of the present disclosure, in-memory computer 10 can be engineered to share the same functional and physical interface as traditional SRAM. This compatibility allows in-memory computer 10 to be seamlessly integrated into existing computer platforms (e.g., processor 70) without necessitating any major architectural changes. For applications with more hardware engineering flexibility, in-memory computer 10 can be instantiated with optional interface signals to control the computation and optimize the system integration. Figs. 6A, 6B, and 7 are hardware block diagrams illustrating embodiments of the present disclosure in which different elements (blocks) of in-memory computer 10 are shown with different shadings. As illustrated in Figs. 6A, 6B, and 7, portions of in-memory computer 10 that can be memory-mode controller 12 (MMC 12) are shown with the lightest shading, the medium shading represents bit cells 22 of compute memory 20, and the darkest shading represents elements of compute engine 30. Some of the blocks are arbitrarily assigned, for example the compute engine 30 buffers can be considered to be a part of compute memory 20, compute engine 30, or MMC 12. Likewise, the configuration and status space can be considered to be a part of compute memory 20 or MMC 12. Data stored in compute memory 20 can be accessed (e.g., read or written) through bit lines 24 (data lines transmitting data values) and word lines 26 transmitting addresses, as shown in Fig. 1.

[0081] Compute memory 20 can comprise multiple independently addressable memories, for example so that two different sets of numbers stored in different memories can be accessed simultaneously. Thus, compute memory 20 can comprise multiple independently addressable memories or in-memory computer 10 can comprise multiple compute memories 20.

[0082] As shown in Figs. 6A, 6B, and 7, an in-memory computer 10 comprising a memorymode controller 12, compute memory 20, and compute engine 30 can be, comprise, or be configured as a dot-product calculator. The dot-product calculator can comprise a controller (e.g., processor 70 or MMC 12), a compute memory 20 (e.g., comprising bit cells 22), a compute engine 30 comprising AND gates 14 (e.g., bit multipliers 14) having two single-bit inputs and providing one single-bit output equal to the product of the input bits, a population counter 34 (e.g., comprising a bit counter 34), and a serial accumulator 38 (e.g., comprising an analog or digital shift-and-accumulate circuit 38 or SAC 38). Compute memory 20 can comprise multiple independently addressable memories or memory banks, e.g., compute memory 20A and compute memory 20B. In some embodiments, the bit product can be stored as a digital value (for example in a flip-flop, register, or other one-bit memory) or as an analog value (e.g., as a charge in a capacitor) and serial accumulator 38 can be a digital accumulator comprising one or more digital adders or an analog accumulator using capacitors. The controller (e.g., MMC 12) can be operable to sequentially provide memory addresses to compute memory 20. In embodiments, MMC 12 simultaneously provides at least two separate addresses to independently addressable compute memories 20A, 20B to read two different sets of data stored at the two separate addresses, e.g., corresponding to two different numbers. Compute memory 20 can be operable to receive (input) the memory addresses from the controller and store at least two vectors (tuples) X and Y. Each vector X and Y can have N numbers (elements in the tuple) and each number in the vector can be individually referenced here with a subscript n (e.g., Xn, Yn) whose value corresponds to the number in an ordered location in the vector (e.g., value of the element in the tuple). Each binary number n can have at least B bits, each bit in the binary number n individually referenced here as b with a subscript corresponding to the bit (e.g., with a subscript whose value corresponds to the place of the bit in the binary number Xnor Yn).

[0083] MMC 12 can organize the vector values in a variety of different ways in compute memory 20. As illustrated in Fig. 6A, the same bit (e.g., having the same place value) of each of the N numbers for vector X is accessible in parallel at a same time, for example stored at a common address in bit cells 22 at least N bits wide (e.g., N bit cells 22 returning N bits per individual address applied to compute memory 20), where each of the numbers in vector X has b bits. Numbers for vector Y can be similarly stored at different addresses in compute memory 20 with the same bit (e.g., having the same place value) of each of the numbers for vector Y is accessible in parallel, where each of the numbers in vector Y has c bits. Compute memory 20 can be at least deep enough to store bits for each of the numbers in vectors X and Y (e.g., 2B bits deep if they have the same number of bits and therefore have compute memory 20 for at least 2B different addresses). In an example, numbers in vector X and vector Y have eight bits each, for example eight bits for at least one value Xnand eight bits for at least one value Yn, totaling sixteen bits and requiring sixteen different memory addresses. MMC 12 can control compute memory 20 to simultaneously access bits having a common place value for all of the N values in vector X and access bits having the same place value for all of the values in vector Y. (In contrast, a conventional read provides all of the bits for a stored number.) Corresponding bits (e.g., having the same place value b) are then applied to the array of bit-multipliers (AND gates) 14. The bit products are counted with population counter 34, shifted according to the bit place b (e.g., bit position b), and accumulated with serial accumulator 38. This process is repeated for each bit b from zero to B (here B is the largest number of bits in numbers of either vector X or vector Y) to provide a dot-product of vectors X and Y.

[0084] Fig. 6B illustrates embodiments with more than two compute memories 20 (e.g., 20A, 20B, 20C...) accessible at the same time with different memory addresses. In some such embodiments, multiple arrays of bit multipliers 14 (AND gates 14) for bits having different bit places in numbers of vectors X and Y can be provided. Different bits having different bit places can be computed and counted at a same time by storing the bits with different bit places in different compute memories 20 responsive to different memory addresses (e.g., in different memory banks) and applying them to different bit multiplier 14 arrays. Each array of bit multipliers 14 provides products that are counted at the same time by a different population counter 34. Fig. 6B illustrates separate compute memories 20 only for numbers in vector Y, but separate compute memories 20 can be provided for numbers in each of the vectors whose dot product is calculated (e.g., also for numbers in vector X). (Fig. 6B does not completely show the connections from the additional bit multipliers 14 to the bits for the numbers in the X vector, but they are indicated and labeled ‘X’.) Once the bits from each separate compute memory 20 are multiplied, they are counted (e.g., with a separate population counter 34), each of the counts is shifted to accommodate the bit place of the products, and the counts are accumulated. If a different compute memory 20 and population counter 34 is provided for each bit place, then all of the counts can be accumulated at once in parallel. If only some of the bits are multiplied and counted at a time, then the remaining bits can be serially accumulated. For example, if a number has eight bits stored in two different simultaneously accessible compute memories 20 (e.g., with two different memory banks), then two bits for each number in the vector can be multiplied and counted at a time. The multiple bits require four (eight divided by two) cycles, and serial accumulator 38 serially accumulates four times. In general, different numbers of individually accessible compute memories 20 can be used to store different bits in a number and the bits in the different compute memories 20 can be multiplied, counted, and accumulated at a same time. The number of accumulation cycles depends on the number of bits and the number of different and separately and simultaneously accessible compute memories 20. Using multiple compute memories 20 (multiple memory banks) can increase throughput because fewer accumulation cycles are needed and can reduce the power used.

[0085] Embodiments of the present disclosure are not limited by the two vector storage illustrations of Figs. 6A and 6B. More generally and as shown in Fig. 7, MMC 12 can store vectors in any desired format and access corresponding bits of numbers in each of vectors X and Y having a same place value in any desired order to sequentially provide, count, and serially accumulate the bit multiplicands and multipliers of each number in vectors X and Y. Figs. 6A, 6B, and 7 illustrate embodiments in which compute engine 30 comprises a onedimensional array of bit multipliers 14 (equivalent to AND gates 14). In further embodiments of the present disclosure and as shown in Fig. 8, compute memory 20 can receive and store multiple vector pairs, e.g., one or more pairs of two-dimensional matrices of numbers. MMC 12 can control compute memory 20 to sequentially apply multiple vectors stored in compute memory 20 to sequentially compute multiple vector operations and provide matrix or tensor operations. As used herein, compute engine 30 can refer to an individual bit multiplier 14, a one-dimensional array of bit multipliers 14, or a two-dimensional array of bit multipliers 14 as shown in Fig. 8 and can include one or more population counters 34 and one or more serial accumulators 38, e.g., one for each vector operation.

[0086] In some embodiments and as shown in Fig. 8, in-memory computer 10 comprises multiple one- or two-dimensional arrays of compute engines 30. The multiple compute engine 30 arrays can, for example under the control of MMC 12, multiply bits of multiple vector pairs whose products are each accumulated to provide multiple dot products. In some embodiments, the bits of the multiple vector pairs are corresponding bits (e.g., have a common place value) of a single number. In some embodiments, the bits of the multiple vector pairs are corresponding bits of different numbers in a common vector pair (as described above). In some embodiments, the bits of the multiple vector pairs are corresponding bits of numbers in different vector pairs. Thus, some embodiments of the present disclosure comprise multiple compute engines 30. Each compute engine 30 (one-dimensional array of bit multipliers 14) can be operable to multiply a bit of each number in two one-dimensional arrays of numbers at a same time. In embodiments, in-memory computers 10 comprises a number of compute engines 30 sufficient to multiply a bit of each number in two two-dimensional arrays (two matrices) of numbers at a same time.

[0087] In some embodiments, compute engine 30 comprises multiple dot-product calculators, for example a two-dimensional array (e.g., a logically connected two-dimensional array) of dot-product calculators. The multiple dot-product calculators can calculate multiple vector dot products at a same time. In some embodiments, the vector pairs in the multiple dot products are in common matrices. In some embodiments, the vector pairs in the multiple dot products are in different matrices. Similarly, computer memory 20 can comprise multiple pairs of independently addressable memories that can each provide bits of pairs of numbers for a dotproduct calculation.

[0088] Fig. 8 provides a logical block diagram illustrating embodiments comprising multiple arrays of compute engines 30. As shown in Fig. 8, compute memory 20 can store, under the direction of MMC12 multiple two-dimensional arrays of values in pairs of simultaneously accessible values in compute memories 20A, 20B, e.g., multiple two-dimensional matrices. MMC 12 can then access compute memory 20 and provide a two-dimensional matrix (or submatrix) of values to a two-dimensional array of bit multipliers 14 (forming at least a portion of compute engine 30). Each one-dimensional array of compute engines 30 (e.g., arrays of bit multipliers 14 together with compute memories 20A, 20B and corresponding arrays of population counters 34 and serial accumulators 38) then compute dot products as described above, producing an entire matrix of scalar results in one operation.

[0089] An example of a 1024-element dot product of eight-bit values in two vectors can be extended, according to embodiments of the present disclosure, to provide multiple arrays of compute engines 30 (or a compute engine 30 comprising multiple arrays of bit multipliers 14, e.g., as illustrated in Fig. 8. For example, in embodiments of the present disclosure, a dot product between two matrices each having 1024 by 1024 eight-bit values can be performed in a single operation, for example in response to a read or write command from external processor 70. In such examples, compute engine 30 can comprise a 1024 by 1024 (equaling 220) array of bit multipliers 14. Compute memory 20 can be organized to provide bits of 1024 x 1024 values in parallel for each matrix in response to a read command from MMC 12 to compute engine 30 one bit at a time. The bits are multiplied (by bit multipliers 14), counted (by population counters 34), and serially accumulated (by SAC 38), to produce a 1024 by 1024 array of scalar results. Given the high density of small transistors in large semiconductor substrates, 220bit multipliers 14, for example as illustrated in Figs. 6B and 9-12, is readily achievable in an array of compute engines 30. Indeed, the number of elements in each vector and the number of bits in each number in the example are merely illustrative. In-memory computers 10 with greater or smaller numbers of compute engines 30, for example 216, 218, 220, 222, 224, or 226bit multipliers 14 and bit lengths of four, eight, twelve, sixteen, twenty, twenty-four, twenty-eight, thirty-two, or sixty-four.

[0090] The dot-product calculator can have at least N logical AND gates 14 connected to compute memory 20, such that each AND gate 14 receives (inputs) a bit b of a number n of vector X and a bit b of a number n of vector Y at a same time in response to a memory address provided by the controller (e.g., MMC 12) to compute memory 20, thereby performing a bit multiplication on pairs of bits from corresponding numbers (elements) in vectors X and Y having the same place (the same bit b). If corresponding numbers having a common ordered location in vectors X and Y are referenced with a subscript n and the bits in each number Xnand Ynare referenced with a subscript b for B bits, then AND gates 14 can multiply Xnb times Ynb for all of n from zero to (N-1 ) at a time for a given b at a time. The multiplication is then repeated for all of b from 0 to B-1 in response to sequential memory addresses provided by the controller, and the products counted and accumulated.

[0091] Population counter 34 is operable to count the number of non-zero bits from N AND gates 14 for each memory address (for each bit b) provided to compute memory 20 by the controller. Population counter 34 can be or comprise one or more digital counters 34 or an analog adder 34 and can have a separate input connected to each of the outputs of AND gates 14. For example, population counter 34 can comprise a series of counters, first for combining pairs of AND-gate outputs (e.g., a one-bit counter providing two-bit sums with values from 0 to 2, or triplets with values from 0 to 3), then for combining the two-bit sums (e.g., a two-bit adder providing three-bit sums with values from 0 to 4), and so on to produce a final count of all of the non-zero bits. Skilled digital designers will appreciate that different logical designs can implement population counter 34.

[0092] For example and with reference to Fig. 6A and Fig. 9, for bits in vectors X and Y, with N=2 and B=2, referenced as Xnb and Ynb,(so that n is the number in the corresponding vector and b is the bit in the corresponding place of the number) in a first cycle and in response to a first address (e.g., arbitrary address 0), AND gates 14 will multiply Xoo by Yoo and multiply Xw by Yw and present the two products to population counter 34 where they are counted if nonzero and the count provided to serial accumulator 38 (thereby initializing SAC 38 with the count value). In a second cycle and in response to a second address (e.g., address 1 ), AND gates 14 can multiply Xoo by Y01 and multiply Xw by Yn and present the two products to population counter 34 where they are counted, shifted by their relative place, and the shifted count provided to serial accumulator 38 which effectively multiplies the new count value by a factor of two (e.g., by shifting the binary count by one place as it has a place value twice that of the existing count value) and adds the second, shifted count value to the existing value. The process is repeated for the second bits of Xo and Yo and Xi and Yi to provide the dot product of vectors X and Y.

[0093] More generally, embodiments of the present disclosure comprise calculating a dot product of a vector X with M values having B bits and a vector Y with N values having C bits. N and M can be the same or different and B and C can be the same or different. At time zero (e.g., using memory address 0), Xoo AND Yoo, Xw AND Y , X20 AND Y20, and so on to X(N-I>O AND Y(N--I)O, are calculated and the number of non-zero bits is counted and stored in serial accumulator 38. At a subsequent time (e.g., using memory address 1 ), X01 AND Yoo, Xn AND Y10, X21 AND Y20, and so on to X(N-I>I AND Y(N-I>O are calculated, the number of non-zero bits is counted, and the count is serially accumulated with serial accumulator 38 (where the count is shifted by one). The process is repeated for each combination of B bits in X and C bits in Y, with a shift corresponding to the bit place, e.g., to multiply the count by 2b. The shifted count is serially accumulated with serial accumulator 38. The final serially accumulated value is then the scalar result of the dot product.

[0094] In some embodiments, the numbers in each of X and Y have the same number of bits B. In some embodiments, the numbers in X have a different number of bits B than the numbers in Y. Furthermore, in some embodiments, vectors X and Y are one-dimensional vectors. In some embodiments, either one or both of vectors X and Y are two-dimensional vectors (e.g., a two-dimensional matrix). More generally, the operands X and Y can be tensors of any dimension having numbers (elements) and MMC 12 and compute engine 30 can organize the operand elements in compute memory 20, control which elements in the operands are computed, and the computation performed. In some embodiments, compute engine 30 can comprise other computing elements (e.g., other than or in addition to bit multipliers 14 (AND gates 14), population counter 34, and serial accumulator 38) to implement other computations (e.g., other vector, matrix, or tensor operations).

[0095] As shown in Fig. 9, the output (the sum of non-zero bits) of population counter 34 is provided as a shifted count to serial accumulator 38 (e.g., an analog or digital shift-and- accumulate circuit 38). The temporally serial accumulator 38 (SAC 38) is operable to sequentially accumulate successive shifted counts (outputs from N AND gates 14 for each bit b). Each count has a place value corresponding to the bit b of the products counted, and the accumulation (sum) of the counts is adjusted for the place of bits b counted by shifting the count by the number of places corresponding to the bit accessed (beginning with zero and effectively multiplying by 2b). The accumulated sum after all B bits are multiplied, shifted, and serially accumulated is the dot product (also inner product or scalar product) of vectors X and Y. The process can be controlled using, for example, a state machine that can be part of compute engine 30.

[0096] As shown in Fig. 10, bits having the same place in numbers of vector X are stored at a common address, e.g., in compute memory 20, so that they are accessible together in parallel at a same time in response to each memory address provided by the controller. Numbers in vector Y are stored similarly at different compute memory 20 addresses. The controller can provide independent addresses at a same time to access bits having a same place in numbers of vector X and numbers of vector Y. Only one set of bit cells 22 is shown in Fig. 10. In some embodiments, numerically successive addresses can access bits having successive places in each number of vectors X, Y, or both X and Y. It is not necessary that the bits having successive place values are stored in compute memory 20 at successive addresses and are successively accessed since SAC 38 can shift each count by an amount corresponding to the place of the multiplied bits regardless of the bit ordering in compute memory 20. However, as will be appreciated by logic designers, such numerically successive memory accesses by the controller, numerically successive bit storage in compute memory 20 addresses, and numerically successive bit-place shifts by SAC 38 can simplify designs of the present disclosure. Knowledgeable designers will also appreciate that numbers in vectors X and Y can be stored in compute memory 20 before calculating the dot product of vectors X and Y and the computed result can be stored in a memory or register, e.g., an accumulation register, or otherwise provided to processor 70 using logical circuit designs. Furthermore, MMC 12 can organize vectors X and Y in any desired arrangement and access numbers in the vectors in any order to sequentially present bits for multiplication, using any necessary intermediate storage. MMC 12 can provide multiple compute memory 20 accesses (e.g., simultaneous different accesses) and corresponding compute engine 30 operations in response to a single read or write operation by processor 70 to any desired location (address) in compute memory 20, including any compute memory 20 location in which vectors X and Y are stored, any compute memory 20 location in which the results of compute engine 30 operations are stored, or any other desired compute memory 20 location, for example as memory mapped by MMC 12.

[0097] Fig. 10 illustrates embodiments of compute memory 20 (comprising bit cells 22) together with address lines (e.g., word lines 26) and data lines (bit lines 24) that, when accessed provide stored data sensed by sense amplifiers (amps) and provide the sensed stored data to AND gates 14 (bit multipliers 14) for processing in compute engine 30. Fig. 10 shows an array of bit cells 22 for one vector X, the other vector Y can be provided by another portion of compute memory 20, a register or memory in compute engine 30, or received from processor 70 through MMC 12 as a write operation that performs or triggers (instigates) a dot- product computation. In some embodiments, a read operation on an output buffer can trigger the dot-product operation by compute engine 30.

[0098] In some embodiments, compute memory 20 can comprise multiple bit cells 22 or groups of bit cells 22 (as shown in Fig. 10) that store multiple multi-bit digital values accessed and processed by a single compute engine 30. Bit cells 22 of compute memory 20 can be connected to a common bit line 24 and to a different word line 26 to enable writing bits into each bit cell 22. In some embodiments, the outputs of each bit cell 22 storing a number (e.g., in a column) can be connected together so that only one bit q of the number is connected to compute engine 30 at a time with a different address on bit line 26. Thus, a single bit q in a bit cell 22 connected to a bit line 24 can be selected by a corresponding word line 26 and operated upon by compute engine 30 at a time. In Fig. 10, each column of bit cells can store the different bits b of a number Xnand the different columns each store a different number Xn.

[0099] As illustrated in the 4-bit example of Figs. 11A and 11 B, each row 21 R of products shown is a multiplication of one bit of value B times the bits of value A. (A and B can be numbers in vectors X and Y, respectively, and can have different numbers of bits although four bits for each is shown in this illustrative example.) The rows are spatially shifted with respect to each other in Figs. 11 A and 11 B to represent the relative magnitude (place) of the products in each row 21 R as is conventional for multiplication manually written on paper, as indicated in Fig. 11 A. The bit products (multiplied values) in each bit column 21 C of products (having the same magnitude or place) can be summed, as shown in Fig. 11 B. Each column sum has a relative magnitude of two (or one half) with respect to a neighboring bit column 21 C. Because each bit column 21 C of products has a different place value (relative magnitude) the values in each column 21 of products must be scaled to multiply them by their place value, e.g., shifted by one to 6 places to multiply them by 2, 4, 8, 16, 32, or 64, before they are added. Scaling and adding the column sums provide a product for the two multi-bit digital binary values A and B.

[0100] In embodiments of the present disclosure, Figs. 12A and 12B illustrate the process of multiplying a first value A having C bits and a second value B having D bits using a bit-serial accumulation of products from bit multipliers 14 for every value in a vector X and Y, where the same bit of each value A in vector X is stored at a common address location and the same bit of each value B in vector Y is stored at a common address location different from that of value A. Figs. 12A and 12B illustrate alternative processes using pseudo-software and a flow diagram, respectively, representing steps in the vector multiplication; those practiced in logical design and software will readily understand that the steps represented in Figs. 12A and 12B can be implemented in a state machine or with software applied to a cpu in MMC 12 and such embodiments are included in the present disclosure. Fig. 12A illustrates a pseudo-code implementation of a dot-product calculation to accumulate each of the sequential rows 21 R of Fig. 11 A. As shown in Fig. 12A, two index variables AJdx and BJdx indicate corresponding bits of each of values A and B in vectors X and Y and are limited by the number of bits in each value A_bit_prec and B_bit_prec, respectively. Each combination of bits in each value A and B are multiplied (with an AND operation), counted, shifted by the sum of the bit places, and accumulated within the two nested loops to provide a dot product value of A and B in Acc.

[0101] Fig. 12B illustrates a state diagram version of the same dot-product calculation to accumulate sequential columns 21C of Fig. 11B. As illustrated in Fig. 12B, three counter variables, i, j, k and the DotProduct are initialized to zero in step 200. In the bit-serial multiplication, i sets the range of bits a of value A (e.g., bit Aa), j sets the range of bits b of value B (e.g., bit Bb), and k is the relative place or position (the shift or multiple of two) in a column 21C of Fig. 11B. The maximum number of bits in A is C and the maximum number of bits in B is D (C and D can be the same, as in Figs. 11 A and 11 B, or different). A serial accumulator value is likewise set to zero in step 205. Bit b is set to j in step 210 and bit a is set to i in step 215, the maximum range of bits accumulated in a given column 21 C. Aais multiplied with Bb for every value A in vector X and value B in vector Y in step 220, using an array of bit multipliers 14, each returning a single bit having a same place for each number A and B in vectors X and Y, respectively. The bits are counted with population counter 34 in step 225 and accumulated in serial accumulator 38 in step 230 to add the total number of bits that equal one in a common bit place, e.g., in a common column 21 C as shown in Fig. 11 B. (The multiple rectangles in step 220 illustrate performing the same operation for the same bit place of different numbers in vectors X and Y.) If b does not equal i (tested in step 235), not all of the bits in the bit-place column 21 C are yet multiplied and accumulated, b is incremented in step 240, a is decremented in step 245, and the process repeats with steps 220, 225, and 230 until all of the bits in the bit-place column 21 C are multiplied and accumulated. The bits (subscripts) of B increment from zero to the maximum i and the bits (subscripts) of A decrement from the maximum i to zero, as shown in bit columns 21 C of Fig. 11B. Once all of the bits in a place (column 21 C) are counted and summed (b=i), the sum for column 21 C is shifted as indicated by counter k (initially zero, equal to i+j) in step 250 and added to the DotProduct value in step 255 and the next column 21 C of the number multiplication can proceed. If the maximum number of bits in a column 21 C is not yet reached (e.g., the bit multiplications still begin with Bo) as tested in step 260, i and k are incremented in steps 265 and 270 (where k=i+j), and the serial accumulation process for the next column 21 C of bits proceeds by first clearing the serial accumulator in step 205. If the maximum number of bits in a column 21 C is reached (e.g., C bits in a column 21 C have been multiplied - the center column 21 C in the multiplication of Fig. 11 B) but not all of columns 21 C have been computed (tested in step 275), i is set to the maximum bit (bit C-1 of value A) in step 280 so that a can decrement from its maximum bit place, j is incremented in step 285 to start b at the proper bit place, and k is incremented in step 270 to indicate the next bit position (e.g., next column 21 C in the product as shown in Fig. 11 B). If all of columns 21 C have been computed (step 275), the process is done, and the value added in step 255 is the dot product of vectors X and Y. All of the operations in Fig. 12B can be performed by compute engine 30 with the bits of A and B accessed by MMC 12 from compute memories 20A and 20B.

[0102] The calculation of a dot product can be done by sequentially computing subsets of numbers A and B in vectors X and Y. Since addition is a commutative operation, the multiplied bits can be counted and added (appropriately shifted by their bit place) in any order. Thus, if the number of bit multipliers 14 is less than the number of values in a vector (the length of the vector), according to embodiments of the present disclosure subsets of the numbers in the vectors can be bit-wise multiplied and their accumulated sums combined to provide a complete bit product. In-memory computer 10 can therefore be applied to vectors, matrices, or tensors having a wide variety of vector lengths and is not limited by the physical implementation of a specific number of bit multipliers 14 in compute engine 30. Similarly, inmemory computer 10 is not limited by the number of bits in multiplied numbers since, according to embodiments of the present disclosure, the bit multiplications are iterative and serially accumulated over time. Larger numbers with more bits can be multiplied using more iterations and time than smaller numbers with fewer bits. Thus, there are no calculation limits on vector lengths and numbers of bits in compute engine 30 of embodiments of the present disclosure.

[0103] According to embodiments of the present disclosure, and as shown in Figs. 13A and 13B, each compute memory 20A, 20B (collectively compute memory 20) can be connected to a common compute engine 30. In particular, compute memory 20 can store multiple bits of one or more multi-bit digital values that can be accessed and processed by compute engine 30. Compute memories 20A and 20B can be independently accessed through separate word lines 26A, 26B. In some embodiments and as shown in Figs. 13A and 13B, bit cells 22 are connected directly to compute engine 30 and can be accessed in parallel at a single time. Accessed bits from each of compute memories 20A and 20B can be multiplied together with bit multipliers 14 and accumulated with SAC 38 using digital logic as shown in Fig. 13A. In some embodiments, bit products and their accumulation can be analog, or as shown in Fig. 13B, SAC 38 can be digital and can accumulate and shift digital data values converted by analog-to-digital converter 36.

[0104] Fig. 14A is a schematic that illustrates digital storage and accumulation. Fig. 14B is a schematic that illustrates analog storage and accumulation. In Fig. 14A, each product storage circuit 16 stores a digital (one bit) value in a register, flipflop, or latch corresponding to a bit of different number pairs A, B of vectors X and Y. Product storage circuits 16 are accessible in parallel to population counter 34 and the counted bits from population counter 34 are digitally shifted and accumulated in SAC 38. Fig. 14B illustrates embodiments in which the processing is done with analog values. In Fig. 14B, each capacitor 16 receives a bit product when switch S (switch 18, e.g., a transistor 50) is in multiply mode (e.g., turned on). Each capacitor 16 is connected together when switch S is in accumulate mode (e.g., turned off). The capacitor values are averaged and stored in an accumulation capacitor 17, and the average values can be converted to a digital value with analog-to-digital converter (ADC) 36 and summed to provide a product of the two multi-bit values with SAC 38. Each time the product values in capacitors 16 are averaged with the charge on accumulation capacitor 17, the product charge is effectively divided by two, providing scaling of each subsequent bit product. (Figs. 15A and 15B only show storage for one bit of each value A, B in vectors X, Y, but in practice multiple bits b are sequentially provided, as illustrate in Fig. 10.

[0105] According to embodiments of the present disclosure, compute engine 30 can comprise multiple bit multipliers 14 that multiply a single bit of each of two numbers at a time. Different numbers can be multiplied by providing the various bits of the two numbers to different bit multipliers 14. In some embodiments, the bits of each of the two numbers are serially multiplied and accumulated for example with a shifter as illustrated in Fig. 9. Fig. 15A shows a simple bit multiplier 14 (AND gate 14). As shown in Fig. 15A, a compute element 14 is operable to receive data from bit cells 22A and 22B. Compute element 14 can comprise a one-bit multiplier 14 (e.g., a switch 50 or transistor 50) that receives input from bit cells 22. One input (e.g., bit cell 22B) is connected to the gate, another input (e.g., bit cell 22A) is connected to the source. When data in both bit cells 22A and 22B (collectively bit cells 22) have a high voltage (e.g., a logical one), the high voltage is transferred to transistor 50 drain and then into a product storage circuit 16 (e.g., a digital flipflop or latch or an analog storage circuit such as a capacitor 16) as the product of bit data stored in bit cells 22A and 22B.

[0106] Fig. 15B illustrates a more complex, electrically efficient, and spatially efficient bit- multiply circuit 14. In some such embodiments, an MOS (metal-oxide semiconductor) transistor pair 15 comprises two transistors 50 driven by complementary outputs from a bit-cell 22. If the output of bit cell 22 is high (e.g., stores a one or a positive charge) a VREFP signal (positive voltage reference) is transferred through MOS transistor pairs 15. Each of two MOS transistor pairs 15 connected in series is connected to bit cell 22A and bit cell 22B, respectively. If the outputs of both are positive, a positive value (e.g., a one or a positive charge or voltage) is transferred to product storage circuit 16 as the product of bit data stored in bit cells 22A and 22B. If the output of either of bit cells 22A or 22B is low, a low or zero charge or voltage value is stored in product storage circuit 16. Bit multiplier 14 shown in Fig. 15B comprises two serially connected pairs of simple MOS (metal-oxide semiconductor) transistors having separate differential inputs and a common output. One of the pair 15 of simple MOS transistors is controlled by a positive control signal and the other by an inverted (negative) version of the same control signal, for example the positive and negative outputs of any single bit cell 22 (e.g., a D-flipflop, latch, or pairs of inverters). Such a series of MOS transistor pairs 15 can require fewer, simpler transistors that operate at a much lower voltage (e.g., one percent or less than one percent, such as 0.624 percent, or 10 mV instead of 1.65 volts) and therefore require much less power.

[0107] A bit-multiplier using a conventional AND gate can require, for example, six relatively large transistors operating at a relatively high voltage to implement a bit-multiply circuit. In contrast and according to embodiments of the present disclosure, bit-multipliers 14 of the present disclosure can comprise serially connected differential MOS transistor pairs 15 that can operate at relatively low voltages (e.g., no greater than 1 V and as low as 10 mV) and low power with, for example, only four relatively small transistors. In embodiments, MOS transistor pairs 15 operate in an analog relatively low-power regime having an analog voltage that is less than a digital relatively high-power regime having a digital voltage. In some embodiments, the analog voltage is no greater than one-half, one quarter, one fifth, one tenth, one twentieth, one fiftieth, or one hundredth (e.g., 50%, 25%, 20%, 10%, 5%, 2%, or 1%) of the digital voltage.

[0108] Fig. 16 illustrates the first stage of population counter 34. Given three bit products Po, Pi , P2 having the same place in three different numbers, a two-bit sum has a higher bit Di that is one if at least two of the bit products are one and a lower bit Do that is one if all three bit products are one or if only one bit product is one. The two-bit sum (having a value of zero, one, two or three) can then be added to another two-bit sum with a three-bit adder, and so on to eventually sum all of the bit products with multiple stages of adders for numbers of increasing bits. In some embodiments, the first stage can provide a three-bit sum (values zero to four) using four bit-products. Those skilled in the art can appreciate that a variety of methods and logical circuits can be used to implement population counter 34.

[0109] In some embodiments, in-memory computer 10 provides an array of dot-product functions that can be a matrix vector product (e.g., where a matrix dimension is one). Each row (or column) of bit cells 22 in an in-memory computer 10 can perform a dot product. Thus, in some embodiments, compute engine 30 comprises a multiplier responsive to bit cells 22 storing two elements A and B, each comprising an arbitrary number of bits. Compute engine 30 is connected to storage element 22 with data lines (bit lines 24) and writes and reads and to and from storage element 22 using control signals. In operation, data is written into storage elements 22 using bit and word lines 24, 26. Compute engine 30 can read data from bit cells 22 and operate on (process) the read data. Storage elements 22 of in-memory computer 10 can be memory mapped to processor (controller) 70. Controller 70 can write data into storage elements 22 in such a way that compute engines 30 each compute the appropriate portion of a multi-bit multiplication. The multiple bit-product values are serially accumulated in accumulator storage circuit 17, storing the product in capacitor 16 when an accumulation switch 18 is turned on and averaging the value stored in accumulator storage circuit 17 with the bit product in capacitor 16 each time an accumulation switch 18 (e.g., MOS transistor pair 15 in a switch 18) is turned off.

[0110] Thus, embodiments of the present disclosure can include a dot-product calculator wherein compute memory 20 is operable to store multiple pairs of vectors X and vectors Y (e.g., can store two matrices each comprising multiple vectors). N AND gates 14 (bit multipliers 14) for each of the vector pairs (e.g., from different matrices) can be connected to compute memory 20, such that each AND gate 14 inputs a bit of a number of vector X in a pair and a corresponding bit (e.g., having the same place value) of a number of vector Y in the same pair at a same time in response to a memory address provided by the controller and outputs the product of the input bits. A population counter 34 for each pair of vectors can be operable to count the number of non-zero bits output from the N AND gates 14 in the pair. A serial accumulator 38 for each population counter 34 can be operable to sequentially accumulate the count from the population counter 34, each count and bit having a place value corresponding to the same address. The multiple pairs of vectors X can form a first two- dimensional matrix of numbers and the multiple pairs of vectors Y can form a second two- dimensional matrix of numbers and the dot-product calculator can be operable to calculate an array of scalar values corresponding to a dot product between the first two-dimensional matrix of numbers and the second two-dimensional matrix of numbers.

[0111] According to embodiments of the present disclosure, a dot-product calculator can comprise a controller (e.g., MMC 12 or external processor 70) operable to sequentially provide memory addresses to a compute memory 20. Compute memory 20 can be operable to receive the memory addresses from the controller and store a first two-dimensional array (a first matrix) of numbers and a second two-dimensional array (a second matrix) of numbers, wherein the same bit of each of the numbers in the first matrix and each of the numbers in the second matrix is accessible in parallel at a same time in response to each memory address. An AND gate 14 for each of the numbers in the first matrix or in the second matrix can be connected to compute memory 20 operable to input a bit from a number in the first matrix and input a bit from a number in the second matrix at a same time in response to the memory address provided by the controller and outputs the product of the input bits. A population counter 34 for each one-dimensional array of numbers in the first matrix or in the second matrix can be operable to count the number of non-zero bits output from AND gates 14 receiving the one-dimensional array of numbers and a serial accumulator 38 for each population counter 34 can be operable to sequentially accumulate the count of the population counter 34, each count and bit having a place value corresponding to the same address.

[0112] More generally, a dot-product calculator can comprise a controller (e.g., MMC 12 or external processor 70) operable to sequentially provide memory addresses, a compute memory 20 can be operable to receive the memory addresses from the controller and store a first n-dimensional array (a first matrix) of numbers and a second n-dimensional array (a second matrix) of numbers. The same bit of each of the numbers in the first matrix and each of the numbers in the second matrix can be accessible in parallel at a same time in response to each memory address provided by the controller to compute memory 20. A compute engine 30 connected to compute memory 20 and operable to compute an array of scalar numbers corresponding to a dot product between the first matrix and the second matrix. In embodiments, n can be two, three, four, five, six, seven, eight, nine, ten, eleven, twelve, or other positive integers. Thus, embodiments of the present disclosure can provide matrix and tensor operations. The dot-product calculators can be comprised in multi-processor computer system 80 or in-memory computer 10 and the controller can be MMC 12 or external processor 70.

[0113] Embodiments of the present disclosure have been designed and modeled. In-memory computer 10 can add linear algebra capabilities to standard memory structures, delivering 50x to 130x performance improvement in latency and energy efficiency in comparison to conventional solutions, particularly in applications that rely heavily on computationally intensive linear algebra operations such as neural networks or digital signal processing. Inmemory computer 10 can be integrated with a variety of processing architectures, including but not limited to x86, ARM, RISC-V, GPUs, NPUs or generic ASICs. By integrating inmemory computer 10 with processor 70, processor 70 can offload computationally expensive workloads, such as dot-product computational primitives, from processor 70. This allows processor 70 to avoid the energy and time cost of executing these primitives while retaining all necessary control to efficiently and rapidly execute higher-order linear algebra routines, such as matrix vector products, convolutional operations and more. In-memory computer 10 ensures seamless backward compatibility with existing linear algebra-heavy libraries and algorithms an provide greatly increased performance.

[0114] In-memory computer 10 was modeled and compared to three ARM Cortex-Mo processor configurations: an ARM Cortex-Mo processor with a serial multiplier (A), an ARM Cortex-Mo processor with a single-cycle multiplier (B), and an ARM Cortex-Mo processor with in-memory computer 10 (C). All configurations were set to operate at a frequency of 250 MHz. The operation was a 1024 element dot product with 8-bit values computed with absolute integer precision (no floating point operations) and rounded to an 8-bit result. Latency is the number of clock cycles required for the Mo core to complete the operation in microseconds. Power is the average power consumed by the system during the operation. Energy is the total energy, including leakage, consumed by the system to complete the operation. For configuration A, latency was 40,710 cycles in 162,840 microseconds, power used was 0.82 mW, and energy used was 134,570 pJ. For configuration B, latency was 8,966 cycles in 35,864 microseconds, power used was 1.45 mW, and energy used was 51 ,987 pJ. For configuration C, latency was 337 cycles in 1 ,348 microseconds, power used was 0.72 mW, and energy used was 1 ,022 pJ.

[0115] Embodiments of the present disclosure are not limited to the specific examples illustrated in the figures and described herein. Skilled designers will readily appreciate that various implementations of analog and digital circuits can be employed to implement the operations described and such implementations are included in embodiments of the present disclosure.

[0116] Embodiments of the present disclosure can be used in neural networks, patternmatching computers, or machine-learning computers and provide efficient and timely processing with reduced power and hardware requirements. Such embodiments can comprise a computing accelerator, e.g., a neural network accelerator, a pattern-matching accelerator, a machine learning accelerator, or an artificial intelligence computation accelerator designed for static or dynamic processing workloads.

[0117] Having described certain implementations of embodiments, it will now become apparent to one of skill in the art that other implementations incorporating the concepts of the disclosure may be used. Therefore, the disclosure should not be limited to certain implementations, but rather should be limited only by the spirit and scope of the following claims.

[0118] Throughout the description, where apparatus and systems are described as having, including, or comprising specific elements, or where processes and methods are described as having, including, or comprising specific steps, it is contemplated that, additionally, there are apparatus and systems of the disclosed technology that consist essentially of, or consist of, the recited elements, and that there are processes and methods according to the disclosed technology that consist essentially of, or consist of, the recited processing steps.

[0119] It should be understood that the order of steps or order for performing certain action is immaterial so long as the disclosed technology remains operable. Moreover, two or more steps or actions in some circumstances can be conducted simultaneously. The disclosure has been described in detail with particular reference to certain embodiments thereof, but it will be understood that variations and modifications can be effected within the spirit and scope of the following claims. PARTS LIST

[0120] F control line I control wire I control signal

[0121] 10 in-memory computer architecture I in-memory computer

[0122] 12 memory-mode controller

[0123] 13 bus / signals

[0124] 13A function line I function wire I function signal

[0125] 13B address line I address wire I address signal

[0126] 13C data line I data wire I data signal

[0127] 14 bit multiplier I bit-multiply circuit I AND gate I compute element

[0128] 15 MOS transistor pair

[0129] 16 product storage circuit I flipflop I latch I capacitor

[0130] 17 accumulation capacitor / accumulation storage circuit

[0131] 18 switch / / S

[0132] 19 analysis circuit

[0133] 20, 20A, 20B compute memory

[0134] 21 C bit column

[0135] 21 R bit row

[0136] 22, 22A, 22B memory element I bit cell I storage element

[0137] 24 bit line / data line

[0138] 26, 26A, 26B word line I address line

[0139] 30 compute engine (CE)

[0140] 34 population counter I bit counter

[0141] 36 analog-to-digital converter (ADC)

[0142] 38 digital shift-and-accumulate (SAC) circuit I serial accumulator

[0143] 50 switch I transistor

[0144] 70 controller I external processor I processor

[0145] 80 multi-processor computer system

[0146] 100 processor write step

[0147] 105 processor read step

[0148] 110 MMC analyze address step

[0149] 115 MMC analyze address and operation step

[0150] 120 memory write step

[0151] 125 memory read step

[0152] 130 compute engine process step

[0153] 140 compute engine write to memory step

[0154] 150 return processed data step

[0155] 200 set counter i =0, j=0, k=0 step 205 set accumulated step

[0156] 210 set b=j step

[0157] 215 set a=i step

[0158] 220 multiplyax Bb step 225 count population step

[0159] 230 serial accumulate step

[0160] 235 b=i? step

[0161] 240 increment b step

[0162] 245 decrement a step 250 shift by j step

[0163] 255 add step

[0164] 260 i=N-1? step

[0165] 265 increment i step

[0166] 270 increment k step 275 j=M-1? step

[0167] 280 set i=N-1 step

[0168] 285 increment j step

[0169] 290 done step

Claims

What is claimed:

1. A multi-processor computer system (80), comprising: an in-memory computer (10) comprising a compute engine (30) and storage elements (22) for storing data, the compute engine (30) operable to process data stored in the storage elements (22) of the in-memory computer (10); and a processor (70) external to the in-memory computer (10) connected to the in-memory computer (10), the processor (70) having a memory address space for writing data into at least some of the storage elements (22) and for reading data from at least some of the storage elements (22), and characterized in that the compute engine (10) is operable to process data stored in a storage element (22) (i) in response to a read operation on a storage element (22) by the processor (70), (ii) in response to a write operation on a storage element (22) by the processor (70), or (iii) both (i) and (ii).

2. The multi-processor computer system of claim 1 , wherein the in-memory computer (10) is operable to return the processed data to the processor (70) in response to the read operation or in response to the write operation.

3. The multi-processor computer system of any of claims 1-2, wherein the read or write operation provides an RW address in the memory address space, the storage element (22) stores the data at the RW address, and the compute engine (30) is operable to process the stored data.

4. The multi-processor computer system of any of claims 1-3, wherein the compute engine (30) is operable to store the processed data in the storage element (22) at the RW address.

5. The multi-processor computer system of any of claims 1-3, wherein the compute engine (30) is operable to store the processed data in the storage element (22) at a process address different from the RW address.

6. The multi-processor computer system of any of claims 1-2, wherein the read or write operation provides an RW address in the memory address space, the storage element (22) stores data at a different address in the memory address space from the RW address, and thecompute engine (30) is operable to process data stored in the storage element (22) at the different address.

7. The multi-processor computer system of any of claims 1-6, wherein the compute engine (30) is operable to store the processed data at the RW address.

8. The multi-processor computer system of any of claims 1-6, wherein the compute engine (30) is operable to store the processed data at a process address in the memory address space different from the RW address.

9. The multi-processor computer system of any of claims 6-8, wherein the compute engine (30) is operable to process data in different ways depending on one of multiple different RW addresses.

10. The multi-processor computer system of any of claims 1-9, wherein the in-memory computer (10) comprises a memory-mode controller (12), the processor (70) is connected to the memory-mode controller (12) with any combination of a function line (13A), one or more address lines, and one or more data lines, and the memory-mode controller (12) is connected to the storage elements (22) and the compute engine (30).11 . The multi-processor computer system of claim 10, wherein the memory-mode controller (12) is responsive to the RW address to control the compute engine (30) to process data stored in the storage element (22).

12. The multi-processor computer system of any of claims 1-10, wherein the memorymode controller (12) comprises an address analysis circuit for analyzing the RW address and controlling the compute engine (30) in response to the address analysis.

13. The multi-processor computer system of any of claims 10-12, wherein the memorymode controller (12) is responsive to the function line (13A) to control the compute engine (30) to process data stored in the storage element (22).

14. The multi-processor computer system of any of claims 10-13, wherein the memorymode controller (12) comprises a function analysis circuit for analyzing the function line (13A) and controlling the compute engine (30) in response to the function-line analysis.

15. The multi-processor computer system of claim 10, wherein the memory-mode controller (12) is operable to (i) read processed data from a storage element (22) or (ii) write processed data to a storage element (22), or (iii) both (i) and (ii).

16. The multi-processor computer system of any of claims 1-15, wherein at least one of the storage elements (22) is mapped to multiple addresses in a memory space of the processor (70).

17. The multi-processor computer system of claim 16, wherein the memory-mode controller (12) is operable to map a write address to a write storage element (22) in response to a write operation from the processor (70) and map a read address to a read storage element (22) different from the write storage element (22) in response to a read operation to the write address from the processor (70).

18. The multi-processor computer system of any of claims 16 and 17, wherein the memory-mode controller (12) is operable to map a write address to a write storage element (22) in response to a write operation from the processor (70) and map a read address to a read storage element (22) different from the write storage element (22) in response to a read operation and the function line (13A).

19. The multi-processor computer system of claim 9, wherein the RW address is mapped to at least three different storage elements (22) and the compute engine (30) processes data differently depending on the function line (13A).

20. The multi-processor computer system of claim 1 , comprising multiple compute engines, wherein each compute engine is operable to multiply a bit of each number in two one-dimensional arrays of numbers at a same time.21 . The multi-processor computer system of claim 20, comprising a number of compute engines sufficient to multiply a bit of each number in two two-dimensional arrays of numbers at a same time.

22. The multi-processor computer system of claim 1 , wherein (i) at least some processing instructions are stored in storage elements within the in-memory-computer address range, (ii) at least some data is stored in storage elements within the in-memory-computer address range, or (iii) both (i) and (ii).

23. A multi-processor computer system (80), comprising: an in-memory computer (10) comprising a compute engine (30) and storage elements (22) for storing data, the compute engine (30) operable to read, write, and process data stored in the storage elements (22) of the in-memory computer (10); and a processor (70) external to the in-memory computer (10) connected to the in-memory computer (10), the processor (70) having a memory address space for writing data into at least some of the storage elements (22) and for reading data from at least some of the storage elements (22), and characterized in that the compute engine (10) is operable to process data stored in a storage element (22) in response to a read or write operation on the storage element (22) by the processor (70) and at least one of the storage elements (22) is mapped to multiple addresses in a memory space of the processor (70).

24. A multi-processor computer system (80), comprising: an in-memory computer (10) comprising a memory-mode controller (12), compute engine (30), and storage elements (22) for storing data, the compute engine (30) operable to read, write, and process data stored in the storage elements (22) of the in-memory computer (10); and a processor (70) external to the in-memory computer (10) connected to the in-memory computer (10), the processor (70) having a memory address space for writing data into at least some of the storage elements (22) and for reading data from at least some of the storage elements (22), and characterized in that the processor (70) is connected to the memory-mode controller (12) with any combination of a function line (13A), one or more address lines (13B), and one or more data lines (13C), and the memory-mode controller (12) is connected to the storage elements (22) and the compute engine (30).

25. A dot-product calculator, comprising: a controller (12, 70) operable to sequentially provide memory addresses; and a memory (22) operable to receive the memory addresses from the controller and store at least one vector X and a vector or matrix Y, each vector having numbers, each number having bits, a bit of each of the numbers for vector X and a bit of each of the numbers for vector Y is accessible in parallel at a same time, characterized by bit multipliers (14) connected to the memory, such that each bit multiplier inputs a bit of a number of vector X and a bit of a number of vector Y at a same time in response to the memory address provided by the controller and outputs the product of the input bits;a population counter (34) operable to count the number of non-zero bits output from the bit multipliers; and a serial accumulator (38) for sequentially accumulating the count, each count and bit having a place value corresponding to the same address.

26. The dot-product calculator of claim 25 wherein the same bit of each of the numbers for both of the vectors X and Y is stored at a same address of the memory addresses provided by the controller.

27. The dot-product calculator of any of claims 25 and 26 wherein the memory addresses in the sequence are numerically adjacent.

28. The dot-product calculator of any of claims 25-27 wherein (i) the population counter is analog, (ii) the serial accumulator is analog, or (iii) both (i) and (ii).

29. The dot-product calculator of any of claims 25-28 wherein (i) the population counter is digital, (ii) the serial accumulator is digital, or (iii) both (i) and (ii).

30. The dot-product calculator of any of claims 25-29, wherein the memory (22) is operable to store multiple pairs of vectors X and Y, and comprising bit multipliers (14) for each of the pairs connected to the memory (22), each bit multiplier (14) operable to input a bit of a number of vector X in a pair and a bit of a number of vector Y in the same pair at a same time and output the product of the input bits; a population counter (34) for each pair operable to count the number of non-zero bits output from the bit multipliers in the pair; and a serial accumulator (38) for each population counter operable to sequentially accumulate all or at least a portion of the count from the population counter, each count and bit having a place value corresponding to the same address.31 . The dot-product calculator of claim 30, wherein the multiple pairs of vectors X form a first two-dimensional matrix of numbers and the multiple pairs of vectors Y form a second two- dimensional matrix of numbers and the dot-product calculator is operable to calculate an array of scalar values corresponding to a dot product between the first two-dimensional matrix and the second two-dimensional matrix.

32. A dot-product calculator, comprising: a controller (12, 70) operable to sequentially provide memory addresses,characterized by a memory (22) operable to receive the memory addresses from the controller and store a first two-dimensional array (a first matrix) of numbers and a second two-dimensional array (a second matrix) of numbers, wherein a bit of each of the numbers in the first matrix and a bit of each of the numbers in the second matrix is accessible in parallel at a same time; a bit multiplier (14) for each of the numbers in the first matrix or in the second matrix connected to the memory operable to input a bit from a number in the first matrix and input a bit from a number in the second matrix in parallel at a same time and output the product of the input bits; a population counter (34) for each one-dimensional array of numbers in the first matrix or in the second matrix operable to count the number of non-zero bits output from the bit multipliers receiving the one-dimensional array of numbers; and a serial accumulator (38) for each population counter operable to sequentially accumulate the count of the population counter, each count and bit having a place value corresponding to the same address.

33. A dot-product calculator, comprising: a controller (12, 70) operable to sequentially provide memory addresses; and a memory (22) operable to receive the memory addresses from the controller and store a first n-dimensional array (a first matrix) of numbers and a second m-dimensional array (a second matrix) of numbers, wherein the same bit of each of the numbers in the first matrix and each of the numbers in the second matrix is accessible in parallel at a same time, characterized by a compute engine connected to the memory operable to compute an array of scalar numbers corresponding to a dot product between the first matrix and the second matrix.

34. The multi-processor computer system of claim 1 , wherein the storage elements comprise bit cells and the compute engine comprises a capacitor associated with pairs of bit cells or comprises a capacitor associated with pairs of groups of multiple bit cells.

35. The multi-processor computer system of claim 1 , wherein the storage elements comprise bit cells and the compute engine comprises binary logic associated with pairs of bit cells or comprises binary logic associated with pairs of groups of multiple bit cells.

36. A multi-processor computer system (80), comprising: an in-memory computer (10) comprising a memory-mode controller (12), compute engine (30), and storage elements (22) for storing data, the compute engine (30) operable toread, write, and process data stored in the storage elements (22) of the in-memory computer (10); and a processor (70) external to the in-memory computer (10) connected to the in-memory computer (10), the processor (70) having a memory address space for writing data into at least some of the storage elements (22) and for reading data from at least some of the storage elements (22), and characterized in that the processor (70) is connected to the memory-mode controller (12), the memorymode controller (12) is connected to the storage elements (22) and the compute engine (30), the memory-mode controller (12) receives memory addresses from the processor (70) and maps the memory addresses from a first memory address to a second memory address different from the first memory address.

37. A multi-processor computer system (80), comprising: an in-memory computer (10) comprising a compute engine (30) and storage elements (22) for storing multi-bit numbers, the compute engine (30) operable to process the multi-bit numbers stored in the storage elements (22) of the in-memory computer (10); and a processor (70) external to the in-memory computer (10) connected to the in-memory computer (10), the processor (70) having a memory address space for writing the multi-bit numbers into at least some of the storage elements (22) and for reading the multi-bit numbers from at least some of the storage elements (22), and characterized in that at least some of the different bits of the multi-bit numbers are stored in the storage elements (22) at different storage element addresses or in different memory banks responsive to a same storage element address.

38. The multi-processor computer system (80) of claim 37, wherein each bit of the multi-bit data is stored in a storage element at a different storage element address or in a different memory bank from any other bit of the multi-bit data.

39. A multi-processor computer system (80), comprising: an in-memory computer (10) comprising a compute engine (30) and storage elements (22) for storing data, the compute engine (30) operable to process data stored in the storage elements (22) of the in-memory computer (10); and a processor (70) external to the in-memory computer (10) connected to the in-memory computer (10), the processor (70) having a memory address space for writing data into at least some of the storage elements (22) and for reading data from at least some of the storage elements (22), andcharacterized in that the compute engine (10) is operable to (i) first process data stored in a storage element (22) in response to a read or write on a storage element (22) at a first address by the processor (70) and (ii) second process data stored in a storage element (22) in response to a read or write on the storage element (22) at a second address by the processor (70), wherein the first address is different from the second address and the first process is different from the second process.

40. A method of multiplying a number A times a number B, A and B having bits each in a bit place, comprising:(i) sequentially multiplying a bit of A times a bit of B to provide a bit product for each combination of bits in A and B having a common bit place and adding the bit products;(ii) shifting the added bit products by the product place;(iii) accumulating the shifted added bit products; and(iv) repeating steps (i)-(iii) for each bit place of A and B to provide a product of A and B.41 . A method of calculating a dot product between a vector X having numbers A and a vector Y having numbers B, A and B having bits in a bit place, comprising:(i) multiplying a bit of A times a bit of B to provide a bit product for a pair of bits in A and B having a common bit place for each A and B in vectors X and Y;(ii) counting the bit products;(iii) accumulating the count;(iv) repeating steps (i) - (iii) for each combination of bits having a common bit place;(v) shifting the accumulated count by the bit place;(vi) adding the shifted accumulated count; and repeating steps (i)-(vi) for each bit place of a product of A and B to provide a dot product of A and B that is the added shifted accumulated count.

42. A multi-processor computer system (80), comprising: an in-memory computer (10) comprising a compute engine (30) and storage elements (22) for storing multi-bit numbers, and characterized in that at least some of the different bits of the multi-bit numbers are stored in the storage elements (22) at different storage element addresses or in different memory banks responsive to a same storage element address.

43. The multi-processor computer system (80) of claim 42, wherein the in-memory computer (10) is operable to provide one or more storage element addresses to the storageelements (22) and retrieve bits of different multi-bit numbers having a common bit place from the storage elements (22).

44. The multi-processor computer system (80) of claim 43, wherein the compute engine (30) comprises bit multipliers (14) to multiply bits of different multi-bit numbers having a common bit place, providing a product for each pair of multiplied bits.

45. The multi-processor computer system (80) of claim 44, wherein the compute engine(30) comprises a population counter (34) for counting the number of non-zero products of the pairs of multiplied bits.

46. The multi-processor computer system (80) of claim 42, wherein all of the different bits of the multi-bit numbers are stored in the storage elements (22) at different storage element addresses or in different memory banks.

47. The multi-processor computer system (80) of claim 42, wherein at least some of the different bits of the multi-bit numbers are stored in the storage elements (22) in different simultaneously accessible storage elements.

Images