Compact in-memory computer architecture

Abstract

A compact in-memory computer architecture includes memory components arranged in rows and columns, bit lines each connecting a row of memory components, and word lines each connecting a column of memory components. Each memory component has a bit unit and a compute engine connected to the bit unit. The bit unit is operable to store a bit, and the compute engine is operable to process the bit. Each bit line connects a corresponding row of memory components and is operable to provide a bit to each memory component in the row of memory components. Each word line connects a corresponding row of memory components and is operable to enable each memory component in the column of memory components to write a bit to each memory component in the column of memory components.

Description

Technical Field

[0001] The present disclosure generally relates to distributed digital memory and computing element architectures, devices, and methods that facilitate matrix multiplication. Background Art

[0002] Matrix multiplication is an important operation in many mathematical calculations. For example, linear algebra can use matrix multiplication to solve linear equations such as differential equations. Such mathematical calculations are used, for example, in pattern matching, artificial intelligence, analytic geometry, engineering, physics, natural sciences, computer science, computer animation, and economics.

[0003] Matrix multiplication is usually performed in a digital computer executing a stored program. The program describes the operations to be performed, and hardware in the computer, such as digital multipliers and adders, performs the operations. The data (matrix) being operated on is stored in a digital memory, such as a static random access memory (SRAM) or a dynamic random access memory (DRAM) accessed through a memory and address bus. The number of bits retrieved in parallel at one time is limited by the bus bit width and corresponds to the number of bits in the memory enabled by the address provided to the memory. In some computing systems, specially designed hardware can speed up the calculation rate.

[0004] In some applications, real-time processing is necessary to provide useful output within a useful amount of time, especially for safety-critical tasks. However, access to data stored in memory is an inherent limitation in traditional digital computing systems. Moreover, applications in portable devices have only limited available power.

[0005] Typically, calculations that require large matrices and high data rates can take longer to solve and use more power than desired. Therefore, there is a need for computational logic and memory architectures that can perform matrix multiplication at higher data rates and with less power. Summary of the invention

[0006] Embodiments of the present disclosure may provide, among other things, a compact in-memory computer architecture suitable for performing matrix multiplication with improved efficiency and speed in a compact design that reduces the amount of physical hardware (e.g., semiconductor die area) required. By limiting the area, costs are reduced and performance is improved. The compact in-memory architecture may provide for massively parallel processing of large numbers of values, such as performing many matrix multiplication operations at the same time.

[0007] According to an embodiment of the present disclosure, a compact in-memory computer architecture includes memory components arranged in rows and columns, bit lines each connecting a row of memory components, and word lines each connecting a column of memory components. Each memory component has a bit cell or a plurality of bit cells and a computing engine connected to the bit cell. The bit cell is operable to store a bit and the computing engine is operable to process the bit. Each bit line connects the corresponding row of the memory component and is operable to provide a bit to each memory component in the row of the memory component. Each word line connects the corresponding column of the memory component and is operable to enable each memory component in the column of the memory component, thereby writing a bit to each memory component in the column of the memory component. The rows and columns of the memory components can form an array of memory components connected in a matrix with bit lines (e.g., in the horizontal row direction) and word lines (e.g., in the vertical column direction). (Horizontal and vertical are arbitrary orthogonal designations.) In some embodiments, the computing engine is operable to process a bit (or a plurality of bits) in the storage element in the memory component in combination with a bit (or a plurality of bits) accessed from outside the compact in-memory computer architecture.

[0008] In some embodiments, each memory component is connected to an external bit line by a memory select (MEMSEL) switch. The memory select switch can isolate the bit cell (and the computing engine) from an external device connected to the bit line. The external device is a device that is spatially and physically external to the memory component connected to the memory component. The externally accessible bit line outside the memory component is an external bit line, and the bit line inside the memory component that isolates the external device from the memory select switch is an internal bit line. The internal bit line and the external bit line are together a bit line. When closed, the memory select switch connects the bit cell to any external device (such as a controller) through the external bit line. When the memory select switch is opened, the bit cell and the internal bit line are electrically isolated from any external device (such as a controller) connected by the external bit line. In some embodiments, the memory select switch of each memory component is controlled in common, for example, electrically connected in common to a common control signal so that all memory select switches (for example, in a row, column, or all of the memory components in the array) operate together with the common control signal.

[0009] According to some embodiments of the present disclosure, each memory component includes a plurality of bit cells connected to a computing engine, and each bit cell of the plurality of bit cells is connected to a common bit line and a different word line so that each computing engine can access a plurality of bits stored in the plurality of bit cells. The plurality of bit cells in the memory component can store a single multi-bit value, such as a byte, a word, or a long word.

[0010] In some embodiments, each bit cell in a memory component is directly connected only to the compute engine of that compute engine so that the compute engine can access all bits stored in the bit cells of the common memory component in parallel. In some such embodiments, the compute engines of the memory components can access one or more bit cells of the memory component serially, such as one bit cell at a time or some groups of bit cells less than all bit cells at a time. Each compute engine in the array of memory components can access the bit cells in the memory components in parallel.

[0011] According to some embodiments, a controller controls memory components in an array.

[0012] According to some embodiments, the memory components are arranged on a substrate and each memory component can be spatially arranged on or above a different portion of the substrate and adjacent to another memory component. The computing engine of each memory component can be spatially arranged adjacent to a bit cell or multiple bit cells of the memory component. At least one of the computing engines in the memory component can be spatially arranged between a bit cell of the memory component and a bit cell of an adjacent memory component so that the bit cell (or group of bit cells) and the computing engine alternate in space in at least one direction.

[0013] According to some embodiments of the present disclosure, each computing engine in a memory component is connected to a computing engine of an adjacent computing engine. In some embodiments, the adjacent computing engine can communicate or transfer data (e.g., processed bits) from one computing engine to the adjacent computing engine. In some embodiments, the adjacent computing engines can be connected together and can share data, such as average data found in adjacent computing engines.

[0014] According to some embodiments of the present disclosure, in at least some memory components, a computing engine is connected to a bit cell with a corresponding bit line (e.g., an internal bit line) such that the bit line on which a bit is transmitted from an external source or an external controller to the bit cell is also the bit line (e.g., an internal bit line) that connects the computing engine to the bit cell.

[0015] According to some embodiments, the computation engine includes a bit multiplier for multiplying the bits stored in the bit cells to calculate the product, and a product storage circuit that is or includes a capacitor for storing the product. In some embodiments, the bit multiplier is a single-bit multiplier. In some embodiments, the bit multiplier is an iterative bit multiplier that effectively scales and accumulates bit products.

[0016] According to some embodiments of the present disclosure, a method of operating a compact in-memory computer architecture includes: providing a bit on each bit line using a controller; enabling a word line of a column of memory components using the controller to store the bit in a bit cell of each memory component in the column of memory components; and processing the stored bit using a computing engine of each memory component in the column of memory components. Each memory component may be connected to a corresponding bit line via a memory select (MEMSEL) switch, and the method of the present disclosure may include turning on the MEMSEL switch before providing the bit on each bit line using the controller, and turning off the MEMSEL switch after providing the bit on each bit line using the controller, and then processing the stored bit using a computing engine of each memory component in the column of memory components.

[0017] Some embodiments of the present disclosure include serially multiplying multiple bits of a first multi-bit value by one bit of a second multi-bit value. Some embodiments of the present disclosure include multiplying multiple bits of a first multi-bit value by one bit of a second multi-bit value in parallel. Some embodiments of the present disclosure include multiplying all bits of a first multi-bit value by one bit of a second multi-bit value in parallel or in series. Some embodiments of the present disclosure include multiplying multiple bits of a first multi-bit value by multiple bits of a second multi-bit value in parallel or in series. Some embodiments of the present disclosure include multiplying all bits of a first multi-bit value by all bits of a second multi-bit value in parallel. In some embodiments, the product of multiple bits of a first value and a single bit of a second value is scaled and accumulated. In some embodiments, the bit product of the first multi-bit value and the second multi-bit value is accumulated, for example by averaging the bit product with a capacitor connected in parallel in which the bit product is stored. In some embodiments, the accumulated bit product is scaled and accumulated.

[0018] Some embodiments of the present disclosure include storing the bit products in capacitors and summing the bit products by connecting the capacitors in parallel. Some embodiments of the present disclosure include iteratively summing and scaling the bit products in an accumulation capacitor.

[0019] In some embodiments of the present disclosure, a multi-processor computer system includes: a compact in-memory computer including memory components, each memory component including a computing engine and a storage element for storing data, the computing engine being operable to read and process data stored only in the storage element of the memory component; and a processor external to the compact in-memory computer, the processor being connected to and operable to write data to each storage element in the compact in-memory computer. In some embodiments, the compact in-memory computer may be or may include a compact in-memory computer architecture. The compact in-memory computer may include an array of memory components in the compact in-memory computer architecture, and may be a compact in-memory computer architecture. Each computing engine may be operable to process data stored in the storage element in response to an operation command. The storage element may include one or more bit cells. The processor may provide the operation command together with the data as part of a storage element write operation, the storage element write operation writing the data to the storage element of the memory component. The operation command may instruct the computing engine to perform an operation or not to perform an operation (e.g., a no operation).

[0020] In some embodiments, each memory component is directly connected to at least one other memory component to directly transmit data to and receive data from the other memory components. In some embodiments, the data is stored in capacitors. In some embodiments, capacitors in different memory components are connected together and the data in the different memory components are averaged together.

[0021] In some embodiments, the storage element is responsive to a compact memory computer address in a compact memory computer address range, and the processor is operable to write data to the storage element in the memory component at the compact memory computer address. In some embodiments, the processor has a processor address space, and the storage element is memory mapped into the processor address space. In some embodiments, a processor memory may be connected to the processor, the processor is operable to write processor data to the processor memory and read processor data from the processor memory, and the processor memory is memory mapped into the processor address space at a processor memory address range different from the compact memory computer address range. The processor memory is operable to store processor instructions.

[0022] Each memory component may include one or more of a bit memory, a multi-bit memory, a single-bit multiplier, or an iterative multi-bit multiplier. The storage element in each memory component may include one or more of a bit memory or a multi-bit memory, such as a bit cell or a multi-bit cell. The computation engine in each memory component may include a single-bit multiplier or an iterative multi-bit multiplier.

[0023] Each computation engine may include a capacitive product storage circuit, a capacitive accumulator storage circuit, or both a product storage circuit and a capacitive accumulator storage circuit. The capacitive product storage circuits of two or more memory components may be connected together, such as the capacitive product storage circuits of multiple pairs of adjacent memory components.

[0024] In some embodiments, the processor includes a controller that controls the computer in the compact memory. The controller can receive analog data, such as charge or voltage, from the memory component. The controller can convert the received analog data into digital data. The controller can accumulate data received from one or more memory components. The controller can include a multiplexer or demultiplexer connected to multiple rows or columns of memory components.

[0025] In some embodiments of the present disclosure, a multi-processor computer system includes: a compact in-memory computer including memory components, each memory component including a computing engine and a storage element for storing data, the computing engine being operable to read and process data stored only in the storage element of the memory component; and a processor external to the compact in-memory computer, the processor being connected to and operable to write data to each storage element in the compact in-memory computer. The computing engine may include a bit multiplier (e.g., a single-bit multiplier or an iterated bit multiplier).

[0026] In some embodiments, the memory components are arranged in an array, where rows of memory components are connected to bit lines and columns of memory components are connected to word lines, or vice versa. In some embodiments, each computation engine includes a capacitive product storage circuit, and the capacitive product storage circuits of a row or column of memory components are connected together. Each computation engine in a row or column of memory components may include an iterative multi-bit multiplier.

[0027] According to some embodiments of the present disclosure, a multi-processor computer system includes: a compact in-memory computer including memory components, each memory component including a computing engine and a storage element for storing data, the computing engine being operable to read data stored only in the storage element of the memory component through a bit line and process the data; and a processor external to the compact in-memory computer, the processor being connected to and operable to write data to each storage element in the compact in-memory computer. The storage element is mapped into the memory space of the processor and is accessible at the memory address of the processor through the bit line. Thus, the storage element is connected to the bit line for writing bits to the storage element by the processor, and the computing engine is connected to the same bit line as the storage element, thereby providing a space-dense construction for the memory component of the compact in-memory computer. In some embodiments, the bit line is an internal bit line connected to an external bit line through a memory selection switch.

[0028] In an embodiment of the present disclosure, a compact in-memory computer includes a multiplexer disposed between and connected to a storage element and a computing engine, and operable to select a bit cell in the storage element so that the computing engine is operable to process data stored in the selected bit cell.

[0029] In some embodiments, the computation engine includes a capacitor or a current source.

[0030] In some embodiments, the computing engine includes an analog-to-digital converter.

[0031] In some embodiments, the computation engine is operable to accumulate data stored in controllably selected bit locations.

[0032] In some embodiments, the computing engine is operable to convert data stored in the bit cells from analog values ​​to digital values, or to process data stored in the bit cells and convert the processed data from analog values ​​to digital values.

[0033] In some embodiments, the computation engine operates using analog circuitry.

[0034] In some embodiments of the present disclosure, a multi-processor computer system includes a processor or controller external to the multi-processor computer system, storage elements of a memory component are memory mapped into a memory space of the processor or controller, and the processor or controller is operable to read data and write data to any subset of the storage elements.

[0035] In some embodiments, at least some of the computation engines include a bit multiplier that stores the bit product in a capacitor, two or more capacitors are electrically connected in parallel and connected to an analog-to-digital converter, and the analog-to-digital converter has a precision less than the maximum possible value of the accumulated bit product stored in the capacitors connected in parallel. In some embodiments, at least some of the computation engines include an iterative bit multiplier that stores the accumulated bit product in a capacitor, the capacitor is electrically connected to the analog-to-digital converter, and the analog-to-digital converter has a precision less than the maximum possible value of the accumulated bit product stored in the capacitors connected in parallel. The analog-to-digital converter can be provided in the computation engine or in a controller or an external processor.

[0036] Some embodiments include a digital adder for adding each partial accumulation sum digitized by the analog-to-digital converter. The digital adder can be provided in the computing engine, in the controller, or in an external processor.

[0037] Embodiments of the present disclosure provide fast, efficient, low power and compact digital storage and computation circuits suitable for matrix multiplication, such as commonly found in pattern matching, machine learning and artificial intelligence applications. The multiplications can be done in parallel at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The foregoing and other objects, aspects, features and advantages of the present disclosure will become more apparent and better understood by referring to the following description taken in conjunction with the accompanying drawings, in which:

[0039] Figure 1 is a schematic block diagram and illustration of a compact computer-in-memory architecture having an array of memory components each including a single compute engine (CE) associated with a bit memory in accordance with an illustrative embodiment of the present disclosure;

[0040] FIG2 is a schematic block diagram of a solid state random access memory (SRAM) according to the prior art;

[0041] Figure 3A is a schematic block diagram of a compact computer-in-memory architecture having an array of memory components each including a MEMSEL switch associated with one byte of memory and a single compute engine (CE) according to an illustrative embodiment of the present disclosure;

[0042] Figure 3B is a detailed schematic block diagram of a controller and a memory component including a bit cell and a computation engine according to an illustrative embodiment of the present disclosure;

[0043] Figure 3C is a detailed schematic block diagram of a controller and a memory component storing a plurality of multi-bit values ​​according to an illustrative embodiment of the present disclosure;

[0044] Figure 3D is a detailed schematic block diagram of a controller and a memory component storing a plurality of multi-bit values ​​via a multiplexer according to an illustrative embodiment of the present disclosure;

[0045] Figure 3E is a detailed schematic block diagram of a controller and a memory component storing a multi-bit value and accessing an external multi-bit value according to an illustrative embodiment of the present disclosure;

[0046] Figure 4 and Figure 5 is a flow chart of a method according to an illustrative embodiment of the present disclosure.

[0047] Fig. 6A is a schematic diagram of a simple bit cell and computational elements according to an illustrative embodiment of the present disclosure;

[0048] Figure 6Bis a schematic diagram of a more complex bit cell and computational elements according to an illustrative embodiment of the present disclosure;

[0049] Figure 7 is a block schematic diagram of an iterative bit multiplier according to an embodiment of the present disclosure;

[0050] Fig. 8A illustrates multiple columns of bits in binary multiplication that are useful in understanding embodiments of the present disclosure;

[0051] Figure 8B illustrates multiple rows of bits in binary multiplication that are useful in understanding embodiments of the present disclosure;

[0052] Fig. 9 is a schematic diagram illustrating a plurality of memory components with connected product capacitors according to an illustrative embodiment of the present disclosure;

[0053] Fig.10 is a simplified schematic diagram illustrating a two-dimensional array of memory components having multiple columns of connected product capacitors and a controller according to an illustrative embodiment of the present disclosure;

[0054] Fig.11 is a simplified schematic diagram illustrating a one-dimensional array of memory components including an iterative bit-product multiplier and a controller according to an illustrative embodiment of the present disclosure;

[0055] Fig.12 is a schematic block diagram of an in-memory computing architecture and a controller for storing multi-bit values ​​according to an illustrative embodiment of the present disclosure;

[0056] Fig.13 and Fig.14 is a flow chart of a method according to an illustrative embodiment of the present disclosure;

[0057] Fig.15 is a block diagram of a multi-processor computer system according to an illustrative embodiment of the present disclosure.

[0058] The features and advantages of the present disclosure will become more apparent through the detailed description set forth below in conjunction with the accompanying drawings, in which like reference numerals identify corresponding elements throughout. In the accompanying drawings, like reference numerals generally indicate identical, functionally similar, and / or structurally similar elements. These figures are not necessarily drawn to scale. DETAILED DESCRIPTION

[0059] Certain embodiments of the present disclosure relate to, among other things, compact in-memory computer architectures, wherein a computing element (CE) is physically and spatially disposed between bit storage elements in a memory disposed above an area such as a substrate, such as a wafer or an integrated circuit substrate, and can provide fast, efficient, low-power, and compact digital storage and computing circuits suitable for use in matrix multiplications, such as those commonly found in pattern matching, machine learning, and artificial intelligence applications. The compact in-memory computer architecture can be a computer including distributed memory and computing elements, such as useful in a pulsating computing system, or a multi-processor computer system. The distributed memory can be memory mapped into the memory space of an external processor, allowing the external processor to directly read and write the distributed memory.

[0060] The term "q" in the text and figures herein designates a bit, and a suffix "B" or "bar" or a bar placed over a value indicates an inverted value, e.g., qB(qBar) designates the inverted value of q (e.g., NOT(q) in Boolean terms).

[0061] like Figure 1 , Figure 3A and Figure 3B As shown in FIG. 1 , the compact in-memory computer architecture 10 includes memory components 40 arranged in rows and columns, bit lines 24 connected to each row of the memory components 40, and word lines 26 connected to each column of the memory components 40. Each memory component 40 includes one or more bit cells 20 and a computing engine 30 (CE) connected to the one or more bit cells 20. Each bit cell 20 is a digital binary bit storage device or circuit that is operable to store a bit q of information (e.g., 1 or 0), and the computing engine 30 is operable to access and process the bit, for example, to read the bit value q stored in the bit cell 20 and perform a computing operation on the bit q. The computing engine 30 can be a hardwired computing engine 30, or can execute a program or state machine. In some embodiments, the computing engine 30 is digital. In some embodiments, the computing engine 30 is or includes an analog circuit, or a combination of analog and digital circuits.

[0062] The bit line 24 is an electrical connection, such as a wire or a trace, operable to provide a bit q to each memory component 40 in a row of memory components 40. The word line 26 is an electrical connection, such as a wire or a trace, operable to provide a select or write signal to each memory component 40 in a column of memory components 40 to enable each memory component 40 in the column of memory components 40 to write the bit q to each memory component 40 in the column of memory components 40. The bit line 24 and the word line 26 can be connected to and controlled by an external processor 82 or controller 70 (as described below with respect to Fig.15Memory component 40 may also include control signals or switches that may be externally controlled by controller 70. In some embodiments, word line 26 may select bit cell 20 and controller 70 may read bit q stored in bit cell 20 with appropriate memory select switch 60 settings on bit line 24 and external bit line 25.

[0063] The bit lines 24 and word lines 26 provide matrix access to the rows and columns of the memory components 40. Thus, the controller 70 may include a row controller operating in combination with a column controller to provide control signals, such as row or column selection, data or write signals, to the array of memory components 40. In some embodiments, the controller 70 provides address and data signals operable to write data to the array of memory components 40 in the compact memory computer architecture 10, for example, using an interface similar to conventional memory.

[0064] Figure 1 Each bit line 24 is illustrated with a subscript value (eg, BL0, BL1, etc.) representing each individual bit line 24. Similarly, Figure 1 Each word line 26 is illustrated with a subscript value (e.g., WL0, WL1, etc.) representing each individual word line 26. The bit cell 20 may include a digital binary bit storage element 22, such as a flip-flop, a latch, or an SRAM cell. Access to the bit storage element 22 may be controlled by a transistor 50 (e.g., an electronic switch) with a gate controlled by the word line 26 and data read or written on the bit line 24, such as by a controller 70. The bit line 24 (or a portion of the bit line 24) may also be connected to the compute engine 30, providing a compact layout for the memory component 40. According to an embodiment of the present disclosure, the compact in-memory computer architecture 10 may utilize a very compact layout for SRAM to reduce the area used by the compact in-memory computer architecture 10.

[0065] FIG. 2 illustrates a prior art SRAM including bit storage elements 22 arranged in rows and columns and connected to bit lines 24 and word lines 26 .

[0066] Figure 3A An embodiment of the present disclosure is illustrated in which each memory component 40 is connected to an external bit line 25 through a memory select (MEMSEL) switch 60 (e.g., transistor 50). The MEMSEL switch 60 can isolate the bit line 24 (e.g., internal bit line 24) of each memory component 40, or connect to an external control or data circuit (e.g., external bit line 25 and controller 70, such as Figure 1 (For clarity, as shown in Figure 1 The controller 70 shown in FIG. Figure 3A Omitted, but can be incorporated Figure 3A ) The MEMSEL switches 60 of each row, column, or entire array of memory components 40 may be electrically connected in common so that a single control signal may isolate or connect the bit lines 24 in the corresponding row, column, or array of memory components 40, e.g., a common control signal is connected to the gates of the MEMSEL switch 60 transistors 50 of the bit cells 20 of multiple rows, columns, or arrays. Thus, the MEMSEL switches 60 may enable external access to the bit cells 20, e.g., as a memory-mapped memory array, through the commonly connected external bit lines 25 in a first mode, and isolate the bit cells 20 from external access in a second mode so that the bit cells 20 may be individually, independently, and separately accessed by the corresponding computation engines 30 in each memory component 40.

[0067] Also like Figure 3A as well as Figure 3B and Figure 3C As shown in the detailed diagram of FIG. 1 , each memory component 40 may include a plurality of bit storage elements 22 or bit cells 20 connected to a common computing engine 30. In particular, the plurality of bit cells 20 may include a bit storage element 22 providing a word storage 28 storing a plurality of bits of one or more multi-bit digital values ​​that may be accessed and processed by the computing engine 30, such as Figure 3B . Word storage 28 may store, for example, any one or more of 4 bits (e.g., half a byte), 8 bits (e.g., a byte), 16 bits (e.g., a word), 24 bits, 32 bits (e.g., a long word), 48 bits, 64 bits, 96 bits, 128 bits, 256 bits, 512 bits, or 1024 bits. In some embodiments, memory components 40 may each include a plurality of bit storage elements 22 or bit cells 20 that store a plurality of multi-bit digital values ​​accessed and processed by a single computing engine 30 in a memory component 40, such as Figure 3C . Each bit cell 20 (or bit storage element 22) in the plurality of bit cells 20 may be connected to a common bit line 24 and a different word line 26 to enable writing of a bit to each bit cell 20. The word line 26 may enable access to a particular bit by the computation engine 30. In some embodiments, the output of each bit cell 20 may be connected together so that only one, or less than the number of bit cells 20, may be connected to the computation engine 30 using a single connection. Thus, a single bit q in a bit cell 20 connected to a bit line 24 may be selected by a corresponding word line 26 and operated on by the computation engine 30 at one time, or a plurality of bits, but less than all bits q, may be selected and operated on by the computation engine 30 at one time, or all bits q in the memory component 40 may be selected and operated on by the computation engine 30 at one time. Thus, in some embodiments, as Figure 3B As shown in FIG, each bit cell 20 is directly connected to a computation engine 30 and can be accessed in parallel at a single time.

[0068] In some embodiments of the present disclosure and as Figure 3C As shown in FIG. 1 , the compact computer-in-memory architecture 10 includes a controller 70 for controlling the memory component 40. In some embodiments and as Figure 3C As shown in , each bit line 24 or word line 26 is connected to a demultiplexer 33 in a controller 70, and the demultiplexer 33 provides a bit or address selection for each row or column of bit cells 20 to the computing engine 30 at a time, so that multiple rows or columns of bit cells 20 are written sequentially, or in some embodiments, data is written to each bit cell 20 at a time. Similarly, the controller 70 may include a multiplexer 32 that receives data from the computing engine 30 in the multiple rows or columns of memory components 40, and can select data from each row or column at a time to selectively input data.

[0069] In some embodiments, and as Figure 3D As shown in , the compact in-memory computer architecture 10 may include a multiplexer 32 (or multiple multiplexers 32) disposed between the storage element 22 and the computing engine 30 controlled by the computing engine 30 or the controller 70. The multiplexer 32 may enable the computing engine 32 to select one or more bit cells 20 in the storage element 22 and process the bits stored in each selected bit cell 20. The multiplexer 32 may be separate and independent of the computing engine 30, or the computing engine 30 may include the multiplexer 32. The selected data or the processed selected data may be converted from analog form to a digital value using an analog-to-digital converter 36. Some embodiments include multiple multiplexers 32 and multiple analog-to-digital converters 36 so that each multiplexer 32 selects data for a separate analog-to-digital converter 36.

[0070] In some embodiments of the present disclosure and as Figure 3E As shown in FIG. 1 , a single-bit or multi-bit value (e.g., memory A) is stored in memory component 40, and a second-bit or multi-bit value (e.g., memory B) is accessed externally by computing engine 30 and processed in combination with memory A under the control of controller 70. Figure 3D In the embodiment illustrated in FIG. 1 , one or more multiplexers 32 may enable the computation engine 30 to select one or more bit cells 20 ( Figure 3E not shown).

[0071] The memory components 40 may be disposed on a substrate (e.g., a wafer such as a silicon wafer or a printed circuit board), and each memory component 40 may be spatially disposed on or above a different portion of the substrate and adjacent to another memory component 40. The computing engine 30 of each memory component 40 may be spatially disposed adjacent to the bit cell 20 of each memory component 40, such as Figure 1 and Figure 3A In some embodiments, at least one computing engine 30 in a memory component 40 may be spatially disposed between a bit cell 20 of a memory component 40 and a bit cell 20 of an adjacent memory component 40, for example, Figure 1 and Figure 3A The bit lines 24 connected to the bit cells 20 may also connect the bit cells 20 to the compute engine 30 in the memory component 40, providing efficient use of space on or in a wafer or integrated circuit and reducing the area required for the memory component 40.

[0072] Adjacent bit cells 20 are bit cells 20 with no other bit cells 20 located between them, and adjacent memory components 40 are memory components 40 with no other memory components 40 located between them. Similarly, adjacent computing engines 30 are computing engines 30 with no other computing engines 30 located between them. In some embodiments, each computing engine 30 is connected to an adjacent computing engine 30, for example, with an electrical connection. Such an arrangement of bit cells 20 and computing engines 30 in a memory component 40 provides a compact and efficient structure, reduces the area used (e.g., silicon area in a wafer or integrated circuit), places circuits close to each other to reduce signal propagation time and improve signal-to-noise ratio, and utilizes, is compatible with, or extends circuit layouts commonly found in highly optimized integrated circuit layouts in integrated circuit foundries or manufacturing facilities. Thus, embodiments of the present disclosure efficiently use semiconductor resources, reduce costs, and provide excellent performance.

[0073] Figure 4 The diagram shows the corresponding Figure 3A In step 100, one or more memory components 40 are provided, for example, an array of memory components 40 connected to bit lines 24 and word lines 26, such as Figure 3A2 . In step 110, the MEMSEL switch 60 is closed (e.g., by the controller 70) to connect the internal bit line 24 to the external bit line 25 and to the controller 70. The controller 70 selects a column of memory components 40 and provides corresponding signals (e.g., bit values ​​q) on the external bit lines 25, which travel to the internal bit lines 24 through the closed MEMSEL switches 60 and are stored in the bit storage elements 22 of each bit cell 20 in step 120. In this mode, the bit cells 20 in the memory components 40 can act as a traditional SRAM, for example, as shown in FIG. 2 . The MEMSEL switch 60 is then opened (e.g., by the controller 70) to isolate the memory components 40 from the external bit lines 25 in step 130 to complete the write step 160. The computing engine 30 can then independently access the connected bit cells 20 in each memory component 40, thereby reading the bit value q in step 140 and then processing the bit value q in step 150.

[0074] Thus, the method of the present disclosure includes providing memory components 40 in step 100 and selecting a row of memory components 40 using the controller 70 in step 200. Figure 5 ; providing a bit q (e.g., providing data) on each bit line 24 in step 210; enabling the word line 26 of each column of memory components 40 in step 220; and storing the bit q in the bit cell 20 of each memory component 40 in the selected row of memory components 40 in step 160. The stored bit q is processed in step 140 using the computing engine 30 of each memory component 40 in the column of memory components 40. Each memory component 40 may be connected to a corresponding bit line 24 by a memory select (MEMSEL) switch 60, and the method of the present disclosure may include opening the MEMSEL switch 60 before using the controller 70 to provide the bit q on each bit line 24, and closing the MEMSEL switch 60 after using the controller 70 to provide the bit q on each bit line 24 (step 160), and then using the computing engine 30 of each memory component 40 in the column of memory components 40 to process the stored bit q in step 140. The processed data may be read, for example, by the controller 70 in step 230.

[0075] According to some embodiments of the present disclosure, the bit cell 20 (e.g., SRAM bit storage) may be implemented with 6 transistors, such that the word storage 28 for a byte (an 8-bit multi-bit digital value) requires 48 transistors, and the word storage 28 for a word (a 16-bit multi-bit digital value) requires 96 transistors. In some embodiments, the computation engine 30 may include 12 transistors and two capacitors, such that integrating the computation engine 30 into an optimized, dense, and efficient SRAM array design from a semiconductor foundry or manufacturing facility results in a relatively optimized, dense, and efficient memory component design.

[0076] As described above, computing engine 30 may include both analog and digital circuit elements, such as capacitors and transistors. Fig. 6A As shown in , the memory component 40 includes a plurality of bit cells 20 (forming a storage element 22) and a computing engine 30 operable to read data from the bit cells 20A and 20B. The computing engine 30 may include a one-bit multiplier 14 (e.g., a switch 50 or a transistor 50) that receives input from the bit cells 20. One input (e.g., the bit cell 20B) is connected to the gate and the other input (e.g., the bit cell 20A) is connected to the source. When the data in both bit cells 20A and 20B are high (e.g., 1), 1 is transferred to the drain of the transistor 50 and accumulated in the product storage circuit 16 (e.g., an analog storage circuit 16, such as a capacitor 16) as the product of the bit data stored in the bit cells 20A and 20B.

[0077] Figure 6B A more complex, electrically more efficient, and space more efficient bit multiplication circuit 14 is illustrated. In some such embodiments, the serial switch circuit 15 includes two transistors 50 driven by complementary outputs from the bit cell 20. If the bit cell 20 is high (e.g., storing a 1 or a positive charge), V is transferred through the serial switch circuit 15. REFP Signal (positive voltage reference). Each of the two serial switch circuits 15 connected in series is connected to the bit cell 20A and the bit cell 20B, respectively. If both are positive, when the switch circuit 18 (switch 18) is high, a positive value (e.g., 1 or positive charge) is deposited in the product storage circuit 16 (e.g., analog storage circuit 16, such as capacitor 16) as the product of the bit data stored in the bit cells 20A and 20B. If the switch circuit 18 is low (e.g., 0), the charge (voltage) in the product storage circuit 16 is output. Thus, the switch circuit 18 is operable to store bit products in a multiplication mode, and is operable to output bit products in an accumulation mode, but not in both modes at the same time.

[0078] Figure 6BThe memory component 40 shown in FIG. 4 includes three serially connected serial switch circuits 15. Each switch circuit 15 includes a pair of simple MOS (metal oxide semiconductor) transistors having separate differential inputs and a common output. One of the pair of simple MOS transistors is controlled by a positive control signal and the other is controlled by an inverted (negative) version of the same control signal, for example, the positive and negative outputs of any single-bit cell 20 (e.g., a D flip-flop or a pair of inverters). Such a series of serial switch circuits 15 may require fewer, simpler transistors that operate at much lower voltages (e.g., one percent or less, such as 0.624%, or 10 mV instead of 1.65 V), and therefore require much less power. The combined (summed) voltage on the analog storage circuit 16 may be:

[0079] V SUM =((n*V REFP )+(Nn)*V REFN )) / N.

[0080] Among them, V REFN =0V:

[0081] V SUM =(n*V REFP ) / N,

[0082] Here, n is the number of capacitors and N is the number of parallel-connected capacitors 16 connected in a row.

[0083] In some embodiments, the bit multiplier 14 controls the current deposited charge on the bit capacitor 16 very accurately over time to maintain the accuracy and precision of the multiplication-accumulation operation. Thus, the bit multiplier 14 can be designed to control the amount of charge deposited on the bit capacitor 16 very accurately, for example, in response to carefully calibrated timing signals and voltages. A bit multiplier 14 using a conventional AND gate may require, for example, six relatively large transistors operating at relatively high voltages to implement a bit multiplication circuit that can adequately control the charge Q (e.g., 1.65-5V) deposited on the analog storage circuit 16. In contrast and in accordance with an embodiment of the present disclosure, the bit multiplier 14 of the present disclosure may include a serially connected serial switch circuit 15 that can operate at a relatively low voltage (e.g., no greater than 1V and as low as 10mV) and low power, and can adequately control the charge Q deposited on the analog storage circuit 16 with, for example, only four relatively small transistors. In an embodiment, the memory component 40 operates in an analog relatively low power state having a relatively high power state less than a digital voltage. In some embodiments, the analog voltage is no greater than one-half, one-quarter, one-fifth, one-tenth, one-twentieth, one-fiftieth, or one-hundredth (e.g., 50%, 25%, 20%, 10%, 5%, 2%, or 1%) of the digital voltage.

[0084] In some embodiments, the bit products are iteratively combined and successively scaled by a factor of 2 to provide a multi-bit multiplication product. Figure 7 As shown in , when switch 18 connects bit multiplier 14 to capacitor 16, the bit product can be stored in product storage circuit 16. When switch 18 connects capacitor 16 to accumulator storage circuit 17 (capacitor 17), the charge is averaged. Each consecutive bit product (0 or 1) will average the accumulator charge to half the charge (if the bit product is 0), or half the difference between the accumulator charge and 1 (if the bit product is 1). Thus, the resulting accumulator charge is a multi-bit product that can be converted to a digital value (scaled by the number of average steps).

[0085] Figure 7 A simple hybrid iterative single-bit multiply-accumulate circuit is shown, which includes Figure 6B 1 , a single-bit multiply-accumulate circuit (shown as logic rather than electrical operation) is shown in which a product storage circuit 16 (capacitor 16) is electrically connected in parallel with an accumulator storage circuit 17 (e.g., a capacitor 17 having the same capacitance as the product storage circuit 16) via a switch 18 serving as an accumulation switch 62. The accumulation switch 62 may be the same, substantially similar, or identical to the differential switch 18 of the serial switch circuit 15. Optionally, the output of the accumulator storage circuit 17 may be connected to an analog-to-digital converter (ADC) 36 via an optional switch 18 (output switch 64).

[0086] In more detail, Figure 7 The multiplication of two single-bit values ​​stored in two corresponding single-bit cells 20 of storage element 22 is shown. When switch 18 is set to multiplication mode (high), the product P is stored in product storage circuit 16 (capacitor 16). When switch 18 is set to accumulation mode (low), any charge stored in product storage circuit 16 is shared (combined) with any charge stored in accumulator storage circuit 17 (capacitor 17). The average of the charges in capacitors 16 and 17 is then stored in both capacitors 16 and 17. Multiple bit products can be accumulated in two capacitors 16, 17 by repeatedly performing the following actions: providing bits in bit cells 20A and 20B; setting switch 18 to multiplication mode; depositing a charge representing the bit product of bit cells 20A and 20B in product storage circuit 16; and setting switch 18 to accumulation mode, thereby combining the charges in capacitor 16 and capacitor 17 (accumulator storage circuit 17). When all bits are multiplied, the result can be output by setting accumulation switch 62 high. The analog charge may then be converted to a digital value and scaled to represent the product of the bits iteratively provided in bit cells 20A and 20B.

[0087] In an embodiment of the present disclosure, iterative bit multiplication proceeds from the least significant bit to the most significant bit. Each time the product values ​​are averaged; they are also divided by 2 so that the next bit will have twice the relative value as the accumulated value. For example, the digital value 11 10(10112) will proceed by clearing the product and accumulator storage circuits 16, 17 (capacitors 16, 17). Given a single bit A equal to 1 (if single bit A equals 0, all products and accumulated charges will be 0), the least significant bit (bit 0) of the multi-bit value B is 1, so the product will be 1, and the 1 value will be transferred to capacitor 16 in the first iteration. (The actual charge is a design choice, and the values ​​described in this example are relative values ​​and charge quantities.) The accumulated value will be one-half (shared between capacitors 16 and 17). The next bit (bit 1) will also produce a product of 1, so capacitor 16 is set to a value of 1, and when combined with the one-half value in accumulation capacitor 17, a value of three-quarters is produced. The next product using zero bit 2 of multi-value B sets capacitor 16 to 0, and when shared with the three-quarters accumulated value, a value of three-eighths is produced (three-quarters divided by two). The last bit of multi-value A (bit 3) is 1, resulting in a capacitor 16 value of 1, which when shared with the three-eighths value in capacitor 17, yields a final product of eleven-sixteenths. The product (scaled by 16 to adjust for the average of each of the four stages) is 11, the product of 11 and 1. The process can then be repeated with another bit of multi-bit value A, calculating all bit products for the two multi-bit values ​​A and B.

[0088] like Fig. 8A and Figure 8B As shown in the 4-bit example of FIG, each row of products shown is the product of a one-bit value B multiplied by a multi-bit value A. Fig. 8A and Figure 8B The bits 21C are spatially displaced relative to each other, indicating the relative magnitude (location) of the products in each row, as in conventional manual multiplication on paper. The bit products (multiplied values) in each bit column 21C of products (having the same magnitude or location) can be summed. Each column and has a relative magnitude of 2 (or half) relative to the adjacent bit column 21C, as Fig. 8A . Because each bit column 21C product has a different locality value (relative magnitude), the values ​​in each column 21 product must be scaled to multiply them by their locality value, e.g., by 1 to 6 localities to multiply them by 2, 4, 8, 16, 32, or 64, before they are summed. The scaling and summing of the column sums provides the product for the two multi-bit digital binary values ​​A and B. Similarly, the bit products (multiplied values) of each bit row 21R product may be appropriately scaled and summed, as shown in FIG. Figure 8B . Each bit product in a row has a relative magnitude of 2 (or half) with respect to the adjacent bit products in that row, and each row has a relative magnitude of 2 (or half) with respect to the adjacent row. Scaling and summing the row sums provides the product for two multi-bit digital binary values ​​A and B.

[0089] Fig. 9The diagram corresponds to Fig. 8A A schematic diagram of an embodiment of the present invention. Fig. 9 In FIG. 4 , each capacitor 16 in a column of memory components 40 is connected together when switch S is in the accumulation mode. The values ​​are averaged, and the average value can be converted to a digital value, scaled and summed to provide the product of two multi-bit values. Fig.10 is a more detailed diagram showing an array of capacitors 16 (simplified as shown) of memory components 40 connected together in a common bit column 21C. Fig. 6A The summed product (output O) of each bit column 21C is converted to a digital value by analog-to-digital converter 36 and then shifted (e.g., with a shift register) or simply by connecting the bits in the shifted arrangement to a digital adder to provide the product P in a digital shift and accumulate circuit 38.

[0090] Fig.11 The diagram shows Figure 7 An array of memory components 40 that iteratively calculate and scale the products of the bit rows 21R. Each memory component 40 iteratively calculates the sum O of the bit rows 21R, and the sum O is converted to a digital value by the ADC 36, and then shifted and summed in the digital shift and accumulate circuit 38 to provide the product P.

[0091] Fig. 9 and Fig.10 Examples of Fig.11 The embodiment of is faster because no iterative calculation is required, instead a two-dimensional array of memory components 40 is required to calculate the product of two multi-bit binary values. Fig.11 The embodiment of requires iterative bit product sums, but only requires a one-dimensional array of memory components 40. In both embodiments, a large array of memory components 40 can calculate many products simultaneously, for example, millions or even billions. (For clarity, Fig.10 and Fig.11 Shows the use of Fig. 6A The memory component 40 of the configuration, but it is also possible to use Figure 6B configuration. )

[0092] The computing engine 30 may include various computing structures, including analog circuits, digital circuits, or a combination of analog and digital circuits. Similarly, the processing operations performed by the computing engine 30 are not limited and may include logic, programming, and mathematical operations. The computing engine 30 may include a control circuit, a state machine, or a programmable machine, including registers, clock signals, and arithmetic structures such as adders and multipliers. In some embodiments, the computing engine 30 may write processed data to the storage element 22, and the process data in the storage element 22 may be read by the processor 70, such as selecting the memory component 40 with the word line 26 and reading the data on the bit line 24 (e.g., by connecting the memory selection switch 60 of the external bit line 25).

[0093] In some embodiments of the present disclosure and as Fig.12 , a computation engine 30 enables multiplication of two multi-bit values ​​stored in a storage element 22, and a compact in-memory computer architecture 10 including a plurality of memory components 40 performs matrix multiplication on the values ​​stored in the storage element 22. In some embodiments, the compact in-memory computer architecture 10 provides an array of dot product functions that can be matrix-vector products (e.g., where the matrix dimension is 1). Each row (or column) of a memory component 40 in the compact in-memory computer 10 can perform a dot product. Thus, in some embodiments, the memory component 40 includes a computation engine 30 and a storage element 22, the computation engine 30 including a multiplier, the storage element 22 having two elements A and B, each containing an arbitrary number of bits. The computation engine 30 is connected to the storage element 22 with a data line (bit line 24), and writes to and reads from the storage element 22 using control signals. In operation, data is written to the storage element 22 using the bit line 24 and the word line 26, the memory select switch 60 ( Figure 3A ). When the memory select switch 60 is not enabled, the computing engine 30 can read data from the storage element 22 and operate (process) the read data. The storage element 22 of the compact memory computer architecture 10 can be memory mapped to the controller 70. The controller 70 can write data to the storage element 22 in such a way that the computing engines 30 each calculate the appropriate part of the multi-bit multiplication, for example, using the demultiplexer 33.

[0094] like Fig. 9 and Fig.10 The circuit diagram and Fig.13As shown in the flowchart of , a single bit A can be multiplied by a multi-bit value B by first providing the memory component 40 in step 100, and then clearing the product storage circuit 16 and the accumulator storage circuit 17 in step 310 (e.g., setting their values ​​to zero, for example, by connecting them to ground with a clear circuit to remove any charge in the capacitors 16, 17). A bit count M is set for each memory component in step 305. Steps 305 and 310 can be completed in any order. The controller 70 selects the single bit value A from the storage element 22 and the multi-bit value B in the storage element 22 in step 315 to select the bit M of the multi-bit value B through the multiplexer 32, and sets the switch 18 to the multiplication mode under the control of the controller 70 in step 320. The bit multiplier 14 multiplies the single bit value A by the bit B in step 325. M The switch 18 is set to an averaging mode under the control of the controller 70 at step 330 to share (average) the charges in the capacitors 16 and 17 at step 335. The averaged value may be converted to a digital value at step 340 and shifted and accumulated at step 345. The accumulated value corresponding to the product may be stored at step 360.

[0095] Fig.14 The diagram shows Fig.12 The circuit is useful for iterative methods and is similar to Fig.13 , except that, instead of averaging, the bit values ​​are iteratively multiplied and accumulated in steps 325 and 335 before conversion to digital values, and accumulated in one of the multi-bit values ​​for each of the multiple bits. Fig.14 In the flowchart of FIG. 1 , a single bit A can be multiplied by a multi-bit value B by first providing the memory component 40 in step 100 and then clearing the product storage circuit 16 and the accumulator storage circuit 17 in step 310 (e.g., setting their values ​​to zero, e.g., by connecting them to ground with a clear circuit, thereby removing any charge from the capacitors 16, 17). The bit count M is set to zero in step 306. Steps 306 and 310 can be performed in any order. The controller 70 selects the single bit value A from the storage element 22 and the multi-bit value B in the storage element 22 in step 315 to select the bit M of the multi-bit value B through the multiplexer 32, and sets the switch 18 to the multiplication mode under the control of the controller 70 in step 320. The bit multiplier 14 multiplies the single bit value A by the bit B in step 325. M. Switch 18 is set to accumulation mode under the control of controller 70 in step 331 so that the charges in capacitors 16 and 17 are shared (averaged and accumulated) in step 335. If all B bits have not been multiplied (step 350), the bit count M is incremented in step 355 and the next bit is selected (step 315) and the process repeats until all bits M are iteratively multiplied and accumulated. The accumulated value can be converted to a digital value in step 340 and shifted and accumulated in step 345. The accumulated value corresponding to the product P is stored in step 360.

[0096] According to an embodiment of the present disclosure, the compact in-memory computer architecture 10 includes many memory components 40 (e.g., thousands, millions, hundreds of millions, or even billions of memory components 40, including both storage elements 22 and compute engines 30). Thus, the compact in-memory computer architecture 10 can perform millions or even billions of bit multiplications at very high rates with very little power. The external processor 82 (see Fig.15 ), for example, a central processing unit (CPU) or an external FPGA, with appropriate control circuitry such as a processor unit or state machine, can write data to the memory component 40 and then receive processed data from the memory component 40 almost immediately, providing a very simple and very fast architecture for processing large amounts of data in parallel. Because the storage elements 22 of the memory component 40 in the compact in-memory computer architecture 10 can be mapped into the memory space of the external CPU or other processor 82, the interface to the compact in-memory computer architecture 10 is very simple and is the same, similar, or substantially the same as the interface to a memory (e.g., DRAM or SRAM). Because there are many compute engines 30 in the compact in-memory computer architecture 10, and because multiplication, summation, analog-to-digital conversion, and shift operations can be analog, data processing can be very fast.

[0097] Embodiments of the present disclosure may be very compact, utilizing or using structures similar to those found in memory chips. In order to provide a dense arrangement of memory components 40, it may be useful to integrate a small and efficient computing engine 30 in a compact in-memory computer architecture 10. In some embodiments, the memory components 40 are arranged in a two-dimensional array (matrix), with rows of memory components 40 (e.g., the storage elements 22 of each memory component 40 in a row of the array) connected to a common bit line 24, and columns of memory components 40 (e.g., the storage elements 22 of each memory component 40 in a column of the array) connected to a common word line 26. In some embodiments, the memory components 40 are arranged in a two-dimensional array (matrix), with rows of memory components 40 (e.g., the storage elements 22 of each memory component 40 in a row of the array) connected to a common word line 26, and columns of memory components 40 (e.g., the storage elements 22 of each memory component 40 in a column of the array) connected to a common bit line 24. Rows and columns are arbitrary designations of orthogonal groups of memory components 40 in an array and may be interchanged.

[0098] In some embodiments, the memory components 40 are interconnected in a matrix. In some embodiments, the memory components 40 are physically and spatially arranged in an array over an area of ​​a substrate on which the memory components 40 are arranged, the rows and columns of the memory components 40 are arranged in a two-dimensional array (matrix), the rows of the memory components 40 (e.g., the storage elements 22 of each memory component 40 in a row of the array) are connected to a common bit line 24, and the columns of the memory components 40 (e.g., the storage elements 22 of each memory component 40 in a column of the array) are connected to a common word line 26. The computing engine 30 of each memory component 40 can be arranged between the storage element 22 of the memory component 40 and the storage element 22 of the adjacent memory component 40, for example, adjacent in the horizontal direction or adjacent in the vertical direction (or both). The adjacent memory components 40 are memory components 40 between which other memory components 40 are arranged without space.

[0099] According to the embodiments of the present disclosure and as Fig.15As shown in the figure, the multi-processor computer system 80 includes a processor 82, the processor 82 includes a controller 70, or the controller 70 can be a processor 82. The processor can be a central processing unit, which is operable to read data from a processor address space and write data to it. In some embodiments, a memory is connected to the central processing unit, and the central processing unit is mapped to a processor memory space (e.g., a processor address space B having a processor memory address range in the processor address space) of the central processing unit for storing programs and data (e.g., a stored program machine). In some embodiments, the processor 82 includes a custom integrated circuit (or multiple circuits), a programmable gate array (e.g., a PGA), a field programmable gate array (e.g., an FPGA), or a state machine containing storage and functional elements. The controller 70 can control or otherwise provide data to the compact memory computer architecture 10 and receive data from it, and can be implemented in the program of the processor 82, or include a peripheral control circuit (e.g., a separate controller 70), which is implemented in any combination of a custom circuit, a programmable gate array, a state machine, or other electronic or optoelectronic circuits. The processor 82 can access the storage elements 22 of the memory component 40 of the compact memory computer architecture 10 as a memory array mapped into the memory space of the processor 82, for example, within an address range corresponding to a processor address space A having an address range different from the address range of the processor address space B.

[0100] In an embodiment, the compact in-memory computer architecture 10 is, or the compact in-memory computer 10 may be or include, a compact in-memory computer architecture 10 that is a distributed memory (e.g., storage elements 22 distributed over an area of ​​a substrate such as a semiconductor wafer substrate), a computing engine 30 (e.g., Figure 1-3C , Figure 6A-78-12 ) are spatially arranged between storage elements 22 of different adjacent memory components 40, thereby providing a compact structure capable of massively parallel processing (e.g., multiplications, such as bit multiplications or iterated bit multiplications that are accumulated to provide the product of two multi-bit values ​​in a matrix multiplication), useful, for example, in machine learning and artificial intelligence applications, with reduced power and increased speed, for example, by using analog operations, analog storage (e.g., capacitors instead of flip-flops or latches), analog summation, or analog scaling (e.g., as part of an iterative multi-bit multiplication). Multiple multiplications of different values ​​in the matrix may be performed in parallel by the processor 82 or the controller 70 and arranging the necessary data in the storage elements 22, or both, for example, by writing two multi-bit values ​​into each storage element 22, by storing a single bit of each of the two multi-bit values ​​into each storage element 22 (e.g., a storage element 22 of a memory component 40 having a commonly connected product storage capacitor 16), or by storing a first multi-bit value into a plurality of storage elements 22 and storing a different bit of a second multi-bit value into each of the plurality of storage elements 22, for example, in a memory component 40 having an iterative multi-bit product circuit computation engine 30. Each different bit stored in a different memory component 40 may be stored in the same location in a storage element 22 of the memory component 40, so that a single operation performed by a different computation engine 30 in a different memory component 40 may perform the same operation using different bits of a multi-bit value (e.g., a second multi-bit value).

[0101] According to an embodiment of the present disclosure, a multi-processor computer system 80 includes a compact in-memory computer 10, which includes a memory component 40 and a processor 82, which is spatially and logically separated, independent and external to the compact in-memory computer 10 connected to the compact in-memory computer 10. Each memory component 40 may include a computing engine 30 and a storage element 22. The storage element 22 may include one bit cell 22 or multiple bit cells 22. The computing engine 30 may include a single bit multiplier 14 and a product storage capacitor 16 or an iterative bit multiplier 14 and a product storage capacitor 16. The bit product may be accumulated in an accumulator storage circuit 17 or a capacitor 17. The capacitors 16 or 17 may be electrically connected together, for example, each through a switch circuit 18. The computing engine 30 may be operable to read data (e.g., a bit or multiple bits of a digital binary value) only from the storage element 22 of the memory component 40 and process the data (e.g., by performing bit multiplication). In some embodiments, the computing engine 30 may write data to the storage element 22 of the memory component 40. The processor 82 may be operable to write data to each storage element 22 in the compact memory computer 10, and in some embodiments read data from each storage element 22 in the compact memory computer 10. The data may be multi-bit values ​​in a matrix that are multiplied to provide a matrix multiplication performed in parallel, whether in a two-dimensional array, in a row of the array, or in a column. Thus, the storage element 22 may be responsive to a compact memory computer address in a compact memory computer address range, and the processor 82 may be operable to write data to the memory component 40 at the compact memory computer address. In some embodiments, the processor 82 may be operable to read data from the storage element 22 at a compact memory computer address in a compact memory computer address range.

[0102] In some embodiments, data is written to the storage element 22 of the computer 10 in the compact memory by controlling the word line 26 as an address line and the bit line 24 as a data line in a memory write operation. The memory write operation may include controlling one or more control bits, for example, a bit that provides a memory select switch control (e.g., opens or closes a memory select switch). In some embodiments, the computing engine 30 may provide two or more different operations, and the control bits may indicate or select the operation of the two or more different operations, for example, an operation command, so that the processor 82 provides the operation command along with the data as part of the storage element 22 write operation to write the data to the storage element 22 of the memory component 40.

[0103] In some embodiments, the controller 70 includes one or more analog-to-digital converters 36, for example, connected to each row or column of the memory component 40, or connected to one or more multiplexers 32 connected to each row or column of the memory component 40, so that the analog-to-digital converters 36 can convert data (e.g., analog values, such as charge or voltage) for multiple rows or columns of the memory component 40 at a time, or select and convert data using the multiplexers 32. The controller 70 may include one or more accumulation circuits, whether digital or analog, scaling circuits, such as binary shift circuits (e.g., place value connections), such as in a shift and accumulate circuit.

[0104] In some embodiments, the computation engine 30 may provide analog computations, for example, by combining full or partial operational amplifier (Op Amp) circuits or differential amplifiers, fully differential amplifiers, and isolation amplifiers that provide arithmetic functions including summation and multiplication. The computation engine 30 may provide functions such as multiplication accumulation functions, dot product functions, and convolution functions. In an embodiment, the computation engine 30 may include one or more analog components, analog current sources, analog storage elements (e.g., capacitors such as product storage circuit 16 and accumulator storage circuit 17), a multiplexing mechanism (e.g., multiplexer 32), and an analog-to-digital converter 36 (e.g., such as Figure 3D ). The computation engine 30 may be operable to accumulate a controllably selected state (e.g., a value or bit) of the bit cell 20 in the storage element 22. The computation engine 30 may perform analog computations (e.g., accumulate values) and, optionally, convert the results of the analog computations on the bit cell data (or directly the bit cell data) into digital values ​​that may be accessed by or transmitted to other computation engines 30 or the controller 70, e.g., replacing the functionality of the analog-to-digital converter 36 in the controller 70, as shown by the dashed element outline.

[0105] In an embodiment of the present disclosure, the analog-to-digital converter 36 may have a relatively low precision, for example, when applied to accumulated values, which may be accumulated iteratively (e.g., as Figure 7 ) or accumulated in parallel (e.g., as Fig. 9). If reduced precision is acceptable, for example, eight-bit values ​​instead of nine-bit values ​​for the accumulated value of 512 bits (data stored in 512 parallel-connected bit cells 20), a reduced-precision analog-to-digital converter 36 can be used to save power and circuit area and increase speed. This design can also be applied to iterative accumulated products. In particular, if it is known that many bit products have a high probability of being equal to zero, fewer bits can be used to store the accumulation of bit products without loss of precision, or at least with a reduced probability of loss of precision. Such a design can be more energy efficient, perhaps by an order of magnitude, and produce acceptable results.

[0106] In embodiments where maintaining accuracy is important, a separate analog-to-digital converter 36 with reduced accuracy may be applied to individual values ​​or partial accumulations and the results then summed digitally (e.g., as in Fig.11 ), thereby providing the accumulated value with full precision. For example, if 256 bits are accumulated, an eight-bit analog-to-digital converter 36 is required to convert the accumulation without loss of precision. Alternatively, four six-bit analog-to-digital converters 36 can convert the corresponding four partial values ​​(each of 64 bits) and the four values ​​digitally summed to provide an eight-bit accumulated value. This design reduces size and power and increases the speed of the analog-to-digital converter 36 at the expense of an additional digital adder.

[0107] More generally, an external device or system (e.g., a process or CPU or controller 70) can write to or read from any subset of the bit cells 20 of the storage elements 22 in any memory component 40 of the compact memory in-computer 10, so that the storage elements 22 (and the bit cells 20) are memory mapped into the memory space of the external device or system. The computation engine 30 of the memory component 40 is operable to compute or process the data stored in the bit cells 20 of the storage elements 22 of the memory component 40.

[0108] Embodiments of the present disclosure provide high-speed operation at relatively low power for compact in-memory computers, and compact in-memory architecture arrays of memory components 40 suitable for matrix multiplication. In some embodiments, the operation is analog and operates at much lower power than in the case of digital calculations. For example, bit products can use capacitor summation, for example, to provide an average function or iterative accumulation, providing averaging and scaling with little power usage or time delay. The bit capacitor 16 (and 17) can be very small to reduce the area of ​​the bit capacitor 16 in the integrated circuit embodiment and the charge required to store or read the value in the capacitor 16. Digital, binary scaling operations can be simply implemented through interconnection, providing relative multiplication of powers of 2 to the summing circuit without additional power cost.

[0109] The operating power for the storage element 22, the bit multiplication circuit 14, and the analog storage circuits 16, 17 can have a voltage of no more than 1V (e.g., no more than 500mV, no more than 100mV, no more than 50mv, or no more than 10mV), which operates at a much lower voltage and power than digital circuits that provide similar functions. The multiplication circuit may include serially connected switches, which include paired MOS transistors, for example, operating in a low voltage, low power state that consumes less power than traditional digital MOS circuits. Therefore, embodiments of the present disclosure can perform many (e.g., billions) bit product and accumulation operations at a time with very low power, providing high-speed, efficient parallel operations for computing tasks such as matrix multiplication computing tasks.

[0110] The embodiments of the present disclosure are not limited to the specific examples illustrated in the drawings and described herein. It should be readily apparent to those skilled in the art that various implementations of analog and digital circuits may be employed to implement the described operations and such implementations are included in the embodiments of the present disclosure.

[0111] Embodiments of the present disclosure may be used for neural networks, pattern matching computers, or machine learning computers, and provide efficient and timely processing with reduced power and hardware requirements. Such embodiments may include computing accelerators, such as neural network accelerators, pattern matching accelerators, machine learning accelerators, or artificial intelligence computing accelerators designed for static or dynamic processing workloads.

[0112] After describing certain embodiments of the embodiment, it will now be apparent to those skilled in the art that other embodiments incorporating the concepts of the present disclosure may be used. Therefore, the present disclosure should not be limited to certain embodiments, but should only be limited by the spirit and scope of the following claims.

[0113] Throughout the description, where devices and systems are described as having, including, or comprising specific elements, or where processes and methods are described as having, including, or comprising specific steps, it is contemplated that there are additional devices and systems of the disclosed technology that essentially consist of or consist of the elements, and processes and methods according to the disclosed technology that essentially consist of or consist of the processing steps.

[0114] It should be understood that the order or sequence of steps used to perform certain actions is immaterial as long as the disclosed technology remains operable. Moreover, two or more steps or actions may be performed simultaneously in some cases. The present disclosure has been described in detail with particular reference to certain embodiments thereof, but it should be understood that variations and modifications may be implemented within the spirit and scope of the following claims.

[0115] Reference numerals list

[0116] O output; P product; 10 compact in-memory computer architecture / compact in-memory computer; 14 bit multiplier / bit multiplication circuit; 15 serial switch circuit; 16 capacitor / analog storage circuit / product storage circuit; 17 capacitor / analog storage circuit / accumulator storage circuit; 18 switch / switch circuit; 20 bit cell; 21C bit column; 21R bit row; 22 storage element; 24 bit line; 25 external bit line; 26 word line; 28 word storage; 30 calculation engine (CE); 32 multiplexer; 33 demultiplexer; 36 analog-to-digital converter; 38 digital shift-accumulate (SAC) circuit; 40 memory component ; 50 switch / transistor; 60 MEMSEL (memory selection) switch; 62 accumulation switch; 64 output switch; 70 controller; 80 multi-processor computer system; 82 processor; 100 provide memory component step; 110 close MEMSEL step; 120 write bit into bit unit step; 130 open MEMSEL step; 140 CE read bit from bit unit step / calculate (process) data step; 150 CE process bit step; 160 write step; 200 select row step; 210 provide data step; 220 enable word line step; 230 read calculated data step; 305 set B M Bit selection step: 306 B M Step 310 of setting the bit count M to zero; Step 310 of clearing the capacitor; Step 315 of selecting the B bit M Step; 320 step of setting the switch to multiplication mode; 325 step of bit multiplication; 330 step of setting the switch to average mode; 331 step of setting accumulation mode; 335 step of accumulation; 340 step of analog-to-digital conversion; 345 step of shift accumulation; 350 step of testing all B bits multiplied; 355 step of setting the B bit count M to M+1; 360 step of storing the product.

Claims

1. A compact in-memory computer architecture (10), comprising: Memory components (40) arranged in rows and columns, each memory component (40) comprising a bit cell (20) and a computation engine (30) connected to the bit cell (20), wherein the bit cell (20) is operable to store a bit and the computation engine (30) is operable to process the bit; bit lines (24), each bit line (24) connected to a corresponding row of the memory components (40), the bit line (24) being operable to provide a bit to each memory component (40) in the row of memory components (40); and Word lines (26), each word line (26) being connected to a corresponding column of the memory components (40), the word lines (26) being operable to enable each memory component (40) in the column of memory components (40) so as to write a bit to each memory component (40) in the column of memory components (40).

2. The compact computer-in-memory architecture of claim 1, wherein: Each memory component (40) is connected to a bit line (24) external to the memory component (40) via a memory select (MEMSEL) switch (60) operable to connect the memory component (40) to the bit line (24) internal to the memory component (40) or to isolate the memory component (40) from the bit line (24) external to the memory component (40).

3. The compact computer-in-memory architecture of claim 2, wherein: The memory select (MEMSEL) switch (60) of each memory component (40) is commonly controlled.

4. A compact computer-in-memory architecture according to any one of the preceding claims, wherein: Each memory component (40) includes a plurality of bit cells (20) connected to the computing engine (30), and each bit cell (20) of the plurality of bit cells (20) is connected to a common bit line (24) and a different word line (26).

5. The compact computer-in-memory architecture of claim 4, wherein: Each bit cell (20) in the memory component (40) is directly connected to the computation engine (30).

6. A compact computer-in-memory architecture according to any one of the preceding claims, comprising a controller (70) for controlling the memory component (40).

7. A compact computer-in-memory architecture according to any one of the preceding claims, comprising a substrate, and wherein: Each memory component (40) is spatially disposed on or over a different portion of the substrate and adjacent to another memory component (40).

8. The compact computer-in-memory architecture of claim 7, wherein: The computing engine (30) of each memory component (40) is arranged to be spatially adjacent to the bit cell (20) or multiple bit cells (20) of the memory component (40).

9. The compact computer-in-memory architecture according to any one of claims 7 and 8, wherein: At least one of the computing engines (30) in the memory component (40) is spatially arranged between the bit cell (20) or multiple bit cells (20) and the adjacent bit cell (20) or multiple bit cells (20) of the memory component (40).

10. The compact computer-in-memory architecture according to any one of claims 7, 8 and 9, wherein: Each computing engine (30) of a memory component (40) is connected to the computing engine (30) of an adjacent memory component (40).

11. A compact computer-in-memory architecture according to any one of the preceding claims, wherein: For each memory component (40), the computation engine (30) is connected to the bit cell (20) via the corresponding bit line (24).

12. A compact computer-in-memory architecture according to any one of the preceding claims, wherein: The calculation engine (30) includes a bit multiplier (14) for multiplying the bits stored in the bit cells (20) to calculate a product, and a product storage circuit (16) which is or includes a capacitor (16) for storing the product.

13. A method of operating a compact computer-in-memory architecture according to any one of claims 7 to 10, comprising: using the controller (70) to provide one bit on each bit line (24); Using the controller (70), the word line (26) of the column of memory components (40) is enabled to store the bit in the bit cell (20) of each memory component (40) in the column of memory components (40); as well as The stored bits are processed using the computation engine (30) of each memory component (40) in the column of memory components (40).

14. The method according to claim 13, wherein: Each memory component (40) is connected to a corresponding bit line (24) via a memory select (MEMSEL) switch (60), and includes using the controller (70) to (i) open the MEMSEL switch (60), and thereafter provide the bit on each bit line (24) using the controller (70), and (ii) close the MEMSEL switch (60) after providing the bit on each bit line (24) using the controller (70), and thereafter process the stored bit using the computation engine (30) of each memory component (40) in the column of memory components (40).

15. The method according to claims 13 and 14, comprising: (i) multiplying a plurality of bits of a first multi-bit value in parallel by a bit of a second multi-bit value; (iii) multiplying multiple bits of the first multi-bit value by one bit of the second multi-bit value in parallel; (iv) multiplying all bits of the first multi-bit value by one bit of the second multi-bit value in parallel; (v) multiplying multiple bits of the first multi-bit value by multiple bits of the second multi-bit value in parallel; (vi) multiplying all bits of the first multi-bit value by all bits of the second multi-bit value in parallel; (vii) storing bit products in capacitors (16) and summing the bit products by connecting the capacitors (16) in parallel; Or (viii) iteratively summing and scaling the bit products in the accumulation capacitor (17).

Images