Computing circuit in memory and operating method thereof

Abstract

The embodiment of the invention provides a computing circuit in a memory and an operation method thereof. The in-memory computing circuit includes a local computing unit. In response to identifying the input data element and the weight data element as a first data type, each local computing unit may provide a first sum comprising (i) a first product of the first input data element and the first weight data element; and (ii) a second product of a second input data element and a second weight data element. In response to identifying the input data element and the weighted data element as a second data type, each local computing unit may provide (i) a second sum of a first portion of a third input data element and a second portion of a third weighted data element; and (ii) a third product of a second portion of a third input data element and a second portion of a third weight data element.

Description

Technical Field

[0001] Embodiments of the present application relate to a computing circuit in a memory and an operation method thereof. Background Art

[0002] Computer artificial intelligence (AI) is built on the foundation of machine learning, such as using deep learning techniques. With machine learning, a computing system organized as a neural network calculates the statistical likelihood that input data matches previously computed data. A neural network is a number of interconnected processing nodes that analyze data, comparing inputs to "trained" data. Trained data refers to computational analysis of properties of known data to develop a model to compare input data to. An example of an application of artificial intelligence and data training is in object recognition, where the system analyzes the properties of many (e.g., thousands or more) images to determine patterns that can be used to perform statistical analysis to identify input objects. Summary of the invention

[0003] According to one aspect of an embodiment of the present application, a computation-in-memory (CIM) circuit is provided, comprising: an input circuit configured to receive a first number of input data elements and a first number of weight data elements; a second number of local computation units operably coupled to the input circuit, wherein each local computation unit is configured to: in response to identifying that the input data elements and the weight data elements are provided as a first data type, provide at least a first sum, the first sum comprising (i) a first product of a first input data element among the input data elements and a first weight data element among the weight data elements; and (ii) a second product of a second input data element among the input data elements and a second weight data element among the weight data elements; and in response to identifying that the input data elements and the weight data elements are provided as a second data type, provide (i) a second sum of a first portion of a third input data element among the input data elements and a first portion of a third weight data element among the weight data elements; and (ii) a third product of a second portion of the third input data element and a first portion of the third weight data element.

[0004] According to another aspect of an embodiment of the present application, a computation-in-memory (CIM) circuit comprises: a first local computation unit operably coupled to a first number of input data elements and a first number of weight data elements, the first local computation unit comprising a first part and a second part operably coupled to each other; wherein, in response to identifying that the input data elements and the weight data elements are provided as integer data types, the first part is configured to provide a first sum of the first number of weight data elements, which causes the second part to (i) select the first number of weight data elements, the first sum, or one of a fixed logic state; and (ii) based on the first number of input data elements, provide a plurality of first multiplication-accumulation (MAC) results of the first number of input data elements and the first number of weight data elements; and wherein, in response to identifying that the input data elements and the weight data elements are provided as floating-point data types, the first part is configured to provide a second sum of the exponent parts of the first number of input data elements and the first number of weight data elements, respectively, so that the second part (i) selects the mantissa parts of the first number of weight data elements; and (ii) based on the mantissa parts of the first number of input data elements, provides a plurality of first products of the corresponding mantissa parts of the first number of output data elements and the first number of weight data elements.

[0005] According to another aspect of an embodiment of the present application, a method for operating a compute-in-memory (CIM) circuit comprises: receiving a plurality of input data elements and a plurality of weight data elements through a memory device; in response to identifying that the input data elements and the weight data elements are provided as integer data types, providing at least a first sum, wherein the first sum comprises (i) a first product of a first input data element among the input data elements and a first weight data element among the weight data elements; and (ii) a second product of a second input data element among the input data elements and a second weight data element among the weight data elements; and in response to identifying that the input data elements and the weight data elements are provided as floating-point data types, providing (i) a second sum of an exponent part of a third input data element among the input data elements and an exponent part of a third weight data element among the weight data elements; and (ii) a third product of a second part of the third input data element and a second part of the third weight data element. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Various aspects of the present disclosure can be best understood from the following detailed description when read in conjunction with the accompanying drawings. It should be emphasized that, in accordance with standard practice in the industry, the various components are not drawn to scale and are only used for illustrative purposes. In fact, the size of the various components may be arbitrarily increased or reduced for clarity of discussion.

[0007] Figure 1 A block diagram of a computation-in-memory circuit is shown in accordance with some embodiments.

[0008] Figure 2 According to some embodiments Figure 1 Block diagram of a local computation unit of an in-memory computation circuit.

[0009] Figure 3 According to some embodiments Figure 2 Schematic diagram of the local computing unit when operating with the INT8 data type.

[0010] Figure 4 According to some embodiments Figure 2 Schematic diagram of the local computing unit when operating with the INT4 data type.

[0011] Figure 5 According to some embodiments Figure 2 Schematic diagram of the local computation unit when operating with the BF16 data type.

[0012] Figure 6 According to some embodiments Figure 1 Schematic diagram of a portion of the input circuit of the memory computing circuit when operating using the INT8 data type.

[0013] Figure 7 According to some embodiments Figure 1 Schematic diagram of a portion of the input circuit of a memory computation circuit when operating using the INT4 data type.

[0014] Figure 8 According to some embodiments Figure 1 Schematic diagram of a portion of the input circuitry of an in-memory computation circuit when operating using the BF16 data type.

[0015] Fig. 9 shows a summary according to some embodiments Figure 2 Table of various operating modes of local computing units.

[0016] Fig.10 According to some embodiments Figure 1 Schematic diagram of the portion of the in-memory computation circuit when operating with the INT8 data type.

[0017] Fig.11 According to some embodiments Figure 1 Schematic diagram of the portion of the in-memory computation circuit when operating using the INT4 data type.

[0018] Fig.12 According to some embodiments Figure 1Schematic diagram of the portion of the in-memory computation circuit when operating using the BF16 data type.

[0019] Fig.13 and Fig.14 Together, the following are shown according to some embodiments Figure 1 An example of an in-memory computation circuit operating with FP16 data type.

[0020] Fig.15 is an example flow chart of a method for operating a computing circuit in a memory according to some embodiments. DETAILED DESCRIPTION

[0021] The following disclosure provides many different embodiments or examples for realizing different features of the present disclosure. Specific embodiments or examples of components and arrangements are described below to simplify the present disclosure. Of course, these are only examples and are not intended to be limiting. For example, in the following description, forming a first component above or on a second component may include an embodiment in which the first component and the second component are directly contacted, and may also include an embodiment in which an additional component may be formed between the first component and the second component so that the first component and the second component may not be in direct contact. In addition, the present disclosure may repeat reference numbers and / or letters in various examples. This repetition is for the purpose of simplicity and clarity, and does not itself indicate the relationship between the various embodiments and / or configurations discussed.

[0022] Additionally, for ease of description, spacing relation terms such as "below," "beneath," "lower," "above," "upper," etc. may be used herein to describe the relationship of one element or component to another element or component as shown in the figures. The spacing relation terms are intended to encompass different orientations of the device in use or in the process of operation in addition to the orientation shown in the figures. The device may be otherwise oriented (rotated 90 degrees or at other orientations) and the spacing relation descriptors used herein may likewise be interpreted accordingly.

[0023] Neural networks calculate "weights" to perform calculations on new data (input data "words"). Neural networks use multiple layers of computational nodes, where deeper layers perform calculations based on the results of calculations performed by higher layers. Machine learning currently relies on the calculation of dot products and vector absolute differences, typically calculated by performing multiply-accumulate (MAC) operations on the parameters, input data, and weights. The calculations of large deep neural networks often involve so many data elements that it is impractical to store them in the processor cache. Therefore, these data elements are usually stored in memory.

[0024] As a result, machine learning is very computationally intensive in calculating and comparing many different data elements. Operations within the processor are computed orders of magnitude faster than data element transfers between the processor and main memory resources. Placing all data elements closer to the processor in a cache is prohibitively expensive for the vast majority of practical systems due to the size of the memory required to store the data elements. As a result, the transfer of data elements becomes a major bottleneck for AI computations. As data sets grow, the time and power / energy used by the computing system to move data elements can end up being multiples of the time and power used to actually perform the computations.

[0025] In this regard, computing-in-memory (CIM) circuits or systems have been proposed to perform such MAC operations. Similar to the human brain, CIM circuits perform in-situ data processing within suitable memory circuits. CIM circuits suppress the latency of data / program reading and output results being uploaded to the corresponding memory (e.g., a memory array), thereby solving the memory (or von Neumann) bottleneck of traditional computers. Another key advantage of CIM circuits is high parallelism, thanks to the specific architecture of the memory array, where calculations can be performed along multiple current paths simultaneously. CIM circuits also benefit from the high density of multiple memory arrays with computing devices, which typically have excellent scalability and 3D integration capabilities. As a non-limiting example, CIM circuits for various machine learning applications can perform MAC operations locally within the memory (i.e., without sending data elements to the main processor) to achieve higher throughput of neuron activation matrices and weight matrix dot products, while still providing higher performance and lower energy compared to the main processor's calculations.

[0026] The data elements processed by CIM circuits have various data types or forms, such as integer data types and floating-point data types. Integer data types (each data type represents a series of mathematical integers) can be of different sizes. For example, integer data types have 4 bits (sometimes called INT4 data types), 8 bits (sometimes called INT8 data types), and so on. Floating-point data types are usually represented by a sign portion, an exponent portion, and a significand (mantissa) portion consisting of the significant digits of the number. For example, the Institute of Electrical and Electronics Engineers One floating-point format is specified as 16 bits in size (sometimes referred to as the FP16 data type), which includes 10 mantissa bits, 5 exponent bits, and 1 sign bit. Another floating-point format is also specified as 16 bits in size, sometimes referred to as the BF16 data type, which includes 7 mantissa bits and 8 exponent bits.

[0027] In machine learning applications, CIM circuits are typically configured to process dot product multiplications based on performing MAC operations on a large number of data elements (e.g., input word vectors and weight matrices), which may be of integer data types or floating point data types, and then process addition (or accumulation) of these dot products. However, in the prior art, few CIM circuits are configured to process data elements of integer data types and floating point data types. For example, dedicated hardware circuit components are typically required to process different data types, which adversely reduces hardware utilization. In turn, such CIM circuits may occupy an additional portion of the valuable space of the substrate. Therefore, existing CIM circuits are not entirely satisfactory in some aspects.

[0028] The present disclosure provides various embodiments of a computation-in-memory (CIM) circuit that can be switched between a first mode and a second mode, wherein the first mode is configured to process multiple input data elements and a corresponding number of weight data elements, the multiple input data elements and the corresponding number of weight data elements are provided as integer data types, and the second mode is configured to process multiple output data elements and a corresponding number of weight data elements provided as floating-point data types. For example, a CIM circuit as disclosed herein can perform memory calculations (e.g., multiply-accumulate (MAC) operations) on input data elements and weight data elements. Based on whether the input / weight data elements are provided as integer or floating-point data types, the CIM circuit can use the same hardware components to perform MAC operations. In various embodiments, the disclosed CIM circuit may include multiple multi-mode local computation units (LCCs). Based on the received or identified data type, each LCC can selectively perform MAC operations on a pair of weight data elements and a pair of input data elements (e.g., when each input / weight data element is provided as an INT8 data type), four weight data elements and four input data elements (e.g., when each input / weight data element is provided as an INT4 data type), or a single weight data element and a single input data element (e.g., if each input / weight data element is provided as an FP16 or BF16 data type).

[0029] Figure 1 1 shows a block diagram of a data calculation circuit 100 according to various embodiments of the present disclosure. Figure 1In the illustrated embodiment, the data computation circuit 100, also referred to as the circuit 100 or the memory circuit 100, includes various components that are collectively configured to perform in-memory computations (e.g., multiply-accumulate (MAC) operations) on an input word vector and a weight matrix. The input word vector may include a plurality of (N) input data elements InDE, and the weight matrix may include a plurality of (N) weight data elements WtDE. In some embodiments, each of the input data element InDE and the weight data element WtDE may be configured or provided with an INT8 data type. In some embodiments, each of the input data element InDE and the weight data element WtDE may be configured or provided with an INT4 data type. In some embodiments, each of the input data element InDE and the weight data element WtDE may be configured or provided with an FP16 data type. In some embodiments, each of the input data element InDE and the weight data element WtDE may be configured or provided with a BF16 data type.

[0030] As shown, memory circuit 100 includes memory circuit 102 , input circuit 104 , a plurality of local computation units 106 , and adder circuit (or adder tree) 108 . Figure 1 Each component (e.g., 102 to 108) shown in is an electronic circuit including a logic circuit configured to perform a corresponding function. In some embodiments, the number of local computing units 106 may correspond to the number of input data elements InDE and weight data elements WtDE. For example, the memory circuit 100 may include, receive, obtain, or otherwise process N weight / input data elements WtDE / InDE, and the number of (e.g., enabled) local computing units 106 may be N / 2, N / 4, or N, depending on the data type of the weight / input data element WtDE / InDE provided or identified. It should be understood that Figure 1 The block diagram of the circuits depicted in is simplified, and thus, memory circuit 100 may include any of a variety of other components while remaining within the scope of the present disclosure.

[0031] Memory circuit 102 may include one or more memory arrays and one or more corresponding circuits. Each memory array is a memory device including a plurality of storage elements 103, each storage element 103 including an electrical, electromechanical, electromagnetic or other device configured to store one or more data elements, each data element including one or more data bits represented by a logical state. In some embodiments, the logical state corresponds to a voltage level of a charge stored in part or all of storage element 103. In some embodiments, the logical state corresponds to a physical property of part or all of storage element 103, such as resistance or magnetic orientation.

[0032] In some embodiments, the storage element 103 includes one or more static random access memory (SRAM) cells. In various embodiments, the SRAM cell includes a plurality of transistors, such as a five-transistor (5T) SRAM cell, a six-transistor (6T) SRAM cell, an eight-transistor (8T) SRAM cell, a nine-transistor (9T) SRAM cell, etc. In some embodiments, the storage element 103 includes one or more dynamic random access memory (DRAM) cells, resistive random access memory (RRAM) cells, magnetoresistive random access memory (MRAM) cells, ferroelectric random access memory (FeRAM) cells, NOR flash memory cells, NAND flash memory cells, conductive bridge connection random access memory (CBRAM) cells, data registers, non-volatile memory (NVM) cells, three-dimensional NVM cells, or other memory cell types capable of storing bit data.

[0033] In addition to the memory array, the memory circuit 102 may also include a plurality of circuits to access or otherwise control the memory array. For example, the memory circuit 102 may include a plurality of (e.g., word line) drivers operably coupled to the memory array. The driver may apply a signal (e.g., voltage) to the corresponding storage element 103 to allow access (e.g., programming, reading, etc.) to these storage elements 103. For another example, the memory circuit 102 may include a plurality of programming circuits and / or reading circuits operably coupled to the memory array.

[0034] The memory arrays of the memory circuit 102 are all configured to store a plurality of weight data elements WtDE. In some embodiments, the programming circuit can write the weight data elements WtDE into the corresponding storage elements 103 of the memory array respectively, and the reading circuit can read the bits written into the storage elements 103 to verify or otherwise test whether the written weight data elements WtDE are correct. The driver of the memory circuit 102 may include or be operably coupled to a plurality of input activation latches, which are configured to receive and temporarily store input data elements InDE. In some other embodiments, such input activation latches may be part of the input circuit 104, which may also include a plurality of buffers, which are configured to temporarily store the weight data elements WtDE retrieved from the memory array of the memory circuit 102. Therefore, the input circuit 104 can receive the input data elements InDE and the weight data elements WtDE.

[0035] In some embodiments, the memory circuit 100 is configured to perform a MAC operation on an input word vector (including, for example, an input data element InDE) and a weight matrix (including, for example, a weight data element WtDE) that can be configured as at least any of the following data types: an INT8 data type, an INT4 data type, an FP16 data type, and a BF16 data type. However, it should be understood that in some other embodiments, each of the input data element InDE and the weight data element WtDE can have any of a variety of other integer or floating-point data types, such as an INT16 data type, an FP32 data type, an FP64 data type, an FP128 data type, etc., while still within the scope of the present disclosure.

[0036] When configured as the INT8 data type, each of the input data element InDE and the weight data element WtDE includes 8 bits. When configured as the INT4 data type, each of the input data element InDE and the weight data element WtDE includes 4 bits. When configured as the FP16 data type, each of the input data element InDE and the weight data element WtDE includes 1 sign bit, 5 exponent bits and 10 mantissa bits. When configured as the BF16 data type, each of the input data element InDE and the weight data element WtDE includes 1 sign bit, 8 exponent bits and 7 mantissa bits.

[0037] Still reference Figure 1 , the input circuit 104 is configured to output all of the input data elements InDE and the weight data elements WtDE to the local computing unit 106. When configured as the INT8 data type, the input circuit 104 is configured to output a pair of input data elements InDE and a pair of weight data elements WtDE to a corresponding one of the local computing units 106. When configured as the INT4 data type, the input circuit 104 is configured to output four input data elements InDE and four weight data elements WtDE to a corresponding one of the local computing units 106. When configured as the BF16 or FP16 data type, the input circuit 104 is configured to output a single input data element InDE and a single weight data element WtDE to a corresponding one of the local computing units 106.

[0038] In response to identifying that the input data element InDE and the weight data element WtDE are provided as integer data types (e.g., INT8 data types), each local computing unit 106 can provide a corresponding pair of input data elements InDE and weight data elements WtDE multiplication accumulation (MAC) results. Such a MAC result is the sum of: (i) the product of the first input data element InDE (e.g., IN0) and the first weight data element WtDE (e.g., W0); and (ii) the product of the second input data element InDE (e.g., IN1) and the second weight data element WtDE (e.g., W1).

[0039] The MAC result can be the cumulative sum of multiple partial MAC results, each partial MAC result representing the sum of: (i) the product of the corresponding bits of the first input data element InDE and the first weight data element WtDE (for example, IN0×W0); and (ii) the product of the corresponding bits of the second input data element InDE and the second weight data element WtDE (for example, IN1×W1).

[0040] In addition, each local computing unit 106 may include a first portion and a second portion, which are generally referred to as an adder portion and a multiplexer portion, respectively. In response to identifying the data type of the input / weight data element, the first portion may sum a pair of weight data elements WtDE and provide it to the second portion, so that the second portion calculates a MAC result based on a logical combination of the pair of input data elements InDE, which will refer to Figure 3 Further detailed discussion.

[0041] In response to identifying that the input data elements InDE and the weight data elements WtDE are provided as another integer data type (e.g., INT4 data type), each local calculation unit 106 can provide four MAC results for a group of corresponding four input data elements InDE and weight data elements WtDE. A first MAC result in the MAC results is the sum of: (i) the product of the first input data element InDE (e.g., IN0) and the first weight data element WtDE (e.g., W0); and (ii) the product of the second input data element InDE (e.g., IN2) and the second weight data element WtDE (e.g., W2). A second MAC result in the MAC results is the sum of: (i) the product of the first input data element InDE and the third weight data element WtDE (e.g., W1); and (ii) the product of the second input data element InDE and the fourth weight data element WtDE (e.g., W3). The third MAC result in the MAC result is the sum of: (i) the product of the third input data element InDE (e.g., IN1) and the first weight data element WtDE; and (ii) the product of the fourth input data element InDE (e.g., IN3) and the second weight data element WtDE. The fourth MAC result in the MAC result is the sum of: (i) the product of the third input data element InDE and the third weight data element WtDE; and (ii) the product of the fourth input data element InDE and the fourth weight data element WtDE.

[0042] The MAC result can be the cumulative sum of multiple partial MAC results, each partial MAC result representing the sum of: (i) the product of corresponding bits of the first / third input data element InDE and the first / third weight data element WtDE (for example, IN0×W0, IN0×W1, IN1×W0 and IN1×W1); and (ii) the product of corresponding bits of the second / fourth input data element InDE and the second / fourth weight data element (for example, IN2×W2, IN2×W3, IN3×W2, IN3×W3).

[0043] In addition, each local computing unit 106 may include a first portion and a second portion, which are generally referred to as an adder portion and a multiplexer portion, respectively. In response to identifying the data type of the input / weight data element, the first portion may add the second and fourth weight data elements WtDE, add the first and third weight data elements WtDE, and provide them to the second portion, so that the second portion calculates a MAC result based on the logical combination of the first and second input data elements InDE and the logical combination of the third and fourth input data elements InDE, which will be in Figure 4 This is discussed in further detail in .

[0044] In response to identifying that the input data element InDE and the weight data element WtDE are provided as floating point data types (e.g., BF16 data types), each local computation unit 106 may provide a pair of MAC elements including the corresponding input data element InDE and the corresponding weight data element WtDE. Such a MAC element includes: (i) an exponent portion of the input data element InDE (e.g., INDE); E0 ) and the exponent portion of the weight data element (e.g., W E0 ) and (ii) the mantissa of the input data element InDE (e.g. IN M0 ) and the mantissa of the weight data element WtDE (e.g. W M0 )

[0045] The MAC element (eg, mantissa product) may be the cumulative sum of multiple partial mantissa products, each of which represents the product of the mantissa portion of the input data element InDE and the corresponding bits of the mantissa portion of the weight data element WtDE (eg, INDE). M0 ×W M0 ).

[0046] In addition, each local computation unit 106 may include a first portion and a second portion, generally referred to as an adder portion and a multiplexer portion, respectively. In response to identifying the data type of the input / weight data element, the first portion may provide a sum of exponent portions and the second portion may provide a product of mantissa portions, which will be referred to below. Figure 5 Further detailed discussion.

[0047] The adder tree 108 may receive MAC results / elements from all local computing units 106 and add them together to generate a final MAC result (PS) of the N input data elements InDE and the N weight data elements WtDE. For example, in response to identifying that the data type of the input / weight data element is INT8, the adder tree 108 may sum the N / 2 MAC results provided by the local computing unit 106, respectively, and provide the PS result through one output channel. In another example, in response to identifying that the data type of the input / weight data element is INT4, the adder tree 108 may sum the N / 4 MAC results provided by the local computing unit 106, respectively, and provide the PS result through four output channels. In another example, in response to identifying that the data type of the input / weight data element is BF16, the adder tree 108 may sum the N MAC elements (mantissa products) provided by the local computing unit 106, respectively, and provide the PS result through one output channel.

[0048] Figure 2A block diagram 200 of a local computing unit 106 (hereinafter referred to as “local computing unit 200”) according to various embodiments of the present disclosure is shown. In short, the local computing unit 200 is configured to receive one or more input data elements InDE and a corresponding number of weight data elements WtDE (e.g., from Figure 1 The input circuit 104) and provides one or more MAC results or MAC elements to the adder tree (e.g., 108). It should be understood that Figure 2 The block diagram of the local computing unit 200 depicted in FIG. 2 is simplified, and thus, the local computing unit 200 may include any of a variety of other components while remaining within the scope of the present disclosure.

[0049] As shown, the local computing unit 200 includes a multi-mode data selector 210, a configurable adder 220, and a plurality of multiplexers (MUX) 230. In various embodiments of the present disclosure, regardless of the data types of the received input data elements InDE and weight data elements WtDE, the local computing unit 200 can use the same hardware components (e.g., 210-230) to process the corresponding input data elements InDE and weight data elements WtDE and provide MAC results / elements. For example, based on the identified data type, components 210 to 230 can respond differently (or operate in different modes) to provide corresponding outputs. Therefore, each hardware component of the local computing unit 200 will be introduced as follows and will be described in detail in the following. Figure 3 , Figure 4 and Figure 5 The operations in different modes are further described in detail.

[0050] In some embodiments, the local computing unit 200 can process the MAC operation of 16-bit data elements at a time (e.g., each clock cycle or each time period). For example, the local computing unit 200 can perform a MAC operation on 2 input data elements InDE and 2 weight data elements WtDE, each element having 8 bits. In another example, the local computing unit 200 can perform a MAC operation on 4 input data elements InDE and 4 weight data elements WtDE, each element having 4 bits. In another example, the local computing unit 200 can perform a MAC operation on 1 input data element InDE and 1 weight data element WtDE, each element having 16 bits. However, the local computing unit 200 can process other numbers of bits while remaining within the scope of the present disclosure. In addition, the number of multiplexers 230 of each local computing unit 200 can correspond to the number of processing bits. For example, the number of multiplexers 230 can be equal to half of the number of processing bits.

[0051] After receiving the input data element InDE and the weight data element WtDE, the local calculation unit 200 can separate the weight data element WtDE into signal A and signal B. When the data element is in the INT8 data type, signal B and signal A can represent the first weight data element WtDE (e.g., W0) and the second weight data element WTDE (e.g., W1), respectively. In addition, in the example where each data element has 16 bits, signal B can have 8 bits, which can be represented as W0[0:7], and signal A can also have 8 bits, which can be represented as W1[0:7]. When the data element is an INT4 data type, signal B can represent the first and second weight data elements WtDE (e.g., W0 and W1), and signal A can represent the third and fourth weight data elements WtDE (e.g., W2 and W3). In the same example where each data element has 16 bits, signal B can have a total of 8 bits, which can be represented as W0[0:3] and W1[0:3], and signal A can also have 8 bits, which can also be represented as W2[0:7] and W3[0:3]. When the data element is of the BF16 data type, the signal B and the signal A may respectively represent the mantissa portion of the weight data input WtDE (e.g., W M0 ) and the exponential part (e.g. W E0 ). Still taking the example of each data element having 16 bits, signal B can have 8 bits, which can be represented as WM0[7:0], and signal A can also have 8 bits, which can be represented as WE0[7:0].

[0052] The multi-mode data selector 210 may receive a signal B and a signal C (which represents the exponent portion of the input data element InDE, such as IN E0 ), and selects one of them as its output based on control signal 211. In the 16-bit example, when the data element is provided in the BF16 data type, signal C (e.g., IN E0 ) can also have 8 bits, which can be represented as IN E0 [7:0]. The control signal 211 may be generated based on the data types identifying the input data element InDE and the weight data element WtDE. For example, when the data type is an integer type (e.g., an INT8 data type, an INT4 data type), the control signal 211 may be represented as "INT", which causes the multi-mode data selector 210 to select the signal B; and when the data type is a floating point type (e.g., an FP16 data type, a BF16 data type), the control signal 211 may be represented as "FP", which causes the multi-mode data selector 210 to select the signal C. The multi-mode data selector 210 may provide the selected signal as a D_SEL signal (e.g., signal B or C) to the configurable adder 220. Continuing with the 16-bit example, the multi-mode data selector 210 may include an 8-bit 2-to-1 multiplexer.

[0053] The configurable adder 220 can add the signal A and the D_SEL signal and output the result as the signal SUM. Continuing the 16-bit example, the signal SUM can have 10 bits. In some embodiments, the configurable adder 220 can have multiple (e.g., 8) full adders, which can be configured differently based on the control signal 221. The control signal 221 can be generated based on the data type identifying the input data element InDE and the weight data element WtDE. For example, when the data element is identified as the INT8 data type, the control signal 221 can be represented as "8b", which causes all 8 full adders to add the 8-bit signal A (e.g., W1[7:0]) and the 8-bit D_SEL signal (e.g., W0[7:0]]. Therefore, the signal SUM can represent W0[7:0]+W1[7:0]. In another example, when the data element is identified as the INT4 data type, the control signal 221 may be indicated as "4b", which causes the first four of the eight full adders to add the first half of the 8-bit signal A (e.g., W2[3:0]) and the first half of the 8-bit D_SEL signal (e.g., W0[3:0]), and causes the last four of the eight full adders to add the last half of the 8-bit signal A (e.g., W3[3:0]) and the last half of the 8-bit D_SEL signal (e.g., W1[3:0]). Thus, the signal SUM may represent W0[3:0]+W2[3:0] and W1[3:0]+W3[3:0]. In another example, when the data element is identified as the BF16 data type, the control signal 221 may be indicated as "X", which causes all eight full adders to add the 8-bit signal A (e.g., W0[3:0]). E0 [7:0]) and an 8-bit D_SEL signal (for example, IN E0 [7:0] are added. Therefore, the signal SUM can represent W E0 [7:0]+IN E0 [7:0], sometimes also called the exponential sum.

[0054] Each multiplexer 230 can select one of signal A, signal B, signal SUM or a fixed voltage (e.g., VSS / ground) based on the corresponding bit of the input data element InDE. Such a bit for controlling the multiplexer 230 may sometimes be referred to as a MUX control bit. In some embodiments, each multiplexer 230 is configured to receive 2 MUX control bits, at least one of which corresponds to the corresponding input data element InDE or the mantissa portion of the corresponding input data element InDE. Based on the MUX control bits, each of the multiplexers 230 can provide an output signal. For example, based on different logical combinations of the 2 MUX control bits, each multiplexer 230 can provide a corresponding output signal, which is a logically processed version of signal A, signal B, signal SUM or VSS. The term "logically processed version" may refer to a signal whose each term / component is multiplied by a corresponding logical value (e.g., 0 or 1).

[0055] In an example where the data element InDE / WtDE is provided as an INT8 data type, the input data element InDE received by the local computing unit 200 may consist of a first input data element (e.g., IN0) and a second input data element (e.g., IN1). The first and second input data elements (e.g., IN0 and IN1) each have 8 bits and may be represented as IN0[7:0] and IN1[7:0], respectively.

[0056] In some embodiments, the two MUX control bits received by each multiplexer 230 may be composed of a corresponding one of the eight bits of IN0 (e.g., IN0[7]) and a corresponding one of the eight bits of IN1 (e.g., IN1[7]). For example, the first of the multiplexers 230 may receive IN0[7] and IN1[7] as its two MUX control bits; the second of the multiplexers 230 may receive IN0[6] and IN1[6] as its two MUX control bits; the third of the multiplexers 230 may receive IN0[5] and IN1[5] as its two MUX control bits; the fourth of the multiplexers 230 may receive IN0[4] and IN1[4] as its two MUX control bits. bits; the fifth of the multiplexers 230 can receive IN0[3] and IN1[3] as its two MUX control bits, respectively; the sixth of the multiplexers 230 can receive IN0[2] and IN1[2] as its two MUX control bits, respectively; the seventh of the multiplexers 230 can receive IN0[1] and IN1[1] as its two MUX control bits, respectively; the eighth of the multiplexers 230 can receive IN0[0] and IN1[0] as its two MUX control bits, respectively.

[0057] After receiving signal A (e.g., W1), signal B (e.g., W0), signal SUM (e.g., W0+W1), and VSS, each multiplexer 230 is configured to select one of the signals and output a signal OUT (e.g., a partial MAC result) by multiplying the selected signal by the MUX control bit. Signal OUT may have 10 bits. For example, each multiplexer 230 is configured to derive a first product by multiplying signal B by a corresponding first MUX control bit (e.g., W0[7:0]×IN0[7]), and derive a second product by multiplying signal A by a corresponding second MUX control bit (e.g., W1[7:0]×IN1[7]), and then add the first product and the second product as signal OUT.

[0058] In other words, each multiplexer 230 can provide a partial MAC result based on the corresponding MUX control bit of the input data element InDE received by the local computing unit 200 and 0 (VSS), signal A, signal B, or signal SUM. Based on this principle, the multiplexer 230 of the local computing unit 200 receiving the input data elements InDE, IN0[7:0], and IN1[7:0] can provide partial MAC results: W0[7:0]×IN0[7]+W1[7:0]×IN1[7], W0[7:0]×IN0[6]+W1[7:0]×IN1[6], W0[7:0]×IN0[5]+W1[7:0]×IN1[5] 5], W0[7:0]×IN0[4]+W1[7:0]×IN1[4], W0[7:0]×IN0[3]+W1[7:0]×IN1[3], W0[7:0]×IN0[2]+W1[7:0]×IN1[2], W0[7:0]×IN0[1]+W1[7:0]×IN1[1], and W0[7:0]×IN0[0]+W1[7:0]×IN1[0].

[0059] In the example where the data element InDE / WtDE is provided as an INT4 data type, the input data element InDE received by the local computing unit 200 may consist of a first input data element (e.g., IN0), a second input data element (e.g., IN1), a third input data element (e.g., IN2), and a fourth input data element (e.g., IN3). The first to fourth input data elements (e.g., IN0, IN1, IN2, and IN3) each have 4 bits and may be represented as IN0[3:0], IN1[3:0], IN2[3:0], and IN3[3:0], respectively.

[0060] In some embodiments, the multiplexers 230 may be grouped into multiple pairs, each pair corresponding to respective bits of the first to fourth input data elements IN0, IN1, IN2, and IN3. Thus, the two MUX control bits received by the first of the first multiplexer pairs 230 may be composed of one of the four bits of IN0 (e.g., IN0[3]) and one of the four bits of IN2 (e.g., IN2[3]), respectively; and the two MUX control bits received by the second of the first multiplexer pairs 230 may be composed of one of the four bits of IN1 (e.g., IN1[3]) and one of the four bits of IN3 (e.g., IN3[3]), respectively. Similarly, the two MUX control bits received by the first of the second multiplexer pairs 230 may be composed of one of the four bits of IN0 (e.g., IN0[2]) and one of the four bits of IN2 (e.g., IN2[2]), respectively; and the two MUX control bits received by the second of the second multiplexer pairs 230 may be composed of one of the four bits of IN1 (e.g., IN1[2]) and one of the four bits of IN3 (e.g., IN3[2]), respectively; and so on. After receiving signal A (e.g., W1 and W3), signal B (e.g., W0 and W2), signal SUM (e.g., W0+W1, W2+W3), and VSS, each multiplexer 230 is configured to select one of these signals and output a signal OUT by multiplying the selected signal by the MUX control bit (e.g., two partial MAC results). Signal OUT may have 10 bits.

[0061] For example, the first of the multiplexer pairs 230 is configured to derive a first product by multiplying the signal B by a corresponding first MUX control bit (e.g., W0[3:0]×IN0[3]), derive a second product by multiplying the signal A by a corresponding second MUX control bit (e.g., W2[3:0]×IN2[3]), and then add the first product and the second product as a first sum (e.g., a first partial MAC result). In addition, the first of the multiplexer pairs 230 is configured to derive a third product by multiplying the signal A by a corresponding first MUX control bit (e.g., W1[3:0]×IN0[3]), derive a fourth product by multiplying the signal B by a corresponding second MUX control bit (e.g., W3[3:0]×IN2[3]), and then add the third product and the fourth product as a second sum (e.g., a second partial MAC result). The first of the multiplexer pairs 230 can provide the first sum and the second sum as a signal OUT. Similarly, the second of the multiplexer pairs 230 can provide a corresponding signal OUT by multiplying the signal A, B, SUM, or VSS by the MUX control bit. Continuing with the same example, the second of the multiplexer pairs 230 can provide a first sum (e.g., W0[3:0]×IN1[3]+W2[3:0]×IN3[3]) and a second sum (e.g., W1[3:0]×IN1[3]+W3[3:0]×IN3[3]) as its signal OUT.

[0062] In other words, each multiplexer 230 may provide a first partial MAC result and a second partial MAC result derived based on the corresponding MUX control bits of two input data elements INDE (e.g., IN0, IN1, IN2, and IN3) received by the local computing unit 200 and 0 (VSS), signal A, signal B, or signal SUM. The multiplexers 230 of each local computing unit 200 may be grouped into a plurality of multiplexer pairs 230. Specifically, one of each multiplexer pair 230 may provide a first pair of partial MAC results based on the input data elements IN0 and IN2, and the other of each multiplexer pair 230 may provide a second pair of partial MAC results based on the input data elements IN1 and IN3. As a non-limiting example, in the multiplexer pair 230, receiving input data elements IN0[3] and IN2[3] as one of its MUX control bits, partial MAC results such as W0[3:0]×IN0[3]+W2[3:0]×IN2[3] and W1[3:0]×IN0[3]+W3[3:0]×IN2[3] can be provided; while in the multiplexer pair 230, receiving input data elements IN1[3] and IN3[3] as the other of its multiplexer pair 230, partial MAC results such as W0[3:0]×IN1[3]+W2[3:0]×IN3[3] and W1[3:0]×IN1[3]+W3[3:0]×IN3[3] can be provided.

[0063] Based on this principle, the multiplexer 230 of the local computing unit 200 receiving the input data elements IN0[2] and IN2[2] can provide partial MAC results, such as W0[3:0]×IN0[2]+W2[3:0]×IN2[2] and W1[3:0]×IN0[2]+W3[3:0]×IN2[2]; and the multiplexer 230 of the local computing unit 200 receiving the input data elements IN1[2] and IN3[2] can provide partial MAC results, such as W0[3:0]×IN1[2]+W2[3:0]×IN3[2] and W1[3:0]×IN1[2]+W3[3:0]×IN3[2]. The multiplexer 230 of the local computing unit 200 receiving the input data elements IN0[1] and IN2[1] may provide partial MAC results, such as W0[3:0]×IN0[1]+W2[3:0]×IN2[1] and W1[3:0]×IN0[1]+W3[3:0]×IN2[1]; and the multiplexer 230 of the local computing unit 200 receiving the input data elements IN1[1] and IN3[1] may provide partial MAC results, such as W0[3:0]×IN1[1]+W2[3:0]×IN3[1] and W1[3:0]×IN1[1]+W3[3:0]×IN3[1]. The multiplexer 230 of the local computing unit 200 receiving the input data elements IN0[0] and IN2[0] may provide partial MAC results, such as W0[3:0]×IN0[0]+W2[3:0]×IN2[0] and W1[3:0]×IN0[0]+W3[3:0]×IN2[0]; and the multiplexer 230 of the local computing unit 200 receiving the input data elements IN1[0] and IN3[0] may provide partial MAC results, such as W0[3:0]×IN1[0]+W2[3:0]×IN3[0] and W1[3:0]×IN1[0]+W3[3:0]×IN3[1].

[0064] In the example where the data element InDE / WtDE is provided as a BF16 data type, the input data element InDE received by the local computing unit 200 may consist of one input data element (e.g., IN0). The input data element (e.g., IN0) may have 16 bits, of which 8 bits represent the mantissa portion of the input data element (e.g., IN0). M0 [7:0]). In addition, the multiplexers 230 of the local computing unit 200 can each receive a corresponding bit of the mantissa portion of the input data element (eg, IN M0 [7:0]) as the first of its two MUX control bits. Each multiplexer 230 can receive VSS as the second of its two MUX control bits. For example, the first of the multiplexers 230 can receive INM0 [7] as one of its two MUX control bits; the second of the multiplexers 230 can receive IN M0 [6] as one of its two MUX control bits; the third of the multiplexers 230 can receive IN M0 [5] as one of its two MUX control bits; the fourth of the multiplexers 230 can receive IN M0 [4] as one of its two MUX control bits; the fifth of the multiplexers 230 can receive IN M0 [3] as one of its two MUX control bits; the sixth of the multiplexers 230 can receive IN M0 [2] as one of its two MUX control bits; the seventh of multiplexer 230 can receive IN M0 [1] as one of its two MUX control bits; and the eighth in multiplexer 230 can receive IN M0 [0] as one of its 2 MUX control bits.

[0065] Upon receiving signal B (e.g. W M0 ) and VSS, each multiplexer 230 is configured to provide a signal OUT (e.g., a MAC element) by multiplying the signal B by the MUX control bit. The signal OUT may have 10 bits. For example, each multiplexer 230 is configured to provide a signal OUT (e.g., a MAC element) by multiplying the signal B by the corresponding first MUX control bit (e.g., W M0 [7:0]×IN M0 [7]) to derive the product, and then output the product as the signal OUT. In other words, each multiplexer 230 can provide a MAC element derived based on the corresponding first MUX control bit and VSS of the input data element InDE received by the local computing unit 200. Based on this principle, the mantissa part IN of the received input data element is M0 The multiplexer 230 of the local computing unit 200 of [7:0] can provide MAC elements, W M0 [7:0]×IN M0 [7] W M0 [7:0]×IN M0 [6] W M0 [7:0]×IN M0 [5] W M0 [7:0]×IN M0 [4] W M0 [7:0]×IN M0 [3] W M0 [7:0]×IN M0 [2] W M0 [7:0]×INM0 [1] and W M0 [7:0]×IN M0 [0] These MAC elements (sometimes called mantissa products) can be obtained by summing their respective exponents (W E0 [7:0]+IN E0 [7:0]) is aligned with the maximum exponent sum in the exponent sum to perform the shift. Next, this shifted MAC element can be added to the exponent of the maximum exponent sum to provide a MAC result.

[0066] First reference Figure 3 , shows that according to some embodiments, when a data element is received or recognized as an INT8 data type, one of the local computation units 106 of the memory circuit 100 (in Figure 2 200). Figure 3 In the illustrative example of , the local computation unit 106 is coupled to the input circuit 104 to receive a pair of input data elements InDE, each having 8 bits, such as IN0[7:0] and IN1[7:0], and receive a pair of weight data elements WtDE, each having 8 bits, such as W0[7:0] and W1[7:0]. Therefore, it should be understood that other local computation units 106 of the memory circuit 100 can each receive a corresponding pair of input data elements InDE, such as IN2[7:0] and IN3[7:0], etc., and a corresponding set of weight data elements WtDE, such as W2[7:0], W3[7:0], etc.

[0067] As shown, the local computing unit 106 can receive W0[7:0] and W1[7:0] as signal B and signal A, respectively. The multi-mode data selector 210 can select signal B and output it as a D_SEL signal (e.g., 8 bits) to the configurable adder 220. The configurable adder 220 can also receive signal A and add signal A and the D_SEL signal to a signal SUM, which can have 10 bits. Each multiplexer 230 can receive signal A, signal B, signal SUM, and VSS. Based on the corresponding MUX control bits provided by the corresponding bits of the first input data element InDE (e.g., IN0[7]) and the corresponding bits of the second input data element InDE (e.g., IN1[7]), the multiplexer 230 can provide a corresponding signal OUT by performing a MAC operation on signal A, signal B, signal SUM, or VSS. In an example where multiplexer 230 receives IN0[7] and IN1[7] as its MUX control bits, multiplexer 230 may provide a signal OUT of W0[7:0]×IN0[7]+W1[7:0]×IN1[7], which may have 10 bits. In some embodiments, multi-mode data selector 210 and configurable adder 220 may operably form a first portion (adder) of local computation unit 106, and multiplexer 230 may operably form a second portion (multiplexer) of local computation unit 206.

[0068] Figure 3 200. As shown, the configurable adder 220 may have eight full adders 220A, 220B, 220C, 220D, 220E, 220F, 220G, and 220H, which are coupled in series with each other, and a multiplexer 220M is coupled between the first half of the full adders 220A-220D and the second half of the full adders 220E-220H. Each full adder 220A-220H may receive and add a corresponding bit of a first signal (e.g., b[0], which is a bit of the D_SEL signal) and a corresponding bit of a second signal (e.g., a[0], which is a bit of the signal A) to output a sum bit (e.g., O0[0], which is a bit of the signal SUM). In addition, each full adder 220A-220H can provide a carry to the next stage on the chain consisting of the full adders 220A-220H and the multiplexer 220M. For example, the full adder 220C can provide a carry to the full adder 220D. In another example, the full adder 220D can provide a carry to the multiplexer 220M. In the INT8 example, the multiplexer 220M can provide the carry provided by the full adder 220D to the next full adder 220E.

[0069] Next reference Figure 4 ,and Figure 3 Similarly, the same schematic diagram of one of the local computation units 106 of the memory circuit 100 is shown, but according to some embodiments, the data element is received or recognized as an INT4 data type. Figure 4 In the illustrative example of , the local computation unit 106 is coupled to the input circuit 104 to receive a set of four input data elements InDE, each having 4 bits, such as IN0[3:0], IN1[3:0], IN2[3:0], and IN3[3:0]; and a set of four weight data elements WtDE, each having 4 bits, such as W0[3:0], W1[3:0], W2[3:0], and W233:0]. Therefore, it should be understood that the other local computation units 106 of the memory circuit 100 can each receive a corresponding set of four input data elements InDE, such as IN4[3:0], IN5[3:0], IN6[3:0], and IN7[3:0], etc., and a corresponding set of four weight data elements WtDE, such as W4[3:0], W5[3:0], W6[3:0], and W7[3:0], etc.

[0070] As shown, the local computing unit 106 can receive W0[3:0] and W2[3:0] as signal B, and W1[3:0] and W3[3:0] as signal A. The multi-mode data selector 210 can select signal B and output it as a D_SEL signal (e.g., 8 bits) to the configurable adder 220. The configurable adder 220 can also receive signal A and add signal A and the D_SEL signal to a signal SUM, which can have 10 bits. In the INT4 example, the multiplexer 220M can be configured to provide VSS to the next full adder 220E. That is, the first half full adders 220A-220D can add W0[3:0] and W2[3:0], and the second half full adders 220E-220H can add W1[3:0] and W3[3:0]. Each multiplexer 230 may receive a signal A, a signal B, a signal SUM, and a VSS. Based on a corresponding MUX control bit provided by a corresponding bit of a first one of four input data elements InDE (e.g., IN0[3]) and a corresponding bit of a second one of four input data elements InDE (e.g., IN2[3]), the multiplexer 230 may provide a corresponding signal OUT by performing a MAC operation on the signal A, the signal B, the signal SUM, or the VSS. In an example where the multiplexer 230 receives IN0[3] and IN2[3] as its MUX control bits, the multiplexer 230 may provide a signal OUT of W0[3:0]×IN0[3]+W2[3:0]×IN2[3], which may have 5 bits, and W1[3:0]×IN0[3]+W3[3:0]×IN2[3], which may also have 5 bits. In some embodiments, the multi-mode data selector 210 and the configurable adder 220 may operably form a first portion (adder) of the local computing unit 106 , and the multiplexer 230 may operably form a second portion (multiplexer) of the local computing unit 206 .

[0071] Then refer to Figure 5 ,and Figure 3 Similarly, the same schematic diagram of one of the local computation units 106 of the memory circuit 100 is shown, but according to some embodiments, the data element is received or recognized as a BF16 data type. Figure 5 In the illustrative example of , the local computing unit 106 is coupled to the input circuit 104 to receive one of the input data elements InDE (which has 16 bits), such as IN0, and another of the weight data elements WtDE (which has 16 bits), such as W0. Therefore, it should be understood that the other local computing units 106 of the memory circuit 100 can each receive a corresponding one of the input data elements InDE, such as IN1, etc., and a corresponding one of the weight data elements WtDE, such as W1, etc.

[0072] As shown, the local computation unit 106 may receive the mantissa portion of the weight data element W0 as a signal A, denoted as W M0 [7:0], and the exponential part of the weight data element W0 as signal B, represented by W E0 [7:0]. The multi-mode data selector 210 can select a signal C (eg, IN0) representing the exponent portion of the input data element IN0. E0 [7:0]), and outputs it as a D_SEL signal (e.g., 8 bits) to the configurable adder 220. The configurable adder 220 may also receive the signal A, and add the signal A and the D_SEL signal to form a signal SUM, which may have 9 bits. The signal SUM may represent the sum of the exponential part of the input data element IN0 and the weight data element W0, which may be represented as IN E0 [7:0]+W E0 [7:0]. Each multiplexer 230 can receive the signal B and VSS. M0 [7]) and VSS provide the corresponding MUX control bit, multiplexer 230 can provide the corresponding signal OUT by performing multiplication on signal B. When multiplexer 130 receives IN M0 [7] and VSS as its MUX control bit, multiplexer 230 can provide signal OUT as W E0 [7:0]×IN M0 [7], which may have 10 bits. In some embodiments, the multi-mode data selector 210 and the configurable adder 220 may operably form a first portion (adder) of the local computing unit 106, and the multiplexer 230 may operably form a second portion (multiplexer) of the local computing unit 206.

[0073] Figure 6 , Figure 7 and Figure 8 is a diagram showing the input circuit 104 ( Figure 1 ) is a schematic diagram of a corresponding configuration of at least a portion of the input circuit 104, the input circuit 104 being configured to store or provide a plurality of weight data elements WtDE. As a brief overview, in Figure 6 In, the input circuit 104 can be configured to provide a pair of weight data elements WtDE, each of which has 8 bits (for example, when the data elements are provided in the INT8 data type); Figure 7 In, the input circuit 104 can be configured to provide a set of four weight data elements WtDE, each of which has 4 bits (for example, when the data elements are provided in the INT4 data type); Figure 8, the input circuit 104 may be configured to provide a weight data element WtDE having 16 bits (eg, when the data element is provided in a BF16 data type).

[0074] exist Figure 6-Figure 8 In each of the memory circuits 102, the input circuit 104 may have 16 storage units (e.g., latches) 120, which may be respectively coupled to different columns included in the memory circuit 102. For example, the storage units 120 are respectively coupled to the columns COL0, COL1, COL2, COL3, COL4, COL5, COL6, COL7, COL8, COL9, COL10, COL11, COL12, COL13, COL14, COL15, COL16, COL17, COL18, COL19, COL20, COL21, COL22, COL23, COL24, COL25, COL26, COL27, COL28, COL29, COL30, COL31, COL32, COL33, COL34, COL35, COL36, COL37, C 10 , COL 11 , COL 12 , COL 13 , COL 14 and COL 15 As a non-limiting example, each storage unit 120 may be operably coupled to a bit line and / or an inverted bit line of a corresponding column. The storage units 120 may each be configured to store one bit of a weight data element WtDE read from a storage element 103 disposed in a corresponding column.

[0075] exist Figure 6 In the embodiment (wherein data elements are provided in the form of INT8 data type), the storage units 120 coupled to the even columns (e.g., COL0, COL2, COL4, etc.) are configured to store 8 bits (e.g., W0[7:0]) of the first weight data element W0, respectively; and the storage units 120 coupled to the odd columns (e.g., COL1, COL3, COL5, etc.) are configured to store 8 bits (e.g., W1[7:0]) of the second weight data element W1, respectively.

[0076] exist Figure 7 In the embodiment of the present invention, in which the data elements are provided in the INT4 data type, the storage units 120 coupled to the first half of the even columns (e.g., COL0, COL2, COL4, and COL6) are configured to store 4 bits (e.g., W0[3:0]) of the first weight data element W0, respectively; the storage units 120 coupled to the first half of the odd columns (e.g., COL1, COL3, COL5, and COL7) are configured to store 4 bits (e.g., W1[3:0]) of the second weight data element W1, respectively; the storage units 120 coupled to the second half of the even columns (e.g., COL8, COL9, and COL10) are configured to store 4 bits (e.g., W1[3:0]) of the second weight data element W1, respectively; 10 , COL 12 and COL 14 ) are configured to store 4 bits of the third weight data element W2 (eg, W2[3:0]); and the second half of the odd columns (eg, COL9, COL 11 , COL 13 and COL15 )The coupled storage units 120 are configured to store 4 bits (eg, W3[3:0]) of the fourth weight data element W3, respectively.

[0077] exist Figure 8 In the example (wherein the data elements are provided in the BF16 data type), the storage units 120 coupled to the even columns (eg, COL0, COL2, COL4, etc.) are configured to store the weight data elements W, respectively. E0 The 8 bits of the exponent part (e.g. W E0 [7:0]); and the storage units 120 coupled to the odd columns (e.g., COL1, COL3, COL5, etc.) are configured to store weight data elements W respectively. M0 The 8 bits of the mantissa (e.g. W M0 [7:0]).

[0078] Fig. 9 Table 900 is shown of various operating modes of the disclosed local computing unit 106 / 200 according to a summary of some embodiments. For example, table 900 includes three operating modes, corresponding to three data types (e.g., INT8, INT4, and FP16) of the received input data elements InDE and weight data elements WtDE, respectively. In addition, within each operation duration (e.g., one clock cycle), the local computing unit 106 / 200 can process a 16-bit signal, which can include two 8-bit data elements, four 4-bit data elements, or one 16-bit metadata element. However, it should be understood that the local computing unit 106 / 200 can be used to process any of a variety of other suitable data types while still within the scope of the present disclosure.

[0079] As shown, when operating using the INT8 data type, the signal B and signal A received by the local computing unit can include a first 8-bit weight data element W0[7:0] and a second 8-bit weight data element W1[7:0], respectively. Based on the control signal 211 (which indicates that the data type is INT), the multi-mode data selector (e.g., 210) of the local computing unit can output signal B as the signal D_SEL. Based on the control signal 221 (which indicates that the integer type is 8b), the configurable adder (e.g., 220) of the local computing unit can use all of its eight full adders to sum the signal A and the signal D_SEL, thereby providing the signal SUM as W0[7:0]+W1[7:0]. Based on the MUX control bits (MUX0[n] and MUX1[n]), which correspond to one of the 8 bits of the first input data element IN0[7:0] and one of the 8 bits of the second input data element IN0[7:0], respectively, each multiplexer (e.g., 230) of the local computing unit can provide the corresponding signal OUT as 0, signal A, signal B, or signal SUM.

[0080] When operating using the INT4 data type, the signal B received by the local computing unit may include a first 4-bit weight data element W0[3:0] and a second 4-bit weight data element W1[3:0], and the signal A received by the local computing unit may include a third 4-bit weight data element W2[3:0] and a fourth 4-bit weight data element W3[3:0]. Based on the control signal 211 (which indicates that the data type is INT), the multi-mode data selector (e.g., 210) of the local computing unit may output the signal B as the signal D_SEL. Based on the control signal 221 (indicating that the integer type is 4b), the configurable adder (e.g., 220) of the local computing unit may use half of its eight full adders to sum W3[3:0] and W1[3:0], and use the other half of the full adders to sum W2[3:0] and W0[3:0], thereby providing a signal SUM of W2[7:0]+W0[7:0] and W3[7:0]+W1[7:0]. Based on the MUX control bits (MUX0[n] and MUX1[n]), which correspond to one of the 4 bits of the first or third input data elements IN0[7:0] or IN2[7:0], and one of the 4 bits of the second or fourth input data elements IN1[7:0] or IN3[7:0], respectively, each multiplexer (e.g., 230) of the local computing unit can provide the corresponding signal OUT as 0, signal A, signal B, or signal SUM.

[0081] When operating with the BF16 data type, the signal B and the signal A received by the local computing unit may each include a weight data element W E0 [7:0] 8-bit exponent part and weight data element W M0 Based on the control signal 211 (indicating that the data type is FP), the multi-mode data selector (e.g., 210) of the local computing unit can output a signal C as the signal D_SEL, which includes the input data element IN E0 Based on the control signal 221 (indicating that the integer type is "X"), the configurable adder (e.g., 220) of the local computing unit can use all eight full adders to sum the signal A and the signal D_SEL, thereby providing a signal SUM of W. E0 [7:0]+IN E0 [7:0]. Based on the VSS and input data elements IN, respectively M0 By setting one of the 8 bits of the mantissa portion of [7:0] of the MUX control bit (MUX0[n] and MUX1[n]), each multiplexer (e.g., 230) of the local computing unit can provide the corresponding signal OUT as 0 or signal A.

[0082] Fig.10 , Fig.11 and Fig.12 FIG. 1 is a schematic diagram showing a portion of a memory circuit 100 that processes a plurality of input data elements InDE and a plurality of weight data elements WtDE, respectively, according to some embodiments. It should be understood that Figure 10-12 The schematic diagrams are provided for illustrative purposes and do not limit the scope of the present disclosure.

[0083] exist Fig.10 In the example of , the memory circuit 100 is configured to process 128 input data elements InDE ("In") and 128 weight data elements WtDE ("W"), each element having 8 bits (e.g., INT8 data type). According to some embodiments, the memory circuit 100 can configure 64 local computing units (e.g., 106-0, 106-1...106-63) to receive data elements from corresponding input circuits 104, respectively, and process MAC operations on these input data elements InDE and weight data elements WtDE. In other words, the memory circuit 100 can use 64 local computing units to perform MAC operations on 64 groups of data elements, each group of data elements consisting of a first group of 8-bit IN and 8-bit W (e.g., IN0[7:0] and W0[7:0]) and a second group of 8-bit IN and 8-bit W (e.g., IN1[7:0] and W1[7:0]). After the 64 local computing units provide their own MAC results, the adder tree can add these MAC results to provide the final MAC result through one output channel (e.g. ). In some embodiments, the 64 local computing units together with the corresponding input circuits 104 can operably form a CIM array or CIM library.

[0084] exist Fig.11 In the example of , the memory circuit 100 is configured to process 256 input data elements InDE ("IN") and 256 weight data elements WtDE ("W"), each element having 4 bits (e.g., INT4 data type). According to some embodiments, the memory circuit 100 can configure 64 local computing units (e.g., 106-0, 106-1...106-63) to receive data elements from corresponding input circuits 104, respectively, and process MAC operations on these input data elements InDE and weight data elements WtDE. In other words, the memory circuit 100 can use 64 local computing units to perform MAC operations on 64 groups of data elements, each group consisting of a first group of 4-bit IN and 4-bit W (e.g., IN 0,0 [3:0] and W 0,0 [3-0]), the second set of 4 bits IN and 4 bits W (e.g., IN 0,1 [3:0] and W0,1 [3.0]), the third group of 4 bits IN and 4 bits W (e.g., IN 1,0 [3:0] and W 1,0 [3:0]), and a fourth set of 4 bits IN and 4 bits W (e.g., IN 1,1 [3:0] and W 1,1 [3:0]). After the 64 local computation units provide their respective MAC results, the adder tree can add these MAC results to provide the final MAC results through four output channels respectively (for example, ). In some embodiments, the 64 local computing units together with the corresponding input circuits 104 can operably form a CIM array or CIM library.

[0085] exist Fig.12 In the example of , the memory circuit 100 is configured to process 64 input data elements InDE ("IN") and 64 weight data elements WtDE ("W"), each element including an 8-bit mantissa portion and an 8-bit exponent portion (e.g., a BF16 data type). According to some embodiments, the memory circuit 100 may be configured with 64 local computing units (e.g., 106-0, 106-1 ... 106-63) to receive data elements from corresponding input circuits 104, respectively, and process MAC operations on these input data elements InDE and weight data elements WtDE. In other words, the memory circuit 100 may utilize 64 local computing units to perform MAC operations on 64 groups of exponent elements and 64 groups of mantissa elements, each group of exponent elements consisting of an 8-bit weight exponent portion and an 8-bit input exponent portion, and each group of mantissa elements consisting of an 8-bit weight mantissa portion and an 8-bit input mantissa portion. The 64 local computing units provide respective MAC elements (e.g., IN M0 ×W M0 ), the adder tree can sum these MAC elements to provide the final MAC result through an output channel (e.g. ). In some embodiments, the 64 local computing units together with the corresponding input circuits 104 can operably form a CIM array or CIM library.

[0086] Fig.13 A schematic diagram of a portion of a memory circuit 100 that processes a plurality of input data elements InDE and a plurality of weight data elements WtDE according to some embodiments is shown, each element being provided as an FP16 data type. That is, each data element InDE / WtDE includes a 10-bit mantissa portion and a 5-bit exponent portion. Fig.14 Shows Fig.13 Equivalent block diagram of the schematic diagram.

[0087] In some embodiments, the memory circuit 100 may configure three CIM banks 1310, 1320, and 1330 to perform MAC operations on data elements InDE / WtDE provided by the FP16 data type. Fig.13 In the example shown in Figure 1, there are 64 input data elements InDE (IN0, IN1, ... IN 63 ), 64 first weight data elements WtDE (“W0”) provided in FP16 data type, and 64 second weight data elements WtDE (“W1”) provided in FP16 data type. In order to process more than 8 bits (e.g., the mantissa portion of the FP16 data type) using the disclosed memory circuit 100, each first weight data element WtDE is a 64 first weight data element WtDE (“W0”) provided in FP16 data type. M0 The mantissa part of can be divided into the first group (for example, W M0 [11:4]) and the second group (e.g. W M0 [3:0]), each second weight data element W M1 The mantissa of can be divided into the first group (for example, W M1 [11:4]) and a second group (e.g., W M1 [3:0]).

[0088] The CIM library 1310 may have 64 local computing units to calculate the weight of the first weight data element W through an adder tree. M0 The first group of mantissas (e.g. W M0 [11:4]), the first weight data element W M0 The exponential part (e.g. W E0 [7:0]) and the input data elements InDE (e.g. IN0, IN1…IN 63 ) provides a first MAC result 1312. The CIM library 1320 may have 64 local computing units to calculate the first weight data element W based on the addition tree. M0 The second group of mantissas (for example, W M0 [3:0]) and the input data elements InDE (e.g. IN0, IN1…IN 63 ) provides a second MAC result 1322, and based on the second weight data element W M1 The second group of mantissas (for example, W M1 [3:0]) and the input data elements InDE (for example, IN0, IN1, ... IN 63 ) to provide a third MAC result 1324. The CIM library 1330 may have 64 local computing units to calculate the weight of the data element W based on the second weight data element W through the adder tree. M1 The first group of mantissas (e.g. W M1 [11:4]), the second weight data element WM1 The exponential part (e.g. W E1 [7:0]) and the input data elements InDE (e.g. IN0, IN1…IN 63 ) provides a fourth MAC result 1332. In some embodiments, the memory circuit 100 may include a global adder tree to add the MAC results 1312 and 1322 as a MAC result 1350 and provide it through a first output channel, and to add the MAC results 1324 and 1332 as a MAC result 1360 and provide it through a second output channel. Specifically, the configuration of the CIM libraries 1310 and 1330 may be similar to the operation mode when processing the BF16 data type (e.g., Figure 5 and Fig.12 ).

[0089] Fig.15 1 is a flowchart of an example method 1500 for operating a computing circuit in memory according to some embodiments. For example, at least some operations of the method 1500 may be performed to cause the memory circuit 100 ( Figure 1 ) generates multiple MAC results. Therefore, in the following discussion of method 1500, the above figure (e.g. Figure 1-Figure 12 ) are used in the reference numerals. It should be noted that method 1500 is merely an example and is not intended to limit the present disclosure. Therefore, it is understood that in Figure 1 Additional operations may be provided before, during, and after method 1500. And some other operations may only be briefly described here.

[0090] Method 1500 begins with operation 1510, where a plurality of input data elements InDE and a plurality of weight data elements WtDE are received by a memory device. In some embodiments, the input data elements InDE may be received by a memory device (e.g., Figure 1 102) receives, and the weight data element WtDE can be stored in each storage element (e.g., 103) of the memory device. In the example of providing 128 input data elements InDE and 128 weight data elements WtDE, the input data elements InDE can be IN0, IN1, IN2 ... IN 127 , the weight data elements WtDE can be W0, W1, W2…W 127 .

[0091] In various embodiments of the present disclosure, the input data element InDE and the weight data element WtDE may be provided or otherwise identified in various data types, such as an INT8 data type, an INT4 data type, a BF16 data type, an FP data type, etc. After identifying the data types of the input data element InDE and the weight data element WtDE, the memory circuit 100 may configure the same hardware component (e.g., the local computing unit 106) in a corresponding mode to process (e.g., MAC) the data element. For example, in response to identifying that the data type is an integer type (e.g., an INT8 data type, an INT4 data type, etc.), the method 1500 may continue to operation 1520; and in response to identifying that the data type is a floating point type (e.g., a BF16 data type, an FP16 data type, etc.), the method 1500 may continue to operation 1530.

[0092] In operation 1520, each local computing unit 106 may provide at least a first sum of the first product and the second product. In addition, when the integer type is an INT8 data type, the local computing unit 106 may provide a corresponding first sum. In some embodiments (e.g., referring to Figure 3 ), the first product may be one of the input data elements (e.g., IN0) multiplied by a corresponding one of the weight data elements (e.g., W0), and the second product may be another input data element (e.g., IN1) multiplied by a corresponding one of the weight data elements (e.g., W1). When the integer type is an INT4 data type (e.g., reference Figure 4 ), in addition to the first sum (e.g., IN0×W0+IN2×W2), the local computation unit 106 may also provide additional sums (e.g., IN0×W1+IN2×W3, IN1×W0+IN3×W2, and IN1×W1+IN3×W3). When configured in a mode for processing integer data types, such sums may sometimes be referred to as partial MAC results. After each local computation unit 106 provides its respective one or more partial MAC results, the memory circuit 100 may use an adder tree (e.g., Figure 1 108) adds them together to provide the final MAC result (PS).

[0093] In operation 1530, each local computing unit 106 may provide at least the second sum and the third product. Figure 5 ), the second sum may be the exponent portion of one of the input data elements (e.g., IN E0 ) plus the exponent portion of a corresponding one of the weight data elements (e.g., W E0 ), the third product may be the mantissa portion of the input data element (e.g., IN M0 ) multiplied by the mantissa of the weight data element (e.g. INM0 ). When configured in a mode for processing floating point data types, such an exponent sum and mantissa product may sometimes be referred to as a pair of MAC elements. After each local computation unit 106 provides its respective pair of MAC elements (e.g., mantissa product), the memory circuit 100 may use an adder tree (e.g., Figure 1 108) adds them together to provide the final MAC result (PS).

[0094] In one aspect of the present disclosure, a computation in memory (CIM) circuit is disclosed. The CIM circuit includes an input circuit configured to receive a first number of input data elements and a first number of weight data elements. The CIM circuit includes a second number of local computation units operably coupled to the input circuit. Each local computation unit is configured to: in response to identifying that the input data elements and the weight data elements are provided as a first data type, provide at least a first sum, the first sum including (i) a first product of a first input data element in the input data elements and a first weight data element in the weight data elements; and (ii) a second product of a second input data element in the input data elements and a second weight data element in the weight data elements. Each local computation unit is configured to: in response to identifying that the input data elements and the weight data elements are provided as a second data type, provide (i) a second sum of a first portion of a third input data element in the input data elements and a first portion of a third weight data element in the weight data elements; and (ii) a third product of a second portion of a third input data element and a first portion of a third weight data element.

[0095] In some embodiments, the second amount is half, one quarter, or equal to the first amount.

[0096] In some embodiments, the first data type includes one of an INT4 data type or an INT8 data type, and the second data type includes one of an FP16 data type or a BF16 data type.

[0097] In some embodiments, the first portion comprises exponent portions of the third input data element and the third weight data element, respectively, and the second portion comprises mantissa portions of the third input data element and the third weight data element, respectively.

[0098] In some embodiments, each local computation unit is further configured to provide a first sum in response to identifying that the input data element and the weight data element are provided as the first data type and each have 8 bits.

[0099] In some embodiments, each local computation unit is further configured to, in response to identifying that the input data elements and the weight data elements are provided as the first data type and each has 4 bits, provide a first sum and a third sum, the third sum comprising: (i) a fourth product of the fifth input data element among the input data elements and the fifth weight data element among the weight data elements; and (ii) a fifth product of the sixth input data element among the input data elements and the sixth weight data element among the weight data elements.

[0100] In some embodiments, each local computing unit includes: a multi-mode data selector; a configurable adder; and a plurality of multiplexers.

[0101] In some embodiments, the multi-mode selector is configured to: select a first weight data element in response to identifying that the input data element and the weight data element are provided as a first data type; and select a first portion of a third input data element in response to identifying that the input data element and the weight data element are provided as a second data type.

[0102] In some embodiments, the configurable adder is configured to: in response to identifying that the input data element and the weight data element are provided as a first data type, sum at least a first weight data element and a second weight data element; and in response to identifying that the input data element and the weight data element are provided as a second data type, sum a first portion of a third input data element and a first portion of a third weight data element.

[0103] In some embodiments, in response to identifying that the input data elements and the weight data elements are provided as a first data type, each of the plurality of multiplexers is configured to: receive a corresponding bit of the first input data element and a corresponding bit of the second input data element; and provide a first sum result based on a logical combination of the corresponding bit of the first input data element and the corresponding bit of the second input data element.

[0104] In some embodiments, in response to identifying that the input data element and the weight data element are provided as a second data type, each of the plurality of multiplexers is configured to: receive corresponding bits of a second portion of a third input data element; and provide a result of a third product based on a logical combination of the corresponding bits of the second portion of the third input data element and a fixed logic state.

[0105] In another aspect of the present disclosure, a computation-in-memory (CIM) circuit is disclosed. The CIM circuit includes: a first local computation unit operably coupled to a first number of input data elements and a first number of weight data elements, the first local computation unit including a first part and a second part operably coupled to each other. In response to identifying that the input data elements and the weight data elements are provided as integer data types, the first part is configured to provide a first sum of the first number of weight data elements, which causes the second part to (i) select one of the first number of weight data elements, the first sum, or a fixed logic state; and (ii) based on the first number of input data elements, provide multiple first multiplication-accumulation (MAC) results of the first number of input data elements and the first number of weight data elements. In response to identifying that the input data elements and the weight data elements are provided as floating point data types, the first part is configured to provide a second sum of the exponent parts of the first number of input data elements and the first number of weight data elements, so that the second part (i) selects the mantissa parts of the first number of weight data elements; and (ii) based on the mantissa parts of the first number of input data elements, provides multiple first products of the corresponding mantissa parts of the first number of output data elements and the first number of weight data elements.

[0106] In some embodiments, the computing circuit in the memory also includes: a second local computing unit, operably coupled to a second number of input data elements and a second number of weight data elements, the second local computing unit including a third part and a fourth part operably coupled to each other; wherein, in response to identifying that the input data elements and the weight data elements are provided as integer data types, the third part is configured to provide a third sum of the second number of weight data elements, so that the fourth part (i) selects the second number of weight data elements, the third sum or one of the fixed logic states; and (ii) based on the second number of input data elements, provides multiple second MAC results of the second number of input data elements and the second number of weight data elements; and wherein, in response to identifying that the input data elements and the weight data elements are provided as floating-point data types, the fourth part is configured to provide a fourth sum of the exponent parts of the second number of input data elements and the second number of weight data elements, so that the fourth part (i) selects the mantissa part of the second number of weight data elements; and (ii) based on the mantissa part of the second number of input data elements, provides multiple second products of the mantissa parts of the second number of input data elements and the second number of weight data elements.

[0107] In some embodiments, the first number is equal to the second number, and the second number is equal to 2 or 4 when the input data elements and the weight data elements are provided as integer data types.

[0108] In some embodiments, the first number is equal to the second number, and the second number is equal to 1 when the input data elements and the weight data elements are provided as floating point data types.

[0109] In some embodiments, the first portion includes a multi-mode data selector and a configurable adder, and the second portion includes a plurality of multiplexers.

[0110] In some embodiments, the multi-mode data selector is configured to: in response to identifying that the input data elements and the weight data elements are provided as integer data types, select one of the first number of weight data elements to provide to a configurable adder to generate a first sum; and in response to identifying that the input data elements and the weight data elements are provided as floating-point data types, select an exponent portion of the first number of input data elements and provide the exponent portion of the first number of input data elements to the configurable adder to generate a second sum.

[0111] In some embodiments, each of the plurality of multiplexers is configured to: receive corresponding bits of the first number of input data elements; and provide a respective one of the first MAC results based on the received bits.

[0112] In some embodiments, each of the plurality of multiplexers is configured to: receive a corresponding bit of a mantissa portion of the first number of input data elements; and provide a respective one of the first products based on the corresponding bit.

[0113] In another aspect of the present disclosure, a method for operating a computation in memory (CIM) circuit is disclosed. The method includes receiving a plurality of input data elements and a plurality of weight data elements through a memory device. The method includes: in response to identifying that the input data elements and the weight data elements are provided as integer data types, providing at least a first sum, wherein the first sum includes (i) a first product of a first input data element in the input data elements and a first weight data element in the weight data elements; and (ii) a second product of a second input data element in the input data elements and a second weight data element in the weight data elements. The method includes: in response to identifying that the input data elements and the weight data elements are provided as floating point data types, providing (i) a second sum of an exponent portion of a third input data element in the input data elements and an exponent portion of a third weight data element in the weight data elements; and (ii) a third product of a second portion of the third input data element and a second portion of the third weight data element.

[0114] In some embodiments, the integer data type includes one of an INT4 data type or an INT8 data type, and the floating point data type includes one of an FP16 data type or a BF16 data type.

[0115] As used herein, the terms "about" and "approximately" generally refer to a value of a given quantity that may vary depending on a particular technology node associated with the subject semiconductor device. Based on the particular technology node, the term "about" may refer to a value of a given quantity that varies, for example, within a range of 10-30% of a value (e.g., +10%, ±20%, or ±30% of a value).

[0116] The features of several embodiments are summarized above so that those skilled in the art can better understand the various aspects of the present disclosure. Those skilled in the art will appreciate that they can easily use the present disclosure as a basis for designing or modifying other processes and structures for achieving the same purpose of the embodiments introduced herein and / or achieving the same advantages thereof. Those skilled in the art will also appreciate that such equivalent structures do not deviate from the spirit and scope of the present disclosure, and that they can make various changes, substitutions, and changes in the present disclosure without departing from the spirit and scope of the present disclosure.

Claims

1. A computing circuit in memory, comprising: input circuitry configured to receive a first number of input data elements and a first number of weight data elements; A second number of local computation units operably coupled to the input circuit, wherein each of the local computation units is configured to: In response to identifying that the input data elements and weight data elements are provided as a first data type, providing at least a first sum, the first sum comprising (i) a first product of a first input data element of the input data elements and a first weight data element of the weight data elements; and (ii) a second product of a second one of the input data elements and a second one of the weight data elements; as well as In response to identifying that the input data elements and the weight data elements are provided as a second data type, providing (i) a second sum of a first portion of a third input data element among the input data elements and a first portion of a third weight data element among the weight data elements; and (ii) a third product of a second portion of the third input data element and a first portion of the third weight data element.

2. The computing-in-memory circuit according to claim 1, wherein: Each of the local computing units is further configured to: In response to identifying that the input data element and the weight data element are provided as the first data type and each have 8 bits, the first sum is provided.

3. The computing-in-memory circuit according to claim 1, wherein: Each of the local computing units is further configured to: In response to identifying that the input data elements and the weight data elements are provided as the first data type and each has 4 bits, the first sum and the third sum are provided, the third sum including: (i) a fourth product of the fifth input data element among the input data elements and the fifth weight data element among the weight data elements; and (ii) a fifth product of the sixth input data element among the input data elements and the sixth weight data element among the weight data elements.

4. The computing-in-memory circuit according to claim 1, wherein: Each of the local computing units comprises: Multi-mode data selector; Configurable adder; and Multiple multiplexers.

5. The computing-in-memory circuit according to claim 4, wherein: The multi-mode data selector is configured to: In response to identifying that the input data element and the weight data element are provided as the first data type, selecting the first weight data element; as well as In response to identifying that the input data element and the weight data element are provided as the second data type, the first portion of the third input data element is selected.

6. The computing-in-memory circuit of claim 4, wherein: The configurable adder is configured as: In response to identifying that the input data element and the weight data element are provided as the first data type, summing at least the first weight data element and the second weight data element; as well as In response to identifying that the input data element and the weight data element are provided as the second data type, the first portion of the third input data element and the first portion of the third weight data element are summed.

7. The computing-in-memory circuit of claim 4, wherein: In response to identifying that the input data element and the weight data element are provided as the first data type, each of the plurality of multiplexers is configured to: receiving corresponding bits of the first input data element and corresponding bits of the second input data element; and A first sum result is provided based on a logical combination of corresponding bits of the first input data element and corresponding bits of the second input data element.

8. A computing circuit in memory, comprising: a first local computation unit operatively coupled to a first number of input data elements and the first number of weight data elements, the first local computation unit comprising a first portion and a second portion operatively coupled to each other; wherein, in response to identifying that the input data elements and the weight data elements are provided as integer data types, the first portion is configured to provide a first sum of the first number of weight data elements, which causes the second portion to (i) select one of the first number of weight data elements, the first sum, or a fixed logic state; and (ii) provide a plurality of first multiplication-accumulation results of the first number of input data elements and the first number of weight data elements based on the first number of input data elements; and Wherein, in response to identifying that the input data elements and the weight data elements are provided as floating point data types, the first part is configured to provide a second sum of the exponent parts of the first number of input data elements and the first number of weight data elements, so that the second part (i) selects the mantissa parts of the first number of weight data elements; and (ii) based on the mantissa parts of the first number of input data elements, provides multiple first products of the corresponding mantissa parts of the first number of output data elements and the first number of weight data elements.

9. The in-memory computing circuit of claim 8, further comprising: a second local computation unit operatively coupled to the second number of input data elements and the second number of weight data elements, the second local computation unit comprising a third portion and a fourth portion operatively coupled to each other; wherein, in response to identifying that the input data elements and the weight data elements are provided as integer data types, the third portion is configured to provide a third sum of the second number of weight data elements, so that the fourth portion (i) selects one of the second number of weight data elements, the third sum, or the fixed logic state; and (ii) based on the second number of input data elements, provides a plurality of second multiplication-accumulation results of the second number of input data elements and the second number of weight data elements; and Wherein, in response to identifying that the input data elements and the weight data elements are provided as floating point data types, the fourth part is configured to provide a fourth sum of the exponent parts of the second number of input data elements and the second number of weight data elements, so that the fourth part (i) selects the mantissa parts of the second number of weight data elements; and (ii) based on the mantissa parts of the second number of input data elements, provides multiple second products of the mantissa parts of the second number of input data elements and the second number of weight data elements.

10. A method for operating a computing circuit in a memory, comprising: receiving, by a memory device, a plurality of input data elements and a plurality of weight data elements; In response to identifying that the input data elements and the weight data elements are provided as integer data types, providing at least a first sum, wherein the first sum comprises (i) a first product of a first input data element of the input data elements and a first weight data element of the weight data elements; and (ii) a second product of a second input data element of the input data elements and a second weight data element of the weight data elements; and In response to identifying that the input data elements and the weight data elements are provided as floating point data types, (i) a second sum of an exponent portion of a third input data element among the input data elements and an exponent portion of a third weight data element among the weight data elements are provided; and (ii) a third product of a second portion of the third input data element and a second portion of the third weight data element are provided.

Images