Computing circuit in memory and operating method thereof

Abstract

A computing in memory (CIM) circuit includes: an input circuit to receive N first and N second inputs; n summing circuits configured to combine the first exponent and the second exponent of one of the N input pairs to generate one of N exponent sums; a selector circuit to select a maximum exponential sum; n subtractor circuits configured to calculate one of N exponent differences, each exponent difference being equal to a difference between one of the N exponent sums and a maximum exponent sum; n comparator circuits configured to: compare the exponent difference with an exponent and a threshold value, and generate one of N control signals based on a result of the comparison; and N multiplier circuits configured to selectively multiply the respective first mantissas by the respective second mantissas of the respective input pairs based on the respective control signals to produce a respective one of the N mantissa products. The embodiment of the invention also discloses a method for operating the computing circuit in the memory.

Description

Technical Field

[0001] Embodiments of the present application relate to a computing circuit in a memory and an operating method thereof. Background Art

[0002] Computer artificial intelligence (AI) is built on the foundation of machine learning, such as using deep learning techniques. With machine learning, a computing system organized as a neural network calculates the statistical likelihood that input data matches previously computed data. A neural network refers to many interconnected processing nodes that enable the analysis of data to compare inputs to "training" data. Training data refers to computational analysis of properties of known data to develop a model to compare input data to. An example of an application of AI and data training is in object recognition, where the system analyzes the properties of many (e.g., thousands or more) images to determine patterns that can be used to perform statistical analysis to identify input objects. Summary of the invention

[0003] According to one aspect of an embodiment of the present application, a computation-in-memory (CIM) circuit is provided, comprising: an input circuit configured to receive: (i) N first inputs and (ii) N second inputs, wherein the first inputs consist of N first symbols, N first exponents, and N first mantissas, and the second inputs consist of N second symbols, N second exponents, and N second mantissas, and wherein each of the second inputs and a corresponding one of the N first inputs form one of the N input pairs; N summing circuits, each of the N summing circuits configured to combine a corresponding first exponent and a corresponding second exponent of a corresponding one of the N input pairs to generate a corresponding one of the N exponent sums; and a selector circuit configured to select one of the N exponent sums from the N exponent sums. The invention relates to a method for generating a plurality of exponential sums of the input signals of the present invention. The method comprises the steps of: selecting a maximum exponent sum among the plurality of exponential sums of the input signals and selecting a maximum exponent sum as the maximum exponent sum; N subtractor circuits, each of the N subtractor circuits being configured to calculate a corresponding exponent difference among the N exponent differences, each of the N exponent differences being equal to a difference between a corresponding one of the N exponent sums and the maximum exponent sum; N comparator circuits, each of the N comparator circuits being configured to: (i) compare the corresponding exponent difference with an exponent sum threshold, and (ii) generate a corresponding one of N control signals based on a comparison between the corresponding exponent difference and the exponent sum threshold; and N multiplier circuits, each of the N multiplier circuits being configured to selectively multiply a corresponding first mantissa of a corresponding input pair by a corresponding second mantissa based on a corresponding control signal, thereby generating a corresponding one of N mantissa products.

[0004] According to another aspect of an embodiment of the present application, a computation-in-memory (CIM) circuit is provided, comprising: an input circuit configured to obtain a first input and a second input, wherein the first input comprises a first exponent part and a first mantissa part, and the second input comprises a second exponent part and a second mantissa part; a first summing circuit configured to generate a first exponent sum by summing the first exponent part and the second exponent part; a subtractor circuit configured to calculate an exponent difference by subtracting the first exponent sum from a maximum exponent sum; a comparator circuit configured to generate a control signal based on comparing the exponent difference with an exponent sum threshold; and a multiplier circuit configured to (i) receive a control signal, and (ii) multiply the first mantissa part by the second mantissa part, wherein one of the first mantissa part or the second mantissa is selectively selected to be zero based on the control signal.

[0005] According to another aspect of an embodiment of the present application, a method for operating a computation circuit in memory is provided, comprising: obtaining a first input and a second input through a computation in memory (CIM) circuit, wherein the first input includes a first exponent part and a first mantissa part, and the second input includes a second exponent part and a second mantissa part; generating a first exponent sum by adding the first exponent part and the second exponent part by the CIM circuit; calculating an exponent difference by subtracting the first exponent sum from a maximum exponent sum by the CIM circuit; generating a control signal by the CIM circuit based on comparing the exponent difference with an exponent sum threshold; and selectively multiplying the first mantissa part by the second mantissa part by the CIM circuit based on the corresponding control signal to generate a mantissa product. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Various aspects of the present disclosure can be best understood from the following detailed description when read in conjunction with the accompanying drawings. It should be emphasized that, in accordance with standard practice in the industry, the various components are not drawn to scale and are only used for illustrative purposes. In fact, the size of the various components may be arbitrarily increased or reduced for clarity of discussion.

[0007] Figure 1 is a block diagram of a data computation circuit for performing MAC operations on floating point numbers with reduced computational resources according to some embodiments.

[0008] Figure 2 is a method for masking the input mantissa according to some embodiments Figure 1 A block diagram of a portion of a data computing circuit.

[0009] Figure 3 According to some embodiments Figure 1 The timing diagram of the data calculation circuit performing MAC operation on floating-point numbers.

[0010] Figure 4is used to mask the weight mantissa according to some embodiments Figure 1 A block diagram of a portion of a data computing circuit.

[0011] Figure 5 is used to mask the multiplier output according to some embodiments Figure 1 A block diagram of a portion of a data computing circuit.

[0012] Figure 6 According to some embodiments Figure 1 A block diagram of a portion of a data calculation circuit for directly outputting the result of a multiplier.

[0013] Figure 7 is a weight mantissa for masking a similarity circuit including a weight exponent according to some embodiments Figure 1 A block diagram of a portion of a data computing circuit.

[0014] Figure 8 is used according to some embodiments to determine whether the index is the same Figure 7 Block diagram of the similarity circuit.

[0015] Fig. 9 is a weight mantissa for masking a similarity circuit including a similarity circuit for input exponents according to some embodiments. Figure 1 A block diagram of a portion of a data computing circuit.

[0016] Fig.10 A flow chart is shown of an example method of performing MAC operations on floating point numbers with reduced computational resources in accordance with various embodiments. DETAILED DESCRIPTION

[0017] The following disclosure provides many different embodiments or examples for realizing different features of the present disclosure. Specific embodiments or examples of components and arrangements are described below to simplify the present disclosure. Of course, these are only examples and are not intended to be limiting. For example, in the following description, forming a first component above or on a second component may include an embodiment in which the first component and the second component are directly contacted, and may also include an embodiment in which an additional component may be formed between the first component and the second component so that the first component and the second component may not be in direct contact. In addition, the present disclosure may repeat reference numbers and / or letters in various examples. This repetition is for the purpose of simplicity and clarity, and does not itself indicate the relationship between the various embodiments and / or configurations discussed.

[0018] Additionally, for ease of description, spacing relation terms such as "below," "beneath," "lower," "above," "upper," etc. may be used herein to describe the relationship of one element or component to another element or component as shown in the figures. The spacing relation terms are intended to encompass different orientations of the device in use or in the process of operation in addition to the orientation shown in the figures. The device may be otherwise oriented (rotated 90 degrees or at other orientations) and the spacing relation descriptors used herein may likewise be interpreted accordingly.

[0019] Neural networks calculate "weights" to perform calculations on new data (input data "words"). Neural networks use multiple layers of computational nodes, where deeper layers perform calculations based on the results of calculations performed by higher layers. Machine learning currently relies on the calculation of dot products and absolute differences of vectors, typically calculated by performing multiply-accumulate (MAC) operations on parameters, input data, and weights. The calculations of large deep neural networks typically involve so many data elements that it is impractical to store them in processor cache. Therefore, these data elements are usually stored in memory.

[0020] As a result, machine learning is extremely intensive in computing and comparing many different data elements. The computation of operations within the processor is orders of magnitude faster than the transfer of data elements between the processor and main memory resources. Placing all data elements in cache memory closer to the processor is prohibitively expensive for the vast majority of practical systems due to the size of the memory required to store the data elements. As a result, the transfer of data elements becomes a major bottleneck for AI computations. As data sets grow, the time and power / energy used by the computing system to move data elements can end up being a multiple of the time and power used to actually perform the computations.

[0021] In this regard, computation-in-memory (CIM) circuits have been proposed to perform such MAC operations. Similar to the human brain, CIM circuits perform in-situ data processing within suitable memory circuits. CIM circuits suppress the latency of data / program reading and output results being uploaded to the corresponding memory (e.g., a memory array), thereby solving the memory (or von Neumann) bottleneck of conventional computers. Another key advantage of CIM circuits is high computational parallelism, thanks to the specific architecture of the memory array, where computations can be performed simultaneously along multiple current paths. CIM circuits also benefit from the high density of multiple memory arrays with computing devices, which typically have excellent scalability and 3D integration capabilities. As a non-limiting example, CIM circuits for various machine learning applications can perform MAC operations locally within the memory (i.e., without sending data elements to the main processor) to achieve higher throughput dot products of neuron activations and weight matrices, while still providing higher performance and lower energy compared to the main processor's computations.

[0022] The data elements processed by CIM circuits are of various types or forms, such as integers and floating-point numbers. Floating-point numbers are usually represented by a sign portion, an exponent portion, and a significand (mantissa) portion consisting of the significant digits of the number. For example, the Institute of Electrical and Electronics Engineers The specified floating-point format size is 32 bits, including 23 mantissa bits, 8 exponent bits, and one sign bit. The other floating-point format size is 16 bits, including 10 mantissa bits, 5 exponent bits, and one sign bit.

[0023] In machine learning applications, CIM circuits are often configured to process dot product multiplications based on performing MAC operations on a large number of data elements (e.g., input word vectors and weight matrices), all of which can be in the form of floating-point numbers, and then process the addition (or accumulation) of these dot products. The multiplication of each floating-point pair typically includes adding the corresponding exponent parts (generating an exponent sum) and multiplying the corresponding mantissa parts (generating a mantissa product). In addition, the exponent sum of each floating-point pair is compared with the maximum exponent sum among multiple floating-point pairs to produce an exponent difference. This exponent difference is used to align the exponent parts of different floating-point pairs, thereby shifting the corresponding mantissa products. The shifted mantissa products are summed, and their exponents take the maximum exponent sum to obtain the final sum.

[0024] In this way, multiplication of floating point numbers (in MAC operations) can be performed without considering the difference in the size of the input values ​​(e.g., floating point numbers). In other words, some circuits can perform calculations even if there is at least one relatively small value / number (e.g., a floating point number with a relatively small exponent number) in the multiplication process of the MAC operation. These calculations can be performed without considering the size (or exponent number) of the input value. However, in the case of floating-point MAC operations, some input value pairs may be small enough compared to other input pairs (e.g., relatively small exponent values ​​compared to the maximum or highest exponent values ​​of various input value pairs), so these input value pairs can be ignored. In this case, during the accumulation process, adding a tiny value (e.g., an input pair with a relatively small exponent value) to other values ​​(e.g., an output pair with a relatively high exponent value) may have a negligible effect on the overall size of the other values. Therefore, performing multiplication using the original input values ​​(e.g., the mantissa portion of the input data and the weight data) may waste computing resources because the effect on the result of the MAC operation (e.g., the result of the accumulation process) is negligible.

[0025] For example, a circuit may perform a MAC operation on a floating point number, including input value pairs (e.g., including input data and weight data) for multiplication and accumulation. The input data and weight data may include respective mantissa portions and exponent portions, wherein the size of the input data or weight data may be based on the exponent portion. If at least one of the input data or weight data is relatively small compared to other input values, the result of the multiplication process of the corresponding input value pair may be small enough (e.g., the value is approximately zero compared to the results of other multiplication pairs) so as not to affect the accumulation process of the MAC operation. Therefore, calculating various floating point numbers without considering their size may result in an increase or excessive consumption of computing resources, increase the number of clock cycles to perform MAC operations, and / or reduce computing efficiency.

[0026] The present disclosure provides various embodiments of a computation in memory (CIM) circuit that can determine whether to apply a mask during a multiplication process. The disclosed CIM circuit may include a component or assembly for detecting whether an input value is small, thereby taking precautions for the multiplier to reduce the calculation / computational resources / power consumption of a MAC operation. In one aspect, the disclosed CIM circuit may mask at least one of the input of the multiplier or the output of the multiplier according to the difference between the exponent of each input value pair and the maximum exponent. Masking the input or output may include changing at least one of the input values ​​to zero or applying zero to the multiplication output according to the exponent difference. Considering the zero product property (e.g., multiplying zero by any number will result in zero), the multiplication calculation can be minimized. On the other hand, the disclosed CIM circuit may directly output a predetermined value (e.g., zero) according to the exponent difference of each input value pair. By applying a (zero) mask or directly outputting zero as the result of multiplying input value pairs with relatively small exponents, computing resources can be reduced when performing MAC operations of floating point numbers, energy efficiency can be improved, and computing delays can be minimized.

[0027] Figure 1 FIG. 1 is a block diagram of a data calculation circuit 100 according to some embodiments of the present disclosure. Figure 1 In the illustrated embodiment, the data computation circuit 100, also referred to as (e.g., CIM) circuit 100 or memory circuit 100, includes various components that are collectively configured to perform in-memory computations (e.g., multiply-accumulate (MAC) operations) on input word vectors and weight matrices. The input word vector may include a plurality (N) of input data elements InDE, and the weight matrix may include a plurality (Nd) of weight data elements WtDE. In various embodiments, each of the input data element InDE and the weight data element WtDE may include a floating point number.

[0028] As shown, circuit 100 includes memory circuit 102, input circuit 104, multiple multiplier circuits 106, multiple summing circuits 108, difference circuit 110 (e.g., sometimes referred to as subtractor circuit 110), shift circuit 112, adder circuit (or adder tree) 114, first converter 116, second converter 118, and comparator circuit 120 (e.g., sometimes referred to as masking circuit 120). In some embodiments, the number of multiplier circuits 106 may correspond to the number of summing circuits 108 or the number of comparator circuits 120. For example, circuit 100 may include N (weight / number of input data elements WtDE / InDE) multiplier circuits 106, N (weight / number of input data elements WtD / InDE) summing circuits 108, and N (weight / number of input data elements WtDE / InDE) comparator circuits 120. It should be understood that Figure 1 The block diagram of the circuit depicted in is simplified, and thus, circuit 100 may include any of a variety of other components while remaining within the scope of the present disclosure.

[0029] Memory circuit 102 may include one or more memory arrays and one or more corresponding circuits. The memory arrays each include a storage device of a plurality of storage elements 103, each storage element 103 including an electrical, electromechanical, electromagnetic or other device configured to store one or more data elements, each data element including one or more data bits represented by a logical state. In some embodiments, the logical state corresponds to a voltage level of a charge stored in a portion or all of storage element 103. In some embodiments, the logical state corresponds to a physical property of a portion or all of storage element 103, such as resistance or magnetic orientation.

[0030] In some embodiments, the storage element 103 includes one or more static random access memory (SRAM) cells. In various embodiments, the SRAM cell includes a plurality of transistors, such as a five-transistor (5T) SRAM cell, a six-transistor (6T) SRAM cell, an eight-transistor (8T) SRAM cell, a nine-transistor (9T) SRAM cell, etc. In some embodiments, the SRAM cell includes a multi-track SRAM cell. In some embodiments, the length of the SRAM cell is at least twice as large as the width.

[0031] In some embodiments, the storage element 103 includes one or more dynamic random access memory (DRAM) cells, resistive random access memory (RRAM) cells, magnetoresistive random access memory (MRAM) cells, ferroelectric random access memory (FeRAM) cells, NOR (NOR) flash memory, NAND (NAND) flash memory, conductive bridging random access memory (CBRAM) cells, data registers, non-volatile memory (NVM) cells, 3D NVM cells, or other memory cell types capable of storing bit data.

[0032] In addition to the memory array, the memory circuit 102 may also include a plurality of circuits to access or otherwise control the memory array. For example, the memory circuit 102 may include a plurality of (e.g., word line) drivers operably coupled to the memory array. The driver may apply a signal (e.g., voltage) to the corresponding storage element 103 to allow access (e.g., programming, reading, etc.) to these storage elements 103. For another example, the memory circuit 102 may include a plurality of programming circuits and / or reading circuits operably coupled to the memory array.

[0033] The memory arrays of the memory circuit 102 are each configured to store a plurality of weight data elements WtDE. In some embodiments, the programming circuit may write the weight data elements WtDE into the corresponding storage elements 103 of the memory array, respectively, and the reading circuit may read the bits written into the storage elements 103 to verify or otherwise test whether the written weight data elements WdDE are correct. The driver of the memory circuit 102 may include or be operably coupled to a plurality of input activation latches configured to receive and temporarily store input data elements InDE. In some other embodiments, such input activation latches may be part of the input circuit 104, which may also include a plurality of buffers configured to temporarily store the weight data elements WtDE retrieved from the memory array of the memory circuit 102. Therefore, the input circuit 104 may receive the input data elements InDE and the weight data elements WtDE.

[0034] In various embodiments of the present disclosure, the circuit 100 is configured to perform a MAC operation on an input word vector (including, for example, an input data element InDE) and a weight matrix (including, for example, a weight data element WtDE) each including a plurality of floating point numbers. Therefore, each of the data element InDE and the weight data element WtDE includes a sign bit, a plurality of exponent bits, and a plurality of mantissa bits (sometimes referred to as fraction bits).

[0035] For example, each of the data element InDE and the weight data element WtDE has a BF16 format, also referred to in some embodiments as a bfloat format or a brain floating point format, in which the first bit represents the sign of the floating point number, the following eight bits represent the exponent of the floating point number, and the last seven bits represent the mantissa or fraction of the floating point number. Because the mantissa is configured to start with a non-zero value, the last seven bits of each stored data element represent an eight-bit mantissa, whose first, most significant bit (MSB) is equal to 1.

[0036] In some embodiments, each of the data element InDE and the weight data element WtDE has a FP16 format, also referred to as a half-precision format, in which the first bit represents the sign of a floating-point number, the following five bits represent the exponent of a floating-point number, and the last ten bits represent the mantissa or fraction of a floating-point number. In this case, the last ten bits of each stored data element represent eleven mantissas, and its first MSB is equal to 1. In some other embodiments, each of the data element InDE and the weight data element WtDE has a floating-point format other than BF16 or FP16 format, such as another 16-bit format, 32-bit, 64-bit, 128-bit or 256-bit format, or a 40-bit or 80-bit extended precision format. The sign and mantissa of the data element representing a floating-point number are collectively referred to as the signed mantissa of a floating-point number. The MSB of the mantissa is referred to as a hidden bit or a hidden MSB.

[0037] Still reference Figure 1 , input circuit 104 is configured to output the entirety of each data element of data elements InDE and WtDE to each of multiplier circuit 106 and summing circuit 108. In some embodiments, input circuit 104 is configured to output the signed mantissa of each data element to multiplier circuit 106 and output the exponent of each data element to summing circuit 108, which will be described below.

[0038] The multiplier circuits 106 are each an electronic circuit, such as an integrated circuit (IC), configured to receive, for example, a sign bit InS and a mantissa InM (collectively referred to as a signed mantissa InS / InM) of each of the N data elements InDE, and a sign bit WtS and a mantissa WtM (collectively referred to as a signed mantissa WtS / WtM) of each of the N data elements WtDE from the input circuit 104. Each multiplier circuit 106 may also receive a signal (e.g., a control signal) from a corresponding one of the comparator circuits 120 to determine whether to mask at least one of the mantissa InM, the mantissa WtM, the multiplier output, or the corresponding product from the multiplier circuit 106, such as in conjunction with but not limited to Figure 2-Figure 10The summing circuits 108 are each an electronic circuit, such as an IC, configured to receive, for example, from the input circuit 104 an exponent InE of each of the N data elements InDE and an exponent WtE of each of the N data elements WtDE.

[0039] The multiplier circuits 106 may each include one or more data registers (not shown) configured to receive instances of the signed mantissas InS / InM and WtS / WtM. Figure 1 In the illustrated embodiment, the multiplier circuit 106 is configured to receive instances of signed mantissas InS / InM and WtS / WtM corresponding to the data elements InDE and WtDE. In some other embodiments, the multiplier circuit 106 includes one or more data registers configured to receive instances of signed mantissas InS / InM and / or WtS / WtM including a hidden MSB. In some embodiments, the multiplier circuit 106 includes one or more data registers configured to add the hidden MSB to the received instances of signed mantissas InS / InM and / or WtS / WtM.

[0040] The multiplier circuit 106 may include a logic circuit (not shown) configured to reformat each instance of the signed mantissa InS / InM into a two's complement mantissa InTC, also referred to as the reformatted mantissa InTC, and to reformat each instance of the signed mantissa WtS / WtM into a two's complement mantissa WtTC, also referred to as the reformatted mantissa WtTC, in an operation. The reformatted mantissa InTC has the same number of bits as the signed mantissa InS / InM, and the reformatted mantissa WtTC has the same number of bits as the signed mantissa WtS / WtM.

[0041] The multiplier circuit 106 may include one or more logic gates M1 configured to multiply some or all instances of the reformatted mantissa InTC with some or all instances of the reformatted mantissa WtTC in an operation to generate N products, such as P[1] to P[N]. In various embodiments, the one or more logic gates M1 include one or more AND or NOR gates or other circuits suitable for performing part or all of the multiplication operation. The one or more logic gates M1 are configured to generate each of the products P[1] to P[N] in an operation as a two's complement data element whose number of bits is equal to twice the number of bits of the reformatted mantissa InTC and WtTC minus one. The one or more logic gates M1 may be referred to as a multiplier configured to multiply some or all instances of the reformatted mantissa InTC with some or all instances of the reformatted mantissa WtTC. In some cases, the multiplier (e.g., one or more logic gates M1) may receive a signed mantissa InS / InM or a signed mantissa WtS / WtM for multiplication.

[0042] The multiplier circuit 106 is configured to generate a number N of products P[1] to P[N] in an operation. For example, the multiplier circuit 106 may generate a number N of products P[1]-P[N] equal to sixteen. In some other embodiments, the multiplier circuit 106 may generate a number N of products P[1]-P[N] less than or greater than sixteen.

[0043] In some embodiments, for example, those embodiments in which the data elements InDE and WtDE have a BF16 format, the multiplier circuit 106 is configured to generate each of the products P[1]-P[N] having a total of 17 bits based on the signed mantissas InS / InM and WtS / WtM and the reformatted mantissas InTC and WtTC each having a total of 9 bits. In some embodiments, for example, those embodiments in which the data elements InDE and WtDE have a FP16 format, the multiplier circuit 106 is configured to generate each of the products P[1]-P[N] having a total of 23 bits based on the signed mantissas InS / InM and WtS / WtM and the reformatted mantissas InTC and WtTC each having a total of 12 bits. Embodiments in which multiplier circuit 106 is configured to generate each of products P[1]-P[N] having other total numbers of bits based on the signed mantissas InS / InM and WtS / WtM and the reformatted mantissas InTC and WtTC each having other total numbers of bits are also within the scope of the present disclosure.

[0044] Thus, the multiplier circuit 106 is configured to perform multiplication and reformatting operations on the sign and mantissa bits of the input data element InDE and the weight data element WtDE in an operation to generate a two's complement product P[1]-P[N]. The multiplier circuit 106 is configured to output the product P[1]-P[N] to the shift circuit 112 via a data bus (not shown).

[0045] In various implementations, the multiplier circuit 106 may include one or more other components to perform the multiplication (or to simplify the multiplication process). For example, the multiplier circuit 106 may include one or more multiplexers (MUXs), switches, or other types of logic components, such as MUX 122, configured to mask at least one of the inputs or outputs of one or more logic gates M1, such as in combination with but not limited to Figure 2 , Figure 4-Figure 7 and Fig. 9 The multiplier circuit 106 may include other types of logic components configured to perform similar functions as the MUX 122, for example, to select one of the multiple inputs as an output based on a control signal. Figure 2 , Figure 4-Figure 7 and Fig. 9 As described in at least one of the above, MUX 122 may include multiple input ports, such as a first input port, a second input port, and a control port. The first input port may receive a predefined value (e.g., zero, sometimes referred to as a mask value) as a first input of MUX 122. The second input port may receive a value from the input circuit 104 (e.g., mantissa InM or WtM or reformatted mantissa InTc or WtTc) or a value from one or more logic gates M1 (e.g., a corresponding product P[n]) as a second input of MUX 122. The second input may be referred to as an original value, corresponding to a value from the input circuit 104 or one or more logic gates M1. The first input and the second input may be interchangeable. The control port of MUX 122 may receive a control signal (e.g., 0 or 1) from a corresponding comparator circuit 120 communicating with the multiplier circuit 106. According to the control signal, MUX 122 may output zero or an original value.

[0046] In another example, one or more logic gates M1 of the multiplier circuit 106 may be configured to receive a third input in addition to the corresponding reformatted mantissa InTc and the reformatted mantissa WtTC. The third input may include or correspond to a control signal from the corresponding comparator circuit 120, including a value of 0 or 1. The one or more logic gates M1 may multiply the reformatted mantissa InTc and the reformatted mantissa WtTC by the control signal. In this case, according to the control signal, the one or more logic gates M1 may output 0 (e.g., control signal=0) as the product P[n], or output the product of the reformatted mantissa InTc and the reformatted mantissa WtTC (e.g., control signal=1). By masking the input or output of one or more logic gates M1 with 0, or multiplying the input of one or more logic gates M1 by 0, the circuit 100 may ignore relatively small values ​​(e.g., values ​​with relatively small exponent values), thereby minimizing resource consumption for performing MAC operations with floating point numbers.

[0047] The summation circuits 108 each include one or more data registers (not shown) configured to receive instances of exponents InE and WtE corresponding to the number of data elements InDE and WtDE discussed above with respect to the multiplier circuits 106 .

[0048] The summing circuits 108 each include one or more logic gates A1 configured to add each instance of the exponent InE to each instance of the exponent WtE in an operation. In various embodiments, the one or more logic gates A1 include one or more full adder gates, half adder gates, ripple carry adder circuits, carry save adder circuits, carry select adder circuits, carry forward adder circuits, or other circuits suitable for performing part or all of the addition operation. Each logic gate A1 of the summing circuit 108 is configured to generate exponent sums S[1]-S[N] as data elements, the total number of bits of which is equal to the number of bits of each of the exponents InE and WtE plus 1.

[0049] Summation circuit 108 is configured to generate exponential sums S[1]-S[N] in an operation having a total number of data elements N and data elements corresponding to the total number of data elements N and data elements in products P[1]-P[N] discussed above with respect to multiplier circuit 106. Thus, for a total of N combinations of data elements InDE and WtDE, each nth combination corresponds to an nth exponent sum S[n] of the exponential sums S[1]-S[N] and an nth product P[n] of the products P[1]-P[N].

[0050] In some embodiments, such as those in which the data elements InDE and WtDE have a BF16 format, the summation circuit 108 is configured to generate each of the sums S[1]-S[N] having a total of 9 bits based on each of the exponents InE and WtE having a total of 8 bits. In some embodiments, such as those in which the data elements InDE and WtDE have an FP16 format, the summation circuit 108 is configured to generate each of the sums S[0]-S[N] having a total of six bits based on each of the exponents InE and WtE having a total of five bits. It is also within the scope of the present disclosure that the summation circuit 108 is configured to generate each of the sums S[1]-S[N] having other total numbers of bits based on each of the exponents InE and WtE having other total numbers of bits. The summation circuit 108 is configured to output the sums S[1]-S[N] to the difference circuit 110 via a data bus (not shown).

[0051] Differential circuit 110 is an electronic circuit, such as an IC, including one or more logic gates L1 (e.g., corresponding to selector circuit 111 or as part of selector circuit 111) and one or more logic gates B1, each of which is configured to receive an exponent sum S[1]-S[N] from summing circuit 108. One or more logic gates L1 may sometimes be referred to as a selector, and one or more logic gates B1 may sometimes be referred to as a subtractor. One or more logic gates L1 are configured to generate a maximum exponent sum MaxExp as a data element in an operation, the value of the maximum exponent sum MaxExp being equal to the maximum value of the data elements of the exponent sum S[1]-S[N], and the number of bits thereof being equal to the number of bits of the data elements of the exponent sum S[1]-S[N]. One or more logic gates L1 are configured to output the maximum exponent sum MaxExp to one or more logic gates B1 and converter circuit 124, as described below.

[0052] The one or more logic gates B1 are configured to generate difference values ​​D[1]-D[N] by subtracting each data element in the exponent sums S[1]-S[N] from the maximum exponent sum MaxExp in the operation. Therefore, the difference values ​​D[1]-D[N] have a total number N and an ordering of data elements corresponding to the total number N and ordering of data elements of the exponent sums S[1]-S[N] and products P[1]-P[N] described above. Figure 1In the illustrated embodiment, one or more logic gates B1 are configured to output the difference values ​​D[1]-D[N] to the shift circuit 112 and the comparator circuit 120 via one or more data buses (not shown). In some embodiments, one or more logic gates B1 are not configured to output the difference values ​​D[1]-D[N] to the multiplier circuit 106, and each of the multiplier circuits 106 is configured to generate each instance P[N] of the products P[1]-P[N] by always performing a multiplication operation. In some other embodiments, one or more logic gates B1 are configured to output the difference values ​​D[1]-D[N] to the multiplier circuit 106, respectively, and each of the multiplier circuits 106 is configured to generate each instance P[n] of the products P[1]-P[N] by selectively performing a multiplication operation based on the corresponding instance D[N].

[0053] Each of the comparator circuits 120 is an electronic circuit, such as an IC, configured to receive, for example, from the difference circuit 110, one of the corresponding difference values ​​D[1]-D[N] representing the difference between at least one of the exponent InE or the exponent WtE and the maximum exponent and MaxExp. The comparator circuit 120 is configured to compare the received difference value D[1]-D[N] with an exponent and a threshold value (e.g., sometimes referred to as an exponent difference threshold) in an operation. The exponent and threshold value may be predefined or preconfigured for a particular machine learning application. The exponent and threshold value may be configured according to the desired accuracy of the MAC operation output.

[0054] In some configurations, the circuit 100 may set the exponent and threshold based on the precision of the mantissa InM or the mantissa WtM (e.g., a portion of the input value) or the format of the input value (e.g., a data element from the input circuit 104). For example, the data elements InDE and WtDE may have an FP16 format, including 1 sign bit, 5 exponent bits, and 10 mantissa bits. The output of the MAC operation (e.g., the output from the converter 118) may have the same or a different format (e.g., an FP32 format, including 1 sign bit, 8 exponent bits, and 23 mantissa bits, or other formats). In this case, the precision may be set to the number of bits (e.g., precision) of the mantissa InM or the mantissa WtM (e.g., 10 mantissa bits). Therefore, the exponent and threshold may be configured to be 10, for example, by applying a mask or directly outputting zero from the multiplier circuit 106, relatively small input values ​​(e.g., corresponding exponent differences greater than or equal to the exponent and threshold) or their products may be ignored. In other words, in this case, for example, if the shift circuit 112 is to perform an 11-bit right shift, the value can be considered to be relatively small.

[0055] In some configurations, the circuit 100 may set the exponent and threshold based on a predetermined rounding value of the least significant bit (LSB), for example, by configuring the exponent and threshold to be the number of mantissa bits plus the number of extra bits. For example, referring to the above example, where the data elements InDE and WtDE may have an FP16 format and the MAC operation output may have an FP32 format, the circuit 100 may set the exponent and threshold to be the precision of the data element plus one or more extra bits. In some cases, the extra bits may be predefined. In some other cases, the extra bits may be based on a specific architecture or implementation of the circuit 100 or CIM, where 6 extra bits may be set for a 64-bit MAC CIM and 5 extra bits may be set for a 32-bit MAC CIM. Taking 6 extra bits as an example, the circuit 100 may set the exponent and threshold to 16 (e.g., 10 mantissa bits and 6 extra bits associated with the data element according to a specific architecture). Therefore, the exponent difference (with the maximum exponent and MaxExp) of at least 16 bits may be considered relatively small, so that a masking process may be performed on the corresponding multiplier circuit 106 or a product P[n] of zero may be generated. Circuit 100 may update additional bits to achieve different precisions.

[0056] Comparator circuit 120 is configured to generate control signals C[1]-C[N] in operation, the total number of which corresponds to the total number N of at least one of multiplier circuit 106, summing circuit 108 and / or difference values ​​D[1]-D[N]. The generated control signals C[1]-C[N] may be based on or in accordance with a comparison of difference values ​​D[1]-D[N] with an index and a threshold value. Each comparator circuit 120 may generate a corresponding instance C[n] of control signals C[1]-C[N]. Comparator circuit 120 may include, for example, one or more components capable of or adapted to perform comparison and generation operations.

[0057] For example, the comparator circuit 120 may generate a control signal C[n] (e.g., by performing a comparison) based on whether the corresponding difference D[n] satisfies an index and a threshold. For example, satisfying the index and the threshold may refer to the difference D[n] being greater than or equal to the index and the threshold. Depending on the comparison result, the control signal C[n] may be 0 or 1. If the difference D[n] is less than the index and the threshold, the comparator circuit 120 may generate a control signal C[n] of 1. If the difference D[n] is greater than or equal to the index and the threshold, the comparator circuit 120 may generate a control signal C[n] of 0. In some configurations, for example, if the difference D[n] is greater than or equal to the index and the threshold, the comparator circuit 120 may generate a control signal C[n] of 1, and if the difference D[n] is less than the index and the threshold, the control signal C[n] of 0 is generated. The comparator circuit 120 may provide the control signal C[n] to the corresponding multiplier circuit 106 or at least one component of the multiplier circuit 106 (e.g., MUX 122 or one or more logic gates M1).

[0058] It should be noted that variables or values, such as exponents and thresholds, input values, formats, etc., are not limited to the examples provided herein, and circuit 100 or other devices or components thereof may similarly use other variables or values ​​(such as different exponents and thresholds, formats, etc.) to perform MAC operations of floating point numbers with reduced computing resources. In addition, it should be noted that more or fewer components and / or different arrangements of one or more components may be implemented to perform the features, operations, or processes discussed herein.

[0059] In various configurations, operations of at least one of the summing circuit 108, the differencing circuit 110, and / or the comparator circuit 120 may be performed before, after, or in parallel with the multiplier circuit 106. In some configurations, operations of the respective summing circuit 108, the differencing circuit 110, or the comparator circuit 120 may be performed sequentially or in parallel.

[0060] Shift circuit 112 is an electronic circuit, such as an IC, including one or more registers and / or logic gates, which is configured to perform a shift operation on each instance P[n] in the product P[1]-P[N] based on the value of the corresponding instance D[n] of the difference value D[1]-D[N].

[0061] Each instance P[n] of the product P[1]-P[N] is based on the sign and mantissa of a corresponding combination of data elements InDE and WtDE, and each instance D[n] of the difference value D[1]-D[N] is based on the exponent sum of the same combination. The shift circuit 112 is configured to right shift each instance P[n] of the product P[1]-P[N] by an amount equal to the corresponding difference value D[n] in operation, thereby generating shifted products SP[1]-SP[N], wherein the sign and mantissa bits are aligned according to the summed exponent used to generate the difference value D[1]-D[N]. Based on the alignment, the shift circuit 112 is configured to generate each instance SP[n] of the shifted product SP[1]-SP[N] having the same exponent using the maximum exponent and MaxExp as a baseline.

[0062] To compensate for the right shift operation, shift circuit 112 may add an instance of the sign bit (zero or one) of each product P[n] as the leftmost bit of the corresponding shifted product SP[n]. The number of added sign bit instances is equal to the right shift amount determined by the corresponding difference value D[n].

[0063] exist Figure 1 In the illustrated embodiment, as described above, multiplier circuit 106 may generate corresponding instances P[n] of products P[1]-P[N] by performing a multiplication operation. Shift circuit 112 may include one or more shifters to receive products P[1]-P[N] from multiplier circuit 106 and selectively output (e.g., shift) one or more of the shifted products SP[1]-SP[N] to adder circuit 114 based on corresponding differences D[1]-D[N]. For example, in Figure 1 , the shift product output to adder circuit 114 may include SP[w]-SP[z], where "w" to "z" may each be one of integers from 1 to N. In one aspect of the present disclosure, the sum of the number of SP[w]-SP[z] may be equal to N. In one aspect of the present disclosure, the sum of the number of SP[w]-SP[z] may be less than N.

[0064] The shift circuit 112 (eg, a shifter) may be configured to shift the bits by comparing the corresponding difference values ​​among the difference values ​​D[1]-D[N] with the difference threshold value ( Figure 1The difference threshold value may be configured based on the distribution of the difference values ​​D[1]-D[N]. In an example where the difference values ​​D[1]-D[N] present a normal distribution, the difference threshold value may be determined at one standard deviation below the mean of the normal distribution. In another example where the difference values ​​D[1]-D[N] still present a normal distribution, the difference threshold value may be determined at two standard deviations below the mean of the normal distribution. In another example where the difference values ​​D[1]-D[N] still present a normal distribution, the difference threshold value may be determined at any standard deviation below the mean of the normal distribution.

[0065] When any difference value (e.g., D[n], where n is an integer between 1 and n) is equal to or less than a difference threshold value (sometimes referred to as a “small exponent difference”), the shift circuit 112 (e.g., a shifter) may be disabled to prevent the corresponding shifted product SP[n] from being received by the adder circuit 114 (e.g., not shifting the corresponding product P[n] or decoupling it from the adder circuit 114). Equivalently, when any difference value (e.g., D[n]) is greater than the difference threshold value (sometimes referred to as a “normal exponent difference”), the shift circuit 112 may be activated to output the corresponding shifted product SP[n] to the adder circuit 114.

[0066] In other words, the shift circuit 112 may shift any one of the products P[1]-P[N] based on comparing the respective difference values ​​D[1]-D[N] with the difference threshold value, and output the shifted products SP[1]-SP[N] to the adder circuit 114. Therefore, the sum of the number of SP[w]-SP[z] may be equal to N. In some configurations, the shift circuit 112 may detect that at least one of the products P[1]-P[N] from the multiplier circuit 106 is zero. In this case, the shift circuit 112 may not shift the corresponding product having a zero value and / or output the product to the adder circuit 114. Therefore, the sum of the number of SP[w]-SP[z] may be less than N.

[0067] In addition, to generate SP[w]-SP[z], shift circuit 112 may right shift each instance P[n] of the product P[w]-P[z] by an amount equal to the corresponding difference value D[n], thereby aligning the sign and mantissa bits according to the summed exponent. In some embodiments, the difference value D[n] may be generated (e.g., by difference circuit 110) based on subtracting each data element of the sum S[w]-S[z] from the maximum exponent and MaxExp. The maximum exponent and MaxExp may correspond to the maximum value of the data elements of the sum S[w]-S[z]. Based on this alignment, shift circuit 112 may use the maximum exponent and MaxExp as a baseline to generate each instance SP[n] of the shifted product SP[w]-SP[z] having the same exponent.

[0068] When any difference value (e.g., D[n], where n is an integer between 1 and n) is equal to or less than a difference threshold value (sometimes referred to as a "small exponent difference"), shift circuit 112 may be disabled to prevent adder circuit 114 from receiving the corresponding (e.g., shifted) product SP[n]. In some embodiments, products P[n] with such large exponent differences may be ignored.

[0069] In other words, the shift circuit 112 may shift all or part of the products P[1]-P[N] based on comparing the corresponding difference values ​​D[1]-D[N] to the difference threshold, and selectively output corresponding ones of the shifted products SP[1]-SP[N] to the adder circuit 114. Therefore, the sum of the numbers SP[w]-SP[z] (output by the shift circuit 112) may be less than or equal to N. The sum is less than N when one or more of the products P[1]-P[N] are ignored (e.g., such that their respective exponent difference values ​​D[N] are equal to or greater than the difference threshold), and the sum is equal to N when none of the products P[1]-P[N] are ignored.

[0070] In some embodiments, multiplier circuit 106 may receive difference D[1]-D[N] from difference circuit 110 to determine whether difference D[n] is greater than or equal to an exponent and a threshold (e.g., sometimes referred to as an exponent difference threshold). Figure 2-Figure 10 At least one of the multiplier circuits 106 may receive a control signal C[1]-C[N] indicating whether the difference value D[n] is greater than or equal to the exponent and the threshold. If the difference value D[n] is greater than or equal to the exponent and the threshold, the corresponding multiplier circuit 106 may perform a masking operation on at least one of the corresponding mantissa InM, the mantissa WtM, the reformatted mantissa InTC, or the reformatted mantissa WtTC, and / or provide a zero output from the multiplier. In some implementations, if the difference value D[n] is greater than or equal to the exponent and the threshold, the corresponding multiplier circuit 106 may omit outputting the corresponding product (e.g., the result of the multiplier or zero) to the shift circuit 112. Therefore, the number of products received by the shift circuit 112 may be less than N, for example, P[1] to P[N] in addition to one or more P[n]. Then, the shift circuit 112 may shift the remaining one of the products P[1]-P[N] based on comparing its corresponding difference value D[1]-D[N] with the difference threshold.

[0071] In some embodiments, such as those in which the data elements InDE and WtDE have a BF16 format, the shift circuit 112 is configured to generate each shift product with a total of 21 bits, such as SP[0]-SP[N], based on each of the products P[0]-P[N] having a total of 17 bits. In some embodiments, such as those in which the data elements InDE and WtDE have an FP16 format, the shift circuit 112 is configured to generate each shift product with a total of 27 bits, such as SP[0]-SP[N], based on each of the products P[0]-P[N] having a total of 23 bits. It is within the scope of the present disclosure for the shift circuit 112 to be configured to generate each shift product SP[0]-SP[N] with other total number of bits, based on each of the products P[0]-P[N] having other total number of bits.

[0072] Based on the product P[0]-P[N] having a two's complement format, the shift circuit 112 is configured to generate a shift product having a two's complement format, such as SP[0]-SP[N]. As described above, Figure 1 In the example shown, the shift circuit 112 is configured to output the shifted product SP[w]-SP[z] to the adder circuit (tree) 114 via a data bus (not shown).

[0073] The adder tree 114 is an electronic circuit, such as an IC, including multiple layers of one or more logic gates (not shown), such as discussed above with respect to the one or more logic gates A1 (of the summing circuit 108). For example, the adder tree 114 may include a first layer configured to receive the shift products SP[w]-SP[z], and a last layer configured to generate a sum 115 as a data element corresponding to the sum of the shift products SP[w]-SP[z]. In some embodiments, each of the one or more consecutive layers between the first layer and the last layer is configured to receive a first number of sum data elements generated by a previous layer, and to generate a second number of sum data elements based on the first number of sum data elements, the second number being half of the first number. Thus, the total number of layers includes the first layer and the last layer and each consecutive layer (if any).

[0074] In some embodiments, the sum PSTC (e.g., corresponding to sum 115) is sometimes referred to as a partial sum PSTC or a mantissa sum PSTC, and the total number of bits corresponds to the number of bits and the number of data elements of the shift product SP[w]-SP[z]. In some embodiments, the number of bits of the sum PSTC is equal to the number of bits of the shift product SP[w]-SP[z] plus the number of bits that can represent the number of data elements of the shift product SP[w]-SP[z]. In some embodiments, the number of bits of the sum PSTC is equal to the number of bits of the shift product SP[w]-SP[z] plus four bits that can represent 16 data elements of the shift product SP[w]-SP[z].

[0075] In some embodiments, such as those in which the data elements InDE and WtDE have a BF16 format, the adder tree 114 is configured to generate a total of 25 bits of the sum PSTC based on each of the shift products SP[w]-SP[z] having a total of 21 bits. In some embodiments, such as those in which the data elements InDE and WtDE have a FP16 format, the adder tree 114 is configured to generate a total of 31 bits of the sum PSTC based on each of the shift products SP[w]-SP[z] having a total of 27 bits. It is within the scope of the present disclosure for the adder tree 114 to be configured to generate a sum PSTC having other total bits of the sum PSTC based on each of the shift products SP[w]-SP[z] having other total bits of the sum PSTC.

[0076] According to various embodiments of the present disclosure, based on the shift product SP[w]-SP[z] having a two's complement format, the adder tree 114 is configured to generate a sum PSTC having a two's complement format. Therefore, the adder tree 114 is configured to output the sum PSTC to the converter 116 through a data bus (not shown). In some other embodiments, the adder tree 114 may output the sum PSTC to a circuit (not shown) outside the circuit 100.

[0077] Converter 116 is an electronic circuit, such as an IC, including logic circuits, which are configured to receive the sum PSTC from the adder tree 114 in operation and convert the sum PSTC from two's complement to a sum PSSM having a sign-plus-mantissa format. Converter 116 is configured to generate a sum PSSM having the same number of bits as the sum PSTC. Figure 1 In the illustrated embodiment, converter 116 is configured to further output the sum PSSM on a data bus (not shown) to converter 118. In some other embodiments, converter 116 may output the sum PSSM to a circuit external to circuit 100 (not shown).

[0078] Converter 118 is an electronic circuit, such as an IC, including logic circuits, which are configured to receive the sum PSSM from converter 116 in an operation, receive the maximum exponent and MaxExp from the difference circuit 110, and convert the sum PSSM from a sign-plus-mantissa format to a sum PS having an output format based on the sum PSSM and MaxExp and different from the sign-plus-mantissa form, such as a sum PS in a floating-point format as described above. In various embodiments of the present disclosure, converter 118 can generate a sum PS that is configured to be compatible with a circuit (not shown) external to circuit 100. For example, converter 118 is configured to output the sum PS to a circuit (not shown) external to circuit 100, such as a memory array or other instance of circuit 100 that is part of a convolutional neural network (CNN). In some arrangements, converter 116 can be part of converter 118, and vice versa.

[0079] Figure 2 According to some embodiments Figure 1 1 is a block diagram of a portion 200 of a data calculation circuit 100 for masking an input mantissa. Portion 200 may include one or more components of circuit 100, such as but not limited to multiplier circuit 106, summation circuit 108, difference circuit 110, and comparator circuit 120. Portion 200 may include other components for performing memory calculations (e.g., MAC operations) on input word vectors and weight matrices. The input word vector may include multiple (N) input data elements InDE (e.g., a first input), and the weight matrix may include multiple (N) weight data elements WtDE (e.g., a second input). In various embodiments, each of the input data elements InDE and the weight data elements WtDE may include a floating point number.

[0080] Each weight data element WtDE and the corresponding one of the input data element InDE can form or be referred to as one of a plurality of (N) input pairs. In various implementations, the input data element InDE and the weight data element WtDE can each include a corresponding symbol part, an exponent part and a mantissa part. For example, the input data element InDE (e.g., the first input) can include a number (N) of first symbols, a number (N) of first exponents and a number (N) of first mantissas. The weight data element WtDE can include a number (N) of second symbols, a number (P) of second exponents and a number (M) of second mantissas.

[0081] The multiplier circuit 106 may receive as inputs the signed mantissas InS / InM (e.g., sometimes referred to as “XIN mantissas”) and WtS / WtM (e.g., stored in a mantissa buffer). In some cases, the multiplier circuit 106 may reformat each instance of the signed mantissas InS / InM and WtS / WtM into their respective complement (e.g., reformatted) mantissas InTC and WtTc. The summation circuit 108 may receive as inputs the exponent InE (e.g., sometimes referred to as “XIN Exp”) and the exponent WtE (e.g., stored in an exponent buffer). For simplicity and purposes of providing examples herein, Figure 2 The mantissas InM and WtM (eg, mantissa portions) are provided as inputs to the multiplier circuit 106 (or logic gate M1), but it should be noted that the signed bits or reformatted mantissas InTC and WtTc may be provided as inputs to the multiplier circuit 106 (or logic gate M1).

[0082] Figure 2 The differential circuit 110 may include, but is not limited to, a combination of Figure 1 The various components described above. For example, the difference circuit 110 can receive the exponential sum (e.g., S[1]-S[N]) from the summing circuit 108. The difference circuit 110 can identify, select, or determine the maximum exponent sum MaxExp from the exponential sums S[1]-S[N]. For the corresponding summing circuit 108, the difference circuit 110 can generate or determine the corresponding exponential difference (e.g., the corresponding difference value D[n]) based on the difference between the maximum exponential sum MaxExp and the corresponding exponential sum. The difference circuit 110 can generate the difference value D[1]-D[N] and provide it to the corresponding comparator circuit 120.

[0083] If at least combined Figure 1 As described, each comparator circuit 120 can receive a difference D[n] (e.g., sometimes referred to as an exponent difference) of a corresponding input pair and compare it to an exponent and a threshold. Comparator circuit 120 can generate a control signal (e.g., C[n]) based on the comparison. For example, if the difference is greater than or equal to the exponent and the threshold, comparator circuit 120 can generate a control signal of 0. In another example, if the difference is less than the exponent and the threshold, comparator circuit 120 can generate a control signal of 1. Comparator circuit 120 can send a control signal to multiplier circuit 106. It should be noted that to provide an example, Figure 2 The components can be Figure 1 The components of circuit 100 are simplified versions of the components of circuit 100, and may include additional or alternative components to perform MAC operations on floating point numbers with reduced computational resources as discussed herein.

[0084] In this case, the multiplier circuit 106 may include a MUX 122 for masking the XIN mantissa to reduce computational resources when the difference is greater than the exponent and the threshold. The input of the MUX 122 may include a predefined value of zero, the XIN mantissa, and a control signal from the comparator circuit 120. For example, if the difference (from the corresponding summing circuit 108) is less than the exponent and the threshold, the MUX 122 may receive a control signal of 1 from the comparator circuit 120. According to the control signal of 1, the MUX 122 may output the XIN mantissa to one or more logic gates M1 of the corresponding multiplier circuit 106 to be multiplied with at least WtM in the mantissa buffer. In this example, the output from the one or more logic gates M1 may be the product of at least the input mantissa (e.g., the XIN mantissa (or InM) and WtM).

[0085] In another example, if the difference is greater than or equal to the exponent and the threshold, the MUX 122 may receive a control signal of 0 from the comparator circuit 120. According to the control signal of 0, the MUX 122 may output 0 (e.g., masking the mantissa or changing the mantissa value to 0) to one or more logic gates M1 of the corresponding multiplier circuit 106 to multiply with at least WtM in the mantissa buffer, thereby reducing resource consumption in the multiplication process (and at least the accumulation process of the adder circuit 114). In this example, the output from the one or more logic gates M1 may be 0.

[0086] Figure 3 According to some embodiments Figure 1 The data calculation circuit 100 performs a MAC operation on a floating point number in a timing diagram 300. The operation associated with the timing diagram 300 may be performed by combining but not limited to at least Figure 1 and Figure 2 For example, at clock cycle 302, a portion of an input data element InDE (e.g., at least an exponent InE and a mantissa InM) may be set or read from input circuit 104. At clock cycle 304, one or more weight exponents (e.g., exponents WtE) may be read and added to the corresponding input exponent InE.

[0087] At clock cycle 306, a maximum exponent sum (e.g., MaxExp) may be determined, as shown in example portion 314. For the corresponding exponent sum, in example portion 316, circuit 100 (e.g., difference circuit 110) may determine a corresponding exponent difference, e.g., D[n]. Based on the exponent difference, circuit 100 (e.g., comparator circuit 120) may determine whether to apply a mask (e.g., clear a mantissa, a multiplier output, or an output of a corresponding multiplier circuit 106) based on a comparison of the exponent difference and an exponent sum threshold.

[0088] At clock cycle 308, circuit 100 (e.g., multiplier circuit 106) may read out weight mantissa WtM. At clock cycle 310, circuit 100 (e.g., multiplier circuit 106) may multiply input mantissa InM by weight mantissa WtM or zero, for example, at portion 318, based on the corresponding exponent difference and comparison of the exponent and the threshold.

[0089] At clock cycle 312, circuit 100 (e.g., multiplier circuit 106) may generate a multiplication result (e.g., from one or more logic gates M1). As shown in portion 320, for example, if the exponent difference is less than the exponent and the threshold, the result of the multiplication may be the product of the input mantissa and the weight mantissa. Additionally, as shown in portion 320, for example, if the exponent difference is greater than or equal to the exponent and the threshold, the result (of multiplier circuit 106) may be zero. It should be noted that clock cycles 302-312 of timing diagram 300 are provided only as non-limiting examples, and different operations performed by circuit 100 may generate different timing diagrams, such as without limitation timing diagram 300.

[0090] Figure 4 According to some embodiments Figure 1 1 is a block diagram of a portion 400 of a data calculation circuit 100 for masking weight mantissas. Portion 400 may include one or more components of circuit 100 similar to or in combination with Figure 1 or Figure 2 At least one of the foregoing is described, but is not limited thereto. Figure 4 Some operations of one or more components of Figure 1 or Figure 2 At least one of the foregoing descriptions is provided, but is not limited thereto. For example, the multiplier circuit 106 may be configured to multiply at least the mantissas of the input value pair (e.g., data elements InDE and WtDE) during operation. The summation circuit 108 may be configured to sum the exponents of the input value pair during operation. The difference circuit 110 may be configured to obtain the maximum exponent sum and the difference between the corresponding exponent sum and the maximum exponent sum (e.g., the exponent difference) during operation. The comparator circuit 120 may be configured to compare the difference value with a (predefined or configured) exponent sum threshold during operation and output a corresponding control signal (e.g., C[1]-C[N]) to the multiplier circuit 106.

[0091] like Figure 4As shown, the multiplier circuit 106 may include a MUX 122 between the mantissa buffer and one or more logic gates M1 (e.g., multipliers). In this case, the MUX 122 may receive a weight mantissa (e.g., WtM), a zero, and a control signal from the mantissa buffer via an input port. The MUX 122 may include an output port coupled to the one or more logic gates M1 (e.g., multipliers) for outputting a selected input based on the control signal. The MUX 122 may be provided to mask the weight mantissa (e.g., WtM) from the mantissa buffer based on a comparison of an exponent difference with an exponent and a threshold.

[0092] For example, the comparator circuit 120 may determine that the difference is greater than or equal to the exponent and the threshold. In response to the determination, the comparator circuit 120 may send a control signal (e.g., 0) to the MUX 122 to mask the weight mantissa WtM by selecting 0 as the output to multiply the corresponding input mantissa InM (e.g., XIN mantissa). In this case, one or more logic gates M1 may receive 0 and the XIN mantissa from the MUX 122, thereby obtaining a product of 0.

[0093] In another example, the comparator circuit 120 may determine that the difference is less than the exponent and the threshold. In response to the determination, the comparator circuit 120 may send a control signal (e.g., 1) to the MUX 122 to not mask the original value from the input circuit 104, for example, to select the weight mantissa WtM as the output to be multiplied with the corresponding XIN mantissa. In this case, one or more logic gates M1 may receive the weight mantissa and the XIN mantissa. Therefore, the corresponding multiplier circuit 106 (e.g., one or more logic gates M1) may output the product between the weight mantissa and the XIN mantissa.

[0094] Figure 5 According to some embodiments Figure 1 1 is a block diagram of a portion 500 of a data calculation circuit for masking a multiplier output. Portion 500 may include one or more components of circuit 100 similar to or in combination with Figure 1 , Figure 2 or Figure 4 At least one of the foregoing is described, but is not limited thereto. Figure 5 Some operations of one or more components of Figure 1 , Figure 2 or Figure 4At least one of the foregoing descriptions is provided, but is not limited thereto. For example, the multiplier circuit 106 may be configured to multiply at least the mantissas of the input value pair (e.g., data elements InDE and WtDE) during operation. The summation circuit 108 may be configured to sum the exponents of the input value pair during operation. The difference circuit 110 may be configured to obtain the maximum exponent sum and the difference between the corresponding exponent sum and the maximum exponent sum (e.g., the exponent difference) during operation. The comparator circuit 120 may be configured to compare the difference with a (predefined or configured) exponent sum threshold during operation and output a corresponding control signal (e.g., C[1]-C[N]) to the multiplier circuit 106.

[0095] like Figure 5 As shown, one or more logic gates M1 of the multiplier circuit 106 can be configured to receive a control signal from the comparator circuit 120. In this case, the control signal from the comparator circuit 120 can be a third input of the multiplier to mask the corresponding mantissa product according to a corresponding control signal based on the corresponding exponent difference compared to the exponent and the threshold. For example, the comparator circuit 120 can determine that the difference is greater than or equal to the exponent and the threshold. In response to the determination, the comparator circuit 120 can output a corresponding control signal (e.g., 0) to one or more logic gates M1 (e.g., a multiplier) to mask the corresponding mantissa product with 0. In this case, the multiplier can further multiply the product of the mantissas InM and WtM by 0 to reduce the computational resources of the MAC operation.

[0096] In another example, the comparator circuit 120 may determine that the difference is less than the exponent and the threshold. In response to the determination, the comparator circuit 120 may output a corresponding control signal (e.g., 1) to the multiplier to not mask the corresponding mantissa product. For example, the multiplier may multiply the product of the mantissas InM and WtM by 1. Therefore, the product from the multiplier (e.g., one or more logic gates M1) may remain unchanged (e.g., InM multiplied by WtM).

[0097] Figure 6 According to some embodiments Figure 1 1 is a block diagram of a portion 600 of a data calculation circuit for directly outputting a multiplier result. Portion 600 may include one or more components of circuit 100 similar to or in combination with Figure 1 , Figure 2 , Figure 4 or Figure 5 At least one of the foregoing is described, but is not limited thereto. Figure 6 Some operations of one or more components of Figure 1 , Figure 2 , Figure 4 or Figure 5At least one of the foregoing descriptions is provided, but is not limited thereto. For example, the multiplier circuit 106 may be configured to multiply at least the mantissas of the input value pair (e.g., data elements InDE and WtDE) during operation. The summation circuit 108 may be configured to sum the exponents of the input value pair during operation. The difference circuit 110 may be configured to obtain the maximum exponent sum and the difference between the corresponding exponent sum and the maximum exponent sum (e.g., the exponent difference) during operation. The comparator circuit 120 may be configured to compare the difference with a (predefined or configured) exponent sum threshold during operation and output a corresponding control signal (e.g., C[1]-C[N]) to the multiplier circuit 106.

[0098] like Figure 6 As shown, the multiplier circuit 106 can include a MUX 122 at the output of one or more logic gates M1 (e.g., a multiplier). In this case, the MUX 122 can receive the corresponding product of the multiplier (e.g., P[n]), zero, and a control signal through an input port. The MUX 122 can include an output port coupled to the shift circuit 112 for outputting the selected input, for example, based on the control signal. The MUX 122 can directly output zero based on the comparison of the exponential difference with the exponential sum threshold.

[0099] For example, the comparator circuit 120 may determine that the difference is greater than or equal to the exponent and the threshold. In response to the determination, the comparator circuit 120 may send a control signal (e.g., 0) to the MUX 122 to directly output zero from the multiplier circuit 106. In this case, the shift circuit 112 may receive 0 from the MUX 122. In some other cases, for example, if the output of the multiplier is 0, or if a control signal of 0 is provided by the comparator circuit 120 when the corresponding difference is greater than or equal to the exponent and the threshold, the multiplier circuit 106 may be turned off, or the clock supply may be stopped / paused for at least one cycle.

[0100] In another example, the comparator circuit 120 may determine that the difference is less than the exponent and the threshold. In response to the determination, the comparator circuit 120 may send a control signal (e.g., 1) to the MUX 122 to output the product (e.g., the original result) from the multiplier, for example, to select the product of the mantissa InM and WtM from the multiplier circuit 106 as the output. In this case, the output selected by the MUX 122 according to the control signal may be provided to the shift circuit 112.

[0101] Figure 7 According to some embodiments Figure 11 is a block diagram of a portion 700 of a data calculation circuit for masking weight mantissas, the portion 700 including a similarity circuit 704 (eg, sometimes referred to as a second comparator circuit) for weight exponents. The portion 700 may include one or more components of the circuit 100 similar to or in combination with Figure 1 , Figure 2 , Figure 4 , Figure 5 or Figure 6 At least one of the foregoing is described, but is not limited thereto. Figure 7 Some operations of one or more components of Figure 1 , Figure 2 , Figure 4 , Figure 5 or Figure 6 At least one of the above is described, but is not limited to this. For example, the multiplier circuit 106 can be configured to multiply at least the mantissa of the input value pair (e.g., data elements InDE and WtDE) during operation. The summation circuit 108 can be configured to sum the exponents of the input value pair during operation. In some cases, due to the FP data format, the summation circuit 108 can output a summation result (e.g., the result of summing the corresponding pair of exponents), such as for aligning and scaling the result according to the FP data format, or providing information about overflow or underflow conditions. The difference circuit 110 can be configured to obtain the maximum exponent sum and the difference between the corresponding exponent sum and the maximum exponent sum (e.g., the exponent difference) during operation. The comparator circuit 120 can be configured to compare the difference with the (predefined or configured) exponent sum threshold during operation, and output a corresponding control signal (e.g., C[1]-C[N]) to the multiplier circuit 106.

[0102] like Figure 7 As shown, the multiplier circuit 106 may include at least a combination of Figure 4 The MUX 122 described above (eg, for masking the weight mantissa WtM based on the control signal). Specifically, the operation or features of the multiplier circuit 106 may be combined with at least Figure 4 In addition, Figure 7 As shown, circuit 100 may include a second MUX (e.g., MUX 702) that is configured in operation to, for example, receive corresponding inputs, select one of the inputs based on a control signal, and output the selected input to differential circuit 110. Although MUX 702 is used for the purpose of providing an example, it should be noted that other logic components may be similarly utilized to output one of the inputs based on a control signal.

[0103] like Figure 7As further shown, the circuit 100 may include a similarity circuit 704. The similarity circuit 704 may be configured to determine whether each of the N indices (e.g., input indices InE or weighted indices WtE) is the same on a data element (e.g., output data element InDE or weighted data element WtDE) during operation, and generate a control signal C2 (e.g., sometimes referred to as a second control signal) based on the determination. For example, based on whether the corresponding indices (e.g., input indices or weighted indices) from the input circuit 104 are the same, the summation circuit 108 (e.g., via the MUX 702) may provide the corresponding index sum S[1]-S[N] or other corresponding indices (e.g., weighted indices or input indices) to the difference circuit 110. In this case, the difference circuit 110 may determine the maximum exponent sum MaxExp and the difference value D[1]-D[N] based on the provided index sum S[1]-S[N] or the provided index (e.g., input indices InE or weighted indices WtE). The features or operations of the similarity circuit 704 may be combined with at least Figure 8 to describe.

[0104] For example, refer to Figure 8 , according to some embodiments, depicting at least Figure 7 800 is a block diagram of a similarity circuit 704 of the input circuit 104 to determine whether the indices are the same. The similarity circuit 704 is an electronic circuit, such as an IC, which is configured to receive, for example, from the input circuit 104 an index InE of each of the N data elements InDE (e.g., to determine whether the input indices are the same) or an index WtE of each of the N data elements WtDE (e.g., to determine whether the weight indices are the same). The similarity circuit 704 can be coupled to the input circuit 104 ( Figure 1 ) to receive an input index or a weight index.

[0105] The similarity circuit 704 may include one or more logic components for comparing pairs of indices to determine whether each pair of indices is the same. The similarity circuit 704 may include multiple layers of one or more logic components. For example, the similarity circuit 704 may include a first logic gate for comparing a first index to a second index and a second logic gate for comparing a third index to a fourth index. The first logic gate and the second logic gate may generate corresponding outputs based on the comparison, such as 0 for different pairs of indices and 1 for similar pairs of indices. The first logic gate and the second logic gate may output corresponding comparison results as a pair of inputs to a third logic gate. Similar processes for comparing indices or comparing previous comparison results may be iterated to arrive at a final logic gate.

[0106] The output of the final logic gate may correspond to a control signal C2, indicating whether all the indices are the same, or whether at least one of the indices is different from the other indices. Thus, for example, if all the indices are the same, the similarity circuit 704 may output a control signal C2 of 1, and if at least one of the indices is different from the other indices, a control signal C2 of 0 may be output. The similarity circuit 704 may provide the control signal C2 to the summing circuit 108 (e.g., MUX 702). If the input of the similarity circuit 704 is the input index InE, the output from the similarity circuit 708 (e.g., the control signal C2) may indicate whether the input index InE is the same. In this case, the control signal C2 may indicate whether the summing circuit 108 outputs the corresponding index sum S[1]-S[N] or the corresponding weight index WtE to the difference circuit 110.

[0107] If the input of similarity circuit 704 is weight index WtE, the output of similarity circuit 708 can indicate whether weight index WtE is the same. In this case, control signal C2 can indicate whether summation circuit 108 outputs the corresponding index sum S[1]-S[N] or the corresponding input index InE to difference circuit 110.

[0108] Return to reference Figure 7 As shown, the similarity circuit 704 can provide a control signal C2 to the MUX 702. In this case, the similarity circuit 704 is used to determine the similarity between the weight indices WtE. Therefore, the input of the MUX 702 can include the corresponding index sum S[n] and the corresponding input index InE from one or more logic gates A1. Based on the control signal C2, the MUX 702 can select one of the index sum S[n] or the input index InE as the output of the difference circuit 110.

[0109] For example, if the similarity circuit 704 determines that the weight indices WtE are all the same, the similarity circuit 704 may send a control signal C2 of 1 to the MUX 702 to select the corresponding input index InE of the difference circuit 110. In another example, if the similarity circuit 704 determines that at least one of the weight indices WtE is different from the other weight indices, the similarity circuit 704 may send a control signal C2 of 0 to the MUX 702 to select the corresponding index sum S[1]-S[N] for the difference circuit 110. Therefore, the difference circuit 110 may use the index sum S[1]-S[N] or the input index InE from the summation circuit 108 to determine the maximum index sum MaxExp and the corresponding index difference, which may be further provided to at least the shift circuit 112 and the comparator circuit 120, for example.

[0110] Fig. 9 According to some embodiments Figure 1A block diagram of a portion 900 of a data calculation circuit 100 for masking weight mantissas, including a similarity circuit 704 for input exponents. Portion 900 may include one or more components of circuit 100, similar to or in combination with Figure 1 , Figure 2 , Figure 4 , Figure 5 , Figure 6 , Figure 7 or Figure 8 At least one of the descriptions, but not limited thereto. Fig. 9 Some operations of one or more components of Figure 1 , Figure 2 , Figure 4 , Figure 5 , Figure 6 , Figure 7 or Figure 8 At least one of the descriptions in, but not limited to, may be used. For example, the multiplier circuit 106 may be configured to multiply at least the mantissas of the input value pair (e.g., data elements InDE and WtDE) during operation. The summation circuit 108 may be configured to sum the exponents of the input value pair during operation. During operation, the difference circuit 110 may be configured to obtain the maximum exponent sum and the difference between the corresponding exponent sum and the maximum exponent sum (e.g., the exponent difference). The comparator circuit 120 may be configured to compare the difference with a (predefined or configured) exponent sum threshold during operation and output a corresponding control signal (e.g., C[1]-C[N]) to the multiplier circuit 106.

[0111] like Fig. 9 As shown, the multiplier circuit 106 may include at least a combination of Figure 4 The MUX 122 described above (eg, for masking the weight mantissa WtM based on the control signal). Specifically, the operation or features of the multiplier circuit 106 may be combined with at least Figure 4 In addition, Fig. 9 As shown, circuit 100 may include MUX 702 that is configured in operation to receive control signal C2, for example, from similarity circuit 704, and select one of the inputs based on control signal C2.

[0112] In this case, the similarity circuit 704 can be configured to determine whether each input index InE is the same as the other input index InE of the input data element InDE in the operation, and generate a control signal C2 based on the determination. Using the similarity circuit 704 to determine the similarity between the input indexes InE, the input of the MUX 702 can include the corresponding index sum S[n] and the corresponding weight index WtE from one or more logic gates A1. Based on the control signal C2, the MUX 702 can select one of the index sum S[n] or the input index InE as the output to the difference circuit 110.

[0113] For example, if the similarity circuit 704 determines that the input indices InE are all the same, the similarity circuit 704 may send a control signal C2 of 1 to the MUX 702 to select the corresponding weight index WtE of the difference circuit 110. In another example, if the similarity circuit 704 determines that at least one of the input indices InE is different from the other indices, the similarity circuit 704 may send a control signal C2 of 0 to the MUX 702 to select the corresponding index sum S[1]-S[N] for the difference circuit 110. Therefore, the difference circuit 110 may use the index sum S[1]-S[N] or the weight index WtE from the summation circuit 108 to determine the maximum exponent sum MaxExp and the corresponding exponent difference, which may be further provided to at least the shift circuit 112 and the comparator circuit 120, for example.

[0114] Fig.10 1 is a flow chart showing an example method 1000 for performing MAC operations on floating point numbers with reduced computational resources according to various embodiments. The example method 1000 may be performed by the circuit 100 or one or more components of the circuit 100. Therefore, the following embodiments of the method 1000 may be combined with but not limited to Figure 1-Figure 9 The illustrated embodiment of method 1000 is provided as an example only and does not limit the scope of the present disclosure. Therefore, it should be understood that any of the various operations of method 1000 can be omitted, reordered, and / or added while still within the scope of the present disclosure.

[0115] According to various embodiments, method 1000 starts with operation 1002 for obtaining a first input and a second input. Circuit 100 (e.g., sometimes referred to as CIM circuit 100) can obtain a first input and a second input from input circuit 104. Each input can include at least an exponent portion and a mantissa portion. For example, the first input can include a first exponent portion and a first mantissa portion. The second input can include a second exponent portion and a second mantissa portion. Each input can also include a sign bit. In various implementations, circuit 100 can obtain multiple first inputs and second inputs. The first input can be composed of a number (N) of first symbols, N first exponents, and N first mantissas. The second input can be composed of N second symbols, N second exponents, and N second mantissas. Each of the second inputs and the corresponding one of the N first inputs form one of the N input pairs. The input can correspond to a data element, such as an input data element InDE or a weight data element WtDE.

[0116] According to various embodiments, method 1000 continues to operation 1004 for generating a first exponential sum. Circuit 100 (e.g., summing circuit 108) may generate a first exponential sum (e.g., S[n]) by summing a first exponential portion of a first input and a second exponential portion of a second input. Circuit 100 may include N summing circuits 108 configured to combine (or add) a corresponding first exponent and a corresponding second exponent of a corresponding one of the N input pairs to generate a corresponding one of the N exponential sums (e.g., S[1]-S[N]).

[0117] According to various embodiments, method 1000 continues to operation 1006 for calculating exponential differences. Circuit 100 (e.g., difference circuit 110, sometimes referred to as a subtractor circuit) may calculate or determine the exponential difference by subtracting the first exponent sum S[n] from the maximum exponential sum (e.g., maximum exponent sum MaxExp). For example, circuit 100 may receive N exponential sums from corresponding summing circuits 108. Circuit 100 may include selector circuit 111 configured to select or identify a maximum one of the N exponential sums as the maximum or largest exponential sum. In response to identifying the maximum exponential sum, each difference circuit 110 (or subtractor circuit) of circuit 100 may calculate or determine a corresponding one of the N exponential differences by subtracting the corresponding exponential sum from summing circuit 108 from the maximum exponential sum. In other words, each of the N exponential differences may be equal to a difference between a corresponding one of the N exponential sums and the maximum exponential sum.

[0118] According to various embodiments, method 1000 continues to operation 1008 for generating a control signal. Circuit 100 (e.g., comparator circuit 120) may generate a control signal (e.g., C[n]) based on comparing the exponential difference with the exponent and the threshold. The exponent and the threshold may be predefined or configured according to an application or implementation. Circuit 100 may include N comparator circuits (e.g., 120), each of which is configured to compare a corresponding exponential difference with the exponent and the threshold, and to generate a corresponding one of N control signals (e.g., C[1]-C[N]) based on a comparison between the corresponding exponential difference and the exponent and the threshold. For example, if the exponential difference is greater than or equal to the exponent and the threshold, comparator circuit 120 may generate a first control signal (e.g., 0). In another example, if the exponential difference is less than the exponent and the threshold, comparator circuit 120 may generate a second control signal (e.g., 1). The N comparator circuits 120 may provide corresponding control signals C[1]-C[N] to corresponding multiplier circuits 106 to perform a masking operation or output a zero value based on the control signals.

[0119] According to various embodiments, method 1000 continues to operation 1010 for selectively multiplying the first mantissa portion by the second mantissa portion. Circuit 100 (e.g., multiplier circuit 106) may selectively multiply the first mantissa portion by the second mantissa portion based on a corresponding control signal (e.g., C[n]) to generate a mantissa product (e.g., zero or the product of the first and second mantissa portions indicated by the control signal). In various configurations, circuit 100 may include N multiplier circuits 106. Each of the N multiplier circuits 106 may be configured to selectively multiply the corresponding first mantissa of the corresponding input pair by the corresponding second mantissa based on a corresponding control signal to generate a corresponding one of the N mantissa products.

[0120] In various implementations, circuit 100 may include N multiplexers (e.g., MUX 122) associated with respective N multiplier circuits 106. Each of the N multiplexers may receive a respective one of the N control signals from a respective one of the N comparator circuits (e.g., 120), a first input, and a second input. The first input may be zero. Depending on the configuration, arrangement, or implementation of the N multiplexers, the second input may be one of the respective first mantissa, the respective second mantissa, or the product of the respective mantissas, such as in combination with Figure 1-Figure 9 In this case, the circuit 100 may output one of the first input or the second input according to the corresponding control signal, for example, based on whether the corresponding index difference is greater than or equal to the index and the threshold.

[0121] In some embodiments, circuit 100 (eg, multiplier circuit 106) may include a multiplexer (eg, MUX 122) for masking the first mantissa (eg, input mantissa InM), such as in combination with at least Figure 2 As described. For example, the multiplexer can receive a corresponding control signal (e.g., C[n]) from the comparator circuit 120, receive a corresponding first mantissa from the input circuit 104, and zero. The multiplexer can select the corresponding first mantissa as an output based on a corresponding control signal based on the corresponding exponent difference being less than the exponent and the threshold, and multiply it with the corresponding second mantissa through one or more logic gates M1 (e.g., a multiplier). The multiplexer can select zero as an output to be multiplied with the corresponding second mantissa based on a corresponding control signal based on the corresponding exponent difference being greater than or equal to the exponent and the threshold. In this case, the multiplexer can mask the first mantissa because the corresponding input pair is considered to be relatively small based on the exponent difference. Therefore, for example, the circuit 100 can reduce computing resources by converting the first mantissa to zero and multiplying the second mantissa by zero.

[0122] In some implementations, circuit 100 (eg, multiplier circuit 106) may include a multiplexer (eg, MUX 122) for masking the second mantissa (eg, weight mantissa WtM), such as in conjunction with at least Figure 4 As described. For example, the multiplexer can receive a corresponding control signal (e.g., C[n]) from the comparator circuit 120, receive a corresponding second mantissa from the input circuit 104, and zero. The multiplexer can select the corresponding second mantissa as an output based on the corresponding control signal based on the corresponding exponent difference being less than the exponent and the threshold, and multiply it with the corresponding first mantissa through one or more logic gates M1 (e.g., a multiplier). The multiplexer can select zero as an output to multiply it with the corresponding first mantissa based on the corresponding control signal based on the corresponding exponent difference being greater than or equal to the exponent and the threshold. In this case, the multiplexer can mask the second mantissa to reduce computing resources.

[0123] In some implementations, the circuit 100 (e.g., the multiplier circuit 106) may include a multiplexer (e.g., MUX 122) for directly outputting zero as an output of the multiplier circuit 106 (e.g., an output of the multiplier or one or more logic gates M1) based on a control signal, such as in combination with at least Figure 6As described. For example, the multiplexer can receive a corresponding control signal (e.g., C[n]) from the comparator circuit 120, receive a corresponding mantissa product from the multiplier, and zero. The multiplexer can select the corresponding mantissa product as the output of the corresponding one of the N multiplier circuits according to the corresponding control signal based on the corresponding exponent difference being less than the exponent and the threshold. The multiplexer can select zero as the output of the corresponding one of the N multiplier circuits according to the corresponding control signal based on the corresponding exponent difference being greater than or equal to the exponent and the threshold. In this case, if the exponent difference is greater than or equal to the exponent and the threshold, the multiplexer can directly output zero to reduce computing resources.

[0124] In some implementations, in addition to at least the first mantissa and the second mantissa, the multiplier (or one or more logic gates M1) of the multiplier circuit 106 may also receive a control signal, such as at least in conjunction with Figure 5 As described. For example, when the comparator circuit 120 determines that the exponent difference is less than the threshold value, the comparator circuit 120 may send a control signal of 1 for multiplying the corresponding mantissa product (e.g., by a multiplier). In this case, the multiplier may output the corresponding mantissa product as a result. In another example, when the comparator circuit 120 determines that the exponent difference is greater than or equal to the threshold value, the comparator circuit 120 may send a control signal of 0 for multiplying the corresponding mantissa product (e.g., by a multiplier). In this case, the product P[n] of the multiplier may be zero. Therefore, each of the N multiplier circuits 106 may mask the corresponding mantissa product with zero according to the corresponding control signal based on the corresponding exponent difference being greater than or equal to the exponent and the threshold value.

[0125] In some configurations, the circuit 100 may include multiple types (or combinations) of masking or implementations of the MUX 122, such as a combination of masking a first mantissa (e.g., InM), masking a second mantissa (e.g., WtM), masking a multiplier output (e.g., one or more logic gates M1), and / or directly outputting zero from the multiplier circuit 106 as the product P[n], etc. For example, the circuit 100 may include N multiplier circuits 106, such as a first multiplier circuit, a second multiplier circuit, a third multiplier circuit, and a fourth multiplier circuit. The first multiplier circuit may include a MUX 122 configured to mask a first input mantissa based on a corresponding control signal. The second multiplier circuit may include a MUX 122 configured to mask a second input mantissa based on a corresponding control signal. The third multiplier circuit may include a MUX 122 configured to directly output zero as the product of the third multiplier circuit based on a corresponding control signal. The fourth multiplier circuit may include a multiplier configured to receive three inputs including a control signal, a first mantissa, and a second mantissa, wherein an output of the multiplier may be masked based on the control signal. Thus, the masking or implementation of MUX 122 may be different for a single N multiplier circuit 106.

[0126] In some implementations, the circuit 100 (e.g., the summing circuit 108) may determine whether to send the corresponding index sum (e.g., S[n]) or one of the corresponding first index or second index to the difference circuit 110 (e.g., the subtractor circuit) based on whether the N first indexes are the same as each other or the N second indexes are the same as each other. Figure 7-Figure 9 At least one of the foregoing is used to describe the determination of sending the corresponding index and or one of the corresponding first index or second index to the difference circuit 110.

[0127] For example, the circuit 100 may include a similarity circuit (eg, 704) configured to compare the N second indices with each other and generate a second control signal based on the comparison between the N second indices, such as at least in combination with Fig. 9As described. In this case, each of the N multiplexers of the circuit 100 can receive a second control signal, a corresponding one of the N exponential sums, and a corresponding first exponent. Each of the N multiplexers can select a corresponding one of the N exponential sums or a corresponding first exponent as an output to at least one of the selector circuit (e.g., 111) or the corresponding subtractor circuit (e.g., 110) according to the second control signal to calculate the corresponding exponent difference. For example, if the N second exponents are equal to each other, the similarity circuit 704 can generate a corresponding second control signal (e.g., C2) to the N multiplexers. Therefore, each of the N multiplexers can select the corresponding first exponent as an output based on the second control signal. Otherwise, each of the N multiplexers can select a corresponding one of the N exponential sums as an output based on at least one of the N second exponents being different from another of the N second exponents.

[0128] In another example, the circuit 100 may include a similarity circuit (eg, 704) configured to compare the N first indices with each other and generate a second control signal based on the comparison between the N first indices, such as at least in combination with Figure 7 As described. In this case, each of the N multiplexers of the circuit 100 can receive a second control signal, a corresponding one of the N exponential sums, and a corresponding second exponent. Each of the N multiplexers can select a corresponding one of the N exponential sums or a corresponding second exponent as an output to at least one of the selector circuit (e.g., 111) or the corresponding subtractor circuit (e.g., 110) according to the second control signal to calculate the corresponding exponent difference. For example, if the N first exponents are equal to each other, the similarity circuit 704 can generate a corresponding second control signal (e.g., C2) to the N multiplexers. Therefore, each of the N multiplexers can select a corresponding second exponent as an output based on the second control signal. Otherwise, each of the N multiplexers can select a corresponding one of the N exponential sums as an output based on at least one of the N first exponents being different from another of the N first exponents.

[0129] In one aspect of the present disclosure, a computation-in-memory (CIM) circuit is disclosed. The computation-in-memory circuit includes: an input circuit configured to receive: (i) a number N of first inputs and (ii) N second inputs, wherein the first inputs consist of N first symbols, N first exponents, and N first mantissas, and the second inputs consist of N second symbols, N second exponents, and N second mantissas, and wherein each of the second inputs and a corresponding one of the N first inputs form one of the N input pairs; N summation circuits, each of the N summation circuits configured to combine the corresponding first exponent and the corresponding second exponent of a corresponding one of the N input pairs to generate a corresponding one of the N exponential sums; a selector circuit configured to select the largest one from the N exponential sums as the maximum exponent sum; exponent sum; N subtractor circuits, each of the N subtractor circuits being configured to calculate a corresponding one of N exponent differences, each of the N exponent differences being equal to a difference between a corresponding one of the N exponent sums and a maximum exponent sum; N comparator circuits, each of the N comparator circuits being configured to: (i) compare the corresponding exponent difference with an exponent sum threshold, and (ii) generate a corresponding one of N control signals based on a comparison between the corresponding exponent difference and the exponent sum threshold; and N multiplier circuits, each of the N multiplier circuits being configured to selectively multiply a corresponding first mantissa of a corresponding input pair by a corresponding second mantissa based on a corresponding control signal, thereby generating a corresponding one of N mantissa products.

[0130] In some embodiments, the in-memory computation circuit includes: N multiplexers, each of the N multiplexers being configured to (i) receive a corresponding one of N control signals from a corresponding one of N comparator circuits, a first input, and a second input, and (ii) output one of the first input or the second input based on the corresponding control signal, wherein the first input is zero and the second input is one of a corresponding first mantissa, a corresponding second mantissa, or a corresponding mantissa product.

[0131] In some embodiments, each of the N multiplier circuits includes a multiplexer configured to: receive a corresponding control signal, a corresponding first mantissa, and zero; and select the corresponding first mantissa as an output to be multiplied with the corresponding second mantissa based on the corresponding control signal based on the corresponding exponent difference being less than the exponent and a threshold; or select zero as an output to be multiplied with the corresponding second mantissa based on the corresponding control signal based on the corresponding exponent difference being greater than or equal to the exponent and the threshold.

[0132] In some embodiments, each of the N multiplier circuits includes a multiplexer configured to: receive a corresponding control signal, a corresponding second mantissa, and zero; and select the corresponding second mantissa as an output to be multiplied with the corresponding first mantissa based on the corresponding control signal based on the corresponding exponent difference being less than the exponent and a threshold; or select zero as an output to be multiplied with the corresponding first mantissa based on the corresponding control signal based on the corresponding exponent difference being greater than or equal to the exponent and the threshold.

[0133] In some embodiments, each of the N multiplier circuits includes a multiplexer configured to: receive a corresponding control signal, a corresponding mantissa product, and zero; and select the corresponding mantissa product as an output of a corresponding one of the N multiplier circuits based on the corresponding control signal based on the corresponding exponent difference being less than the exponent and a threshold; or select zero as an output of a corresponding one of the N multiplier circuits based on the corresponding control signal based on the corresponding exponent difference being greater than or equal to the exponent and the threshold.

[0134] In some embodiments, each of the N multiplier circuits is configured to mask a corresponding mantissa product with zeros according to a corresponding control signal based on a corresponding exponent difference being greater than or equal to the exponent and a threshold.

[0135] In some embodiments, the in-memory calculation circuit includes: a similarity circuit configured to (i) compare N second indices with each other, and (ii) generate a second control signal based on the comparison between the N second indices; and N multiplexers, each of the N multiplexers configured to (i) receive the second control signal, a corresponding one of the N index sums, and a corresponding first index, and (ii) select the corresponding one of the N index sums or the corresponding first index as output to at least one of the selector circuit or the corresponding subtractor circuit based on the second control signal to calculate the corresponding index difference.

[0136] In some embodiments, each of the N multiplexers is configured to: select a corresponding first index as output based on the N second indices being equal to each other; or select a corresponding one of the N indices and as output based on at least one of the N second indices being different from another of the N second indices.

[0137] In some embodiments, the in-memory calculation circuit includes: a similarity circuit configured to (i) compare N first indices with each other, and (ii) generate a second control signal based on the comparison between the N first indices; and N multiplexers, each of the N multiplexers configured to (i) receive the second control signal, a corresponding one of the N exponential sums, and a corresponding second index, and (ii) select the corresponding one of the N exponential sums or the corresponding second index as output to at least one of the selector circuit or the corresponding subtractor circuit based on the second control signal to calculate the corresponding index difference.

[0138] In some embodiments, each of the N multiplexers is configured to: select a corresponding second index as output based on the N first indices being equal to each other; or select a corresponding one of the N indices and as output based on at least one of the N first indices being different from another of the N first indices.

[0139] In another aspect of the present disclosure, a computing circuit in a memory is disclosed, comprising: an input circuit configured to obtain a first input and a second input, wherein the first input includes a first exponent part and a first mantissa part, and the second input includes a second exponent part and a second mantissa part; a first summing circuit configured to generate a first exponent sum by summing the first exponent part and the second exponent part; a subtractor circuit configured to calculate an exponent difference by subtracting the first exponent sum from a maximum exponent sum; a comparator circuit configured to generate a control signal based on comparing the exponent difference with an exponent sum threshold; and a multiplier circuit configured to (i) receive the control signal, and (ii) multiply the first mantissa part by the second mantissa part, wherein one of the first mantissa part or the second mantissa is selectively selected to be zero based on the control signal.

[0140] In some embodiments, the input circuit is configured to obtain a third input and a fourth input, wherein the third input includes a third exponent part and a third mantissa part, and the fourth input includes a fourth exponent part and a fourth mantissa part, and the calculation circuit in the memory further includes: a second summation circuit, configured to generate a second exponent sum by summing the third exponent part and the fourth exponent part; and a selector circuit, configured to select the largest one from the first exponent sum and the second exponent sum as the maximum exponent sum.

[0141] In some embodiments, the in-memory computing circuit includes: a multiplexer configured to (i) receive a control signal from a comparator circuit, a third input, and a fourth input, and (ii) output one of the third input or the fourth input based on the control signal, wherein the third input is zero and the fourth input is one of the first mantissa part, the second mantissa part, or the product of the first mantissa part multiplied by the second mantissa part.

[0142] In some embodiments, the multiplier circuit includes a multiplexer configured to: receive a control signal, a first mantissa portion, and zero; and select the first mantissa portion as an output to be multiplied with the second mantissa portion based on the control signal based on the exponent difference being less than the exponent and a threshold; or select zero as an output to be multiplied with the second mantissa portion based on the control signal based on the exponent difference being greater than or equal to the exponent and the threshold.

[0143] In some embodiments, the multiplier circuit includes a multiplexer configured to: receive a control signal, a second mantissa portion, and zero; and select the second mantissa portion as an output to be multiplied with the first mantissa portion based on the control signal based on the exponent difference being less than the exponent and a threshold; or select zero as an output to be multiplied with the first mantissa portion based on the control signal based on the exponent difference being greater than or equal to the exponent and the threshold.

[0144] In some embodiments, the multiplier circuit includes a multiplexer configured to: receive a control signal, a mantissa product of the first mantissa portion multiplied by the second mantissa portion, and zero; and, based on the control signal based on the exponent difference being less than the exponent and a threshold, select the mantissa product as an output of the multiplier circuit; or, based on the control signal based on the exponent difference being greater than or equal to the exponent and the threshold, select zero as the output of the multiplier circuit.

[0145] In some embodiments, the multiplier circuit is configured to mask the mantissa product with zeros to prevent multiplication of the first mantissa portion by the second mantissa portion based on a control signal based on the exponent difference being greater than or equal to the exponent and a threshold.

[0146] In another aspect of the present disclosure, a method for operating a computing circuit in a memory is disclosed, comprising: obtaining a first input and a second input through the computing circuit in the memory, wherein the first input includes a first exponent part and a first mantissa part, and the second input includes a second exponent part and a second mantissa part; generating a first exponent sum by adding the first exponent part to the second exponent part by the computing circuit in the memory; calculating an exponent difference by subtracting the first exponent sum from a maximum exponent sum by the computing circuit in the memory; generating a control signal by the computing circuit in the memory based on comparing the exponent difference with an exponent sum threshold; and selectively multiplying the first mantissa part by the second mantissa part based on the corresponding control signal by the computing circuit in the memory to generate a mantissa product.

[0147] In some embodiments, the method also includes: receiving a third input and a fourth input by a computing circuit in the memory, wherein the third input includes a third exponent part and a third mantissa part, and the fourth input includes a fourth exponent part and a fourth mantissa part; generating a second exponent sum by adding the third exponent part and the fourth exponent part by the computing circuit in the memory; and selecting a largest one from the first exponent sum and the second exponent sum as the maximum exponent sum by the computing circuit in the memory.

[0148] In some embodiments, the method includes: the computing circuit in the memory outputs one of the first mantissa part, the second mantissa part or the mantissa product according to a control signal based on the exponent difference being less than the exponent and a threshold; or, the computing circuit in the memory outputs zero according to a control signal based on the exponent difference being greater than or equal to the exponent and a threshold.

[0149] As used herein, the terms "about" and "approximately" generally refer to a value of a given quantity that may vary depending on a particular technology node associated with the subject semiconductor device. Based on the particular technology node, the term "about" may refer to a value of a given quantity that varies, for example, within a range of 10-30% of a value (e.g., +10%, ±20%, or ±30% of a value).

[0150] The features of several embodiments are summarized above so that those skilled in the art can better understand the various aspects of the present disclosure. Those skilled in the art will appreciate that they can easily use the present disclosure as a basis for designing or modifying other processes and structures for achieving the same purpose of the embodiments introduced herein and / or achieving the same advantages thereof. Those skilled in the art will also appreciate that such equivalent structures do not deviate from the spirit and scope of the present disclosure, and that they can make various changes, substitutions, and changes in the present disclosure without departing from the spirit and scope of the present disclosure.

Claims

1. A computing circuit in memory, comprising: an input circuit configured to receive: (i) a number N of first inputs and (ii) N second inputs, wherein the first inputs consist of N first symbols, N first exponents, and N first mantissas, and the second inputs consist of N second symbols, N second exponents, and N second mantissas, and wherein each of the second inputs and a corresponding one of the N first inputs form one of the N input pairs; N summing circuits, each of the N summing circuits configured to combine a respective first exponent and a respective second exponent of a respective one of the N input pairs to generate a respective one of the N exponential sums; A selector circuit configured to select a largest one from among the N exponential sums as a maximum exponential sum; N subtractor circuits, each of the N subtractor circuits configured to calculate a corresponding one of N exponential differences, each of the N exponential differences being equal to a difference between a corresponding one of the N exponential sums and the maximum exponential sum; N comparator circuits, each of the N comparator circuits being configured to: (i) compare a corresponding exponential difference with an exponent and a threshold, and (ii) generate a corresponding one of N control signals based on a comparison between the corresponding exponential difference and the exponent and the threshold; and N multiplier circuits, each of the N multiplier circuits is configured to selectively multiply the corresponding first mantissa of the corresponding input pair by the corresponding second mantissa based on the corresponding control signal to generate a corresponding one of N mantissa products.

2. The computing-in-memory circuit according to claim 1, wherein: Each of the N multiplier circuits includes a multiplexer configured to: receiving a corresponding control signal, a corresponding first mantissa, and zero; as well as selecting a corresponding first mantissa as output to be multiplied with a corresponding second mantissa according to a corresponding control signal based on the corresponding exponent difference being less than the exponent and a threshold; or According to the corresponding control signal based on the corresponding exponent difference being greater than or equal to the exponent and a threshold, zero is selected as an output to be multiplied with the corresponding second mantissa.

3. The computing-in-memory circuit according to claim 1, wherein: Each of the N multiplier circuits is configured to mask a corresponding mantissa product with zeros according to a corresponding control signal based on a corresponding exponent difference being greater than or equal to the exponent and a threshold.

4. The in-memory computing circuit according to claim 1, comprising: a similarity circuit configured to (i) compare the N second indices to each other, and (ii) generate a second control signal based on the comparison between the N second indices; as well as N multiplexers, each of the N multiplexers is configured to (i) receive the second control signal, a corresponding one of the N exponential sums, and the corresponding first index, and (ii) select the corresponding one of the N exponential sums or the corresponding first index as output to at least one of the selector circuit or the corresponding subtractor circuit according to the second control signal to calculate the corresponding exponent difference.

5. The computing-in-memory circuit according to claim 4, wherein: Each of the N multiplexers is configured as: Based on the N second indices being equal to each other, selecting the corresponding first index as the output; or Based on at least one of the N second indices being different from another of the N second indices, a corresponding one of the N sum of indices is selected as the output.

6. A computing circuit in memory, comprising: An input circuit configured to obtain a first input and a second input, wherein the first input includes a first exponent portion and a first mantissa portion, and the second input includes a second exponent portion and a second mantissa portion; a first summing circuit configured to generate a first exponential sum by summing the first exponential portion and the second exponential portion; a subtractor circuit configured to calculate an exponential difference by subtracting the first exponential sum from a maximum exponential sum; a comparator circuit configured to generate a control signal based on comparing the index difference with an index and a threshold; and A multiplier circuit is configured to (i) receive the control signal, and (ii) multiply the first mantissa portion by the second mantissa portion, wherein one of the first mantissa portion or the second mantissa is selectively selected to be zero based on the control signal.

7. The computing-in-memory circuit of claim 6, wherein: The input circuit is configured to obtain a third input and a fourth input, wherein the third input includes a third exponent portion and a third mantissa portion, and the fourth input includes a fourth exponent portion and a fourth mantissa portion, The computing circuit in the memory also includes: a second summing circuit configured to generate a second exponential sum by summing the third exponential portion and the fourth exponential portion; and A selector circuit is configured to select a largest one from among the first exponential sum and the second exponential sum as the maximum exponential sum.

8. The in-memory computing circuit according to claim 6, comprising: A multiplexer is configured to (i) receive the control signal, a third input and a fourth input from the comparator circuit, and (ii) output one of the third input or the fourth input according to the control signal, wherein the third input is zero and the fourth input is one of the first mantissa part, the second mantissa part or the product of the first mantissa part multiplied by the second mantissa part.

9. The computing-in-memory circuit of claim 6, wherein: The multiplier circuit comprises a multiplexer configured to: receiving the control signal, a mantissa product of the first mantissa portion multiplied by the second mantissa portion, and zero; and selecting the mantissa product as an output of the multiplier circuit in response to the control signal based on the exponent difference being less than the exponent and a threshold; or Zero is selected as an output of the multiplier circuit in accordance with the control signal based on the exponent difference being greater than or equal to the exponent and a threshold value.

10. A method of operating a computing circuit in a memory, comprising: Obtaining a first input and a second input through a computing circuit in a memory, wherein the first input includes a first exponent portion and a first mantissa portion, and the second input includes a second exponent portion and a second mantissa portion; generating, by a calculation circuit in the memory, a first exponent sum by adding the first exponent part to the second exponent part; calculating, by a calculation circuit in the memory, an exponential difference by subtracting the first exponential sum from the maximum exponential sum; generating, by the in-memory computation circuit, a control signal based on comparing the index difference with the index and a threshold; and Based on the corresponding control signal, the calculation circuit in the memory selectively multiplies the first mantissa portion by the second mantissa portion to generate a mantissa product.

Images