Computing circuit in memory and method for executing multiply-accumulate operation

Abstract

A computing circuit in memory (CIM) circuit includes an input circuit configured to receive N first inputs and N second inputs; n multiplier circuits, each configured to multiply a respective input pair to generate a respective one of the N products; a shift circuit configured to align the N products according to the maximum exponent sum to generate a corresponding one of the N aligned products; an adder circuit configured to sum a respective pair of the N aligned products to generate a summing result; and a fill circuit configured to (i) determine a fill number based on a bit position of a maximum non-zero value in the summation result, (ii) shift the summation result by a number of bits corresponding to the fill number to generate a shifted summation result, and (iii) apply a fill pattern having a fill number length to the shifted summation result to generate a fill sum. The embodiment of the invention further discloses a method for executing the multiplication and accumulation operation.

Description

Technical Field

[0001] Embodiments of the present application relate to a computing circuit in a memory and a method for performing a multiplication-accumulation operation. Background Art

[0002] Computer artificial intelligence (AI) is built on the foundation of machine learning, such as using deep learning techniques. With machine learning, a computing system organized as a neural network calculates the statistical likelihood that input data matches previously computed data. A neural network is a number of interconnected processing nodes that analyze data, comparing inputs to "training" data. Training data refers to computational analysis of properties of known data to develop a model to compare input data to. An example of the application of AI and data training is in object recognition, where the system analyzes the properties of many (e.g., thousands or more) images to determine patterns that can be used to perform statistical analysis to recognize input objects. Summary of the invention

[0003] According to one aspect of an embodiment of the present application, a computation in memory (CIM) circuit is provided, comprising: an input circuit configured to receive: (i) N first inputs, and (ii) N second inputs, wherein each of the N second inputs and a corresponding one of the N first inputs form an input pair among N input pairs; N multiplier circuits, each of the N multiplier circuits being configured to multiply the corresponding input pair to generate a corresponding one of N products; and a shift circuit configured to align each of the N products according to a maximum exponent and a shift position. to generate a corresponding one of N aligned products; an adder circuit configured to sum a corresponding pair of the N aligned products to generate a corresponding summation result, wherein the summation result consists of a sign part, an integer part, and a fractional part; and a padding circuit configured to: (i) determine a padding number based on the bit position of the largest non-zero value in the summation result, (ii) shift the summation result by a number of bits corresponding to the padding number to generate a shifted summation result, and (iii) apply a padding pattern having a length of the padding number to the shifted summation result to generate a padded sum.

[0004] According to another aspect of an embodiment of the present application, a computing-in-memory (CIM) circuit is provided, comprising: an input circuit configured to receive a first input, a second input, a third input, and a fourth input; a first multiplier circuit configured to multiply the first input by the second input to generate a first product; a second multiplier circuit configured to multiply the third input by the fourth input to generate a second product; a shift circuit configured to align the first product and the second product according to a maximum exponent sum to generate a first aligned product and a second aligned product, respectively; an adder circuit configured to add the first aligned product and the second aligned product to generate a sum result consisting of a sign part, an integer part, and a fractional part; and a filling circuit configured to: (i) determine a filling number based on the bit position of the maximum non-zero value in the sum result, (ii) shift the sum result by the number of bits corresponding to the filling number to generate a shifted sum result, and (iii) apply a filling pattern having a length of the filling number to the shifted sum result to generate a filled sum.

[0005] According to another aspect of an embodiment of the present application, a method for performing a multiplication-accumulation operation is provided, comprising: obtaining a first input, a second input, a third input, and a fourth input through a computation-in-memory (CIM) circuit, wherein the first input and the second input form a first input pair, and wherein the third input and the fourth input form a second input pair; generating a first product by multiplying the first input pair by the CIM circuit; generating a second product by multiplying the second input pair by the CIM circuit; aligning the first product and the second product according to a maximum exponent sum by the CIM circuit; generating a sum result by adding the aligned first product and the aligned second product by the CIM circuit; determining a padding number by the CIM circuit based on the bit position of the maximum non-zero value in the sum result; shifting the sum result by the CIM circuit by the number of bits corresponding to the padding number; and generating a padding sum by applying a padding pattern having a length of the padding number to the shifted sum result. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Various aspects of the present disclosure can be best understood from the following detailed description when read in conjunction with the accompanying drawings. It should be emphasized that, in accordance with standard practice in the industry, the various components are not drawn to scale and are only used for illustrative purposes. In fact, the size of the various components may be arbitrarily increased or reduced for clarity of discussion.

[0007] Figure 1 is a block diagram of a data computation circuit for performing MAC operations on floating point numbers with improved computation-in-memory (CIM) precision, according to some embodiments.

[0008] Figure 2 According to some embodiments, Figure 1 An example of a data calculation circuit shift and sum result.

[0009] Figure 3 is a mapping diagram according to some embodiments, mapping the number of bits filled and Figure 1 The corresponding value of the decimal point of the summation result of the data calculation circuit.

[0010] Figure 4 According to some embodiments, Figure 1 A flow chart of a method for filling bits of a data calculation circuit.

[0011] Figure 5 According to some embodiments, Figure 4 Flowchart of the target filling curve method.

[0012] Figure 6 According to some embodiments, Figure 1 A flow chart of a method for filling the summation result of a data calculation circuit.

[0013] Figure 7 According to some embodiments, Figure 6 A flow chart of a method for determining fill data.

[0014] Figure 8 According to some embodiments Figure 1 A block diagram of a search component of a data computation circuit.

[0015] Fig. 9 According to some embodiments Figure 1 Block diagram of the bit extraction component of the data calculation circuit.

[0016] Fig.10 According to some embodiments Figure 1 A block diagram of the associated components of a data computing circuit.

[0017] Fig.11 A flow chart is shown of an example method of performing MAC operations on floating point numbers with improved compute-in-memory (CIM) precision in accordance with various embodiments. DETAILED DESCRIPTION

[0018] The following disclosure provides many different embodiments or examples for realizing different features of the present disclosure. Specific embodiments or examples of components and arrangements are described below to simplify the present disclosure. Of course, these are only examples and are not intended to be limiting. For example, in the following description, forming a first component above or on a second component may include an embodiment in which the first component and the second component are directly contacted, and may also include an embodiment in which an additional component may be formed between the first component and the second component so that the first component and the second component may not be in direct contact. In addition, the present disclosure may repeat reference numbers and / or letters in various examples. This repetition is for the purpose of simplicity and clarity, and does not itself indicate the relationship between the various embodiments and / or configurations discussed.

[0019] Additionally, for ease of description, spacing relation terms such as "below," "beneath," "lower," "above," "upper," etc. may be used herein to describe the relationship of one element or component to another element or component as shown in the figures. The spacing relation terms are intended to encompass different orientations of the device in use or in the process of operation in addition to the orientation shown in the figures. The device may be otherwise oriented (rotated 90 degrees or at other orientations) and the spacing relation descriptors used herein may likewise be interpreted accordingly.

[0020] Neural networks calculate "weights" to perform calculations on new data (input data "words"). Neural networks use multiple layers of computational nodes, where deeper layers perform calculations based on the results of calculations performed by higher layers. Machine learning currently relies on the calculation of dot products and absolute differences of vectors, typically calculated by performing multiply-accumulate (MAC) operations on the parameters, input data, and weights. The calculations of large deep neural networks often involve so many data elements that it is impractical to store them in the processor cache. Therefore, these data elements are usually stored in memory.

[0021] As a result, machine learning is extremely intensive in computing and comparing many different data elements. Operational computations within the processor are orders of magnitude faster than the transfer of data elements between the processor and main memory resources. Placing all data elements in cache memory closer to the processor is prohibitively expensive for the vast majority of practical systems due to the size of the memory required to store the data elements. As a result, the transfer of data elements becomes a major bottleneck for AI computations. As data sets grow, the time and power / energy used by the computing system to move data elements can end up being multiples of the time and power used to actually perform the computations.

[0022] In this regard, computation-in-memory (CIM) circuits have been proposed to perform such MAC operations. Similar to the human brain, CIM circuits perform in-situ data processing within suitable memory circuits. CIM circuits suppress the latency of data / program reading and output results being uploaded to the corresponding memory (e.g., a memory array), thereby solving the memory (or von Neumann) bottleneck of conventional computers. Another key advantage of CIM circuits is high computational parallelism, thanks to the specific architecture of the memory array, where computations can be performed simultaneously along multiple current paths. CIM circuits also benefit from the high density of multiple memory arrays with computing devices, which typically have excellent scalability and 3D integration capabilities. As a non-limiting example, CIM circuits for various machine learning applications can perform MAC operations locally within the memory (i.e., without sending data elements to the main processor) to achieve higher throughput dot products of neuron activations and weight matrices, while still providing higher performance and lower energy compared to the main processor's computations.

[0023] The data elements processed by CIM circuits are of various types or forms, such as integers and floating-point numbers. Floating-point numbers are usually represented by a sign portion, an exponent portion, and a significand (mantissa) portion consisting of the significant digits of the number. For example, the Institute of Electrical and Electronics Engineers The specified floating point format size is 32 bits, including 23 mantissa bits, 8 exponent bits, and one sign bit. The other floating point format size is 16 bits, including 10 mantissa bits, 5 exponent bits, and one sign bit.

[0024] In machine learning applications, CIM circuits are often configured to process dot product multiplications based on performing MAC operations on a large number of data elements (e.g., input word vectors and weight matrices), all of which can be in the form of floating-point numbers, and then process the addition (or accumulation) of such dot products. The multiplication of each floating-point number pair typically includes adding the corresponding exponent parts (generating an exponent sum) and multiplying the corresponding mantissa parts (generating a mantissa product). In addition, the exponent sum of each floating-point number pair is compared with the maximum exponent sum among multiple floating-point number pairs to produce an exponent difference. This exponent difference is used to align the exponent parts of different floating-point number pairs, thereby shifting the corresponding mantissa products. The shifted mantissa product is added to the exponent of the maximum exponent sum to obtain the final sum.

[0025] In this manner, the summation of one or more pairs of products may produce or result in a relatively small output (e.g., the summation of shifted mantissa products). Because the output may be relatively small (e.g., a decimal), the summation may experience loss of digits within the constraints of the predefined bit counts (depending on the format) assigned to the corresponding values. Because the output of a summation of a pair of products is relatively small, the potential occurrence of loss of digits may introduce errors in calculating the final sum, or result in loss of information.

[0026] For example, when a relatively small sum of products (e.g., non-zero) is obtained, some of the smaller bits may be ignored due to the maximum number of bits of the data elements (e.g., 8 bits, 16 bits, 32 bits, 64 bits, etc.). In this case, the relatively small sum may be shifted to conform to a predefined or specified format, such as, but not limited to, the FP16, FP32, or FP64 format, for example, so that the integer portion of the sum is occupied by a non-zero value (shifted from the fractional portion of the sum). However, in some systems, each shifted bit may be automatically filled with zeros, which may not accurately represent the actual value of the corresponding product pair summation result. Therefore, automatically filling the shifted bits with zeros may result in an erroneous summation result, and the potential error level (e.g., the difference between the calculated result and the expected result) may be further increased based on at least the number of bits shifted (or the number of zero paddings), the number of relatively small values ​​generated by the summation, or the number of iterations to obtain the final sum (e.g., the number of elements to be added to each other).

[0027] The present disclosure provides various embodiments of a computation in memory (CIM) circuit that can determine whether to fill a summation result (e.g., the sum of shifted mantissa products) with a non-zero value after a summation and shifting process. The disclosed CIM circuit can include one or more features or components for detecting a bit position of at least one non-zero value (e.g., a "1" bit) from the summation result to determine whether to perform a filling process. For example, in order to meet a predefined format (e.g., setting the integer part to 1), the disclosed CIM circuit can shift the summation result left and fill one or more least significant bits (LSBs) corresponding to the number of bits shifted with a filling pattern. The filling pattern can include one or more non-zero values ​​to compensate for or minimize information loss in the (e.g., mantissa) product summation process. The filling pattern can be predetermined, configured, updated, or adjusted according to a configured target curve (e.g., a desired output of a fractional part). The disclosed CIM circuit can include a strategy for filling bits. For example, if the number of padding bits is relatively small (e.g., relatively small error), a relatively small padding value may be applied, and the padding value may be gradually increased as the number of padding bits increases (e.g., relatively large error). Thus, by applying or associating one or more non-zero values ​​rather than all zero values ​​to the shifted summation result (e.g., in the case of floating point operations for CIM applications), the disclosed CIM circuit may reduce the level of error that may be caused by information loss and improve / optimize the accuracy of the final sum (e.g., the padded sum).

[0028] Figure 1 FIG. 1 is a block diagram of a data calculation circuit 100 according to some embodiments of the present disclosure. Figure 1In the illustrated embodiment, the data computation circuit 100, also referred to as (e.g., CIM) circuit 100 or memory circuit 100, includes various components that are collectively configured to perform memory computations (e.g., multiply-accumulate (MAC) operations) on an input word vector and a weight matrix. The input word vector may include a plurality of (N) input data elements InDE, and the weight matrix may include a plurality of (Nd) weight data elements WtDE. In various embodiments, each input data element InDE and weight data element WtDE may include a floating point number.

[0029] As shown, circuit 100 includes memory circuit 102, input circuit 104, multiple multiplier circuits 106, multiple summing circuits 108, difference circuit 110 (e.g., sometimes referred to as subtractor circuit 110), shift circuit 112, one or more adder circuits (or adder trees) 114w-114z (e.g., sometimes referred to as adder circuit 114), at least one adder circuit or adder tree 116, one or more fill circuits 118w-118z (sometimes referred to as fill circuit 118), a first converter 120 and a second converter 122. Circuit 100 may include additional or alternative circuits, components, or devices not limited to those described herein. In some embodiments, the number of multiplier circuits 106 may correspond to the number of summing circuits 108. For example, circuit 100 may include N (weight / number of input data elements WtDE / InDE) multiplier circuits 106 and N (weight / number of input data elements WtD / InDE) summing circuits 108. It should be understood that Figure 1 The block diagram of the circuit depicted in is simplified, and thus, circuit 100 may include any of a variety of other components while remaining within the scope of the present disclosure.

[0030] The memory circuit 102 may include one or more memory arrays and one or more corresponding circuits. The memory arrays are each a memory device including a plurality of storage elements 103, each storage element 103 including an electrical, electromechanical, electromagnetic or other device configured to store one or more data elements, each data element including one or more data bits represented by a logical state. In some embodiments, the logical state corresponds to a voltage level of a charge stored in a portion or all of the storage elements 103. In some embodiments, the logical state corresponds to a physical property (e.g., resistance or magnetic orientation) of a portion or all of the storage elements 103.

[0031] In some embodiments, the storage element 103 includes one or more static random access memory (SRAM) cells. In various embodiments, the SRAM cell includes a plurality of transistors, such as a five-transistor (5T) SRAM cell, a six-transistor (6T) SRAM cell, an eight-transistor (8T) SRAM cell, a nine-transistor (9T) SRAM cell, etc. In some embodiments, the SRAM cell includes a multi-track SRAM cell. In some embodiments, the length of the SRAM cell is at least twice the width.

[0032] In some embodiments, the storage element 103 includes one or more dynamic random access memory (DRAM) cells, resistive random access memory (RRAM) cells, magnetoresistive random access memory (MRAM) cells, ferroelectric random access memory (FeRAM) cells, or non-flash cells, NAND flash cells, conductive bridging random access memory (CBRAM) cells, data buffers, non-volatile memory (NVM) cells, 3D NVM cells, or other memory cell types capable of storing bit data.

[0033] In addition to the memory array, the memory circuit 102 may also include a plurality of circuits to access or otherwise control the memory array. For example, the memory circuit 102 may include a plurality of (e.g., word line) drivers operably coupled to the memory array. The driver may apply a signal (e.g., voltage) to the corresponding storage element 103 to allow access (e.g., programming, reading, etc.) to these storage elements 103. For another example, the memory circuit 102 may include a plurality of programming circuits and / or reading circuits operably coupled to the memory array.

[0034] The memory arrays of the memory circuit 102 are all configured to store a plurality of weight data elements WtDE. In some embodiments, the programming circuit can write the weight data elements WtDE into the corresponding storage elements 103 of the memory array respectively, and the reading circuit can read the bits written into the storage elements 103 to verify or otherwise test whether the written weight data elements WdDE are correct. The driver of the memory circuit 102 may include or be operably coupled to a plurality of input activation latches, which are configured to receive and temporarily store input data elements InDE. In some other embodiments, such input activation latches may be part of the input circuit 104, which may also include a plurality of buffers, which are configured to temporarily store the weight data elements WtDE retrieved from the memory array of the memory circuit 102. Therefore, the input circuit 104 can receive the input data elements InDE and the weight data elements WtDE.

[0035] In various embodiments of the present disclosure, the input word vector (including, for example, input data element InDE) and the weight matrix (including, for example, weight data element WtDE) for performing MAC operations by the circuit 100 both include multiple floating point numbers. Therefore, each of the data element InDE and the weight data element WtDE includes a sign bit, multiple exponent bits, and multiple mantissa bits (sometimes referred to as fraction bits).

[0036] For example, each of the data element InDE and the weight data element WtDE has a BF16 format, also referred to in some embodiments as a bfloat format or a brain floating point format, in which the first bit represents the sign of the floating point number, the following eight bits represent the exponent of the floating point number, and the last seven bits represent the mantissa or decimal of the floating point number. Because the mantissa is configured to start with a non-zero value, the last seven bits of each stored data element represent an eight-bit mantissa with the first, most significant bit (MSB) equal to 1.

[0037] In some embodiments, each of the data element InDE and the weight data element WtDE has an FP16 format, also known as a half-precision format, in which the first bit represents the sign of the floating point number, the next five bits represent the exponent of the floating point number, and the last ten bits represent the mantissa or decimal of the floating point number. In this case, the last ten bits of each stored data element represent an eleven-bit mantissa, whose first MSB is equal to 1. In some other embodiments, each of the data element InDE and the weight data element WtDE has a floating point format other than BF16 or FP16 format, such as another 16-bit format, a 32-bit, 64-bit, 128-bit or 256-bit format, or a 40-bit or 80-bit extended precision format. The sign and mantissa of the data element representing a floating point number are collectively referred to as the signed mantissa of the floating point number. The MSB of the mantissa is called a hidden bit or a hidden MSB. To provide examples herein, for example in combination with at least Figure 2-11 As described, FP32 (eg, 32-bit) format may be used as an exemplary format, but it should be noted that other formats may also be similarly used to perform or obtain the benefits of the features or operations discussed herein.

[0038] Still reference Figure 1 , input circuit 104 is configured to output the entirety of each data element of data elements InDE and WtDE to each of multiplier circuit 106 and summing circuit 108. In some embodiments, input circuit 104 is configured to output the signed mantissa of each data element to multiplier circuit 106 and output the exponent of each data element to summing circuit 108, which will be described below.

[0039] The multiplier circuits 106 are each an electronic circuit, such as an integrated circuit (IC), configured to receive, for example, a sign bit InS and a mantissa InM (collectively referred to as a signed mantissa InS / InM) of each of the N data elements InDE and a sign bit WtS and a mantissa WtM (collectively referred to as a signed mantissa WtS / WtM) of each of the N data elements WtDE from the input circuit 104. The summation circuits 108 are each an electronic circuit, such as an IC, configured to receive, for example, an exponent InE of each of the N data elements InDE and an exponent WtE of each of the N data elements WtDE from the input circuit 104.

[0040] The multiplier circuits 106 may each include one or more data registers (not shown) configured to receive instances of the signed mantissas InS / InM and WtS / WtM. Figure 1 In the illustrated embodiment, multiplier circuit 106 is configured to receive instances of signed mantissas InS / InM and WtS / WtM corresponding to data elements InDE and WtDE. In some other embodiments, multiplier circuit 106 includes one or more data registers configured to receive instances of signed mantissas InS / InM and / or WtS / WtM including a hidden MSB. In some embodiments, multiplier circuit 106 includes one or more data registers configured to add the hidden MSB to the received instances of signed mantissas InS / InM and / or WtS / WtM.

[0041] The multiplier circuit 106 may include a logic circuit (not shown) configured to reformat each instance of the signed mantissa InS / InM into a two's complement mantissa InTC, also referred to as the reformatted mantissa InTC, and to reformat each instance of the signed mantissa WtS / WtM into a two's complement mantissa WtTC, also referred to as the reformatted mantissa WtTC, in the operation. The reformatted mantissa InTC has the same number of bits as the signed mantissa InS / InM, and the reformatted mantissa WtTC has the same number of bits as the signed mantissa WtS / WtM.

[0042] The multiplier circuit 106 may include one or more logic gates M1 configured to multiply some or all instances of the reformatted mantissa InTC with some or all instances of the reformatted mantissa WtTC in an operation, thereby generating N products, such as P[1] to P[N]. In various embodiments, the one or more logic gates M1 include one or more AND gates or NOR gates, or other circuits suitable for performing part or all of the multiplication operation. The one or more logic gates M1 are configured to generate each of the products P[1] to P[N] in an operation as a two-code data element whose number of bits is equal to twice the number of bits of the reformatted mantissa InTC and WtTC minus 1. The one or more logic gates M1 may be referred to as a multiplier configured to multiply some or all instances of the reformatted mantissa InTC with some or all instances of the reformatted mantissa WtTC. In some cases, the multiplier (e.g., one or more logic gates M1) may receive a signed mantissa InS / InM or a signed mantissa WtS / WtM for multiplication.

[0043] The multiplier circuit 106 is configured to generate a number N of products P[1] to P[N] in an operation. For example, the multiplier circuit 106 may generate a number N of products P[1]-P[N] equal to sixteen (e.g., sixteen elements). In some other embodiments, the multiplier circuit 106 may generate a number N of products P[1]-P[N] less than or greater than sixteen (e.g., eight, thirty-two, sixty-four, etc.).

[0044] In some embodiments, for example, those embodiments in which the data elements InDE and WtDE have a BF16 format, the multiplier circuit 106 is configured to generate products P[1]-P[N] having a total of 17 bits based on the signed mantissas InS / InM and WtS / WtM and the reformatted mantissas InTC and WtTC having a total of 9 bits. In some embodiments, for example, those embodiments in which the data elements InDE and WtDE have a FP16 format, the multiplier circuit 106 is configured to generate each of the products P[1]-P[N] having a total of 23 bits based on each of the signed mantissas InS / InM and WtS / WtM and the reformatted mantissas InTC and WtTC having a total of 12 bits. Embodiments in which multiplier circuit 106 is configured to generate each of products P[1]-P[N] having other total numbers of bits based on signed mantissas InS / InM and WtS / WtM and reformatted mantissas InTC and WtTC having other total numbers of bits are within the scope of the present disclosure.

[0045] Thus, the multiplier circuit 106 is configured to perform a multiplication and reformatting operation on the sign and mantissa bits of the input data element InDE and the weight data element WtDE in an operation to generate a two's complement product P[1]-P[N]. The multiplier circuit 106 is configured to output the product P[1]-P[N] to the shift circuit 112 on a data bus (not shown).

[0046] The summation circuits 108 each include one or more data registers (not shown) configured to receive instances of exponents InE and WtE corresponding to the number of data elements InDE and WtDE discussed above with respect to the multiplier circuits 106 .

[0047] The summing circuits 108 each include one or more logic gates A1 configured to add each instance of the exponent InE to each instance of the exponent WtE in an operation. In various embodiments, the one or more logic gates A1 include one or more full adder gates, half adder gates, ripple carry adder circuits, carry save adder circuits, carry select adder circuits, carry lookahead adder circuits, or other circuits suitable for performing a partial or full addition operation. Each logic gate A1 of the summing circuit 108 is configured to generate an exponent sum S[1]-S[N] as a data element having a total number of bits equal to the number of bits in each of the exponents InE and WtE plus 1.

[0048] Summation circuit 108 is configured to generate exponential sums S[1]-S[N] in an operation having a total number N and data element ordering corresponding to the total number N and data element ordering of products P[1]-P[N] discussed above with respect to multiplier circuit 106. Thus, for a total of N combinations of data elements InDE and WtDE, each nth combination corresponds to an nth exponent sum S[N] of the exponential sums S[1]-S[N] and an nth product P[n] of the products P[1]-P[N].

[0049] In some embodiments, such as those in which the data elements InDE and WtDE have a BF16 format, the summation circuit 108 is configured to generate a corresponding one of each of the exponent sums S[1]-S[N] having a total of 9 bits based on each of the exponents InE and WtE having a total of 8 bits. In some embodiments, such as those in which the data elements InDE and WtDE have an FP16 format, the summation circuit 108 is configured to generate each of the exponent sums S[0]-S[N] having a total of six bits based on each of the exponents InE and WtE having a total of five bits. It is within the scope of the present disclosure that the summation circuit 108 is configured to generate each of the exponent sums S[1]-S[N] having other total numbers of bits based on each of the exponents InE and WtE having other total numbers of bits. The summation circuit 108 is configured to output the exponent sums S[1]-S[N] to the difference circuit 110 on a data bus (not shown).

[0050] Differential circuit 110 is an electronic circuit, such as an IC, including one or more logic gates L1 (e.g., corresponding to or as part of selector circuit 111) and one or more logic gates B1, each of which is configured to receive an exponent and S[1]-S[N] from summing circuit 108. One or more logic gates L1 may sometimes be referred to as a selector, and one or more logic gates B1 may sometimes be referred to as a subtractor. One or more logic gates L1 are configured to generate a maximum exponent and MaxExp as a data element in an operation, the value of which is equal to the maximum value of the data elements of the exponent and S[1]-S[N], and the number of bits of which is equal to the number of bits of the data elements of the exponent and S[1]-S[N]. One or more logic gates L1 are configured to output the maximum exponent and MaxExp to one or more logic gates B1 and converter circuit 124, as described below.

[0051] The one or more logic gates B1 are configured to generate a difference value D[1]-D[N] by subtracting each data element of the exponent sum S[1]-S[N] from the maximum exponent and MaxExp in the operation. Therefore, the difference value D[1]-D[N] has a total number N and data element order corresponding to the total number N and data element order of the above-mentioned exponent sum S[1]-S[N] and product P[1]-P[N]. Figure 1In the illustrated embodiment, one or more logic gates B1 are configured to output the difference values ​​D[1]-D[N] to the shift circuit 112 on one or more data buses (not shown). In some embodiments, one or more logic gates B1 are not configured to output the difference values ​​D[1]-D[N] to the multiplier circuit 106, and each of the multiplier circuits 106 is configured to generate each instance P[N] of the product P[1]-P[N] by always performing a multiplication operation. In some other embodiments, one or more logic gates B1 are configured to output the difference values ​​D[1]-D[N] to the multiplier circuit 106, respectively, and each of the multiplier circuits 106 is configured to generate each instance P[n] of the product P[1]-P[N] by selectively performing a multiplication operation based on the corresponding instance D[n].

[0052] In various arrangements, operations of at least one of summing circuit 108 and / or differencing circuit 110 may be performed before, after, or in parallel with multiplier circuit 106. In some configurations, operations of each summing circuit 108 or differencing circuit 110 may be performed sequentially or in parallel.

[0053] Shift circuit 112 is an electronic circuit, such as an IC, including one or more registers and / or logic gates, which is configured to perform a shift operation on each instance P[n] in the product P[1]-P[n] based on the value of the corresponding instance D[n] of the difference value D[1]-D[n].

[0054] Each instance P[n] of the products P[1]-P[n] is based on the sign and mantissa of a corresponding combination of data elements InDE and WtDE, and each instance D[n] of the differences D[1]-D[n] is based on the sum of the exponents of the same combination. The shift circuit 112 is configured to, in operation, right shift each instance P[n] of the products P[1]-P[N] by an amount equal to the corresponding difference D[n], thereby generating shifted products SP[1]-SP[N], wherein the sign and mantissa bits are aligned according to the summed exponents used to generate the differences D[1]-D[N]. Based on the alignment, the shift circuit 112 is configured to generate each instance SP[n] of the shifted products SP[1]-SP[N] having the same exponent using the maximum exponent and MaxExp as a baseline.

[0055] To compensate for the right shift operation, shift circuit 112 may add an instance of the sign bit (zero or one) of each product P[n] as the leftmost bit of the corresponding shifted product SP[n]. The number of added instances of the sign bit is equal to the right shift amount determined by the corresponding difference D[n].

[0056] exist Figure 1In the illustrated embodiment, as described above, multiplier circuit 106 may generate corresponding instances P[n] of products P[1]-P[N] by performing a multiplication operation. Shift circuit 112 may include one or more shifters to receive products P[1]-P[N] from multiplier circuit 106 and selectively output (e.g., shift) one or more of the shifted products SP[1]-SP[N] to one or more adder circuits 114 based on corresponding differences D[1]-D[N]. For example, in Figure 1 , the shift products output to the one or more adder circuits 114 may include SP[w]-SP[z], where "w" through "z" may each be one of integers from 1 to N. In some configurations, a corresponding pair of shift products (e.g., a first product and a second product) may be output to or received by at least one adder circuit 116. In some other configurations, a plurality of products (e.g., SP[w]-SP[z] or more than two products) may be output to or received by at least one adder circuit 114. In one aspect of the invention, the sum of the number of SP[w]-SP[x] may be equal to N. In another aspect of the invention, the sum of the number of SP[w]-SP[z] may be less than N.

[0057] The shift circuit 112 (eg, a shifter) may be configured to shift the bits by comparing the corresponding difference values ​​among the difference values ​​D[1]-D[N] with the difference threshold value ( Figure 1 The difference threshold value may be configured based on the distribution of the difference values ​​D[1]-D[N]. In an example where the difference values ​​D[1]-D[N] present a normal distribution, the difference threshold value may be determined at one standard deviation below the mean of the normal distribution. In another example where the difference values ​​D[1]-D[N] still present a normal distribution, the difference threshold value may be determined at two standard deviations below the mean of the normal distribution. In another example where the difference values ​​D[1]-D[N] still present a normal distribution, the difference threshold value may be determined at any standard deviation below the mean of the normal distribution.

[0058] When any difference value (e.g., D[n], where n is an integer between 1 and N) is equal to or less than a difference threshold value (sometimes referred to as a “small exponent difference”), the shift circuit 112 (e.g., a shifter) may be disabled to prevent the corresponding shifted product SP[n] from being received by the at least one adder circuit 114 (e.g., not shifting the corresponding product P[n] or decoupling from the at least one adder circuit 114). Equivalently, when any difference value (e.g., D[n]) is greater than the difference threshold value (sometimes referred to as a “normal exponent difference”), the shift circuit 112 may be activated to output the corresponding shifted product SP[n] to the at least one adder circuit 114.

[0059] In other words, the shift circuit 112 may shift any of the products P[1]-P[N] and output the shifted products SP[1]-SP[N] to at least one adder circuit (tree) 114 based on comparing the corresponding difference values ​​D[1]-D[N] to the difference threshold. Thus, the sum of the numbers SP[w]-SP[z] may be equal to N. In some configurations, the shift circuit 112 may detect that at least one of the products P[1]-P[N] from the multiplier circuit 106 is zero. In this case, the shift circuit 112 may not perform a shift on the corresponding product having a zero value and / or output the product to the adder circuit 114. Thus, the sum of the numbers SP[w]-SP[z] may be less than N.

[0060] In addition, to generate SP[w]-SP[z], shift circuit 112 may right shift (or in some cases left shift) each instance P[n] of the product P[w]-P[z] by an amount equal to the corresponding difference DA[n], thereby aligning the sign and mantissa bits according to the summed exponent. In some embodiments, difference DA[n] may be generated (e.g., by difference circuit 110) based on subtracting each data element of sum S[w]-S[z] from a maximum exponent and MaxExp. The maximum exponent and MaxExp may correspond to a maximum value of the data elements of sum S[w]-S[z]. Based on this alignment, shift circuit 112 may use the maximum exponent and MaxExp as a baseline to generate each instance SP[n] of the shifted product SP[w]-SP[z] having the same exponent.

[0061] When any difference value (e.g., D[n], where n is an integer between 1 and N) is equal to or less than a difference threshold value (sometimes referred to as a "small exponent difference"), shift circuit 112 may be disabled to prevent adder circuit 114 from receiving the corresponding (e.g., shifted) product SP[n]. In some embodiments, products P[n] with such large exponent differences may be ignored.

[0062] In other words, the shift circuit 112 may shift all or part of the products P[1]-P[N] and selectively output corresponding ones of the shifted products SP[1]-SP[N] to the adder circuit 114 based on comparing the corresponding differences D[1]-D[N] to the difference threshold. Thus, the sum of the numbers SP[w]-SP[z] (output by the shift circuit 112) may be less than or equal to N. The sum is less than N when one or more of the products P[1]-P[N] are ignored (e.g., such that their corresponding exponent differences D[n] are equal to or greater than the difference threshold), and the sum is equal to N when none of the products P[1]-P[N] are ignored.

[0063] In some embodiments, such as those in which the data elements InDE and WtDE have a BF16 format, the shift circuit 112 is configured to generate each shift product (e.g., SP[0]-SP[N]) with a total of 21 bits based on each of the products P[0]-P[N] with a total of 17 bits. In some embodiments, such as those in which the data elements InDE and WtDE have an FP16 format, the shift circuit 112 is configured to generate each shift product (e.g., SP[0]-SP[N]) with a total of 27 bits based on each of the products P[0]-P[N] with a total of 23 bits. It is within the scope of the present disclosure for the shift circuit 112 to be configured to generate each shift product SP[0]-SP[N] with other total number of bits based on each of the products P[0]-P[N] with other total number of bits.

[0064] Based on the product P[0]-P[N] having a two's complement format, the shift circuit 112 is configured to generate a shift product having a two's complement format, such as SP[0]-SP[N]. As described above, Figure 1 In the example shown, the shift circuit 112 is configured to output the shifted products SP[w]-SP[z] to the adder circuit (tree) 114 on a data bus (not shown).

[0065] Each of the adder trees 114, 116 is an electronic circuit, such as an IC, including multiple layers of one or more logic gates (not shown), such as discussed above with respect to one or more logic gates A1 (of the summing circuit 108). For example, the adder trees 114, 116 may include a first layer configured to receive the shift products SP[w]-SP[z], and a last layer configured to generate the sums 115, 117 (e.g., summation results) as data elements corresponding to the sums of the shift products SP[w]-SP[z]. In some embodiments, each of the one or more consecutive layers between the first layer and the last layer is configured to receive a first number of summation data elements generated by a previous layer, and to generate a second number of summation data elements based on the first number of summation data elements, the second number being half of the first number. Thus, the total number of layers includes the first layer and the last layer and each consecutive layer (if any).

[0066] In some embodiments, one or more adder trees 114 may represent a first layer that is configured to receive shift products SP[w]-SP[z] and generate multiple summed data elements for subsequent layers. Adder tree 116 may represent a last layer that is configured to receive multiple summed data elements generated by a previous layer (e.g., from one or more adder trees 114). Although two layers are shown to provide an example, it should be noted that there may be one or more consecutive layers between the first and last layers. In some cases, there may be a layer, such as the first layer, for summing the shift products SP[w]-SP[z]. For example, circuit 100 may include at least one adder tree (circuit) 114 for summing the shift products SP[w]-SP[z] without including adder tree (circuit) 116.

[0067] In some configurations, at least one fill circuit 118 may be included between each layer of the adder trees 114, 116. In some cases, at least one fill circuit 118 may be included after at least one of the adder trees 114, 116. For example, each fill circuit 118 may be included after each adder tree 114 or before the adder tree 116. In some cases, at least one fill circuit 118 may be included after at least one adder tree 114 (but not other adder trees).

[0068] In some embodiments, the sum 115 output by the adder tree 114w may be provided to a fill circuit 118w. Each fill circuit 118 is an electronic circuit, such as an IC, that includes one or more registers, logic gates, and / or components configured to perform a fill operation on the sum 115, thereby generating a filled sum 115S. Similarly, the sum 117 output by the adder tree 114z may be provided to a fill circuit 118z to generate a filled sum 117S. For example, each fill circuit 118 may shift the corresponding sum 115, 117 so that the shifted sum 115, 117 satisfies a predefined format (e.g., an integer part having a value of "1"). The operation of the fill circuit 118 may include, but is not limited to, shifting the sum result (e.g., sum 115, 117), determining whether to fill the sum result, determining a fill mode, and / or filling (or associating) the sum result with a fill mode. In some cases, for example, the fill circuit 118w may perform a shift operation so that the sum 115 is aligned with the sum 117. One or more features or operations of fill circuit 118 may be combined with, but not limited to, Figure 2-Figure 11 to describe at least one of .

[0069] For example, Figure 2 Depicted is a diagram of a process according to some embodiments of the present invention. Figure 1An example 200 of shifting the summation result of the circuit 100. In an exemplary operation, the padding circuit 118 may receive the summation result (e.g., summation 115 or summation 117) from the corresponding adder tree 114 to perform at least one padding operation. As shown in the example block 202, the summation result may include at least an integer part and a fractional part. The summation result may include a sign part (not shown). In the FP32 format, the summation result may include 8 bits of the integer part and 23 bits of the fractional part. To provide an example of the operation of the padding circuit 118, the summation result may be a relatively small value, such as 0.000…1 obtained by summing 1.0000xxxx-1.0000xxxx.

[0070] In a further example, the filling circuit 118 can identify the value in each bit position of the summation result. Specifically, the filling circuit 118 can identify whether the value in each bit position is "0" or "1", for example, in combination with at least Figure 8 As described. In example block 204, the padding circuit 118 may identify that there is a "1" in bit position 0 and that the other bit positions are "0". This may represent a worst case scenario because after shifting the "1" to the integer part to satisfy a predefined format (e.g., FP32 format), the fractional part may be filled with all padding bits, which may result in potentially large errors (e.g., rounded values) if the padding bits are assumed to be zero. Because the padding circuit 118 identifies a non-zero value in the fractional part and the integer part is zero, the padding circuit 118 may pad the sum result with at least one non-zero value based on the padding mode determination. The padding mode determination may be combined with Figure 3-7 or Fig. 9 For example, as shown in block 206, potential rounding values ​​(e.g., fill mode) may include a range between all zeros and all ones. At least one may be combined Figure 3 Describes examples of result values ​​(eg, result fill sums) using different fill modes.

[0071] For example, Figure 3 Depicted is a diagram 300 that maps the number of bits of padding and the number of bits from Figure 1 14. As shown, the example graph 300 shows the result value of the fractional portion (y-axis) of the padded sum based on the number of padded bits (x-axis). The example graph 300 includes lines 302-308 corresponding to different paddings for the fractional portion (e.g., for the shifted sum result in block 206).

[0072] For example, when "1" is filled as the most significant bit (MSB) and "0" is filled as the other bits, line 302 can represent a fixed value of 0.5. When a first mode, such as filling the fractional part with all "1", is used, line 304 can represent a value range from 0.5 to about 1. For example, as the fill number (e.g., the number of bits to be filled in the fractional part) increases, such as from 1 to 23 in the FP32 format, the value of the fraction can increase from 0.5 to about 1. When a second mode is used to fill the MSB of the fractional part with "0" and fill the other bits with "1", line 306 can represent a value range from 0 to about 0.5. In this example, as the fill number increases, the value of the fraction can increase from 0 to about 0.5.

[0073] In a further example, when the third mode is used, line 308 can represent the range of values ​​between lines 304 and 306. The third mode can correspond to the second mode plus an offset, such as 01111 ... 111 + offset value (e.g., 23'h20_0000). The offset value can be a predefined or configurable value. In some cases, the offset value (or fill pattern) can be selected from a table or array according to the number of fills. Adding the offset value to the second mode causes the amplitude of the value associated with the second mode to increase (e.g., the value of the fill and the corresponding mode is increased). In some cases, the offset value may be a negative value subtracted from the first pattern. For example, subtracting the offset value from the first mode causes the amplitude of the value associated with the first mode to decrease (e.g., the value of the fill and the corresponding mode is reduced). By applying "1" filling with or without an offset, the error level of information loss can be minimized.

[0074] In some configurations, a portion of the padding pattern (e.g., a certain number of LSBs) may be a fixed value of "0" or "1". For example, portion 310 of example diagram 300 illustrates a specific number of LSBs padded for a fractional portion. As shown, padding these LSBs (given the size of the padding number) may not significantly affect the overall result of, for example, the padded sum. Therefore, fixed values ​​may be configured for these LSBs (e.g., for portion 310) to minimize resource consumption, including but not limited to memory reduction or computational resource reduction (if the padded sum is used for subsequent calculations). Other operations of padding circuit 118 (or each component of padding circuit 118) may be combined with, but not limited to, Figure 4-Figure 11 to describe at least one of .

[0075] It should be noted that while two exemplary adder trees 114 (e.g., adder tree 114w and adder tree 114z) are shown for purposes of providing an example, a greater or lesser number of adder trees 114 may be used or included in the operation. For example, the circuit 100 may include one adder tree 114 (e.g., adder tree 114w, for outputting the sum 115 without the adder tree 114z). In another example, the circuit 100 may include more than two adder trees 114 to sum the shift products SP[w]-SP[z]. In some configurations, the number of fill circuits 118 may be the same as the number of adder trees 114. In some other configurations, the number of fill circuits 118 may be greater or less than the number of adder trees 114, for example, the fill circuit 118w may be included after the adder tree 114w, while the fill circuit 118z may not be included after the adder tree 114z.

[0076] Return to reference Figure 1 , adder circuit (tree) 116 is an electronic circuit, such as an IC, that includes multiple layers of one or more logic gates (not shown), such as discussed above with respect to one or more logic gates A1 (of summation circuit 108). For example, adder tree 116 may include a first layer configured to receive sums 117S and 115S, and a last layer configured to generate sum PSTC as a data element corresponding to the sum of shift products SP[w]-SP[x] and SP[y]-SP[z]. In some embodiments, each of one or more consecutive layers between the first layer and the last layer is configured to receive a first number of sum data elements generated by a previous layer, and to generate a second number of sum data elements based on the first number of sum data elements, the second number being half the first number. Thus, the total number of layers includes the first layer and the last layer and each consecutive layer (if present).

[0077] In some embodiments, adder circuit 116 may receive all shift products SP[w]-SP[z] directly from shift circuit 112, e.g., adder circuit 114 may not be included. In this case, adder circuit 116 may sum the shift products SP[w]-SP[z] and generate a summed result. Fill circuit 118 may be included after adder circuit 116, where adder circuit 116 may determine whether to fill the summed result from adder circuit 116. In this case, fill circuit 118 may perform a fill operation and generate a sum PSTC.

[0078] In some embodiments, the sum PSTC (e.g., corresponding to the sum of sum 115S and sum 117S) is sometimes referred to as a partial sum PSTC or a mantissa sum PSTC, and the total number of bits corresponds to the number of bits and the number of data elements of the shift product SP[w]-SP[z]. In some embodiments, the number of bits of the sum PSTC is equal to the number of bits of the shift product SP[w]-SP[z] plus the number of bits that can represent the number of data elements of the shift product SP[w]-SP[z]. In some embodiments, the number of bits of the sum PSTC is equal to the number of bits of the shift product SP[w]-SP[z] plus four bits that can represent 16 data elements of the shift product SP[w]-SP[z].

[0079] In some embodiments, such as those in which the data elements InDE and WtDE have a BF16 format, the adder tree 114 is configured to generate a total of 25 bits of sum PSTC based on each of the shift products SP[w]-SP[z] having a total of 21 bits. In some embodiments, such as those in which the data elements InDE and WtDE have a FP16 format, the adder tree 114 is configured to generate a total of 31 bits of sum PSTC based on each of the shift products SP[w]-SP[z] having a total of 27 bits. It is within the scope of the present disclosure that the adder tree 114 is configured to generate a sum PSTC based on each of the shift products SP[w]-SP[z] having other total bits.

[0080] According to various embodiments of the present disclosure, based on the shift product SP[w]-SP[z] having a two's complement format, the adder tree 114 is configured to generate a sum PSTC having a two's complement format. Therefore, the adder tree 114 is configured to output the sum PSTC to the converter 116 on a data bus (not shown). In some other embodiments, the adder tree 114 may output the sum PSTC to a circuit (not shown) outside the circuit 100.

[0081] Converter 116 is an electronic circuit, such as an IC, that includes logic circuitry configured to receive the sum PSTC from adder tree 114 in an operation and convert the sum PSTC from two's complement to a sum PSSM having a sign-plus-mantissa format. Converter 116 is configured to generate a sum PSSM having the same number of bits as the sum PSTC. Figure 1 In the illustrated embodiment, converter 116 is configured to further output the sum PSSM on a data bus (not shown) to converter 118. In some other embodiments, converter 116 may output the sum PSSM to a circuit external to circuit 100 (not shown).

[0082] Converter 118 is an electronic circuit, such as an IC, that includes logic circuitry configured to receive the sum PSSM from converter 116, receive the maximum exponent and MaxExp from differential circuit 110, and convert the sum PSSM from a sign-plus-mantissa format to a sum PS having an output format based on the sum PSSM and MaxExp and different from the sign-plus-mantissa format, such as a floating-point format as described above, in an operation. In various embodiments of the present disclosure, converter 118 may generate a sum PS that is configured to be compatible with a circuit (not shown) external to circuit 100. For example, converter 118 is configured to output the sum PS to a circuit (not shown) external to circuit 100, such as a memory array or other instance of circuit 100 that is part of a convolutional neural network (CNN). In some configurations, converter 116 may be part of converter 118, and vice versa.

[0083] Figure 4 According to some embodiments, Figure 1 Flowchart of method 400 for filling bits of circuit 100 of FIG. Example method 400 may be performed by circuit 100 or one or more components of circuit 100. Therefore, the following embodiments of method 400 may be combined with but not limited to Figure 1-Figure 3 and / or Figure 5-Figure 11 The illustrated embodiment of method 400 is provided as an example only and does not limit the scope of the present disclosure. Therefore, it should be understood that any of the various operations of method 400 may be omitted, reordered, and / or added while still within the scope of the present disclosure. Method 400 is not limited to the arrangement of the operations discussed herein, such that certain operations may be performed before, during, or after other operations.

[0084] Method 400 begins with operation 402, which is used to determine the maximum number of bits to be filled. The maximum number of bits to be filled can be user-defined, pre-configured, or updated based on the expected value of the fill sum (e.g., the expected rounding value). In some cases, the maximum number of bits to be filled can be based on the format of the floating point number. For example, depending on the format, the maximum number of bits to be filled can be set according to the width of the mantissa portion, such as 23 bits for the FP32 format (mode). For FP16, FP8, etc., the maximum number of fill bits can be set to different values.

[0085] The method 400 continues to operation 404, which is used to determine the number of bits to be set to a fixed value (e.g., a fixed "0" or "1" value). The number of bits to be set can be predetermined, configured, or updated according to user preferences or CIM applications. The number of bits set to a fixed value can be located at different positions, such as setting 6 LSBs to "1", setting 4 MSBs to "1", setting a series of bit positions to "0", etc. For example, other lengths can be considered depending on the CIM application. Setting 6 LSBs to a fixed value As an example, in the FP32 format, the table storing the bit pattern can be reduced from a 23-bit table to a 17-bit table because 6 of the 23 bits are set to a fixed value. Therefore, the logic circuit occupancy rate can be reduced in proportion to the number of bits set to a fixed value. To provide an example, multiple LSBs can be set to a fixed value.

[0086] The method 400 continues to operation 406 for determining an offset value to be added. The offset value may be a user defined value or may be set according to a CIM application. The offset value may include a bit width corresponding to the maximum number of bits to be filled. In some cases, the offset value may include a bit width corresponding to the maximum number of bits to be filled minus the number of bits set to a fixed value. The offset value may be added as part of generating / creating a fill pattern (e.g., a fill data pattern). In some cases, for example, the offset value may not be added according to a user configuration or according to a CIM application.

[0087] Method 400 continues to operation 408 for setting a target fill curve. Setting a target fill curve may refer to or include setting a fill pattern to obtain a desired curvature or value of at least a portion of the fill sum (e.g., the example curvature shown in example graph 300, or at least one of lines 302-308). The fill pattern may be stored in a table or array. Operation 408 for setting a target fill curve may be combined with at least Figure 5 to describe.

[0088] Figure 5 According to some embodiments, Figure 4 408. For example, operation 408 begins with operation 502, which is used to determine whether to set the target fill to 0.5. Operation 502 can be performed after determining whether there is an offset value to be added. The target fill can be user-defined or based on the CIM application. When the fill summation result (for example, combined with Figure 2 ), the target fill can represent the expected value.

[0089] At operation 502, if the target padding is set to 0.5, operation 408 proceeds to operation 504 for setting the MSB of the table (e.g., padding mode) to "1" and the other bits to "0". In other words, at operation 504, the padding value may be set to the MSB to "1" and the other bits to "0". In this case, the value obtained by padding the summation result may be associated with line 302, as shown in conjunction with Figure 3 If the target padding is not set to 0.5, operation 408 proceeds to operation 506 to determine whether the target padding is set to a range from 0.5 to about 1. If the target padding is set to a range from 0.5 to about 1, operation 408 may proceed to operation 508 to set all bits of a table (e.g., padding mode) to "1", e.g., setting the bits of the padding value to all "1".

[0090] If the target fill is not set to a range from 0.5 to about 1, operation 408 may continue to operation 510 to determine whether the target fill is set to a range from 0 to about 0.5. If the target fill is set to a range from 0 to about 0.5, operation 408 may continue to operation 512 for setting the MSB of the fill pattern to "0" and the other bits to "1". In other words, the fill value may be set to the MSB to "0" and the other bits to "1". Otherwise, if the target fill is not set, operation 408 may continue to operation 514 for setting the bits in the table (e.g., fill pattern) according to a predetermined pattern or a user-defined pattern.

[0091] Operation 408 continues to operation 516 for determining whether to add the offset value to a table, for example, representing the fill mode. Figure 4 At least operation 406 of 408 may be used to describe the determination of whether to add an offset value. For example, if the offset value is defined by the user, operation 408 may continue to operation 518, such as adding the offset value to a table. In this case, the fill mode (e.g., the target fill curve) may correspond to the sum of the fill value (from one of operations 504, 508, 512, 514) and the offset value.

[0092] If the offset value is not set, operation 408 can directly set one of the fill values ​​(from one of operations 504, 508, 512, 514) as the fill pattern for filling the summation result. The set target fill curve or fill pattern can be stored in a memory array, table, or memory device local or remote to the circuit 100. The fill circuit 118 can retrieve or access the stored fill pattern to fill the summation result. In various implementations, the fill pattern can be defined by the user. For example, the fill pattern can be retrieved or obtained from a memory device, such as a memory device local to the circuit 100 or remote from the circuit 100. The fill circuit 118 can use the obtained fill pattern to fill the summation result.

[0093] Figure 6 According to some embodiments, Figure 1 Flowchart of method 600 for filling the summation result of circuit 100. Example method 600 may be performed by circuit 100 or one or more components of circuit 100 (eg, filling circuit 118). Therefore, the following embodiments of method 600 may be combined with but not limited to Figure 1-Figure 5 and / or Figure 7-Figure 11 The illustrated embodiment of method 600 is provided as an example only and does not limit the scope of the present disclosure. Therefore, it should be understood that any of the various operations of method 600 may be omitted, reordered, and / or added while still within the scope of the present disclosure. Method 600 is not limited to the arrangement of the operations discussed herein, such that certain operations may be performed before, during, or after other operations.

[0094] Method 600 begins with operation 602, which is used to search for a "1" bit in the integer portion of the summation result. To provide an example, Figure 8 Method 600 is described. Figure 8 According to some embodiments Figure 1 Block diagram 800 of a search component 802 of circuit 100. A search component 802 can be part of the filling circuit 118. A search component 802 can be an electronic circuit or component, such as an IC, including one or more registers, logic gates and / or components configured to search for non-zero values ​​and determine the filling number. A search component 802 may include a detector 804, a shift number decoder and an output selector 808. A detector 804 can be configured to detect one or more non-zero values ​​(e.g., "1") in a bit (Bit) of the summation result. The shift number decoder 806 can be configured to determine the number of bits of the shift summation result. The output selector 808 can be configured to select the output of the filling number.

[0095] Corresponding to operation 602, a detector 804 may be configured to detect a "1" in at least one of the integer part and / or the fractional part of the summation result. A detector 804 may include one or more logic gates, such as an OR gate, to generate and output corresponding signals of the integer part and the fractional part based on whether the value of each part is zero or non-zero.

[0096] For example, the bits of the integer part (e.g., bits [30:23] of the FP32 format) can be input to one or more logic gates of a detector 804. If at least one bit of the integer part is a "1" bit, the one or more logic gates can generate a "1" signal or a "true" indication as an output of detecting one (DetOneInt) in the integer part. A "1" DetOneInt can indicate that the value of the integer part is non-zero. If the bits of the integer part are all "0", the one or more logic gates can generate a "0" signal or a "false" indication for DetOneInt, indicating that the value of the integer part is zero. The one or more logic gates can output the DetOneInt signal to the output selector 808 to select a value as the padding number (PadNum). The output selector 808 can output the padding number to determine the number of bits of the padding pattern. The padding number can be based on at least one of the DetOneInt signal, the DetOneFra signal, and / or the one detection (OneDet) signal. If the summation result is not padded, the output selector 808 can output the padding number as zero. The output selector 808 may output the OneDet value as a fill number, for example indicating the number of bits used to fill the sum result and to shift the sum result.

[0097] In response to searching for "1" in the integer portion, the method 600 continues to operation 604 to determine whether a non-zero value (e.g., "1") is found in the integer portion of the summation result. If a non-zero value is found in the integer portion of the summation result (e.g., DetOneInt=1), the method 600 continues to operation 606. At operation 606, the output selector 808 can generate a PadNum of zero based on the integer portion including the non-zero value. For example, because a "1" is found in the integer portion, one search component 802 can determine that the summation result is greater than or equal to 1 and that padding is not required. Therefore, the output selector 808 can output zero as PadNum because no shift is required, or a right shift (rather than a left shift) can be performed to satisfy a predefined format.

[0098] If "1" is not found in the integer part, the method 600 continues to operation 608 to search for "1" in the fractional part of the summation result. A search component 802 can use a detector 804 to search for "1" in the fractional part. A detector 804 may include one or more logic gates to perform the operations discussed herein, such as one or more logic gates similar to searching for "1" in the integer part. For example, in the FP32 format, the fractional part may include bits [22:0]. Bits [22:0] may be provided as inputs to one or more logic gates, such as at least one OR gate. One or more logic gates of a detector 804 may generate or output a signal DetOneFra based on whether at least one bit in the fractional part includes a non-zero value (e.g., "1"). A "1" in DetOneFra may indicate that the fractional part contains a non-zero value. A DetOneFra of "0" may indicate that all bits in the fractional part are zero (e.g., "1" is not found).

[0099] Assuming that the value of the integer part is zero, the value of the fractional part being zero may indicate that all of the summation result (e.g., all bits of the summation result) are zero, and therefore no padding is performed. For example, if all bits [22:0] in the fractional part are zero, one or more logic gates of a detector 804 may generate a signal DetOneFra of "0" indicating that the summation result is not to be padded (e.g., because no left shift is required). A detector 804 may send the DetOneFra signal to the output selector 808. In addition, assuming that the value of the integer part is zero, a non-zero value in the fractional part may indicate that at least one left shift is to be performed, and therefore, the summation result needs to be padded. For example, if at least one bit [22:0] in the fractional part is non-zero, one or more logic gates of a detector 804 may generate a signal DetOneFra of "1" indicating that the summation result is padded with a non-zero value (e.g., to compensate for information loss). In this case, when the DetOneFra signal is sent to the output selector 808, the output selector 808 may output OneDet as the padding number.

[0100] After searching for a non-zero value (e.g., "1") in the fractional portion (in operation 608), the method 600 continues to operation 610 to determine whether a non-zero value is found in the fractional portion. If "1" is not found in the fractional portion, for example, one or more logic gates of a detector 804 of the fractional portion return "0" DetOneFra, the method 600 continues to operation 614. In operation 614, a search component 802 can determine that the sum result is a real zero (e.g., all bits are zero) and that padding is not required. In this case, a search component 802 (e.g., output selector 808) can output zero as the padding number.

[0101] If a "1" is found in the fractional portion, e.g., one or more logic gates of a detector 804 of the fractional portion return "1" DetOneFra, then the method 600 proceeds to operation 612 to determine padding data. For example, at operation 612, a search component 802 may determine that the sum result is to be left-shifted (e.g., the integer portion is zero and the fractional portion is non-zero), and determine padding data (or padding pattern) for association. In this case, a search component 802 (e.g., output selector 808) may output OneDet as the padding number. For example, the padding data determination of operation 612 may be combined with at least Figure 7 to describe.

[0102] Figure 7 According to some embodiments Figure 6 Flowchart of operation 612 for padding data determination of FIG. Operation 612 may be performed by padding circuit 118, a component of padding circuit 118 (e.g., a search component 802), or other components of circuit 100. Operation 612 for determining padding data begins with operation 702 for searching for a maximum "1" in the fractional portion of the summed result.

[0103] Corresponding to operation 702, a shift bit decoder 806 may receive a fractional value (e.g., bits of the fractional portion of the summed result) as input. The shift bit decoder 806 may be an electronic circuit or component, such as an IC, that includes one or more registers, logic gates, and / or components configured to search for the largest "1" in the fractional portion. For example, Figure 8 As shown, the shift number decoder 806 may include a plurality of (N) multiplexers (MUXs). Each of the N MUXs may be associated with a corresponding bit position of the fractional part. In this case, the number of MUXs included in the shift number decoder 806 may correspond to the number of bits of the fractional part. Each of the N MUXs may correspond to a corresponding continuous layer for determining the number of bits (or filler number) of the shift sum result.

[0104] Each MUX can receive three inputs. The first input can include the corresponding bit position value of the fractional part. The first input can be used as a control signal to select one of the second input or the third input as the output of the MUX, for example, to be provided to the next consecutive layer (or as OneDet). For example, if the corresponding bit position value (e.g., the first input) is the highest / largest "1" in the fractional part (e.g., the most significant "1' bit), the second input can include the number of bits to be shifted. The third input can include zero (for the first layer of consecutive layers) or the output value carried from the previous layer.

[0105] For example, in the case of FP32 format, the shift bit decoder 806 may include 23 MUXs. More or fewer MUXs (or other components configured to perform similar tasks) may be included, not limited to 23 MUXs, such as for FP16, FP8, etc. Each successive layer of the 23 MUXs may be associated with a corresponding bit position. Figure 8 As shown, the first MUX can receive the value at bit position 0, the second MUX can receive the value at bit position 1, etc., and the last MUX of the shift number decoder 806 can receive the value at bit position 22. If the value is "1" at a given bit position, the corresponding MUX can output the corresponding number of bits for shifting. For example, if the value at bit position 0 is "1", the corresponding MUX can output 23, indicating a left shift of 23 bits. In another example, if bit position 19 is "1", the corresponding MUX can output 4, indicating a left shift of 4 bits. If the value is "0" at a given bit position, the output from the previous consecutive layer (or the previous MUX) can be forwarded to the next MUX (or if the MUX is the final / last MUX of the shift number decoder 806, as OneDet). Therefore, the shift number decoder 806 can determine the bit position (number) of the largest "1" in the fractional part, and output the corresponding fill number of that number to shift the sum result (e.g., OneDet). The shift bit decoder 806 may output OneDet to the output selector 808 , where the output selector 808 may select OneDet as PadNum in response to receiving “0” DetOneInt and “1” DetOneFra from one detector 804 , for example.

[0106] Corresponding to operation 704 (continuing from operation 702), a search component 802 can determine a fill number (PadNum) based on the number of bits of the largest "1" in the fractional part. For example, the shift number decoder 806 can output OneDet to the output selector 808, indicating the number of bits of the shifted sum result. In response to receiving "0" DetOneInt and "1" DetOneFra from a detector 804, the output selector 808 can select OneDet as PadNum. Therefore, the fill number can correspond to the shift number (e.g., the number of bits of the shifted sum result).

[0107] In some configurations, the operation of the shift number decoder 806 may be performed after receiving a signal from one detector 804. For example, if one detector 804 outputs "1" DetOneInt and / or "0" DetOneFra to the output selector 808, the output selector 808 may (e.g., directly) output zero as the fill number. In this case, the shift number decoder 806 may not decode the shift number. Otherwise, if one detector 804 outputs "0" DetOneInt and "1" DetOneFra, the operation of the shift number decoder 806 may be activated / enabled. The output selector 808 may receive the output (e.g., OneDet) from the shift number decoder 806 and output the fill number as OneDet (e.g., the shift number). In some other configurations, the operation of the shift number decoder 806 may be performed in parallel with one detector 804, or may be independent of the output of one detector 804.

[0108] In some embodiments, the shift decoder 806 may include one or more components or features similar to the differential circuit 110, for example, to determine the difference between the maximum non-zero value (e.g., "1") in the fractional portion and the maximum number of bits to be filled. For example, the shift decoder 806 may receive the bits of the fractional portion and identify the maximum non-zero value in the fractional portion. The shift decoder 806 may receive the maximum number of bits to be filled. The shift decoder 806 may subtract the maximum number of bits to be filled for the bit position of the maximum non-zero value in the fractional portion, thereby generating a difference value. The difference value may represent the shift sum result and the number of bits of the fill number.

[0109] Continuing to operation 706, the padding circuit 118 may be configured to extract padding data having a padding number length from a padding data pattern (eg, a padding pattern). The padding data may be at least a subset of the padding pattern. The extraction of the padding data may be combined with at least Fig. 9 For example, Fig. 9 According to some embodiments Figure 1 1 is a block diagram 900 of a bit extraction component 902 of the circuit 100 of FIG. The bit extraction component 902 may be part of the padding circuit 118. The bit extraction component 902 may be an electronic circuit or component, such as an IC, that includes one or more registers, logic gates, and / or components configured to extract padding data (or padding pattern or bit pattern) and output the padding pattern for the padding circuit 118 to pad (or associate) the shift-and-sum result.

[0110] The bit extraction component 902 may include or store N bit patterns corresponding to the maximum number of bits to be filled, such as 23 patterns in the FP32 format. The bit extraction component 902 may include a MUX configured to receive each of the N bit patterns as input and receive an output from one of the search components 802 as a control signal. Each of the N bit patterns may be, for example, in combination with, but not limited to, 4- Figure 5 For example, each bit pattern may include a padding value with or without an offset. Based on the padding number (e.g., the length of the bit pattern), the MUX of the bit extraction component 902 can extract the corresponding bit pattern and output a bit pattern (e.g., padding pattern) for padding the (shifted) summation result. The process for padding the summation result can be combined with at least Fig.10 to describe.

[0111] In some implementations, the bit extraction component 902 may include or store a single bit pattern having a maximum number of bits to be padded. In this case, the bit extraction component 902 may include one or more components configured to extract one or more bits from the bit pattern based on or according to a padding number (e.g., a desired length of the padding pattern). The bit extraction component 902 may output at least a portion of the bit pattern having the length of the padding number. Fig.10 As described, the bit extraction component 902 can output the extracted bit pattern (or fill pattern) to the association component 1002.

[0112] Fig.10 According to some embodiments Figure 1 1000 of an associating component 1002 of a circuit 100 of FIG. The associating component 1002 may be part of the padding circuit 118. The associating component 1002 may be an electronic circuit or component, such as an IC, that includes one or more registers, logic gates, and / or components configured to associate (or pad) an extracted bit pattern (e.g., a padding pattern) with a summation result. The associating component 1002 may include at least a shifter circuit 1004 and an adder circuit 1006 to perform the operations discussed herein.

[0113] The shifter circuit 1004 may be configured to shift the summation result. For example, the shifter circuit 1004 may receive the summation result from the corresponding adder tree 114. In some cases, the shifter circuit 1004 may receive a fill number from a search component 802, where the fill number corresponds to the shift number (e.g., the number of bits of the shifted summation result). In other cases, the fill number may be different from the shift number. To provide an example, the shift number may correspond to the fill number or the number of bits of the fill data.

[0114] The shifter circuit 1004 may shift the sum result to the left according to the shift amount. The shifter circuit 1004 may generate a shifted sum result in response to the shifting of the sum result. The shifter circuit 1004 may output the shifted sum result to the adder circuit 1006.

[0115] The adder circuit 1006 may be configured to associate, add, or fill the shift-sum result with the extracted bit pattern. For example, the adder circuit 1006 may receive the shift-sum result from the shifter circuit 1004 and the bit pattern from the bit extraction component 902 as input. The adder circuit 1006 may include one or more logic components to associate the shift-sum result with the extracted bit pattern to generate a padding sum. The adder circuit 1006 may output the padding sum to other circuits or components, such as, but not limited to, the adder tree 116 (or other adder trees 114) or the converter 120. The output of the adder circuit 1006 may be the output of the padding circuit 118. Therefore, by generating a padding sum for subsequent summing or for the converter 120, potential information loss may be compensated and potential error levels may be reduced.

[0116] In some implementations, the fill pattern may include a fixed value portion (e.g., "0" or "1") and a bit pattern portion (e.g., a combination of "1" and "0"). For example, to provide an example, the fixed value portion may include a user-defined number of bits, such as a fixed value of 7 bits. The fixed value may be located in the LSB portion of the fill pattern (e.g., the least significant 7 bits). In the case of the FP32 format, the 16 other bits may be a predefined bit pattern (e.g., the most significant 16 bits of the fill pattern). In this case, if the fill number is lower than the user-specified fixed value, the fill circuit 118 may associate the shift-sum result with the fixed value. Otherwise, if the fill number is equal to or higher than the user-specified fixed value, the fill circuit 118 may associate the shift-sum result with a combination of the bit pattern and the fixed value.

[0117] For example, the user-specified value may be 7. If the fill number of the sum result is 3, the fill circuit 118 may fill the 3 LSBs of the shifted sum result with a fixed value. If the fill number is 12, the fill circuit 118 may fill the 7 LSBs of the shifted sum result with a fixed value and fill the 5 subsequent LSBs of the shifted sum result with a portion of a predefined bit pattern.

[0118] Fig.11 1 is a flow chart showing an example method 1100 for performing MAC operations on floating point numbers with increased CIM precision according to various embodiments. The example method 1100 may be performed by the circuit 100 (eg, sometimes referred to as the CIM circuit 100) or one or more components of the circuit 100. Thus, the following embodiments of the method 1100 may be combined with, but not limited to, Figure 1-Figure 10 The illustrated embodiment of method 1100 is provided as an example only and does not limit the scope of the present disclosure. Therefore, it should be understood that any of the various operations of method 1100 can be omitted, reordered, and / or added while remaining within the scope of the present disclosure.

[0119] Method 1100 starts with operation 1102 for obtaining multiple inputs. Circuit 100 (e.g., input circuit 104) can obtain / receive multiple (N) first inputs and N second inputs. Each of the N second inputs and a corresponding one of the N first inputs form one of the N input pairs. For example, the N first inputs may include a first input and a third input. The N second inputs may include a second input and a fourth input. The first pair of inputs may include a first input and a second input. The second pair of inputs may include a third input and a fourth input. Each input may include a symbol portion, an exponent portion, and a mantissa portion. For example, the N first inputs may consist of N first symbols, N first exponents, and N first mantissas. The N second inputs may consist of N second symbols, N second exponents, and N second mantissas.

[0120] The method 1100 continues to operation 1104 for generating N products. Each of the N products can be calculated based on a corresponding pair of inputs (e.g., mantissa portions of the inputs). For example, the circuit 100 (e.g., the multiplier circuit 106) can generate a first product by multiplying a first input pair, e.g., the product of a corresponding first mantissa of the first input and a corresponding second mantissa of the second input. The circuit 100 can generate a second product by multiplying a second input pair, e.g., the product of a corresponding third mantissa of the third input and a corresponding fourth mantissa of the fourth input. The circuit 100 can multiply one or more other input pairs to generate corresponding one or more of the N products.

[0121] The method 1100 continues to operation 1106 for aligning products, such as each of the N products. Taking the generated first product and the second product as an example, the circuit 100 (e.g., the shift circuit 112) can align the first product and the second product according to the maximum exponent sum of the N products. By aligning the N products with the maximum exponent sum, the circuit 100 can generate corresponding N aligned products. The aligned first product and the second product can form a pair of aligned products.

[0122] In various implementations, circuit 100 (e.g., summing circuit 108 and selector circuit 111) may be configured to determine or select the largest exponential sum. For example, circuit 100 (e.g., summing circuit 108) may combine the exponents of each pair of inputs, such as the respective first exponent and the respective second exponent of the respective one of the N input pairs, to generate a respective one of the N exponential sums. Circuit 100 (e.g., selector circuit 111) may select the largest one of the N exponential sums as the largest exponential sum.

[0123] In some embodiments, the circuit 100 (e.g., the subtractor circuit or the difference circuit 110) can be configured to determine or calculate N exponent differences, where each of the N exponent difference values ​​corresponds to one of the N input pairs. For example, the circuit 100 (e.g., the difference circuit 110) can calculate a corresponding one of the N exponent differences based on a difference between a maximum exponent and a corresponding one of the exponent sums associated with the corresponding input pair. In this case, for example, each of the N exponent differences can be equal to a difference between a corresponding one of the N exponent sums and the maximum exponent sum. In response to obtaining the exponent differences, the circuit 100 (e.g., the shift circuit 112) can align the N products by shifting each of the N products based on a corresponding one of the N exponent differences.

[0124] The method 1100 continues to operation 1108 for generating a summation result. The circuit 100 (e.g., the adder circuit (tree) 114, 116) can generate a summation result by summing a corresponding pair of the N aligned products. For example, the circuit 100 can generate the summation result by adding the aligned first product and the aligned second product. The summation result can consist of a sign part, an integer part, and a fractional part.

[0125] Circuit 100 (e.g., padding circuit 118) can determine whether to pad the sum result, for example, to compensate for information loss. For example, circuit 100 (e.g., padding circuit 118) can identify a first value associated with an integer portion of the sum result and a second value associated with a fractional portion. The first value can represent the value of one or more bits in the integer portion (e.g., indicating whether there is a non-zero value in the integer portion). The second value can represent the value of one or more bits in the fractional portion (e.g., indicating whether there is a non-zero value in the fractional portion). Circuit 100 can identify the first and second values ​​by identifying the bit values ​​associated with the integer portion and the fractional portion, respectively. If the first value is a non-zero value (e.g., greater than zero) or the second value is zero, circuit 100 can determine that a left shift operation can not be performed on the sum result. In this case, circuit 100 can determine not to pad the sum result. On the other hand, if the first value is zero and the second value is non-zero (e.g., greater than zero), circuit 100 can determine to perform a left shift and pad operation on the sum result because the sum result is a relatively small value.

[0126] When it is determined that the sum result is to be padded, the method 1100 proceeds to operation 1110 to determine the padding number. The circuit 100 (e.g., the padding circuit 118) may determine the padding number based on the bit position of the largest non-zero value (e.g., "1") in the sum result, such as at least in combination with Figure 8 As described. For example, to determine the number of fills, the circuit 100 can determine the bit position of the largest non-zero value in the fractional portion of the summation result. The circuit 100 can determine the number of fills based on the difference between the bit position (of the largest non-zero value) and the predetermined value. The predetermined value can be the maximum number of bits to be filled, which can be user-defined according to the digital format, etc. In some cases, the circuit 100 can identify the number of bits to shift the summation result based on the difference between the bit position of the largest non-zero value and the maximum number of bits to be filled.

[0127] In various embodiments, circuit 100 (e.g., padding circuit 118) may receive a plurality of padding patterns. Each of the plurality of padding patterns may have a corresponding length. For example, the length of a first padding pattern may be 1 bit, the length of a second padding pattern may be 2 bits, the length of a third padding pattern may be 3 bits, and so on. The total number of padding patterns may correspond to the maximum number of bits to be filled. Circuit 100 may extract or select a padding pattern from a plurality of padding patterns for association or padding based on the length of the padding pattern corresponding to the padding number. In some cases, circuit 100 may include a padding pattern having a length of the maximum number of bits to be filled. In this case, circuit 100 may extract at least a portion of a padding pattern for association based on the padding number.

[0128] Method 1100 continues to operation 1112 for shifting the sum result. Circuit 100 (e.g., padding circuit 118) may shift (e.g., left shift) the sum result by an amount of bits corresponding to the pad number (e.g., shift number). Circuit 100 may generate a shifted sum result in response to shifting the sum result.

[0129] The method 1100 continues to operation 1114 for generating a padding sum. The circuit 100 (e.g., padding circuit 118) may generate a padding sum by associating a padding pattern having a padding number length to a shift-and-sum result. For example, after the shift-and-sum result, the shift-and-sum result may be added, padded, or associated with a padding pattern (e.g., a bit pattern) to generate a padding sum. The padding sum may include the padding pattern as one or more LSBs (associated with the length of the padding number).

[0130] In some implementations, the fill mode may be predefined, configured, or updated based on a desired target curve (e.g., as combined with Figure 3-Figure 5At least one of the above, etc.). For example, the circuit 100 (e.g., the filling circuit 118) may receive a maximum number of bits to be filled. The circuit 100 may receive a plurality of bits that are set to a fixed value (e.g., "1" or "0"). The circuit 100 may receive an offset value. The number of bits to be set to a fixed value and / or an offset value may be user-defined or pre-configured according to a CIM application. The circuit 100 may generate a second filling pattern having a length of the maximum number of bits to be filled based on the sum of the number of bits set to a fixed value and the offset value. In this case, the filling pattern may correspond to at least a portion of the second filling pattern according to the length of the filling number. In some other cases, the circuit 100 may generate a plurality of filling patterns associated with respective filling numbers, each of the plurality of filling patterns including at least one of the number of bits set to a fixed value and / or an offset value added to the fixed value.

[0131] In some embodiments, the circuit 100 may include a second adder circuit (e.g., another adder circuit (tree) 114) configured to sum another pair of the N aligned products to generate a corresponding second summation result. In this case, the circuit 100 (e.g., the second padding circuit) may determine the second padding number based on the bit position of the largest non-zero value in the second summation result. The circuit 100 may shift the second summation result by the number of bits corresponding to the second padding number to generate a second shifted summation result. The circuit 100 may associate a padding pattern having a length of the second padding number to the second shifted summation result to generate a second padding sum. The circuit 100 (e.g., the adder circuit (tree) 116 or the third adder circuit) may add the padding sum to the second padding sum to generate an accumulated result.

[0132] In one aspect of the present disclosure, a compute-in-memory (CIM) circuit is disclosed. The CIM circuit includes an input circuit configured to receive: (i) N first inputs, and (ii) N second inputs, wherein each of the N second inputs and a corresponding one of the N first inputs form one of the N input pairs; N multiplier circuits, each of the N multiplier circuits configured to multiply the corresponding input pair to generate a corresponding one of the N products; a shift circuit configured to align each of the N products according to a maximum exponent to generate a corresponding one of the N aligned products; an adder circuit configured to sum a corresponding pair of the N aligned products to generate a corresponding summation result, wherein the summation result consists of a sign part, an integer part, and a fractional part; and a padding circuit configured to: (i) determine a padding number based on a bit position of a maximum non-zero value in the summation result, (ii) shift the summation result by a number of bits corresponding to the padding number to generate a shifted summation result, and (iii) apply a padding pattern having a length of the padding number to the shifted summation result to generate a padded sum.

[0133] In some embodiments, the filling circuit is also configured to: identify a first value associated with the integer part and a second value associated with the fractional part; and determine an output summation result based on the first value being greater than zero or the second value being zero; and determine to fill the summation result based on the first value being zero and the second value being greater than zero.

[0134] In some embodiments, the padding circuit is further configured to: determine a bit position of a maximum non-zero value in a fractional portion of the summation result; and determine a padding number based on a difference between the bit position and a predetermined value.

[0135] In some embodiments, the padding circuit is further configured to: receive a plurality of padding patterns, each padding pattern having a corresponding length; and extract a padding pattern from the plurality of padding patterns for association based on the length of the padding pattern corresponding to the padding number.

[0136] In some embodiments, the filling circuit is also configured to: receive a maximum number of bits to be filled; receive a number of bits set to a fixed value; receive an offset value; and generate a second filling pattern having a length of the maximum number of bits to be filled based on the sum of the number of bits set to the fixed value and the offset value, wherein, based on the length of the filling number, the filling pattern corresponds to at least a portion of the second filling pattern.

[0137] In some embodiments, the N first inputs consist of N first symbols, N first exponents, and N first mantissas, and the N second inputs consist of N second symbols, N second exponents, and N second mantissas.

[0138] In some embodiments, the CIM circuit further includes: N summing circuits, each of the N summing circuits being configured to combine a corresponding first exponent and a corresponding second exponent of a corresponding one of the N input pairs to generate a corresponding one of the N exponential sums; and a selector circuit being configured to select a largest one from the N exponential sums as a maximum exponential sum.

[0139] In some embodiments, the CIM circuit further comprises: N subtractor circuits, each of the N subtractor circuits being configured to calculate a corresponding one of N exponent differences, each of the N exponent differences being equal to a difference between a corresponding one of the N exponent sums and a maximum exponent sum, wherein the shift circuit is configured to shift each of the N products based on the corresponding one of the N exponent differences to align each of the N products.

[0140] In some embodiments, each of the N multipliers is configured to multiply a corresponding first mantissa by a corresponding second mantissa of a corresponding input pair to generate a corresponding one of the N products.

[0141] In some embodiments, the N first inputs include a first input and a third input forming a first input pair, the N second inputs include a second input and a fourth input forming a second input pair, and wherein generating a corresponding one of the N products includes: (i) a first multiplier circuit configured to multiply the first input and the second input of the first input pair to generate a first product; and (i) a second multiplier circuit configured to multiply the third input and the fourth input of the second input pair to produce a second product.

[0142] In some embodiments, the shift circuit is configured to align the first product and the second product according to a maximum exponent sum, and wherein the aligned first product and the aligned second product form a pair of aligned products.

[0143] In some embodiments, the CIM circuit further includes: a second adder circuit configured to sum another pair of the N aligned products to generate a corresponding second summation result; a second fill circuit configured to: (i) determine a second fill number based on the bit position of the largest non-zero value in the second summation result, (ii) shift the second summation result by a number of bits corresponding to the second fill number to generate a second shifted summation result, and (iii) associate a fill pattern having a length of the second fill number to the second shifted summation result to generate a second fill sum; and a third adder circuit configured to add the fill sum to the second fill sum to generate an accumulated result.

[0144] In another aspect of the present disclosure, a computation in memory (CIM) circuit is disclosed. The CIM circuit includes an input circuit configured to receive: a first input, a second input, a third input, and a fourth input; a first multiplier circuit configured to multiply the first input by the second input to generate a first product; a second multiplier circuit configured to multiply the third input by the fourth input to generate a second product; a shift circuit configured to align the first product and the second product according to a maximum exponent sum to generate a first aligned product and a second aligned product, respectively; an adder circuit configured to add the first aligned product and the second aligned product to generate a sum result consisting of a sign part, an integer part, and a fractional part; and a padding circuit configured to: (i) determine a padding number based on a bit position of a maximum non-zero value in the sum result, (ii) shift the sum result by a number of bits corresponding to the padding number to generate a shifted sum result, and (iii) apply a padding pattern having a padding number length to the shifted sum result to generate a padding sum.

[0145] In some embodiments, the filling circuit is also configured to: identify a first value associated with the integer part and a second value associated with the fractional part; and determine an output summation result based on the first value being greater than zero or the second value being zero; and determine to fill the summation result based on the first value being zero and the second value being greater than zero.

[0146] In some embodiments, to determine the fill number, the fill circuit is further configured to: determine the bit position of the largest non-zero value in the fractional part of the summation result; and determine the fill number based on a difference between the bit position and a predetermined value.

[0147] In some embodiments, the padding circuit is further configured to: receive a plurality of padding patterns, each of the plurality of padding patterns having a corresponding length; and extract a padding pattern from the plurality of padding patterns for association based on the length of the padding pattern corresponding to the padding number.

[0148] In some embodiments, the filling circuit is also configured to: receive a maximum number of bits to be filled; receive the number of bits set to a fixed value; receive an offset value; and generate a second filling pattern having a length of the maximum number of bits to be filled based on the sum of the number of bits set to the fixed value and the offset value, wherein, based on the length of the filling number, the filling pattern corresponds to at least a portion of the second filling pattern.

[0149] In another aspect of the present disclosure, a method for performing MAC operations on floating-point numbers with improved memory calculation precision (CIM) is disclosed. The method includes: obtaining a first input, a second input, a third input, and a fourth input by a calculation in memory (CIM) circuit, wherein the first input and the second input form a first input pair, and wherein the third input and the fourth input form a second input pair; generating a first product by the CIM circuit by multiplying the first input pair; generating a second product by the CIM circuit by multiplying the second input pair; aligning the first product and the second product according to a maximum exponent sum by the CIM circuit; generating a sum result by adding the aligned first product and the aligned second product by the CIM circuit; determining a padding number by the CIM circuit based on the bit position of the maximum non-zero value in the sum result; shifting the sum result by the CIM circuit by the number of bits corresponding to the padding number; and generating a padding sum by applying a padding pattern having a length of the padding number to the shifted sum result by the CIM circuit.

[0150] In some embodiments, the method includes: receiving, by the CIM circuit, a maximum number of bits to be filled; receiving, by the CIM circuit, a number of bits to be set to a fixed value; receiving an offset value by the CIM circuit; and generating, by the CIM circuit, a second fill pattern having a length of the maximum number of bits to be filled based on the sum of the number of bits set to the fixed value and the offset value, wherein, according to the length of the fill number, the fill pattern corresponds to at least a portion of the second fill pattern.

[0151] In some embodiments, the first input, the second input, the third input and the fourth input consist of corresponding signs, corresponding exponents and corresponding mantissas, and the method further includes: combining, by the CIM circuit, each pair of exponents associated with the corresponding input pair to generate a corresponding one of a plurality of exponential sums; and selecting, by the CIM circuit, a largest one from the N exponential sums as the largest exponential sum.

[0152] As used herein, the terms "approximately" and "approximately" generally refer to a value of a given quantity that may vary depending on a particular technology node associated with the subject semiconductor device. Based on the particular technology node, the term "approximately" may refer to a value of a given quantity that varies, for example, within a range of 10-30% of a value (e.g., +10%, ±20%, or ±30% of a value).

[0153] The features of several embodiments are summarized above so that those skilled in the art can better understand the various aspects of the present disclosure. Those skilled in the art will appreciate that they can easily use the present disclosure as a basis for designing or modifying other processes and structures for achieving the same purpose and / or achieving the same advantages of the embodiments introduced herein. Those skilled in the art will also appreciate that such equivalent structures do not deviate from the spirit and scope of the present disclosure, and they can make various changes, substitutions and changes in the present disclosure without departing from the spirit and scope of the present disclosure.

Claims

1. A computing circuit in memory, comprising: An input circuit configured to receive: (i) N first inputs, and (ii) N second inputs, wherein each of the N second inputs and a corresponding one of the N first inputs form one input pair of the N input pairs; N multiplier circuits, each of the N multiplier circuits configured to multiply a corresponding pair of inputs to generate a corresponding one of N products; a shift circuit configured to align each of the N products according to a maximum exponent to generate a corresponding one of the N aligned products; an adder circuit configured to sum a corresponding pair of the N aligned products to generate a corresponding summation result, wherein the summation result consists of a sign portion, an integer portion, and a fractional portion; and A padding circuit is configured to: (i) determine a padding number based on the bit position of the largest non-zero value in the summation result, (ii) shift the summation result by the number of bits corresponding to the padding number to generate a shifted summation result, and (iii) apply a padding pattern having a length of the padding number to the shifted summation result to generate a padded sum.

2. The computing-in-memory circuit according to claim 1, wherein: The filling circuit is further configured to: identifying a first value associated with the integer portion and a second value associated with the fractional portion; Determine to output the summation result based on the first value being greater than zero or the second value being zero; as well as Padding the summed result is determined based on the first value being zero and the second value being greater than zero.

3. The computing-in-memory circuit according to claim 1, wherein: The filling circuit is further configured to: determining the bit position of the largest non-zero value in the fractional portion of the summed result; and The fill number is determined based on a difference between the bit position and a predetermined value.

4. The computing-in-memory circuit according to claim 3, wherein: The filling circuit is further configured to: receiving a plurality of padding patterns, each padding pattern having a corresponding length; and Based on the length of the padding pattern corresponding to the padding number, a padding pattern is extracted from the plurality of padding patterns for association.

5. The computing-in-memory circuit of claim 1, wherein: The filling circuit is further configured to: Receive the maximum number of bits to fill; Receive the number of bits set to a fixed value; Receive the offset value; as well as generating a second padding pattern having a length of a maximum number of bits to be padded based on the sum of the number of bits set to a fixed value and the offset value, Wherein, according to the length of the filling number, the filling pattern corresponds to at least a part of the second filling pattern.

6. The computing-in-memory circuit of claim 1, wherein: The N first inputs include a first input and a third input forming a first input pair, the N second inputs include a second input and a fourth input forming a second input pair, and wherein generating a corresponding one of the N products includes: (i) a first multiplier circuit configured to multiply the first input and the second input of the first input pair to generate a first product; and (i) a second multiplier circuit configured to multiply the third input and the fourth input of the second input pair to generate a second product.

7. The computing-in-memory circuit of claim 6, wherein: The shift circuit is configured to align the first product and the second product according to the maximum exponent sum, and wherein the aligned first product and the aligned second product form a pair of aligned products.

8. The in-memory computing circuit of claim 1 , further comprising: a second adder circuit configured to sum another pair of the N aligned products to generate a corresponding second summation result; a second padding circuit configured to: (i) determine a second padding number based on the bit position of the largest non-zero value in the second summation result, (ii) shift the second summation result by an amount of bits corresponding to the second padding number to generate a second shifted-sum result, and (iii) associate a padding pattern having a length of the second padding number to the second shifted-sum result to generate a second padding sum; as well as A third adder circuit is configured to add the padding sum and the second padding sum to generate an accumulation result.

9. A computing circuit in memory, comprising: an input circuit configured to receive a first input, a second input, a third input, and a fourth input; a first multiplier circuit configured to multiply the first input by the second input to produce a first product; a second multiplier circuit configured to multiply the third input by the fourth input to produce a second product; a shift circuit configured to align the first product and the second product according to a maximum exponent sum to generate a first aligned product and a second aligned product, respectively; an adder circuit configured to add the first aligned product and the second aligned product to generate a summation result consisting of a sign part, an integer part, and a fractional part; as well as A padding circuit is configured to: (i) determine a padding number based on the bit position of the largest non-zero value in the summation result, (ii) shift the summation result by the number of bits corresponding to the padding number to generate a shifted summation result, and (iii) apply a padding pattern having a length of the padding number to the shifted summation result to generate a padded sum.

10. A method for performing a multiplication-accumulation operation, comprising: Obtaining, by the in-memory computation circuit, a first input, a second input, a third input, and a fourth input, wherein the first input and the second input form a first input pair, and wherein the third input and the fourth input form a second input pair; generating, by the in-memory computation circuit, a first product by multiplying the first input pair; generating, by the in-memory computation circuit, a second product by multiplying the second pair of inputs; aligning the first product and the second product by a calculation circuit in the memory according to a maximum exponent sum; generating a summation result by adding the aligned first product and the aligned second product by the calculation circuit in the memory; Determining the filling number by a calculation circuit in the memory based on the bit position of the maximum non-zero value in the summation result; The calculation circuit in the memory shifts the sum result by the number of bits corresponding to the fill number; and A pad sum is generated by the in-memory computation circuit by applying a pad pattern having a length of the pad number to the shifted sum result.

Images