In-memory operation device and index storage operation module and mantissa storage operation module thereof

Abstract

The invention provides an in-memory operation device and an index storage operation module and a mantissa storage operation module thereof. The in-memory operation device is used for in-memory operation of floating-point number data and comprises an index storage operation module and a mantissa storage operation module. The index storage operation module comprises a plurality of weight index storage circuits, a plurality of index operation circuits and a comparison circuit. The exponent operation circuits are used for obtaining a plurality of exponent product data. The mantissa storage operation module comprises a digit displacement circuit, a plurality of weight mantissa storage circuits, a plurality of mantissa operation circuits, a displacement and addition circuit, a plurality of weight sign storage circuits, a plurality of sign operation circuits and a summing circuit. The mantissa operation circuits and the displacement and addition circuits are used for obtaining a plurality of mantissa product data. The sign operation circuits are used for obtaining a plurality of sign product data.

Description

Technical Field

[0001] This application relates to an in-memory computing device for floating-point data memory operations, its exponent storage and operation module, and its mantissa storage and operation module. Background Art

[0002] The Computing-in-memory (CIM) technology is regarded as one of the effective technologies to solve the memory wall. It uses the operations within the memory to reduce the number of data transfers, which can significantly improve the operation speed to hundreds or even thousands of times that of the traditional architecture. Currently, a large part of the energy of large AI networks (such as DNNs) is consumed in data transfers. Through the CIM technology, the wasted energy can be significantly reduced, which can be said to be a potential technology for future AI that combines increased computing power and reduced power consumption.

[0003] The potential of the CIM technology has led many manufacturers and research institutions to invest in and publish many novel technologies. However, these technologies can only perform integer operations, and the analog sensing used may cause problems such as noise or process variations. The currently proposed CIM cannot support floating-point operations. Therefore, researchers are working on developing an in-memory computing architecture that supports floating-point numbers. Summary of the Invention

[0004] This application relates to an in-memory computing device for floating-point data memory operations, its exponent storage and operation module, and its mantissa storage and operation module. It integrates the floating-point operation circuit into the memory, avoiding data input and output. Therefore, it has the advantage of fast operation, can reduce power consumption, and improves energy efficiency.

[0005] According to one aspect of the present application, an in-memory computing device for floating-point data memory operations is provided. The in-memory computing device includes an exponent storage and computing module and a mantissa storage and computing module. The exponent storage and computing module includes a plurality of weighted exponent storage circuits, a plurality of exponent computing circuits, and a comparison circuit. These weighted exponent storage circuits are used to store the exponent parts of a plurality of weighted data. These exponent computing circuits are used to perform addition operations on the exponent parts of a plurality of input data and the exponent parts of these weighted data to obtain a plurality of exponent product data. The comparison circuit is used to compare these exponent product data to obtain the maximum exponent product data. The mantissa storage and computing module includes a bit shift circuit, a plurality of weighted mantissa storage circuits, a plurality of mantissa computing circuits, a shift and addition circuit, a plurality of weighted sign storage circuits, a plurality of sign computing circuits, and a summation circuit. The bit shift circuit is used to shift the mantissa parts of these input data according to the maximum exponent product data. These weighted mantissa storage circuits are used to store the mantissa parts of these weighted data. These mantissa computing circuits are used to perform multiplication operations on the mantissa parts of these input data and the mantissa parts of these weighted data to obtain a plurality of intermediate mantissa product data. The shift and addition circuit is used to shift and then sum these intermediate mantissa product data to obtain a plurality of mantissa product data. The plurality of weighted sign storage circuits are used to store the sign parts of these weighted data. The plurality of sign computing circuits are used to perform exclusive OR operations on the sign parts of these input data and the sign parts of these weighted data to obtain a plurality of sign product data. The summation circuit is used to integrate these sign product data, these maximum exponent product data, and these mantissa product data to obtain the input and weighted product sum data.

[0006] According to another aspect of the present application, an exponent storage and computing module is provided. The exponent storage and computing module includes a plurality of weighted exponent storage circuits, a plurality of exponent computing circuits, and a comparison circuit. These weighted exponent storage circuits are used to store the exponent parts of a plurality of weighted data. These exponent computing circuits are used to perform addition operations on the exponent parts of a plurality of input data and the exponent parts of these weighted data to obtain a plurality of exponent product data. The comparison circuit is used to compare these exponent product data to obtain the maximum exponent product data.

[0007] According to yet another aspect of the present application, a mantissa storage and computing module is provided. The mantissa storage and computing module includes a plurality of weighted mantissa storage circuits, a plurality of mantissa computing circuits, and a shift and addition circuit. These weighted mantissa storage circuits are used to store the mantissa parts of a plurality of weighted data. These mantissa computing circuits are used to perform multiplication operations on the mantissa parts of a plurality of input data and the mantissa parts of these weighted data to obtain a plurality of intermediate mantissa product data. The shift and addition circuit is used to shift and then sum these intermediate mantissa product data to obtain a plurality of mantissa product data.

[0008] To have a better understanding of the above and other aspects of the present invention, specific embodiments are given below and described in detail in conjunction with the accompanying drawings as follows. Description of the Drawings

[0009] Figure 1 The example illustrates the multiplication operation of floating-point data in an embodiment of the present application;

[0010] Figure 2 Illustrates the storage method of floating-point data according to an embodiment of the present application;

[0011] Figure 3 Illustrates the architecture diagram of the in-memory computing device according to an embodiment of the present application;

[0012] Figure 4 Illustrates the data flow of the in-memory computing device performing floating-point operations according to an embodiment of the present application;

[0013] Figure 5 Illustrates the schematic diagram of the weight index storage circuit and the exponent operation circuit according to an embodiment of the present application;

[0014] Figure 6 Illustrates the schematic diagram of the comparison circuit according to an embodiment of the present application;

[0015] Figure 7 Illustrates the schematic diagram of the comparator according to an embodiment of the present application;

[0016] Figure 8 Illustrates the schematic diagram of the bit shift circuit according to an embodiment of the present application;

[0017] Figure 9 Illustrates the schematic diagram of the mantissa operation circuit according to an embodiment of the present application;

[0018] Figure 10 Illustrates the schematic diagram of the dot-by-dot multiplier according to an embodiment of the present application;

[0019] Figure 11 The example illustrates the operation of the shift and addition circuit according to an embodiment of the present application;

[0020] Figure 12 Illustrates the schematic diagram of the sign operation circuit according to an embodiment of the present application;

[0021] Figure 13 The example illustrates the multiplication operation of integer data in an embodiment of the present application;

[0022] Figure 14 Illustrates the data flow of the in-memory computing device performing integer operations according to an embodiment of the present application.

[0023] Among them, reference numerals:

[0024] 100: In-memory computing device

[0025] AC: Accumulation

[0026] AD: Adder

[0027] AL: Alignment

[0028] AWIN, BWIN: Judgment result

[0029] A<0>, A<1>, A<2>, B<0>, B<1>, B<2>: Front bits

[0030] BL0, BLB0, BL7, BLB7: Bit lines

[0031] COMP: Comparison circuit

[0032] CP: Comparator

[0033] CP1: First judgment circuit

[0034] CP2: Second judgment circuit

[0035] CP3: Third judgment circuit

[0036] C<0>, C<1>, C<2>: Exclusive OR result

[0037] E: Exponent part

[0038] EN: Enable signal

[0039] EP: Exponent storage operation module

[0040] GBL<0>, GBL<7>, GBLB<0>, GBLB<7>: Global bit lines

[0041] GD: Ground terminal

[0042] IN: Input data

[0043] INT: Integer part

[0044] IN[0], IN[6], IN[7]: Input terminals

[0045] IN_E: Exponent part of input data

[0046] IN_M: Mantissa part of input data

[0047] IN_M’: Shifted mantissa part

[0048] IN_S: Sign part of input data

[0049] LBL<0>,LBL<7>,LBLB<0>,LBL<7>: Region bit line

[0050] LCCE: Exponent operation circuit

[0051] LCCM: Mantissa operation circuit

[0052] LCCS: Sign operation circuit

[0053] LSB[0]: Least significant bit

[0054] M: Mantissa part

[0055] M0,M1,M2,M3,M4,M5,M6: Numerical value

[0056] MAC: Input and weight product sum data

[0057] ML: Product data

[0058] MLP: Product

[0059] ML_E: Exponent product data

[0060] ML_E_max: Maximum exponent product data

[0061] ML_M: Mantissa product data

[0062] ML_M_im: Mantissa product intermediate data

[0063] ML_S: Sign product data

[0064] MSA: Summing circuit

[0065] MSB[7]: Most significant bit

[0066] MT: Mantissa storage operation module

[0067] OF: Displacement amount data

[0068] OUT0[0],OUT0[6],OUT0[7],OUT7[0],OUT7[6],OUT7[7]: Output terminal

[0069] PWM: Point-by-point multiplier

[0070] S: Sign part

[0071] SAP: Switching and precharging circuit

[0072] SB: Subtractor

[0073] SH: Shifter

[0074] SHT: Bit shift circuit

[0075] SHTA: Shift and Addition Circuit

[0076] SR: Static Random Access Memory

[0077] SRE: Weight Exponent Storage Circuit

[0078] SRM: Weight Mantissa Storage Circuit

[0079] SRS: Weight Sign Storage Circuit

[0080] TR, TRB: Transistor

[0081] WT: Weight Data

[0082] WT_E: Exponent Part of Weight Data

[0083] WT_M: Mantissa Part of Weight Data

[0084] WT_S: Sign Part of Weight Data

[0085] XOR: Exclusive OR Operator Detailed Implementation Manner

[0086] Please refer to Figure 1 , which illustrates the multiplication operation of floating - point data in an embodiment of this application by way of example. The floating - point data consists of a sign part S, an exponent part E, and a mantissa part M. Taking the 16 - bit FP16 operation architecture as an example, the sign part S occupies 1 bit, the exponent part E occupies 8 bits, and the mantissa part M occupies 7 bits. When the 7 bits of the mantissa part M are in sequence of the numerical values M6, M5, M4, M3, M2, M1, M0, the numerical content of this floating - point data is (-1) S ×1.M6M5M4M3M2M1M0×2 E-127 .

[0087] When the sign part S is 0, it represents a positive value; when the sign part S is 1, it represents a negative value. The range that the exponent part E can represent is 2 -127 ~2 128 . The range that the mantissa part M can represent is 1.0~1.9921875.

[0088] For example Figure 1As shown, both the input data IN and the weight data WT can adopt the FP16 arithmetic architecture. After the input data IN and the weight data WT are multiplied, the product data ML can be obtained. The product data ML will also adopt the FP16 arithmetic architecture. When the input data IN and the weight data WT are multiplied, an addition operation is performed on the exponent part E, a multiplication operation is performed on the mantissa part M, and an exclusive-OR operation is performed on the sign part S.

[0089] Please refer to Figure 2 , which shows the storage method of floating-point data according to an embodiment of the present application. In one embodiment, the exponent part E, the sign part S, and the mantissa part M of the floating-point data can be arranged in sequence and stored in the memory.

[0090] Please refer to Figure 3 , which shows the architecture diagram of the floating-point computing-in-memory device 100 according to an embodiment of the present application. The computing-in-memory device 100 includes an exponent storage and operation module EP and a mantissa storage and operation module MT. The exponent storage and operation module EP is used to store and operate the exponent part E of the floating-point data (shown in Figure 1 ); the mantissa storage and operation module MT is used to store and operate the mantissa part M of the floating-point data (shown in Figure 1 ).

[0091] The exponent storage and operation module EP includes a plurality of weight exponent storage circuits SRE, a plurality of exponent operation circuits LCCE, and a comparison circuit COMP. The mantissa storage and operation module MT includes a bit shift circuit SHT, a plurality of weight sign storage circuits SRS, a plurality of sign operation circuits LCCS, a plurality of weight mantissa storage circuits SRM, a plurality of mantissa operation circuits LCCM, a shift and addition circuit SHTA, and a summing circuit MSA.

[0092] In the computing-in-memory device 100, storage units (such as the weight exponent storage circuit SRE, the weight sign storage circuit SRS, and the weight mantissa storage circuit SRM) and operation units (such as the exponent operation circuit LCCE, the comparison circuit COMP, the bit shift circuit SHT, the sign operation circuit LCCS, the mantissa operation circuit LCCM, the shift and addition circuit SHTA, and the summing circuit MSA) are integrated. Therefore, when performing floating-point operations, frequent input and output of data can be avoided, so it has the advantage of fast operation, can reduce power consumption, and improves energy efficiency.

[0093] Please also refer to Figure 3 and Figure 4 ,Figure 4 Illustrates the data flow of the in-memory computing device 100 performing floating-point operations according to an embodiment of the present application. The data flow of the in-memory computing device 100 performing floating-point operations includes the alignment AL of the exponent part E (illustrated in Figure 1 ), the multiplication MLP of the mantissa part M (illustrated in Figure 1 ), and the accumulation AC of the product data ML (illustrated in Figure 1 ). The alignment AL of the exponent part E is completed by the exponent operation circuit LCCE, the comparison circuit COMP of the exponent storage operation module EP, and the bit shift circuit SHT of the mantissa storage operation module MT. The multiplication MLP of the mantissa part M is completed by the mantissa operation circuit LCCM and the shift and addition circuit SHTA of the mantissa storage operation module MT. The accumulation AC of the product result is completed by the summation circuit MSA of the mantissa storage operation module MT.

[0094] When performing the multiplication operation of floating-point data, an addition operation is performed on the exponent part E. As Figure 4 shown, the exponent operation circuit LCCE is used to perform addition operations on the exponent part IN_E of multiple input data IN and the exponent part WT_E of multiple weight data WT respectively to obtain multiple exponent product data ML_E. The exponent part WT_E of the weight data WT is stored in the Figure 3 weight exponent storage circuit SRE.

[0095] The comparison circuit COMP is connected to the exponent operation circuit LCCE. The comparison circuit COMP is used to compare these exponent product data ML_E to obtain the maximum exponent product data ML_E_max.

[0096] The bit shift circuit SHT is connected to the exponent operation circuit LCCE and the comparison circuit COMP. The bit shift circuit SHT shifts the mantissa part IN_M of these input data IN according to the maximum exponent product data ML_E_max to obtain the shifted mantissa part IN_M'. The mantissa part WT_M of the weight data WT is stored in the Figure 3 weight mantissa storage circuit SRM.

[0097] The mantissa operation circuit LCCM is connected to the bit shift circuit SHT. The mantissa operation circuit LCCM is used to perform a multiplication operation on the mantissa part IN_M' of the input data IN and the mantissa part WT_M of the weight data WT to obtain multiple mantissa product intermediate data ML_M_im. The mantissa product intermediate data ML_M_im is the data obtained by multiplying each bit of the mantissa part WT_M with the mantissa part IN_M' point by point in the multiplication operation.

[0098] The shift and addition circuit SHTA is used to shift and then sum up these intermediate data ML_M_im of the mantissa products to obtain the mantissa product data ML_M.

[0099] The sign operation circuit LCCS is used to perform an exclusive-OR operation on the sign part IN_S of the input data IN and the sign part WT_S of the weight data WT to obtain the sign product data ML_S. The sign part WT_S of the weight data WT is stored in Figure 3 the weight sign storage circuit SRS.

[0100] The summation circuit MSA is used to integrate the sign product data ML_S, the maximum exponent product data ML_E_max, and the mantissa product data ML_M to obtain the input and weight product sum data MAC.

[0101] The detailed structures and operations of each component are further described in detail below.

[0102] Please refer to Figure 5 , which shows a schematic diagram of the weight exponent storage circuit SRE and the exponent operation circuit LCCE according to an embodiment of the present application. The weight exponent storage circuit SRE includes a plurality of static random-access memories (SRAM) SR. Each static random-access memory SR includes six transistors (i.e., 6T-SRAM). The weight exponent storage circuit SRE has global bit lines GBL<0> to GBL<7>, GBLB<0> to GBLB<7>, and local bit lines LBL<0> to LBL<7>, LBLB<0> to LBLB<7>, for example. The exponent part WT_E of a set of weight data WT is stored in a horizontal row of static random-access memories SR. When a horizontal row of static random-access memories SR is enabled, a set of exponent parts WT_E of the weight data WT can be input to the exponent operation circuit LCCE via the local bit lines LBL<0> to LBL<7>.

[0103] The exponent operation circuit LCCE includes a plurality of switching and precharging circuits SAP and adders AD. The switching and precharging circuits SAP are connected to the weight exponent storage circuit SRE. The switching and precharging circuits SAP are used to receive the exponent part WT_E of the weight data WT. The adder AD is connected to the switching and precharging circuits SAP to receive the exponent part WT_E of the weight data WT. The adder AD is used to perform an addition operation on the exponent part IN_E of the input data IN and the exponent part WT_E of the weight data WT to obtain the exponent product data ML_E.

[0104] Please refer to Figure 6, which shows a schematic diagram of a comparison circuit COMP according to an embodiment of the present application. The comparison circuit COMP includes a plurality of comparators CP. The comparator CP is used to compare two pieces of exponent product data ML_E of the exponent product data ML_E. Through hierarchical pairwise comparison, the maximum exponent product data ML_E_max can be obtained.

[0105] Please refer to again Figure 7 , which shows a schematic diagram of a comparator CP according to an embodiment of the present application. The comparator CP of this embodiment includes a first judgment circuit CP1, a second judgment circuit CP2, and a third judgment circuit CP3. The first judgment circuit CP1 is used to compare the front bits A<0> to A<2>, B<0> to B<2> of the exponent product data ML_E. When the front bit A<2> is compared with the front bit B<2>, the exclusive-OR judge is activated through the enable signal EN, and the exclusive-OR result C<2> is output. The exclusive-OR results C<2> to C<0> can output a judgment result AWIN or a judgment result BWIN through the judgment of the judge. The judgment result AWIN represents that the front bits A<0> to A<2> are larger than the front bits B<0> to B<2>. If the size can be judged in the first judgment circuit CP1, there is no need to activate the subsequent second judgment circuit CP2 and third judgment circuit CP3.

[0106] The second judgment circuit CP2 is connected to the first judgment circuit CP1. The second judgment circuit CP2 is used to compare the middle bits of the exponent product data ML_E. If the size can be judged in the second judgment circuit CP2, there is no need to activate the subsequent third judgment circuit CP3.

[0107] The third judgment circuit CP3 is connected to the second judgment circuit CP2. The third judgment circuit CP3 is used to compare the rear bits of the exponent product data ML_E.

[0108] Through the three-stage judgment circuit design of the comparator CP, the comparison of many exponent product data ML_E can omit the activation of the second judgment circuit CP2 and the third judgment circuit CP3, or omit the activation of the third judgment circuit CP3. Therefore, the power consumption can be greatly saved, and the comparison speed can be accelerated.

[0109] Please refer to Figure 8 , which shows a schematic diagram of a bit shift circuit SHT according to an embodiment of the present application. The bit shift circuit SHT includes a plurality of subtractors SB and a plurality of shifters SH. The subtractor SB is connected to the comparison circuit COMP. The subtractor SB is used to perform a subtraction operation on the maximum exponent product data ML_E_max and the exponent product data ML_E to obtain the displacement amount data OF.

[0110] The shifter SH is connected to the subtractor SB. The shifter SH is used to shift the mantissa part IN_M of the input data IN according to the shift amount data OF to obtain the shifted mantissa part IN_M'.

[0111] Please refer to Figure 9 , which shows a schematic diagram of the mantissa arithmetic circuit LCCM according to an embodiment of the present application. The weight mantissa storage circuit SRM includes a plurality of static random-access memories (SRAM) SR. Each static random-access memory SR includes six transistors (i.e., 6T-SRAM). The weight mantissa storage circuit SRM has global bit lines GBL<0> to GBL<7>, GBLB<0> to GBLB<7> and local bit lines LBL<0> to LBL<7>, LBLB<0> to LBLB<7>, for example. The mantissa part WT_M of a set of weight data WT is stored in a row of the static random-access memories SR. When a row of the static random-access memories SR is enabled, the mantissa part WT_M of a set of weight data WT can be input to the mantissa arithmetic circuit LCCM via the local bit lines LBL<0> to LBL<7>.

[0112] The mantissa arithmetic circuit LCCM includes a plurality of switching and precharging circuits SAP and a point-wise multiplier PWM. The switching and precharging circuit SAP is connected to the weight mantissa storage circuit SRM. The switching and precharging circuit SAP is used to receive the mantissa part WT_M of the weight data WT. The point-wise multiplier PWM is connected to the switching and precharging circuit SAP to receive the mantissa part WT_M of the weight data WT. The point-wise multiplier PWM is used to perform a multiplication operation on the mantissa part IN_M of the input data IN and the mantissa part WT_M of the weight data WT to obtain the mantissa product data ML_M.

[0113] Please refer to Figure 10 , which shows a schematic diagram of the point-wise multiplier PWM according to an embodiment of the present application. The point-wise multiplier PWM is composed of a plurality of transistors TR, TRB. The static random-access memory SR is used to store the bit values of the mantissa part WT_M of the weight data WT. The leftmost static random-access memory SR corresponds to the most significant bit MSB[7], for example, and the rightmost static random-access memory SR corresponds to the least significant bit LSB[0], for example.

[0114] The bit lines BL0 to BL7 of the static random access memory SR storing the mantissa part WT_M of the weight data WT are connected to the cascaded transistors TR. The bit lines BLB0 to BLB7 of the static random access memory SR are connected to the cascaded transistors TRB. Both ends of the transistor TR are connected to the input terminals IN[0] to IN[7] and the output terminals OUT0[0] to OUT0[7], and both ends of the transistor TRB are connected to the ground terminal GD and the output terminals OUT0[0] to OUT0[7]. The mantissa part IN_M of the input data IN is input from the input terminals IN[0] to IN[7].

[0115] According to the circuit architecture of the pointwise multiplier PWM, when the mantissa part WT_M of the weight data WT inputs 1 from the bit line BL7 and the mantissa part IN_M of the input data IN inputs 1 from the input terminal IN[7], the output terminal OUT7[7] outputs 1. When the mantissa part WT_M of the weight data WT inputs 1 from the bit line BL7 and the mantissa part IN_M of the input data IN inputs 0 from the input terminal IN[6], the output terminal OUT7[6] outputs 0. When the mantissa part WT_M of the weight data WT inputs 0 from the bit line BL0 and the mantissa part IN_M of the input data IN inputs 0 from the input terminal IN[7], the output terminal OUT0[7] outputs 0. When the mantissa part WT_M of the weight data WT inputs 0 from the bit line BL0 and the mantissa part IN_M of the input data IN inputs 1 from the input terminal IN[6], the output terminal OUT0[6] outputs 0.

[0116] Through the above circuit architecture of the pointwise multiplier PWM, the pointwise product result of the mantissa part WT_M of the weight data WT and the mantissa part IN_M of the input data IN can be obtained. These product results are the aforementioned mantissa product intermediate data ML_M_im.

[0117] Please refer to Figure 11 , which exemplarily illustrates the operation of the shift and addition circuit SHTA according to an embodiment of the present application. The shift and addition circuit SHTA is used to shift and then sum the mantissa product intermediate data ML_M_im to obtain the mantissa product data ML_M.

[0118] Please refer to Figure 12, which shows a schematic diagram of a sign operation circuit LCCS according to an embodiment of the present application. The weight sign storage circuit SRS includes a plurality of static random-access memories (SRAM) SR. Each static random-access memory SR includes six transistors (i.e., 6T-SRAM). The weight sign storage circuit SRS has, for example, global bit lines GBL<7>, GBLB<7> and local bit lines LBL<7>, LBLB<7>. A certain static random-access memory SR stores the sign part WT_S of a set of weight data WT. When a certain static random-access memory SR is enabled, the sign part WT_S of a set of weight data WT can be input to the sign operation circuit LCCS via the local bit line LBL<7>.

[0119] The sign operation circuit LCCS includes a switching and precharging circuit SAP and an exclusive OR calculator XOR.

[0120] The switching and precharging circuit SAP is connected to the weight sign storage circuit SRS. The switching and precharging circuit SAP is used to receive the sign part WT_S of the weight data WT. The exclusive OR calculator XOR is connected to the switching and precharging circuit SAP to receive the sign part WT_S of the weight data WT. The exclusive OR calculator XOR is used to perform an exclusive OR operation on the sign part IN_S of the input data IN and the sign part WT_S of the weight data WT to obtain the sign product data ML_S.

[0121] According to the above description, the in-memory computing device 100 can support the FP16 computing architecture. In other embodiments, the in-memory computing device 100 also supports the INT8 computing architecture at the same time. Please refer to Figure 13 , which exemplarily illustrates the product operation of integer data according to an embodiment of the present application. The sign part S and the mantissa part M of the floating-point data form the integer part INT of the integer data. The exponent part E is not used. The integer part INT occupies 8 bits. The range that the input data IN can represent is 0 to 255. The range that the weight data WT can represent is -128 to 127. After the input data IN and the weight data WT are subjected to a product operation, the product data ML can be obtained.

[0122] Please refer to Figure 14, which shows the data flow of the in-memory computing device 100 performing integer operations according to an embodiment of the present application. The data flow of the in-memory computing device 100 performing integer operations includes a multiply MLP and an accumulate AC. The multiply MLP is completed by the mantissa arithmetic circuit LCCM and the shift and add circuit SHTA of the mantissa storage arithmetic module MT. The accumulation AC of the product data ML is completed by the summation circuit MSA of the mantissa storage arithmetic module MT. In this way, the in-memory computing device 100 can also support the INT8 computing architecture at the same time.

[0123] The above disclosure provides different features for implementing some embodiments or examples of the present application. The specific examples (such as the numerical values or names mentioned) of the described components and configurations are used to simplify / schematize some embodiments of the present application. Of course, such components and configurations are only examples and are not intended to be restrictive. In addition, some embodiments of the present application may repeatedly refer to reference signs and / or letters in various examples. This repetition is for the purpose of simplicity and clarity, and does not itself indicate the relationship between the various embodiments and / or configurations discussed.

[0124] According to the above embodiment, in the in-memory computing device 100, a storage unit (such as the weight exponent storage circuit SRE, the weight sign storage circuit SRS, the weight mantissa storage circuit SRM) and an arithmetic unit (such as the exponent arithmetic circuit LCCE, the comparison circuit COMP, the bit shift circuit SHT, the sign arithmetic circuit LCCS, the mantissa arithmetic circuit LCCM, the shift and add circuit SHTA, the summation circuit MSA) are integrated. Therefore, when performing floating-point operations, frequent input and output of data can be avoided, so it has the advantage of fast operation, can reduce power consumption, and improves energy efficiency.

[0125] The exponent storage arithmetic module EP and / or the mantissa storage arithmetic module MT proposed in the present application are all within the scope of protection of the present application. If the exponent storage arithmetic module EP of the present application is implemented alone and the rest is combined with other circuit designs, it still does not deviate from the spirit and scope of the present application. If the mantissa storage arithmetic module MT of the present application is implemented alone and the rest is combined with other circuit designs, it still does not deviate from the spirit and scope of the present application.

[0126] In summary, although the present application has been disclosed above with embodiments, it is not intended to limit the present application. Those of ordinary skill in the technical field to which the present application pertains can make various modifications and refinements without departing from the spirit and scope of the present application. Therefore, the protection scope of the present application is defined by the scope of the patent application.

Claims

1. An in-memory computing device for in-memory computing of floating-point data, characterized in that, Comprising: An exponential storage operation module, comprising: A plurality of weight exponential storage circuits for storing the exponential parts of a plurality of weight data; A plurality of exponential operation circuits for performing addition operations on the exponential parts of a plurality of input data and the exponential parts of the plurality of weight data to obtain a plurality of exponential product data; and A comparison circuit for comparing the plurality of exponential product data to obtain the maximum exponential product data; and A mantissa storage operation module, comprising: A digit shift circuit for shifting the mantissa parts of the plurality of input data according to the maximum exponential product data; A plurality of weight mantissa storage circuits for storing the mantissa parts of the plurality of weight data; A plurality of mantissa operation circuits for performing multiplication operations on the mantissa parts of the plurality of input data and the mantissa parts of the plurality of weight data to obtain a plurality of intermediate mantissa product data; A shift and addition circuit for shifting and then summing the plurality of intermediate mantissa product data to obtain a plurality of mantissa product data; A plurality of weight sign storage circuits for storing the sign parts of the plurality of weight data; A plurality of sign operation circuits for performing exclusive OR operations on the sign parts of the plurality of input data and the sign parts of the plurality of weight data to obtain a plurality of sign product data; and A summing circuit for integrating the plurality of sign product data, the plurality of maximum exponential product data, and the plurality of mantissa product data to obtain the input and weight product sum data.

2. The in-memory computing device according to claim 1, wherein Each of the weight exponential storage circuits includes a plurality of static random access memories.

3. The in-memory computing device according to claim 2, wherein Each of the static random access memories includes six transistors.

4. The in-memory computing device according to claim 1, wherein Each of the exponential operation circuits includes: A plurality of switching and precharging circuits connected to the plurality of weight exponential storage circuits and for receiving the exponential parts of the plurality of weight data; and An adder connected to the plurality of switching and precharging circuits and for receiving the exponential parts of the plurality of weight data to perform the addition operation on the exponential parts of the plurality of input data and the exponential parts of the plurality of weight data to obtain the plurality of exponential product data.

5. The in-memory computing device according to claim 1, wherein The comparison circuit is connected to the plurality of exponential operation circuits, and the comparison circuit includes: A plurality of comparators for comparing two pieces of exponential product data among the plurality of exponential product data.

6. The in-memory computing device according to claim 5, wherein Each of the comparators includes: A first judgment circuit for comparing the leading bits of the plurality of exponential product data; A second judgment circuit connected to the first judgment circuit and for comparing the middle bits of the plurality of exponential product data; and A third judgment circuit connected to the second judgment circuit and for comparing the trailing bits of the plurality of exponential product data.

7. The in-memory computing device according to claim 1, wherein The digit shift circuit includes: A plurality of subtractors connected to the comparison circuit and for performing subtraction operations on the maximum exponential product data and the plurality of exponential product data to obtain a plurality of displacement data; and A plurality of shifters connected to the plurality of subtractors and for shifting the mantissa parts of the plurality of input data according to the plurality of displacement data.

8. The in-memory computing device according to claim 1, wherein Each of the weight mantissa storage circuits includes a plurality of static random access memories.

9. The in-memory computing device according to claim 1, wherein Each of the mantissa operation circuits includes: Multiple switching and precharging circuits, connected to the multiple weight mantissa storage circuits and used to receive the mantissa parts of the multiple weight data; and A point-by-point multiplier, connected to the multiple switching and precharging circuits and used to receive the mantissa parts of the multiple weight data, to perform the multiplication operation on the mantissa parts of the multiple input data and the mantissa parts of the multiple weight data, so as to obtain the multiple mantissa product data.

10. The in-memory computing device according to claim 1, wherein Each of the weight sign storage circuits includes a plurality of static random access memories.

11. The in-memory computing device according to claim 10, wherein, Each of the static random access memories includes six transistors.

12. The in-memory computing device according to claim 1, wherein, Each of the sign operation circuits includes: A switching and precharging circuit, connected to the multiple weight sign storage circuits and used to receive the sign parts of the multiple weight data; and An exclusive-OR operator, connected to the switching and precharging circuit and used to receive the sign parts of the multiple weight data, to perform the exclusive-OR operation on the sign parts of the multiple input data and the sign parts of the multiple weight data, so as to obtain the multiple sign product data.

13. An exponential storage operation module, characterized in that, Includes: Multiple weight exponent storage circuits, used to store the exponent parts of multiple weight data; Multiple exponent operation circuits, used to perform an addition operation on the exponent parts of multiple input data and the exponent parts of the multiple weight data, so as to obtain multiple exponent product data; And A comparison circuit, used to compare the multiple exponent product data to obtain the maximum exponent product data.

14. The exponential storage operation module according to claim 13, wherein Each of the weight exponent storage circuits includes a plurality of static random access memories.

15. The exponential storage operation module according to claim 13, wherein Each of the exponent operation circuits includes: Multiple switching and precharging circuits, connected to the multiple weight exponent storage circuits and used to receive the exponent parts of the multiple weight data; and An adder, connected to the multiple switching and precharging circuits and used to receive the exponent parts of the multiple weight data, to perform the addition operation on the exponent parts of the multiple input data and the exponent parts of the multiple weight data, so as to obtain the multiple exponent product data.

16. The exponential storage operation module according to claim 13, wherein The comparison circuit is connected to the multiple exponent operation circuits, and the comparison circuit includes: Multiple comparators, used to compare two pieces of exponent product data among the multiple exponent product data.

17. The exponential storage operation module according to claim 16, wherein Each of the comparators includes: A first judgment circuit, used to compare the front bits of the multiple exponent product data; A second judgment circuit, connected to the first judgment circuit and used to compare the middle bits of the multiple exponent product data; and A third judgment circuit, connected to the second judgment circuit and used to compare the rear bits of the multiple exponent product data.

18. A mantissa storage operation module, characterized in that, Includes: Multiple weight mantissa storage circuits, used to store the mantissa parts of multiple weight data; Multiple mantissa operation circuits, used to perform a multiplication operation on the mantissa parts of multiple input data and the mantissa parts of the multiple weight data, so as to obtain multiple mantissa product intermediate data; And A shift and addition circuit, used to perform a shift on the multiple mantissa product intermediate data and then perform an addition to obtain multiple mantissa product data.

19. The mantissa storage operation module according to claim 18, wherein Each of the weight mantissa storage circuits includes a plurality of static random access memories.

20. The mantissa storage operation module according to claim 19, wherein Each of the mantissa operation circuits includes: Multiple switching and precharging circuits, connected to the multiple weight mantissa storage circuits and used to receive the mantissa parts of the multiple weight data; and A point-by-point multiplier, connected to the plurality of switching and precharging circuits, and used to receive the mantissa parts of the plurality of weight data, so as to perform the multiplication operation on the mantissa parts of the plurality of input data and the mantissa parts of the plurality of weight data to obtain the plurality of mantissa product data.

Images