Finger-storage-multiply-accumulate three-stacked floating-point number in-memory computing system

Abstract

The invention discloses a finger-storage-multiply-accumulate three-stacked floating-point number in-memory computing system, and belongs to the field of application-specific integrated circuit design. According to the system, a three-stack type floating-point number calculation framework is designed, a pointing shift and mantissa multiply-accumulate module is designed, and floating-point number multiply-accumulate calculation of global pointing is supported. Compared with a traditional parallel floating-point multiply-accumulate circuit, the parallel floating-point multiply-accumulate circuit has the advantages that a calculation process of global alignment shifting and mantissa multiply-accumulate in sequence and a three-stack type calculation framework are adopted, similar calculation precision is achieved with smaller hardware overhead, low hardware overhead and high calculation precision are both considered, and the comprehensive performance of the calculation framework and the circuit is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of application-specific integrated circuit design, and in particular to a three-stack floating-point in-memory computing system for pointer-storage-multiply-accumulate. Background Art

[0002] Driven by the demands for high performance and high computing density, Computing in Memory (CIM) technology has gradually become an important direction in processor design. By integrating multiply and accumulate (MAC) operations directly into the memory array, CIM significantly reduces the energy consumption caused by repeated data transfers between the processor and memory, alleviating the bottleneck problem of the "memory wall" in the von Neumann architecture. In the fine-tuning and training of edge AI and the inference training of cloud AI, certain tasks require very high CIM computational accuracy. The integer number system no longer meets the high-precision requirements. Current architectures that support floating-point CIM have problems with low input multiplexing and high hardware overhead. Therefore, a floating-point in-memory computing architecture that is CIM-friendly, highly parallel, and has low circuit overhead is needed. Summary of the Invention

[0003] The present invention provides an index-store-multiply-accumulate triple-stack floating-point in-memory computing system to solve the technical problems of limited floating-point computing input reuse and high computing overhead in the existing in-memory computing structure.

[0004] An embodiment of the present invention provides a three-stack floating-point in-memory computing system with finger-storage-multiplication-accumulation, comprising a plurality of three-stack arrays, each of which is composed of a finger shift module, a storage module, and a mantissa multiplication-accumulation module arranged in sequence from top to bottom on a layout;

[0005] a pair-of-index shift module, configured to sum multiple pairs of input exponents and weight exponents to obtain multiple exponential sums, use the maximum value of the multiple exponential sums as the exponential part of the multiplication-accumulation result, and decrement the maximum value in each cycle, obtain a shift control signal by comparing it with other exponential sums, use the shift control signal to control whether each input mantissa is shifted, and input the shifted input mantissa into the mantissa multiplication-accumulation module;

[0006] A storage module, configured to store the exponent and mantissa of the floating-point weight in binary original code format, send the exponent of the weight to the index shift module, and send the mantissa of the weight to the mantissa multiplication and accumulation module;

[0007] The mantissa multiplication and accumulation module is used to perform multiplication and accumulation of the input mantissa and the weight mantissa to obtain the mantissa part of the floating-point multiplication and accumulation result.

[0008] Optionally, in one embodiment of the present invention, the index shift module includes a parallel exponential processing module and a serial shift module, the parallel exponential processing module is used to parallelly calculate the exponential sum of multiple input exponents and weights, and find the maximum exponential sum as the exponential part of the multiplication and accumulation result, and the control circuit generates a corresponding shift control signal per cycle; the serial shift module determines whether to shift the input mantissa according to the shift control signal. When shifting is not required, the serial shift module outputs the sign bit of the input mantissa to the mantissa multiplication and accumulation module. When shifting is required, the serial shift module shifts the input mantissa and outputs it to the mantissa multiplication and accumulation module.

[0009] Optionally, in one embodiment of the present invention, the parallel exponential processing module includes multiple exponential adders and a maximum value search module; multiple pairs of input exponents and weight exponents are input into the exponential adder to obtain multiple exponential sums, which are output to the maximum value search module. The maximum value search module searches for the maximum value of the multiple exponential sums in parallel as the exponential part of the multiplication and accumulation result.

[0010] Optionally, in one embodiment of the present invention, each storage module includes a 6T SRAM storage cell array and a pre-charging circuit thereof, and pre-stores binary format floating-point weights, which will participate in multiplication and accumulation calculations with floating-point inputs.

[0011] Optionally, in one embodiment of the present invention, each SRAM storage array is composed of multiple 6T-SRAM storage blocks, each 6T-SRAM block is composed of multiple rows and columns of 6T-SRAM storage cells and read-write circuits, and during data mapping, multi-bit wide floating-point weights are stored in the 6T-SRAM storage blocks by row, and each 6T-SRAM storage block stores multiple weights.

[0012] Optionally, in one embodiment of the present invention, the mantissa multiplication and accumulation module includes multiple multipliers, an adder tree and an accumulator, each multiplier calculates the product of the shifted input mantissa and the weight mantissa per cycle; the adder tree adds the product results of the multiple multipliers to obtain a partial sum, and then outputs it to the accumulator; the accumulator accumulates the partial sums of multiple cycles as the mantissa part of the multiplication and accumulation result.

[0013] The three-stacked floating-point in-memory computing system of finger-storage-multiplication-accumulation of the embodiment of the present invention is a three-stacked floating-point multiplication-accumulation computing architecture with scalable input multiplexing. This architecture takes into account both computing accuracy and area efficiency and is suitable for AI devices at the edge or in the cloud. Compared with traditional parallel floating-point multiplication-accumulation circuits, the present invention adopts a computing process of first global finger shifting and then mantissa multiplication-accumulation and a three-stacked computing architecture, which achieves similar computing accuracy with smaller hardware overhead, takes into account low hardware overhead and high computing accuracy, and improves the comprehensive performance of the computing architecture and circuit.

[0014] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0016] Figure 1 A schematic diagram of the structure of a three-stack floating-point in-memory computing system for index-store-multiply-accumulate provided according to an embodiment of the present invention;

[0017] Figure 2 Schematic diagram of a finger shift module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.

[0019] Figure 1 The present invention provides a schematic structural diagram of a three-stack floating-point in-memory computing system for pointer-store-multiply-accumulate according to an embodiment of the present invention.

[0020] like Figure 1 As shown, the finger-storage-multiplication-accumulation three-stack floating-point in-memory computing system includes multiple three-stack arrays, each of which is composed of a finger shift module, a storage module, and a mantissa multiplication-accumulation module arranged in sequence from top to bottom on the layout;

[0021] The index shift module is used to sum multiple pairs of input exponents and weight exponents to obtain multiple exponential sums, use the maximum value of the multiple exponential sums as the exponential part of the multiplication and accumulation result, and decrement the maximum value in each cycle. By comparing with other exponential sums, a shift control signal is obtained, and the shift control signal is used to control whether each input mantissa is shifted, and the shifted input mantissa is input into the mantissa multiplication and accumulation module;

[0022] A storage module is used to store the exponent and mantissa of the floating-point weight in binary original code format, send the exponent of the weight to the index shift module, and send the mantissa of the weight to the mantissa multiplication and accumulation module;

[0023] The mantissa multiplication and accumulation module is used to perform multiplication and accumulation of the input mantissa and the weight mantissa to obtain the mantissa part of the floating-point multiplication and accumulation result.

[0024] The three-stacked floating-point in-memory computing system of finger-storage-multiplication-accumulation of the embodiment of the present invention can support the design of digital domain floating-point in-memory computing macro units and accelerators. Based on the three-stacked floating-point computing architecture, finger shift and mantissa multiplication-accumulation modules are designed to support global finger floating-point multiplication-accumulation calculations. In this architecture, by distributing the finger shift modules and mantissa multiplication-accumulation modules on both sides of the storage array, while supporting global finger floating-point multiplication-accumulation calculations, it also supports the expansion of input multiplexing. At the same time, the three modules are compactly arranged from top to bottom to form a three-stacked structure. Each three-stacked structure supports expansion in the vertical direction to achieve scalable input multiplexing.

[0025] In an embodiment of the present invention, Figure 2 As shown, the index shift module includes a parallel exponential processing module and a serial shift module. The parallel exponential processing module is used to parallelly calculate the exponential sum of multiple input exponents and weights, and find the maximum exponential sum as the exponential part of the multiplication and accumulation result. The control circuit generates a corresponding shift control signal per cycle; the serial shift module decides whether to shift the input mantissa according to the shift control signal. When shifting is not required, the serial shift module outputs the sign bit of the input mantissa to the mantissa multiplication and accumulation module. When shifting is required, the serial shift module shifts the input mantissa and outputs it to the mantissa multiplication and accumulation module.

[0026] Specifically, the functions of the index shift module are as follows: summing several pairs of input exponents and weight exponents to obtain several exponential sums; finding the maximum value among these exponents and using it as the exponential part of the multiplication and accumulation result; decreasing the maximum value in each cycle and comparing it with other exponential sums to obtain a shift control signal; the shift control signal controls whether each mantissa input is shifted.

[0027] In an embodiment of the present invention, the parallel exponential processing module includes multiple exponential adders and a maximum value search module; multiple pairs of input exponents and weight exponents are input into the exponential adder to obtain multiple exponential sums, which are output to the maximum value search module. The maximum value search module searches for the maximum value of the multiple exponential sums in parallel as the exponential part of the multiplication and accumulation result.

[0028] In an embodiment of the present invention, each storage module includes a 6T SRAM storage cell array and a pre-charge circuit thereof, and pre-stores binary format floating-point weights, which will participate in multiplication and accumulation calculations with floating-point inputs.

[0029] In an embodiment of the present invention, each SRAM storage array is composed of multiple 6T-SRAM storage blocks, each 6T-SRAM block is composed of multiple rows and columns of 6T-SRAM storage cells and read-write circuits. During data mapping, multi-bit floating-point weights are stored in the 6T-SRAM storage blocks by row, and each 6T-SRAM storage block stores multiple weights.

[0030] In an embodiment of the present invention, a mantissa multiplication and accumulation module includes multiple multipliers, an adder tree and an accumulator, each multiplier calculates the product of the shifted input mantissa and the weight mantissa in each cycle; the adder tree adds the product results of the multiple multipliers to obtain a partial sum, which is then output to the accumulator; the accumulator accumulates the partial sums of multiple cycles as the mantissa part of the multiplication and accumulation result.

[0031] like Figure 1 As shown, a total of 16 three-stack modules are shown. These 16 three-stack modules share 16 global inputs (each three-stack module has 16 inputs, and all three-stack modules have the same input). Each global input consists of an input exponent and an input mantissa. The index shift module indexes the input mantissa based on the sum of the input exponent and the weight exponent, and serially outputs it to the mantissa multiplication and accumulation module for calculation. The index shift module takes the global input exponent and the weight exponent as input. It shifts the input mantissa based on the exponent information and then feeds the shifted mantissa into the mantissa multiplication and accumulation module. The output of the index shift module is the shifted input mantissa. The mantissa multiplication and accumulation module takes the shifted input mantissa and the weight mantissa as input, and outputs the partial sum of the multiplication and accumulation. Its function is to multiply the shifted input mantissa and the weight mantissa and then add them together to obtain the partial sum. The storage module stores the weight mantissa and exponent.

[0032] like Figure 2 As shown, each finger-paired shift module consists of 16 exponent adders, a maximum search module, and 16 serial shift modules. The finger-paired shift module operates as follows: 16 pairs of input exponents and weight exponents are fed into the exponential adders to generate 16 exponential sums, which are then output to the maximum search module. The maximum search module then searches for the maximum of these 16 exponential sums in parallel and uses this as the exponential portion of the multiplication-accumulation result. Each cycle, the maximum value is then decremented and compared with each exponential sum. If the comparison results are identical, the corresponding serial shift module is controlled to begin shifting; if the comparison results are different, shifting is not required for this cycle. The serial shift module determines whether to shift the input mantissa based on a shift control signal. If shifting is not required, the serial shift module outputs the sign bit of the input mantissa to the multiplication-accumulation module. If shifting is required, the serial shift module shifts the input mantissa and outputs it to the mantissa multiplication-accumulation module.

[0033] In this embodiment, each SRAM memory array consists of 16 6T-SRAM memory blocks, each of which consists of 16 rows and 16 columns of 6T-SRAM memory cells and read / write circuits. During data mapping, 16-bit floating-point weights are stored row by row in the 6T-SRAM memory blocks, and each 6T-SRAM memory block can store 16 weights.

[0034] In this embodiment, the mantissa multiplication and accumulation module consists of 16 multipliers, one adder tree, and one accumulator. Each multiplier multiplies an 8-bit weighted mantissa by a 2-bit input mantissa per cycle, outputting a 10-bit result to the adder tree. The adder tree adds the 16 10-bit multiplication results and outputs them to the accumulator. The accumulator accumulates the multi-cycle calculation results as the mantissa portion of the multiplication and accumulation result.

[0035] According to the embodiment of the present invention, a three-stacked floating-point in-memory computing system of indexing-storage-multiplication-accumulation is proposed. The computing process of first global index shifting and then mantissa multiplication-accumulation and a three-stacked computing architecture are adopted to achieve similar computing accuracy with smaller hardware overhead, taking into account low hardware overhead and high computing accuracy, and improving the comprehensive performance of the computing architecture and circuit.

[0036] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.

[0037] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "N" means at least two, such as two, three, etc., unless otherwise specifically defined.

Claims

1. A three-stack floating-point in-memory computing system with pointer-store-multiply-accumulate, characterized in that: It includes multiple three-stack arrays, each of which is composed of a finger shift module, a storage module and a mantissa multiplication and accumulation module arranged in sequence from top to bottom on the layout; a pair-of-index shift module, configured to sum multiple pairs of input exponents and weight exponents to obtain multiple exponential sums, use the maximum value of the multiple exponential sums as the exponential part of the multiplication-accumulation result, and decrement the maximum value in each cycle, obtain a shift control signal by comparing it with other exponential sums, use the shift control signal to control whether each input mantissa is shifted, and input the shifted input mantissa into the mantissa multiplication-accumulation module; A storage module, configured to store the exponent and mantissa of the floating-point weight in binary original code format, send the exponent of the weight to the index shift module, and send the mantissa of the weight to the mantissa multiplication and accumulation module; The mantissa multiplication and accumulation module is used to perform multiplication and accumulation of the input mantissa and the weight mantissa to obtain the mantissa part of the floating-point multiplication and accumulation result.

2. The system according to claim 1, wherein: The index shift module includes a parallel exponential processing module and a serial shift module. The parallel exponential processing module is used to parallelly calculate the exponential sum of multiple input exponents and weights, and find the maximum exponential sum as the exponential part of the multiplication and accumulation result. The control circuit generates a corresponding shift control signal per cycle; the serial shift module determines whether to shift the input mantissa according to the shift control signal. When shifting is not required, the serial shift module outputs the sign bit of the input mantissa to the mantissa multiplication and accumulation module. When shifting is required, the serial shift module shifts the input mantissa and outputs it to the mantissa multiplication and accumulation module.

3. The system according to claim 2, characterized in that The parallel exponential processing module includes multiple exponential adders and a maximum value search module; multiple pairs of input exponents and weight exponents are input into the exponential adder to obtain multiple exponential sums, which are output to the maximum value search module. The maximum value search module searches for the maximum value of the multiple exponential sums in parallel as the exponential part of the multiplication and accumulation result.

4. The system according to claim 1, wherein: Each storage module contains a 6T SRAM storage cell array and its pre-charge circuit, which pre-stores binary format floating-point weights. The floating-point weights will participate in multiplication and accumulation calculations with floating-point inputs.

5. The system according to claim 4, characterized in that Each SRAM storage array consists of multiple 6T-SRAM storage blocks, each of which consists of multiple rows and columns of 6T-SRAM storage cells and read-write circuits. During data mapping, multi-bit floating-point weights are stored in the 6T-SRAM storage blocks by row, and each 6T-SRAM storage block stores multiple weights.

6. The system according to claim 1, wherein: The mantissa multiplication and accumulation module includes multiple multipliers, an adder tree and an accumulator, each multiplier calculates the product of the shifted input mantissa and the weight mantissa in each cycle; the adder tree adds the product results of the multiple multipliers to obtain a partial sum, which is then output to the accumulator; The accumulator accumulates the partial sums of multiple cycles as the mantissa of the multiplication-accumulation result.

Images