136 results about "Partial product" patented technology

The invention relates to the technical field of artificial intelligence and natural language processing, in particular to a KV cache compression and eviction lexical element recovery method and system for large language model reasoning, and the method comprises the steps: for a query vector of an ith lexical element calculated by a current Transform layer, calculating attention scores of the query vector and all key vectors; executing a V cache dynamic updating operation based on the attention score; and for the ith lexical element, after the V cache dynamic updating operation is completely completed, performing attention calculation by using the updated value vector set in the V cache storage pool and the pre-calculated attention score partial product P. According to the technical scheme, the technical problem that in the prior art, collaborative optimization of video memory occupancy and calculation efficiency is difficult is solved, and the method has the advantages that KV cache video memory occupancy is dynamically managed, the model reasoning quality is kept, and the calculation efficiency is improved.

Integrated circuits with dot product circuitry are provided. The dot product circuitry may be configured to generate partial products of different ranks based on the inputs. The partial products may be organized into corresponding groups based on their ranks. Each group of partial products having the same rank can then be compressed using a compressor / reduction tree. At least some of the compressed partial product values may be shifted between the different groups to maintain the proper offset. Each partial product may have an associated one's to two's complement conversion bit. The conversion bits of the various partial product groups can be separately aggregated and then injected into the compressor tree at one or more locations.

The invention discloses an approximate floating point multiplier, a chip and computing equipment, an approximate mantissa multiplier of the approximate floating point multiplier comprises an AND logic unit and a compressor unit, and the AND logic unit is used for performing AND operation on two input operands bit by bit to generate a partial product array with the size of 11 rows and 21 columns; the compressor unit is used for compressing the 11th column to the 21st column by column to obtain a final approximate mantissa, the compressor unit comprises two novel approximate 4-2 compressors ignoring carry design, and the error rate of the approximate 4-2 compressors is within an acceptable range by utilizing mutual compensation inside the compressors. The invention aims to excavate and use the characteristics of floating point multiplication to further improve the energy efficiency of floating point multiplication, realize the optimization of the approximate floating point multiplier in the overhead aspects of precision, power consumption, area and the like, and solve the problems of relatively complex circuit and low compression efficiency of the traditional approximate 4-2 compressor.

The invention discloses a compressor based on a cascade full adder for multiplication and a partial product array compression method, which are used for solving the technical problem that the delay of a compression step is relatively high because most of the existing compressors carry out optimization calculation on an input partial product array based on the idea of a traditional Dada tree. The compressor comprises a first parallel 3-2 compressor circuit, a first parallel cascade 3-2 compressor circuit, a second parallel 3-2 compressor circuit, a second parallel cascade 3-2 compressor circuit and a serial 3-2 / 2-2 compressor circuit; the input partial product array is compressed through a first parallel 3-2 compressor circuit, a first parallel cascade 3-2 compressor circuit, a second parallel 3-2 compressor circuit and a second parallel cascade 3-2 compressor circuit, after a fourth initial compression result is obtained, a serial 3-2 / 2-2 compressor circuit is adopted for accumulation, and a target compression result is generated.

The invention relates to a multiplier based on radix-4Booth coding and an improved Wallace compression tree, and the multiplier comprises a partial product generation module which recodes an input multiplier into a plurality of partial products through employing a Radix-4Booth algorithm, and forms a compact partial product array; the partial product compression module is used for quickly compressing the partial product through two compressors by utilizing an improved Wallace tree compression structure, and reducing the calculation path delay; and the final summation module is used for summing the compressed partial product by using a high-speed summator to obtain a final multiplication result. According to the invention, a radix-4Booth coding algorithm is adopted, so that a partial product array is regularly arranged, the circuit area is reduced, a new tree type compression structure in which a 3-2 compressor and a 4-2 compressor are mixed is adopted to realize a Wallace tree, the compression efficiency is improved, and the key delay is reduced.

The invention discloses a CNN-oriented signed approximate multiply-add calculation unit based on error mutual compensation, and belongs to the technical field of calculation, reckoning or counting. The approximate multiplication and addition calculation unit comprises a sign bit processing module, a sub multiplier module and a partial accumulation module. And the symbol processing module performs XOR operation on the symbol bits of the two input operands to obtain the symbol bit of a calculation result. The sub-multiplier module multiplies the numerical value part of the operand to obtain a partial product; wherein a precise multiplier is used when the numeric high order is multiplied by the numeric high order, and an approximate multiplier is used in other cases. The partial accumulation module uses an accurate adder at a high position and uses an approximate adder at a low position. And meanwhile, when the approximate multiplier and the approximate adder are designed, errors of the approximate multiplier and the approximate adder are mutually compensated, so that the energy efficiency of the circuit is improved, the influence of approximate calculation on the precision of the neural network is reduced, and the requirement of the neural network on calculation precision is met.

The invention discloses a high-speed fixed-point multiplication circuit. The multiplication circuit is mainly composed of a multiplier coding module, a partial product generation module, a partial product compression module and a traveling wave carry adder module. The multiplier coding module is composed of a radix-4-Booth coding algorithm and an opposite number generation module, and is used for carrying out three-bit block coding on input multiplication data and generating an opposite number of a multiplicand in advance. The partial product generation module generates a plurality of groups of partial products with symbol extension according to the multiplicand coded signal. And the partial product compression module adopts an improved Wallace compression structure to perform layered compression on the partial product. And the traveling wave carry adder module sums the two groups of results output by compression and outputs a multiplication result. According to the invention, multiplication accumulation series can be reduced, the switching times of invalid signals in the circuit can be reduced, and the longest delay path in an operation link can be shortened, so that fixed-point multiplication which is high in speed, low in power consumption and more favorable for a comprehensive tool in structure is realized.

A system and method for providing a tunable floating-point multiply-accumulate (MAC) unit are disclosed. The unit maintains full arithmetic precision while enabling dynamic elimination of ineffectual computation through operand decomposition and selective activation of partial product generation logic. The disclosed MAC unit is suitable for drop-in replacement in existing deep-learning accelerators and improves energy efficiency without requiring architectural changes.

The invention discloses multi-precision dot product operation hardware which is characterized in that an input processing unit carries out unpacking according to an input data format and obtains a symbol, an index and a mantissa distribution pattern; judging a special value of a dot product result, and calculating a product symbol; the dot product arithmetic unit is used for calculating dot product indexes and product shift values according to the index distribution patterns; calculating a product or a partial product according to the symbol, the mantissa distribution pattern and the product symbol, performing shift alignment and summation according to an input data format and / or a shift value, and summing a dot product operation intermediate result to obtain a dot product result; and the output processing unit rounds and normalizes the dot product mantissa according to the standardized output format, the rounding type and the index of the dot product result, and packages and outputs the normalized processing result according to the special value of the dot product result. The hardware supports input data formats with various precisions, the application scene of the hardware is expanded, the design area and power consumption are reduced by multiplexing the dot product arithmetic unit, and the hardware resource utilization rate and the calculation efficiency are improved.

The invention discloses an approximate multiplier implementation method for a neural network and an approximate multiplier, and the key points of the technical scheme are that the method comprises the steps: inputting a first operand and a second operand, and splitting the first operand into a low-order part and a high-order part; performing first coding on the low-order part to obtain a plurality of groups of first coding numbers, and performing second coding on the high-order part to obtain a plurality of groups of second coding numbers; assembling the plurality of groups of first coding numbers and the plurality of groups of second coding numbers to form an assembly operand; according to the assembly operand and the second operand, generating a partial product matrix and a truncation error compensation value corresponding to the partial product matrix; based on the partial product matrix and the truncation error compensation value, an approximate multiplication result of the first operand and the second operand is generated, approximate high-base coding and truncation compensation are combined for use, and the performance of an approximate multiplier is greatly improved on the premise that the calculation precision and the final network reasoning precision are guaranteed.

The invention discloses a circuit for approximate floating point fusion dot product operation. The circuit comprises an extraction module used for extracting sign bits, mantissa bits and exponent bits from four single-precision floating point numbers; the symbol integration module is used for integrating symbol bits and correcting mantissa bits; the index comparison module is used for calculating two groups of dot product effective indexes according to the index bits and generating three control quantities; the first multiplication module is used for generating a first group of approximate partial products according to a correction mantissa digit corresponding to a first control quantity larger value; the second multiplication module is used for generating a second group of approximate partial products according to the correction mantissa digit corresponding to the smaller value of the first control quantity, and performing arithmetic displacement and adding symbol compensation bits according to the size of the second control quantity; the fusion compression module is used for performing fusion compression on the first group of approximate partial products, the second group of approximate partial products after arithmetic shift and the symbol compensation bits to obtain a sum sequence and a carry sequence; and the result output module is used for generating a dot product operation result in combination with the third control quantity.

The invention relates to the technical field of digital filters, and provides an improved FIR filter structure based on a distributed algorithm (DA). The structure comprises a DA table, a partial product selection module and a shift accumulator. Wherein the DA table generates all possible partial products according to input; the partial product selection module selects an actual partial product according to the address sequence; the shift accumulator realizes accumulation and summation of partial products. Compared with the traditional DA-FIR filter, the method disclosed by the invention has the advantage that the consumed lookup table (LUT) and register (FF) resources are obviously reduced.

The invention relates to a neural network acceleration method and device based on algorithm and hardware cooperation and an accelerator, and the method comprises the steps: processing a value-level index of a quantization tensor based on a sparsity threshold value, so as to generate layer metadata, and dynamically adjusting a sparsity mode; identifying the positions and the number of non-zero bits according to symbol conversion and a bit lookup table, and putting the processed data into bit groups to eliminate non-contribution bits; processing the bit groups, distributing the related input and tensor of the self-adaptive indexing mechanism extracted from the layer metadata to an elastic calculation unit, and executing shift addition calculation according to the number of non-zero bits; and on the basis of the column type calculation array, instructing all the arrays to complete multiplication operation in a preset period, and storing partial products and data streams corresponding to respective bit widths. According to the method, the accelerator controller can efficiently skip the zero positions in the calculation process, and meanwhile, the performance and applicability of the bit-level accelerator are improved.

The invention discloses a reusable double-domain multiplier design method and system, and the method comprises the steps: double-domain non-logic base-4Booth coding: setting a coding logic which simultaneously supports the complement operation of a prime number domain and a binary extension domain and a symbol extension bit, and easily realizing the generation of a partial product under double domains through extension; dual-domain partial product generation: generating a partial product array based on the coded values; performing Wallace tree-shaped compression: on the basis of a compressor of a Wallace tree-shaped compression structure, realizing dual-domain compression by introducing a domain control bit; and adder operation: obtaining two partial products based on compression, and adding the two partial products through an adder to obtain a final multiplication result. According to the method, Booth coding, double-domain partial product generation and double-domain compression logic on the prime number domain and the binary extension domain are designed, the symbol extension bit preprocessing technology and the complement plus one-bit optimization technology are combined, multiplication on the double domains can be supported at the same time, and the method has the remarkable advantages of being small in resource occupation and high in performance.

The invention discloses an approximate multiplier, an operation method thereof, a processor and a chip, and belongs to the field of integrated circuits. The approximate multiplier comprises a high-order processing circuit, a low-order processing circuit, a high-low-order fusion circuit and an error compensation circuit, and the high-order processing circuit adopts a plurality of negative deviation approximate addition units, performs approximate addition on a high-order area of a partial product step by step and outputs an error signal; the low-order processing circuit adopts a positive deviation addition unit to process the low-order area to generate an intermediate result of positive deviation; the fusion circuit merges the high and low position results and outputs an initial multiplication result; the error compensation circuit compensates the result high order according to the error signal to offset the deviation. According to the approximate multiplier provided by the invention, the comprehensive performance of the multiplication unit in the aspects of area, power consumption and speed can be improved while the basic calculation precision is ensured.

A hardware module for performing dot product operations includes receiver circuitry receiving a first vector and a second vector, each comprising at least two elements of a binary encoded integer. Logic generates an array of partial products of N rows of bits for a dot product operation between the first vector and the second vector. Grouping circuitry groups bits of the elements of the second vector into a binary number, wherein each binary number is associated with a respective row of the N rows of bits, and selector circuitry selects a partial product value for each of the N rows of bits based on the binary number that is associated with the respective row, such that one partial product is generated per binary number. The hardware module also comprises adder circuitry configured to perform adding the N rows of bits together to compute an output associated with the dot product operation between the first and second vectors.

Multiplication circuitry comprises adder sub-arrays which each add partial products derived from first / second operands. Sub-array result values generated by the adder sub-arrays are added in a result assembly addition to generate at least one multiplication result value representing a result of signed multiplication of the first operand and the second operand. Sign extension emulation is performed for a sign-extension-emulated sub-array result value added in the result assembly addition, by applying a default zero extension to the sign-extension-emulated sub-array result value regardless of its sign and emulating an effect of sign extending the sign-extension-emulated sub-array result value using another of the assembled values. Another example of multiplication circuitry applies default zero extension to a third signed operand being added to a product of first / second signed operands, and emulates sign extension of the third signed operand by adjusting one of the partial products derived from the first / second signed operands.

The present disclosure provides implementations of a filter suitable for use in quantum computing systems and other low-power, high-speed applications. In some aspects, a filter circuit includes a series-connected arrangement of unit delays and summers in an alternating pattern. The filter circuit further includes a plurality of coefficient multipliers, each having a respective output connected with one or more of the summers, and each including a multiplexing stage including one or more multiplexers addressed using one or more bits of a respective input coefficient vector. A first coefficient multiplier of the plurality of coefficient multipliers includes a partial product stage configured to provide a plurality of integer partial products of an input data vector to the multiplexing stages of the plurality of coefficient multipliers.

The application provides a kind of approximate precoding convolution operation method and system based on booth algorithm, comprising: step S1: multiplier and multiplicand input multiplier are separated by approximate precoding module to partial product, and the partial product related to the multiplicand after separation is input to addition tree module to complete compression, and the partial product related to the multiplicand after compression and the partial product related to the multiplier are added to obtain multiplier result;Step S2: the multiplier result is extended to sign bit, and the extended multiplier result is input into accumulation module to carry out shift and addition operation, and the shift addition operation result is carried out carry compensation.

The invention discloses a processor control method, a processor and electronic equipment. The method comprises the following steps: inputting a first mantissa of first data into a first partial product generation unit in the processor, and inputting a second mantissa of second data into a second partial product generation unit; generating a first mantissa and a partial product of the first mantissa through a first partial product generation unit to obtain a first group of partial products, and generating a second mantissa and a partial product of the second mantissa through a second partial product generation unit to obtain a second group of partial products; performing addition processing on the first group of partial products through the first CSA to obtain a first carry value and a first summation value, and performing addition processing on the second group of partial products through the second CSA to obtain a second carry value and a second summation value; and processing the first index, the second index, the first carry value, the first summation value, the second carry value and the second summation value to obtain a target processing result. The technical problem that the processing efficiency of a processor in the related technology is low is solved.

The invention discloses a structured sparse matrix multiplier realized based on FPGA primitives, and belongs to the technical field of FPGA hardware acceleration and deep learning computing architecture. The multiplier is composed of a register area and a multiplication and addition area, and the core design thought is that a circuit is built in a customized mode by directly calling bottom layer physical resources based on FPGA primitives; the register area stores a dense matrix B and supports parallel reading of elements by utilizing the characteristic that LUT in an SLICEM can be configured to be double SRL16E; the multiplication and addition area refers to a partial product generation unit based on the 4-Booth algorithm and an improved GPC (4: 2) compressor structure, and efficient generation and rapid accumulation of partial products are achieved. According to the method, redundant loss caused by high-level HDL logic synthesis is avoided through precise physical resource binding of primitives, fine wiring constraint and structured sparse data characteristic adaptation.

The present disclosure relates to a multiplier, a multiply-accumulate circuit, and a convolution operation unit. The multiplier includes: one or more selection circuits, each of the one or more selection circuits respectively configured to select a target preset multiple of a first operand from a preset multiple of a first operand as a fourth operand according to a corresponding third operand, wherein the target preset multiple is equal to a value of the third operand; and a partial product summing circuit, each of one or more input terminals of the partial product summing circuit respectively connected to an output terminal of corresponding one of at least one or more selection circuits, wherein the partial product summing circuit is configured to calculate a partial product sum of one or more fourth operands from the one or more selection circuits.

Computer computing device with optimized cache memory for matrix calculation Computer computing device (100), comprising: - a computing unit (102); - a main memory (104); - a cache memory (106) comprising a computing circuit (108) for reduction operations between partial products from values ​​of a sparse matrix (118) and an input vector, and an output vector; in which the cache memory has a first N-way associative memory region (112) storing, with a first granularity of TD words, results of reduction operations carried out from partial products from a dense region (120) of the matrix, and a second fully associative or M-way associative memory region (114) storing, with a second granularity of TS words, results of reduction operations carried out from partial products from a sparse region (124) of the matrix. Figure for the abridged version: Fig. 1

The invention provides an in-memory calculation circuit for realizing high-speed multiplication. The in-memory calculation circuit comprises a dual-output Raidx-8Booth encoder array, an in-memory calculation array and a partial product post-processing module, the dual-output Raidx-8Booth encoder array comprises m rows and z columns of dual-output Raidx-8Booth encoders, the dual-output Raidx-8Booth encoders encode a multiplier I and generate a control signal, and the calculation array in the memory is controlled to generate an intermediate result expressed by redundant numbers; data carrying is reduced, and the throughput rate and the energy efficiency of the system are improved. Meanwhile, a redundant coding method is applied to an in-memory computing structure, which is beneficial for eliminating carry transfer in the traditional multiplication operation by using parallel processing and a mode of compressing partial products by using a carry-free addition tree.

A method, computer readable medium, and processor are disclosed for performing matrix multiply and accumulate (MMA) operations. The processor includes a datapath configured to execute the MMA operation to generate a plurality of elements of a result matrix at an output of the datapath. Each element of the result matrix is generated by calculating at least one dot product of corresponding pairs of vectors associated with matrix operands specified in an instruction for the MMA operation. A dot product operation includes the steps of: generating a plurality of partial products by multiplying each element of a first vector with a corresponding element of a second vector; aligning the plurality of partial products based on the exponents associated with each element of the first vector and each element of the second vector; and accumulating the plurality of aligned partial products into a result queue utilizing at least one adder.

Multiplier Circuit This description relates to a circuit configured to perform a multiplication operation between a first value and a second value, the first and second values ​​each comprising up to N pieces of bits respectively, the circuit comprising: - an NMUL number of multiplier subcircuits configured to generate first partial products, each first partial product corresponding to a multiplication between a piece from among the pieces of the first value and a piece from among the pieces of the second value; - an adding part configured to generate a first intermediate sum and first excess bit values, from the first partial products;and a parallel adder configured to generate a value corresponding to the product between the first and second values ​​from the first intermediate sum and the first integer bit generated by the first adder circuit and the second processing circuit. Figure for the abbreviation: Fig. 5;

The invention provides an approximate multiplier based on a field programmable gate array (FPGA), which comprises an asymmetric partial product generation module, a signal processing module and a signal processing module, and is characterized in that the asymmetric partial product generation module is used for processing a received N-bit multiplier and a received N-bit multiplicand to generate an asymmetric approximate partial product matrix of which the line number is halved; the single-step compression module is used for compressing the asymmetric approximate partial product matrix into two rows at one time; and the final summing module is used for summing the two rows of compressed data and outputting a final product. Through the collaborative design of asymmetric partial product generation and single-step compression, high-energy-efficiency and low-delay approximate multiplication is realized on an FPGA platform, and the method is suitable for application scenes such as resource-limited edge calculation and the like.

A method and apparatus for reducing computation time are disclosed herein which provides in at least one embodiment a computer-implemented method for reducing computation time, the computer-implemented method including the steps of: receiving one or more bits from a multiplier, generating a shift signal by using the one or more bits, performing a shift operation by n bits (where n is a natural number) based on the shift signal to obtain a partial product, and generating an output product by using the partial product and then outputting the output product.

The present invention discloses an approximate squarer, which includes an accurate Booth folding encoder and decoder circuit, an approximate Booth folding encoder and decoder circuit, an accurate partial product compression circuit, an approximate partial product compression circuit, and a carry-ripple adder circuit; the accurate Booth folding encoder and decoder divide the high n-k bits of the input operand into groups of three bits, generate the accurate partial products of part P and the accurate partial products of part C, and output them to the accurate partial product compression circuit; the approximate Booth folding encoder and decoder divide the low k bits of the input operand into groups of three bits, generate the approximate partial products of part P and the approximate partial products of part C, and output them to the approximate partial product compression circuit. The present invention can achieve lower power consumption and area, lower cost; and has a faster calculation speed, which is more advantageous for communication and machine learning applications.

This invention discloses a vector floating-point multiply-adder suitable for floating-point operations of various precisions, comprising a first operation module, a second operation module, a third operation module, and a fourth operation module. The first operation module includes a partial product generation module, a Wallace network, a first inversion module, an exponent alignment module, a mantissa compound right shifter, a sticky logic module, and an exception pre-judgment module. The second operation module includes a 3:2 CSA adder, a CPA adder, an increment circuit, a GRS logic module, and a sign pre-judgment module. The third operation module includes a second inversion module, a leading zero detection module, a trailing zero detection module, a normalized compound left shifter module, a normalization correction module, a rounding preprocessing module, a fast GRS solver module, and an exponent adjustment module. The fourth operation module includes a mantissa increment logic module, an exponent increment logic module, a sign judgment module, an exception judgment module, and a control logic output module. This invention balances computational speed and chip area, enabling parallel execution of floating-point multiply-add operations of various precisions.