76 results about "Partial product" patented technology

The invention relates to the technical field of artificial intelligence and natural language processing, in particular to a KV cache compression and eviction lexical element recovery method and system for large language model reasoning, and the method comprises the steps: for a query vector of an ith lexical element calculated by a current Transform layer, calculating attention scores of the query vector and all key vectors; executing a V cache dynamic updating operation based on the attention score; and for the ith lexical element, after the V cache dynamic updating operation is completely completed, performing attention calculation by using the updated value vector set in the V cache storage pool and the pre-calculated attention score partial product P. According to the technical scheme, the technical problem that in the prior art, collaborative optimization of video memory occupancy and calculation efficiency is difficult is solved, and the method has the advantages that KV cache video memory occupancy is dynamically managed, the model reasoning quality is kept, and the calculation efficiency is improved.

The invention discloses a high-speed fixed-point multiplication circuit. The multiplication circuit is mainly composed of a multiplier coding module, a partial product generation module, a partial product compression module and a traveling wave carry adder module. The multiplier coding module is composed of a radix-4-Booth coding algorithm and an opposite number generation module, and is used for carrying out three-bit block coding on input multiplication data and generating an opposite number of a multiplicand in advance. The partial product generation module generates a plurality of groups of partial products with symbol extension according to the multiplicand coded signal. And the partial product compression module adopts an improved Wallace compression structure to perform layered compression on the partial product. And the traveling wave carry adder module sums the two groups of results output by compression and outputs a multiplication result. According to the invention, multiplication accumulation series can be reduced, the switching times of invalid signals in the circuit can be reduced, and the longest delay path in an operation link can be shortened, so that fixed-point multiplication which is high in speed, low in power consumption and more favorable for a comprehensive tool in structure is realized.

A system and method for providing a tunable floating-point multiply-accumulate (MAC) unit are disclosed. The unit maintains full arithmetic precision while enabling dynamic elimination of ineffectual computation through operand decomposition and selective activation of partial product generation logic. The disclosed MAC unit is suitable for drop-in replacement in existing deep-learning accelerators and improves energy efficiency without requiring architectural changes.

The invention discloses a circuit for approximate floating point fusion dot product operation. The circuit comprises an extraction module used for extracting sign bits, mantissa bits and exponent bits from four single-precision floating point numbers; the symbol integration module is used for integrating symbol bits and correcting mantissa bits; the index comparison module is used for calculating two groups of dot product effective indexes according to the index bits and generating three control quantities; the first multiplication module is used for generating a first group of approximate partial products according to a correction mantissa digit corresponding to a first control quantity larger value; the second multiplication module is used for generating a second group of approximate partial products according to the correction mantissa digit corresponding to the smaller value of the first control quantity, and performing arithmetic displacement and adding symbol compensation bits according to the size of the second control quantity; the fusion compression module is used for performing fusion compression on the first group of approximate partial products, the second group of approximate partial products after arithmetic shift and the symbol compensation bits to obtain a sum sequence and a carry sequence; and the result output module is used for generating a dot product operation result in combination with the third control quantity.

The invention discloses an approximate multiplier, an operation method thereof, a processor and a chip, and belongs to the field of integrated circuits. The approximate multiplier comprises a high-order processing circuit, a low-order processing circuit, a high-low-order fusion circuit and an error compensation circuit, and the high-order processing circuit adopts a plurality of negative deviation approximate addition units, performs approximate addition on a high-order area of a partial product step by step and outputs an error signal; the low-order processing circuit adopts a positive deviation addition unit to process the low-order area to generate an intermediate result of positive deviation; the fusion circuit merges the high and low position results and outputs an initial multiplication result; the error compensation circuit compensates the result high order according to the error signal to offset the deviation. According to the approximate multiplier provided by the invention, the comprehensive performance of the multiplication unit in the aspects of area, power consumption and speed can be improved while the basic calculation precision is ensured.

A hardware module for performing dot product operations includes receiver circuitry receiving a first vector and a second vector, each comprising at least two elements of a binary encoded integer. Logic generates an array of partial products of N rows of bits for a dot product operation between the first vector and the second vector. Grouping circuitry groups bits of the elements of the second vector into a binary number, wherein each binary number is associated with a respective row of the N rows of bits, and selector circuitry selects a partial product value for each of the N rows of bits based on the binary number that is associated with the respective row, such that one partial product is generated per binary number. The hardware module also comprises adder circuitry configured to perform adding the N rows of bits together to compute an output associated with the dot product operation between the first and second vectors.

The present disclosure provides implementations of a filter suitable for use in quantum computing systems and other low-power, high-speed applications. In some aspects, a filter circuit includes a series-connected arrangement of unit delays and summers in an alternating pattern. The filter circuit further includes a plurality of coefficient multipliers, each having a respective output connected with one or more of the summers, and each including a multiplexing stage including one or more multiplexers addressed using one or more bits of a respective input coefficient vector. A first coefficient multiplier of the plurality of coefficient multipliers includes a partial product stage configured to provide a plurality of integer partial products of an input data vector to the multiplexing stages of the plurality of coefficient multipliers.

The application provides a kind of approximate precoding convolution operation method and system based on booth algorithm, comprising: step S1: multiplier and multiplicand input multiplier are separated by approximate precoding module to partial product, and the partial product related to the multiplicand after separation is input to addition tree module to complete compression, and the partial product related to the multiplicand after compression and the partial product related to the multiplier are added to obtain multiplier result;Step S2: the multiplier result is extended to sign bit, and the extended multiplier result is input into accumulation module to carry out shift and addition operation, and the shift addition operation result is carried out carry compensation.

The invention discloses a structured sparse matrix multiplier realized based on FPGA primitives, and belongs to the technical field of FPGA hardware acceleration and deep learning computing architecture. The multiplier is composed of a register area and a multiplication and addition area, and the core design thought is that a circuit is built in a customized mode by directly calling bottom layer physical resources based on FPGA primitives; the register area stores a dense matrix B and supports parallel reading of elements by utilizing the characteristic that LUT in an SLICEM can be configured to be double SRL16E; the multiplication and addition area refers to a partial product generation unit based on the 4-Booth algorithm and an improved GPC (4: 2) compressor structure, and efficient generation and rapid accumulation of partial products are achieved. According to the method, redundant loss caused by high-level HDL logic synthesis is avoided through precise physical resource binding of primitives, fine wiring constraint and structured sparse data characteristic adaptation.

Computer computing device with optimized cache memory for matrix calculation Computer computing device (100), comprising: - a computing unit (102); - a main memory (104); - a cache memory (106) comprising a computing circuit (108) for reduction operations between partial products from values ​​of a sparse matrix (118) and an input vector, and an output vector; in which the cache memory has a first N-way associative memory region (112) storing, with a first granularity of TD words, results of reduction operations carried out from partial products from a dense region (120) of the matrix, and a second fully associative or M-way associative memory region (114) storing, with a second granularity of TS words, results of reduction operations carried out from partial products from a sparse region (124) of the matrix. Figure for the abridged version: Fig. 1

The invention provides an in-memory calculation circuit for realizing high-speed multiplication. The in-memory calculation circuit comprises a dual-output Raidx-8Booth encoder array, an in-memory calculation array and a partial product post-processing module, the dual-output Raidx-8Booth encoder array comprises m rows and z columns of dual-output Raidx-8Booth encoders, the dual-output Raidx-8Booth encoders encode a multiplier I and generate a control signal, and the calculation array in the memory is controlled to generate an intermediate result expressed by redundant numbers; data carrying is reduced, and the throughput rate and the energy efficiency of the system are improved. Meanwhile, a redundant coding method is applied to an in-memory computing structure, which is beneficial for eliminating carry transfer in the traditional multiplication operation by using parallel processing and a mode of compressing partial products by using a carry-free addition tree.

Multiplier Circuit This description relates to a circuit configured to perform a multiplication operation between a first value and a second value, the first and second values ​​each comprising up to N pieces of bits respectively, the circuit comprising: - an NMUL number of multiplier subcircuits configured to generate first partial products, each first partial product corresponding to a multiplication between a piece from among the pieces of the first value and a piece from among the pieces of the second value; - an adding part configured to generate a first intermediate sum and first excess bit values, from the first partial products;and a parallel adder configured to generate a value corresponding to the product between the first and second values ​​from the first intermediate sum and the first integer bit generated by the first adder circuit and the second processing circuit. Figure for the abbreviation: Fig. 5;

The invention provides an approximate multiplier based on a field programmable gate array (FPGA), which comprises an asymmetric partial product generation module, a signal processing module and a signal processing module, and is characterized in that the asymmetric partial product generation module is used for processing a received N-bit multiplier and a received N-bit multiplicand to generate an asymmetric approximate partial product matrix of which the line number is halved; the single-step compression module is used for compressing the asymmetric approximate partial product matrix into two rows at one time; and the final summing module is used for summing the two rows of compressed data and outputting a final product. Through the collaborative design of asymmetric partial product generation and single-step compression, high-energy-efficiency and low-delay approximate multiplication is realized on an FPGA platform, and the method is suitable for application scenes such as resource-limited edge calculation and the like.

A method and apparatus for reducing computation time are disclosed herein which provides in at least one embodiment a computer-implemented method for reducing computation time, the computer-implemented method including the steps of: receiving one or more bits from a multiplier, generating a shift signal by using the one or more bits, performing a shift operation by n bits (where n is a natural number) based on the shift signal to obtain a partial product, and generating an output product by using the partial product and then outputting the output product.

This invention discloses a vector floating-point multiply-adder suitable for floating-point operations of various precisions, comprising a first operation module, a second operation module, a third operation module, and a fourth operation module. The first operation module includes a partial product generation module, a Wallace network, a first inversion module, an exponent alignment module, a mantissa compound right shifter, a sticky logic module, and an exception pre-judgment module. The second operation module includes a 3:2 CSA adder, a CPA adder, an increment circuit, a GRS logic module, and a sign pre-judgment module. The third operation module includes a second inversion module, a leading zero detection module, a trailing zero detection module, a normalized compound left shifter module, a normalization correction module, a rounding preprocessing module, a fast GRS solver module, and an exponent adjustment module. The fourth operation module includes a mantissa increment logic module, an exponent increment logic module, a sign judgment module, an exception judgment module, and a control logic output module. This invention balances computational speed and chip area, enabling parallel execution of floating-point multiply-add operations of various precisions.

The invention provides a floating-point number multiplier, a near memory computing circuit, a high-bandwidth magnetic computing chip system and electronic equipment, and relates to the technical field of circuits, and the floating-point number multiplier comprises a compression circuit and a partial product generation circuit; the floating-point number multiplier is used for realizing mantissa multiplication of a known floating-point number of a multiplier; the partial product generation circuit comprises: a one-out-of-four selector; the compression circuit comprises a compressor and a summator. The one-out-of-four selector is used for generating a partial product of a known multiplier and a multiplicand after NR4SD + coding; the compressor is used for carrying out parallel compression on the partial product output by the one-out-of-four selector; and the adder is used for accumulating the parallel compression results output by the compressor and outputting an accumulation result to finish mantissa multiplication of the known multiplier and the multiplicand. The storage efficiency can be improved, and the area and power consumption of a hardware circuit are reduced.

The invention relates to the technical field of integrated circuit design, in particular to a processing circuit architecture supporting multi-calculation precision dynamic switching, which comprises a control port module, a partial product generation module, a symbol compression module, an addition compression tree module and a final adder module. And single-cycle dynamic switching of various precisions is realized. The partial product generation module adopts a Booth coding algorithm to split an operation vector, and cooperates with boundary symbol selection logic to solve the problem of symbol expansion and auxiliary bit overlapping; the symbol compression module compresses the redundant extension bits through a preset coding mode; the addition compression tree module is formed by cascading multiple stages of compressors and inserting carry blocking logic; the final adder module is composed of a plurality of carry lookahead adders, and the output bit width is dynamically controlled through blocking logic. The architecture optimizes the parallel operation efficiency and the resource utilization rate, adapts to the deep learning training and reasoning full scene, and has the advantages of real-time performance and low power consumption.

The invention discloses a method for realizing multiplication operation based on an FPGA, computer equipment and a storage medium, and the method comprises the steps: traversing each multiplication unit in a circuit netlist, obtaining an input signal and an output signal of each multiplication unit, comparing the bit width of the input signal, and determining a multiplier signal; traversing the multiplier signal according to a sequence from low to high, wherein each continuous three-bit signal of the multiplier signal generates a corresponding coding logic unit; each continuous three-bit signal of the multiplier signal generates a corresponding partial product signal, and the partial product signal is stored in a partial product signal set C; performing iterative summation on the partial product signals in the partial product signal set C; carrying out traveling wave carry addition on partial product signals in the partial product signal set C; and deleting the multiplication unit in the circuit netlist. The multiplication units in the circuit netlist are processed through FPGA traversal, multiplication operation in the circuit is achieved, consumption of basic logic resources is reduced, and the application performance of the circuit is improved.

The invention belongs to the field of heterogeneous approximate calculation, and relates to a high-precision random calculation method based on a binary partial product and a multiplier. The high-precision random calculation method based on the binary partial product directly uses the weight of a bit corresponding to a binary number to generate a random bit stream. Bit streams are combined in a uniform distribution mode, phase and sum are carried out bit by bit, and all phase and results are added to obtain the random calculation multiplier with the optimal precision. According to the method, binary weights are combined, a shifting and splicing mode is used for replacing addition, partial product addition is used for achieving random calculation, and an approximate parallel counter is adopted for optimization. The multiplier provided by the invention fully combines the advantages of random calculation and binary calculation, the hardware cost is less than half of that of a binary multiplier, and the delay is lower. When the provided multiplier is applied to neural network reasoning, the precision of model loss can be almost ignored.

The invention provides a partial product decoupling-based dynamic precision approximate multiplication implementation method and a multiplier, and the multiplier comprises a significance evaluation unit which splits a first operand and a second operand into high digits and low digits, and correspondingly generates a first amplitude and a second amplitude; different precision instructions are generated based on the amplitudes and the threshold values; the dynamic precision unit performs zero setting processing on the second low digit based on the precision instruction to form an approximate number, and calculates a partial product of the first high digit and the approximate number; the accurate calculation unit accurately calculates products of the other three parts; and the final addition unit sums the products of the four parts according to the weight. The calculation unit can be used for constructing or optimizing a multiplication calculation array in a neural network hardware accelerator, is particularly suitable for scenes with dense fixed-point number operation such as convolution, and performs efficient approximate operation on the input fixed-point number through a data driving mechanism by utilizing the common sparsity and amplitude difference characteristics of neural network data, so that the calculation efficiency is improved. And self-adaptive management of power consumption is realized.

The present description concerns a circuit configured to perform a multiplication between first and second values each comprising up to a number N of chunks of n bits, comprising: a number NMUL of multiplier sub-circuits configured to generate partial products corresponding to a multiplication between one chunk of the first value and one chunk of the second value; an adder part configured to generate an intermediate sum, corresponding to the addition of partial products of same significance, and excess bit values; an input crossbar circuit configured to select at least one chunk of the first value and at least one chunk of the second value and to provide them to one of the multiplier sub-circuits; a control unit configured to indicate which chunks to supply to which multiplier sub-circuit; and a parallel adder configured to generate a value corresponding to the product.

The invention discloses an in-memory multiply-accumulate and similarity calculation circuit based on dynamic logic, which is used for supporting the design of an in-memory calculation macro cell and an accelerator in a digital domain, and provides an in-memory calculation circuit architecture which supports a Booth coding rule for inputting similarity calculation. On the basis, a local input sharing calculation unit and a decoding and addition tree unit are designed, and element-by-element multiply-accumulate calculation and similarity calculation of fixed data are supported. Compared with a traditional multiply-accumulate circuit, the multiply-accumulate circuit disclosed by the invention adopts a calculation mode of firstly carrying out Booth coding on input data, then generating a multiply-accumulate partial product and finally carrying out decoding accumulation, and meanwhile, a partial product generation and similarity calculation circuit is designed based on dynamic logic, so that the partial product is multiplexed, and multiply-accumulate calculation is realized element by element with smaller calculation overhead; the face effect and the energy efficiency are both considered, and the comprehensive performance of the computing architecture and the circuit is improved.

A method of a neural network processor performing a multiplication operation and a neural network processor. In some embodiments, the method includes forming a first set of products and forming a second set of products. Forming the first set of products can include multiplying, in a first multiplier, a first activation value with a least significant subword and a most significant subword of a first weight to form a first partial product and a second partial product, and adding the first partial product and the second partial product. Forming the second set of products can include multiplying, in the first multiplier, a second activation value with a first subword and a second subword of a mantissa to form a third partial product and a fourth partial product, and adding the third partial product and the fourth partial product.

This invention provides an approximate 4-bit lookup table multiplier, relating to the field of integrated circuits, comprising: a conversion module, a first-stage multiplication module, and a second-stage multiplication module. During the multiplication process, for two compressed partial products with the same input, a 6-input, 2-output lookup table (LUT6_2) is used for simultaneous calculation. For three compressed partial products with the same input, the precision loss of each compressed partial product is calculated, and a 6-input, 2-output lookup table (LUT6_2) is used to simultaneously calculate the two compressed partial products with the smallest precision loss. The compressed partial product with the largest precision loss is calculated separately using a 6-input lookup table (LUT6). For compressed partial products with an input number exceeding the lookup table's input limit, and for unprocessed partial products, a partial carry-truncation method is used to calculate the product result. This invention can improve multiplier performance while reducing multiplier precision loss.

The embodiment of the invention provides a method for executing polynomial multiplication and acceleration hardware. The acceleration hardware comprises an input transformation module, a convolution kernel transformation module, a modular multiplication module, an output transformation module and a partial product processing module. The method comprises the following steps: respectively segmenting a first polynomial coefficient vector and a second polynomial coefficient vector to obtain T first sub-vectors and K second sub-vectors; the input transformation module and the convolution kernel transformation module transform the first sub-vector and the second sub-vector into an input square matrix and a convolution kernel square matrix respectively based on a two-dimensional Winograd input transformation matrix and a convolution kernel transformation matrix; and executing the first operation for K times. Any ith first operation comprises the steps that a modular multiplication module multiplies T input square matrixes and an ith convolution kernel square matrix element by element to obtain T Hadamard product square matrixes; the output transformation module transforms the Hadamard product matrix into partial product sub-vectors based on a two-dimensional Winograd output transformation matrix; a partial product processing module accumulates partial product sub-vectors to corresponding sub-vectors in the product polynomial coefficient vector in a specified manner.

The present disclosure relates to the technical field of artificial intelligence and natural language processing, in particular to a KV cache compression and word piece recovery method and system for large language model inference, the method comprising: calculating the attention score of the query vector of the i-th word piece calculated by the current Transformer layer with all key vectors; performing a V cache dynamic update operation based on the attention score; for the i-th word piece, after the V cache dynamic update operation is completed, performing attention calculation using the updated value vector set in the V cache storage pool and the pre-calculated attention score partial product P. The above technical solution solves the technical problem that the existing technology is difficult to optimize the memory occupation and the calculation efficiency, and has the advantages of dynamically managing the KV cache memory occupation, maintaining the model inference quality, and improving the calculation efficiency.

A method, computer readable medium, and processor are disclosed for performing matrix multiply and accumulate (MMA) operations. The processor includes a datapath configured to execute the MMA operation to generate a plurality of elements of a result matrix at an output of the datapath. Each element of the result matrix is generated by calculating at least one dot product of corresponding pairs of vectors associated with matrix operands specified in an instruction for the MMA operation. A dot product operation includes the steps of: generating a plurality of partial products by multiplying each element of a first vector with a corresponding element of a second vector; aligning the plurality of partial products based on the exponents associated with each element of the first vector and each element of the second vector; and accumulating the plurality of aligned partial products into a result queue utilizing at least one adder.

The invention provides an integer multiplier, a processor and computing equipment, and the integer multiplier comprises a partial product generation circuit which is used for receiving a multiplicand and a multiplier to calculate a partial product; the partial product compression circuit is used for compressing the partial product to generate a compressed partial product; the low-order calculation circuit is used for calculating a low-order multiplication result and a carry signal according to the low-order compression partial product; and the high-order calculation circuit is used for calculating according to the high-order compression partial product and the carry signal to obtain a high-order multiplication result. The multiplication result of the common multiplication instruction type can be output in advance, the overall delay of the integer multiplier is reduced, the overall throughput of the integer multiplier is improved, the area of the integer multiplier can be reduced as much as possible, and space is saved.

The invention discloses a zero-frequency signal amplitude reduction method and system based on a multi-cut full-parallel FIR filter, and belongs to the technical field of chip digital design, and the method comprises the steps: executing the CSD multiplication of a filter coefficient and input data, and obtaining a plurality of partial products; carrying out cutting processing on the plurality of partial products respectively; calculating a compensation value of a truncation error based on the symbol attributes of the plurality of partial products and the state of the least significant bit after truncation; and accumulating the plurality of partial products after cutting processing and the compensation value to obtain filtering output. The invention solves the problems that the zero-frequency signal amplitude of the filter is obviously increased and the resource efficiency and the signal precision are difficult to consider at the same time due to the fact that the truncation digits are increased for reducing hardware resources in the prior art.

This invention provides an approximate floating-point fused point integral step-by-step alignment circuit, relating to the field of approximate circuit design. The invention generates two sets of effective dot product exponents based on the comparison results, and remaps the inputs of the first and second multiplication modules. The difference between the two sets of effective dot product exponents is applied step-by-step to the arithmetic shift of the approximate partial product, reducing the area of ​​the shift unit and the logic depth. A fine-grained step-by-step shift compression architecture is adopted, integrating alignment and compression operations within two processing units, avoiding large-width shifters becoming critical path bottlenecks. The processed first set of approximate partial products, the second set of approximate partial products after two compressions and shifts of the remaining partial products, and the sign compensation bit are fused and compressed, improving the efficiency of the fusion compression module and effectively reducing circuit area and delay.