45 results about "Adder tree" patented technology

The invention provides a neural network multiply-accumulate operation device and a neural network multiply-accumulate operation method, and the method comprises the steps: carrying out the product operation of input data and a pre-distributed corresponding weight based on a primitive code coding rule through a preprocessing module, generating a positive and negative product pair, and transmitting the positive and negative product pair to a positive and negative product temporary storage region; the adder tree sequentially calculates the numerical value of one bit position of each pair of positive and negative product pairs according to the order from the high order to the low order of the positive and negative product pairs, and accumulates the numerical values; the operation module performs subtraction operation on the positive and negative dynamic sums, and accumulates an intermediate value calculated by a current bit and an intermediate value calculated by a previous bit; the determiner determines whether to terminate the subsequent calculation of the current bit or continue the accumulation of the next bit in advance; the neuron calculation module executes corresponding activation function operation or neuron dynamics updating operation according to the type of the neural network, an operation mode of bit-by-bit calculation from a high bit to a low bit is used, the data processing amount is reduced, and the trend of a calculation result can be predicted.

According to the technical scheme, the large-scale logic optimization multiplier verification method is characterized in that a four-stage pipeline processing mode is adopted, the first stage is a partial adder tree recovery stage, and the second stage is a partial adder tree recovery stage; the second stage is an architecture generation stage; the third stage is a function verification stage; and the fourth stage is an equivalence checking stage. The verification capability is improved in a breakthrough manner and is far better than that of an existing method; the innovative three-stage decomposition strategy significantly reduces the calculation complexity, the ILP algorithm optimizes resource allocation, the multi-stage scheduling balances the calculation load, and the QACO algorithm efficiently explores the design space, so that the RefSCAT-2.0 framework provided by the invention becomes the most effective solution for verifying a large-scale logic optimization multiplier at present. From the perspective of practice, the technical blank of large-scale logic optimization multiplier verification is filled up, and important tool support is provided for reliability guarantee of key computing systems such as artificial intelligence chips, high-performance CPUs and GPUs.

The invention provides a Softmax accelerator realized based on segmentation approximation and a design method. The Softmax accelerator comprises a maximum value module, a subtraction module, a data segmentation module, an index module, an adder tree module and a logarithm module. The input of the maximum value module is an input vector xi of Softmax, i = 1, 2, 3... n, and the maximum value xmax in the input vector is selected through a multi-path parallel comparator; the input of the subtraction module is xi and xmax, subtraction operation is completed through n parallel adders, and n output results xi-xmax are obtained; the data segmentation module segments the xi-xmax data into xi-xmax = a negative integer + a positive decimal; the index module is used for fitting an exponential function through a piecewise approximation method according to data segmentation result calculation; the input of the adder tree module is output, the input of the logarithm module is fitting a logarithmic function through a piecewise approximation method, and the output is realized. On the premise of ensuring the calculation precision, the exponential operation is simplified through the piecewise approximation method, and the division operation is optimized to reduce the hardware complexity, so that the reasoning speed of a deep learning model is improved.

The present application provides a deep learning convolution acceleration method using bit-level sparsity and a processor. Comprises: selecting the maximum sum of the exponents from all data pairs to be convolved as a maximum exponent; arranging mantissas of the original weights in a computation sequence to form a weight matrix, and uniformly aligning each row of the weight matrix to the maximum exponent and removing slack bits to obtain a reduced matrix, allowing essential bits in each column of the reduced matrix to fill the vacancies according to the computation sequence, after removing null rows in the intermediate matrix, placing zeros at vacancies of the matrix to obtain an interleaved weight matrix, sending the weight segments in each row of the interleaved weight matrix and the mantissa of the corresponding activation to an adder tree for processing summation, by shifting and adding the sum result to obtain a convolution result.

The disclosure describes systems and methods for mixed‑mode, mixed‑precision computation using a configurable compute engine for vector–matrix operations. A processing system selects an operating mode including at least a floating‑point mode or an integer mode for the compute engine. Operand storage and format preprocessing route differently sized element representations through shared lanes. In floating‑point mode, compute circuitry generates a mantissa product and a product exponent, and alignment applies power‑of‑two control through a selection network, including as part of a fused multiply‑accumulate path. Parallel adder trees feed a hierarchical output accumulator sized to the selected precision. Normalization operates at a wider intermediate precision, applies block‑level scaling with dequantization, and, in lower‑precision operation, performs groupwise normalization with an intermediate‑precision combine to preserve a constant compute‑block size. Implementations integrate the compute engine within a memory macro with double buffering or deploy the compute engine as a standalone accelerator.

Systems, apparatuses, and methods are disclosed for improved matrix–vector operations in accelerators that may be useful or heavy AI training and inference workloads. The disclosed technology provides arrangements that permit more efficient computation by, for example, performing calculations without repeated conversion between numeric domains. In some implementations, a compute-in-memory (CIM) macro is configured to perform a vector matrix multiplication (VMM) operation between a vector of activation values and a matrix of weight values in floating point formats. The CIM macro has a functional block configured to align mantissa bits of primitive products between the activation values and the weight values by shifting the mantissa bits and an adder tree configured to output an accumulation value of the primitive products in an integer format by adding the shifted mantissa bits.

Disclosed in the embodiments of the present application are an internal memory, a chip and a related electronic device. A target processing unit (202) of an internal memory (102) comprises a plurality of groups of primary multipliers, a plurality of adder trees and a plurality of secondary multipliers, wherein one group of primary multipliers is connected to one secondary multiplier by means of one adder tree; one group of primary multipliers comprises a plurality of pairs of first multipliers; each adder tree comprises a first adder, a second adder and a third adder; each pair of first multipliers separately receives data to perform multiplication operations, and separately inputs output results into one first adder for addition operations; each second adder receives output results of a plurality of first adders or the upper-level second adder, so as to perform addition operations, and separately inputs the output results into the lower-level second adder or one third adder; and each third adder receives output results of a plurality of second adders to perform addition operations, and inputs the output results into one secondary multiplier for multiplication operations. Implementing the embodiments of the present application can improve the processing capability of the internal memory.

Systems, apparatuses, and methods are disclosed for improved matrix-vector operations in accelerators that may be useful or heavy AI training and inference workloads. The disclosed technology provides arrangements that permit more efficient computation by, for example, performing calculations without repeated conversion between numeric domains. In some implementations, a compute-in-memory (CIM) macro is configured to perform a vector matrix multiplication (VMM) operation between a vector of activation values and a matrix of weight values in floating point formats. The CIM macro has a functional block configured to align mantissa bits of primitive products between the activation values and the weight values by shifting the mantissa bits and an adder tree configured to output an accumulation value of the primitive products in an integer format by adding the shifted mantissa bits.

The invention provides a neural network calculation acceleration method and system, and the method comprises the steps: extracting an amplitude bit of input vector data and an amplitude bit of weight vector data, and generating a zero tag through logic operation; the zero label represents whether the product of the input vector data and the weight vector data is zero or not; dynamically aggregating the non-zero products according to the zero label to form an aggregated product; and dynamically adjusting the calculation scale of the adder tree according to the number of non-zero products in the aggregated products, and only performing accumulation operation on the non-zero products. According to the method, invalid calculation can be effectively reduced, the calculation efficiency and the utilization efficiency of hardware resources are improved, and the method is suitable for accelerated calculation of various neural network models.

Methods and apparatuses for performing deep learning operations are provided. The computing apparatus includes an adder-tree-based tensor kernel and a multiplier-accumulator (MAC)-based vector kernel. The adder-tree-based tensor kernel is configured to perform tensor operations, and the multiplier-accumulator (MAC)-based vector kernel is configured to perform vector operations using the output of the tensor kernel as input.

The invention belongs to the technical field of in-memory computing, and discloses a fine-grained structured sparse-oriented in-memory computing matrix multiplication acceleration system, which comprises an in-memory computing array group and an add tree, the in-memory computing array group consists of M rows and N columns and consists of a plurality of in-memory computing arrays, and each in-memory computing array comprises a storage module mem, a computing unit and a ping-pong pulsation input transmission chain PPSIC, the PPSIC is used for transmitting input data to the calculation module cmp and transmitting the input data to the PPSIC of the next column; and the computing units of the in-memory computing arrays on the same column are connected with the same add tree. According to the technical scheme, the internal structure of the digital in-memory computing array is optimized, the flexibility of the in-memory computing array is improved, and N: M fine-grained structured sparsity can be efficiently supported.

The present invention provides a hybrid precision MAC (multiple-and-multiple-accumulation) tree structure for maximizing memory bandwidth usage to accelerate the operation of a generative large scale language model. The present invention provides a hybrid precision MAC (multiple-and-multiple-accumulation) tree structure for maximizing memory bandwidth usage to accelerate the operation of the generative large scale language model. A MAC tree-based arithmetic unit according to one embodiment may include: a plurality of floating point multipliers connected in parallel and processing multiplication of data transferred from an external memory; a plurality of first converters for converting the output of each of the plurality of floating point multipliers from a floating point to a fixed point; a fixed-point adder tree which is connected to the plurality of first converters and processes the addition of the multiplication results of the plurality of floating-point multipliers; a fixed-point accumulator that accumulates the output of the fixed-point adder tree; and a second converter that converts the output of the fixed-point accumulator from a fixed point to a floating point.

A memory device is provided, including a non-volatile memory array including multiple memory cells, in which the memory cells arranged in a same row are configured to store corresponding weight data and are coupled to a same word line in multiple word lines; a word line driver configured to transmit multiple word line signals according to multiple input data signals to the word lines to perform a compute-in-memory (CIM) operation of the input data signals and the weight data stored in the non-volatile memory array; and an adder tree circuit coupled to the memory cells. Each of the memory cells in the same row is configured to generate a corresponding output voltage of the CIM operation to the adder tree circuit.

The application discloses a double-bit input feature optimization DCIM circuit based on SRAM and belongs to the technical field of electronic circuits. The circuit comprises an input preprocessing module, an SRAM memory and computing array, a split adder tree, a result correction module and an uninterrupted accumulator, supports feature optimization of multiple groups of double-bit data, reverses low-sparse input data to improve sparsity, and corrects calculation errors generated in the subsequent correction. After sparsification, the input data is reorganized into two groups of input data with lower average reversal rate. Feature optimization can significantly improve sparsity and reduce reversal rate. The hybrid computing unit and the split adder tree embedded in each SRAM unit of the SRAM memory and computing array are used to complete the multiplication and addition operation of the reorganized input data and weights. The uninterrupted accumulator completes accumulation in the least period. The circuit aims to improve the operation energy efficiency and data throughput of the DCIM architecture for edge AI devices.

A neural network accelerator includes 2n multiplier circuits, 2n shifter circuits and an adder tree circuit. Each respective multiplier circuit multiplies a first value by a second value to output a first product value. Each respective first value is represented by a first predetermined number of bits beginning at a most significant bit of the first value having a value equal to 1. Each respective second value is represented by a second predetermined number of bits, and each respective first product value is represented by a third predetermined number of bits. Each respective shifter circuit receives the first product value of a corresponding multiplier circuit and left shifts the corresponding product value by the first predetermined number of bits to form a respective second product value. The adder circuit adds each respective second product value to form a partial-sum value represented by a fourth predetermined number of bits.

A multiplier unit and a method and apparatus for calculating a dot product of floating point values are disclosed. The apparatus includes an array of multiplier units each including integer logic to multiply integer values of corresponding elements of two vectors, exponent logic to add exponent values of corresponding elements of the two vectors to form un-biased exponent values, and a local shifter to form a first shifted value by shifting an integer product value in a predetermined direction by a number of bits based on a difference between the un-biased exponent value corresponding to the integer product value and a maximum un-biased exponent value of the array of multiplier units. An adder tree adds the shifted values output from the local shifters of the array of multiplier units to form an output, and an accumulator accumulates the output of the adder unit.

The invention relates to a device for supporting matrix transpose multiplication calculation, which is particularly suitable for matrix multiplication operation in artificial intelligence algorithms such as a neural network. The device comprises an X vector input driving circuit, a Y matrix input driving circuit, a multiplication array and a configurable adder tree circuit. The adder tree circuit dynamically reconstructs and supports matrix multiplication calculation and matrix transpose multiplication calculation through a data distribution network. By multiplexing most circuits, hardware overhead is reduced, hardware efficiency is improved, power consumption is greatly reduced, different addition paths can be flexibly configured through the selector to support matrix multiplication and matrix transpose multiplication calculation, and therefore the effect of efficient and low-power-consumption calculation performance is achieved.

The invention discloses an adder tree circuit and a floating point calculation method, and relates to the technical field of chips. The adder tree circuit comprises an analysis circuit used for determining a plurality of floating point input data corresponding to a function operator and analyzing the plurality of floating point input data to obtain indexes and mantissas corresponding to the plurality of floating point input data; wherein the function operator is used for executing additive operation on a plurality of floating point input data; the order matching processing circuit is used for performing order matching on the indexes corresponding to the floating point input data based on the indexes and mantissas corresponding to the floating point input data to obtain mantissas after order matching corresponding to the floating point input data; the additive operation circuit is used for performing additive operation on the mantissa after order based on the plurality of floating point input data to obtain a first operation result; and the normalization processing circuit is used for performing normalization processing on the first operation result to obtain a second operation result. The area overhead of the logic operation circuit can be reduced, and the accuracy of the operation result is ensured.

The application discloses a kind of SCA-SAT synergistic effect realizes formal verification method of optimized multiplier, include: reverse engineering algorithm systematically from optimized multiplier restores adder tree;2) constraint satisfaction algorithm is only used by adder through constraint condition to complete the generation of reference multiplier;3) the verification method based on SCA and SAT combines the complementary advantages of SCA and SAT.In the foregoing verification framework, the application introduces a benchmark multiplier generator, which is used to generate a correct benchmark multiplier that has both similar structure to the optimized multiplier and clear adder boundaries.Clear adder boundaries enable the use of SCA-based verification to prove its correctness.Using the structural similarity of the benchmark multiplier and the optimized multiplier, the benchmark multiplier is then used as a known correct model for SAT-based verification of the optimized multiplier.

The application provides an SRAM-based digital domain in-memory computing method and device. First, input data and bit high-low order of the input data are acquired, and SRAM in-memory computing arrays corresponding to the input data respectively are determined according to the bit high-low order of the input data. Different preset approximate adder trees are included in different SRAM in-memory computing arrays, the different preset approximate adder trees include preset approximate full adders with different calculation accuracies, and the structure of the adder tree is a Wallace tree structure. Finally, the input data are subjected to approximate operation through the preset approximate adder tree, and a data approximate operation result is obtained. Through the above method, in combination with the application of the preset approximate full adder and the Wallace tree structure, the preset approximate adder tree with different accuracies can be selected to perform data calculation according to the bit high-low order of the input data while reducing the area and power consumption of the adder tree, and the calculation accuracy is improved.

The present application provides an anti-aliasing downsampling circuit, a circuit module, a chip and a signal processing device. The circuit includes a register, a module I Counter, multiplier, adder tree, multiplexer, adder, accumulator register and latch register; registers store different data in different clock cycles; module I The value of the counter variable increases with the change of the clock cycle. The filter coefficient selected according to the value of the count variable and the data stored in the preset register are input to the corresponding multiplier; the adder tree performs addition operation on the output result of the multiplier; under the control of the value of the count variable, the multiplexer selects to output 0 or the output result of the accumulator register, and the adder performs addition operation on the output result of the adder tree and the result selected by the multiplexer; the accumulator register stores the output result of the adder and uses the stored result as I The down-sampled result is fed into the adder via a multiplexer.

The application relates to a hierarchical addition tree structure module with fine-grained accurate reconfigurable approximate calculation, comprising a multi-data input module, an adder tree structure generation module, a calculation and result output module; the multi-data input module receives addends and defines approximate adder calculation, and receives user required precision configuration; the adder tree structure generation module input end receives addition numbers; after initialization, the number of addition tree layers will be transmitted to the approximate adder tree structure module to complete generation of the fine-grained accurate reconfigurable approximate adder module; the calculation and result output module performs approximate addition operation on the approximate adder generated in the previous stage to complete the final approximate calculation task, and controls the precision required by each layer in the approximate calculation process. The application realizes approximate addition calculation on a data vector, higher energy efficiency and appropriate precision, and solves the problem that the existing approximate adder configuration scheme cannot cope with data vector calculation.

In some aspects of the present disclosure, an adder tree circuit is disclosed. In some aspects, the adder tree circuit includes a plurality of full adders (FAs) including: a first subgroup of FAs, wherein each FA of the first subgroup includes a first number of transistors; and a second subgroup of FAs, wherein each FA of the second subgroup includes a second number of transistors, the first number being greater than the second number; wherein each FA of the first subgroup receives a first input from a first one of the second subgroup of FAs and a second input from a second one of the second subgroup of FAs, and each FA provides a first output to a third one of the second subgroup of FAs and a second output to a fourth one of the second subgroup of FAs.

The invention discloses a sparse linear optimization-oriented fast floating point multiply-accumulator, which comprises a complement module for complementing a mantissa part and a sign bit of input data; the order adding module is used for carrying out additive operation on the order part of the input data; the coding module is used for coding the complement data; the adder tree module comprises a 5-3 carry-save adder and is used for summing the partial products; the shifting module is used for performing shifting alignment on the Sum and the Carry according to the order code data; the accumulator module is used for accumulating the shifted Sum and Carry and a feedback result; the carry propagation adder module is used for carrying out carry propagation addition operation on the accumulation result; and the normalization module is used for carrying out normalization processing on the accumulated fixed-point format result. According to the sparse linear optimization-oriented fast floating point multiply accumulator, when the 5-3 carry-save adder is adopted to be realized on the FPGA, only one LUT is needed, and the compression ratio is improved.

A deconvolution calculation method, calculation apparatus (100) and calculation system (300), and a storage medium (400). The deconvolution calculation method comprises: by means of a systolic array (10), calculating each weight of each row of a convolution kernel with a first feature matrix, so as to obtain a second feature matrix corresponding to each weight of each row of the convolution kernel; and by means of an adder tree (20), calculating a plurality of second feature matrices corresponding to a plurality of weights of a plurality of rows of the convolution kernel, so as to obtain a third feature matrix.

An apparatus with deep learning includes: a systolic adder tree including adder trees connected in row and column directions; and an input multiplexer connected to an input register of at least one of the adder trees and configured to determine column directional data movement between the adder trees based on operation modes.

A calculation circuit, a memory device including the calculation circuit, and a calculation method are provided. The calculation circuit comprises an input allocator receiving and dividing n-bit input data (where n is a natural number equal to or greater than 2) into a plurality of operation elements based on a data type of the input data, an adder tree performing a multiplication operation between the operation elements, and an accumulator generating a first output value by adding an output value of the adder tree to a value stored in an accumulation register, wherein the first output value includes a sign bit and data bits, and the accumulator includes a first lightweight normalizer that performs bit shifting on the first output value by comparing a value of the sign bit with values of m bits (where m is a natural number) among the data bits.

A method and apparatus for performing deep learning operations. A computation apparatus includes an adder tree-based tensor core configured to perform a tensor operation, and a multiplier and accumulator (MAC)-based vector core configured to perform a vector operation using an output of the tensor core as an input.

The application relates to a variable bit width adder tree generation system based on multiple types of approximate calculation units, which comprises a signal input module, an adder tree construction module and a calculation and result output module. The input end A of the signal input module receives two groups of addends and defines the bit width; the input end B receives the required precision bit number and initializes the precision comparison module. The input end of the adder tree construction module receives two groups of addends; the approximate calculation unit library and the Boolean gate logic unit library are initialized and called respectively; in the iteration process, different types of approximate calculation units are selected in different levels according to the precision requirement input by the user. The calculation and result output module performs approximate addition operation on the approximate adder tree module generated in the last stage, and finally completes the approximate calculation task. The application realizes dynamic configuration of the bit width of the approximate addition operation, makes the selection of the precision and the power consumption more flexible, and solves the technical problem of poor precision of the existing approximate addition dynamic configuration scheme.

A representative reconfigurable processing circuit and a reconfigurable arithmetic circuit are disclosed, each of which may include input reordering queues; a multiplier shifter and combiner network coupled to the input reordering queues; an accumulator circuit; and a control logic circuit, along with a processor and various interconnection networks. A representative reconfigurable arithmetic circuit has a plurality of operating modes, such as floating point and integer arithmetic modes, logical manipulation modes, Boolean logic, shift, rotate, conditional operations, and format conversion, and is configurable for a wide variety of multiplication modes. Dedicated routing connecting multiplier adder trees allows multiple reconfigurable arithmetic circuits to be reconfigurably combined, in pair or quad configurations, for larger adders, complex multiplies and general sum of products use, for example.