13 results about "Variable precision" patented technology

A fused dot-product multiply-accumulate (MAC) circuit may support variable precision of floating-point data elements to perform computations in deep learning operations (e.g., MAC operations). The operating mode of the circuit may be selected based on the accuracy of the input element. The mode of operation may be an FP16 mode or an FP8 mode. In the FP8 mode, a product index may be calculated based on an index of a floating point input element. A maximum index may be selected from the one or more product indexes. A global maximum index may be selected from a plurality of maximum indexes. A product mantissa may be calculated based on a difference between the global maximum exponent and a corresponding maximum exponent and aligned with another product mantissa. The adder tree may accumulate the aligned product mantissas and compute the partial and mantissas. The portions and mantissas may be normalized using a global maximum index.

The invention provides a precision target optimization method and system adaptive to a variable precision arithmetic logic unit, a medium, a terminal and a program product. The method comprises the following steps: acquiring an output feature set of each group of an upper layer; the precision generation network layer generates a corresponding precision target according to the output feature set, and the ALU calculation layer generates a prediction result according to the generated precision target; the teacher model generates a reference target and a real label according to the output feature set; constructing a total loss function according to the calculated task loss, precision generation loss and adversarial loss; performing back propagation optimization on the student model based on the constructed total loss function; repeatedly and iteratively training the student model until convergence to obtain a final student model; and deploying the final student model to generate an optimal precision target corresponding to each group. According to the method provided by the invention, the fine precision adjustment of the bit granularity can be realized, the adaptive ability of the model is enhanced, and the matching degree of the precision and the task demand is improved.

A tunable performance parameter is used to control the level of accuracy of operations f the computation unit, such as a large language model (LLM) in a digital and / or analog neural network. The tuna

The invention discloses a precision-variable Tanh activation function fitting method, which comprises the following steps of: dividing the positive axis input of a Tanh function into a linear region (0, 1.156) and a saturation region (1.156, + infinity) based on the characteristics of the Tanh function, and setting target absolute precision epsilon 0 to determine a demarcation point x0 of a linear function y = x and a trapezoidal function, y = x fitting is adopted in a linear region, it is ensured that the error does not exceed epsilon 0 through segmented iteration in a trapezoidal region, optimization constant tempset fitting is adopted in a saturation region, and negative semi-axis fitting or negative number rejection processing is achieved through the property of a Tanh odd function. By means of the mode, it can be guaranteed that the accuracy of the AI model is reduced within a certain range, hardware resources are greatly reduced, and the requirements for low-power-consumption, small-area and high-speed deployment of the neural network terminal are met.

The key-value (LV) cache in this application accelerates inference in large language models (LLMs) by allowing attention operations to scale linearly, rather than quadratically, with the total sequence length. Since context lengths are long in modern LLMs, the KV cache size may exceed the model size, potentially negatively impacting throughput. To address this issue, a multi-granularity clustering-based scheme for KV cache compression is implemented. Clusters created at different clustering levels with variable precision approximate the key and value tensors corresponding to less important terms. Precision loss is reduced by using proxies generated at finer-grained clustering levels of subsets of more salient attention heads. More salient attention heads have a greater impact on model accuracy than less salient attention heads. When the impact on accuracy is low, latency is improved by retrieving proxies from a subset of less salient attention heads from faster memory.

The present disclosure relates to a floating-point computation circuit comprising: an internal memory (104, 114) storing one or more floating-point values in a first format; status registers (124) defining a plurality of floating-point number format types associated with corresponding identifiers, each format type indicating at least a maximum size (BIS, MBB); and a load and store unit (108, 118) for loading floating-point values from and storing floating-point values to an external memory (120, 122), the load and store unit (108, 118) being configured: to receive, in relation with a first store operation, a first floating-point value from the internal memory (104, 114) and a first of said identifiers; and to convert the first floating-point value from the first format to a first external memory format having a maximum size (BIS, MBB) defined by the floating-point number format type designated by the first identifier.

Disclosed are methods for coding, transmitting, decoding and processing elements in a vector space or semantic data with variable precision. The invention relates to a method for coding an element (SD) belonging to a vector space comprising a plurality of subsets respectively identified by sequences of bits, in which the element (SD) is coded in the form of a sequence of bits (S) identifying a subset containing the element (SD), and to a transmission method carried out from a transmitting entity (100), the transmission method comprising: - transmitting, to a receiving entity (200), a sequence of bits (S) obtained by coding an element (SD) in a vector space in accordance with the coding method according to the invention; and to associated decoding and processing methods.

The invention provides a tensor arithmetic unit based on an RISC-V instruction set, and the tensor arithmetic unit comprises a microinstruction splitting and scheduling unit which receives a decoded macroscopic tensor instruction and an operation size parameter thereof, splits the macroscopic tensor instruction into microinstruction sequences according to a fixed physical scale of a reconfigurable calculation array, and transmits the microinstruction sequences to the reconfigurable calculation array; a zigzag traversal sequence and microinstruction scheduling are achieved through quintuple hardware circulation, and the loading, using and replacing sequence of the data blocks in the block register array is planned; the reconfigurable computing array is used for executing matrix multiply-accumulate operation of various data formats by designing a reconfigurable data path and fusing and multiplexing a floating point multiplier and a multi-precision accumulation tree under data paths with different bit widths; and the hardware multi-buffer unit is a multi-buffer architecture automatically managed by hardware and is matched with a zigzag traversal sequence to realize pipeline overlapping of data prefetching and calculation execution. The invention further provides an intelligent processor. Therefore, the invention can efficiently execute tensor operation with variable precision and variable scale.

The invention relates to the field of artificial intelligence and brain-like chips, in particular to a variable-precision in-memory computing macro circuit and a forward reasoning method, and the method comprises the steps: mapping a pulse neural network (full connection or convolutional neural network) formed based on LIF neurons into the variable-precision in-memory computing macro circuit, and selecting signals according to different precisions, and forward reasoning processes with different precisions are realized in the storage and calculation array, so that the energy consumption and delay of network calculation are greatly reduced, and the energy efficiency of the processor is greatly improved.

The application discloses a kind of parallelization multiplication operation methods of floating point number based on FPGA, comprising the following steps: based on the representation under IEEE754 different precision, design a kind of low-precision storage mode that can be blocked, including the design of exponent block and floating block;Then design arbitrary variable bit floating block fixed-point addition, variable bit floating block fixed-point multiplication is realized by the way of multiplication pool;Then using FPGA, the multiplication calculation result between the sign bit and the exponent bit of the multiplicand and the multiplier is obtained by exclusive or operation and fixed-point addition, the multiplication calculation result of the significand bit of the multiplicand and the multiplier is obtained by fixed-point multiplication;Finally, the obtained multiplication calculation result is normalized.The application can be effectively applied in memory computing, with the increase of the number of blocks, the data calculation delay can be significantly reduced, and the variable precision can improve the flexibility of calculation.