108 results about "Mixed precision" patented technology

The invention discloses a heterogeneous computing power cooperative scheduling system and method for mixed precision training, and belongs to the technical field of artificial intelligence computing. The system comprises a computational graph analysis and operator portrait module which is used for analyzing and dividing a model computational graph and extracting operator features; the heterogeneous hardware capability sensing and matching module is used for managing performance files and real-time states of heterogeneous hardware in the cluster and matching optimal execution hardware for each calculation partition; and the data flow coordination and pipeline parallel controller is used for generating a global execution plan, managing cross-device data dependence and communication and calculating overlapping optimization execution efficiency through communication. According to the method, the problem of low scheduling efficiency of mixed precision training in a heterogeneous environment is solved, automatic and accurate mapping from a calculation task to heterogeneous hardware is realized, the training speed is remarkably improved, the training cost is reduced, and the overall resource utilization rate of a cluster is improved.

The invention belongs to the technical field of storage and calculation integration, and particularly relates to a low-bit-width and high-energy-efficiency floating point storage and calculation integrated circuit based on a partial pre-alignment framework. The circuit comprises a memory array, a pre-calculation unit, an adder tree, a configurable arithmetic unit and a normalization unit, and supports mixed precision operation of FP8MACFP4 and FP8MACFP8. The method is characterized in that a partial pre-alignment strategy dominated by an activation value is adopted, the maximum index of the activation value is dynamically counted, the mantissa of the maximum index is aligned, and multiple partial pre-alignment intermediate results are pre-calculated and latched for reuse; in combination with a customized lookup table and a multiplexer, a pre-calculation result is directly selected to replace real-time multiplication and displacement; and through the reconfigurable hardware, the FP8MACFP8 high-precision operation is realized by utilizing the FP8MACFP4 unit combination. According to the method, complete online floating point multiplication and addition operation is realized, and excellent energy efficiency ratio and operation speed are obtained while high precision is kept.

The invention provides a lightweight large model operation method based on end-side deployment. The lightweight large model operation method comprises the following steps: acquiring equipment operation data and storing a model file in a mixed precision format according to quantization precision supported by terminal equipment; determining a mixing precision quantification strategy and carrying out strategy analysis on the equipment operation data; a Key-Value cache file obtained through model reasoning in the historical dialogue is reserved; calculating a semantic embedding vector of the second user request, calculating a cosine similarity between the semantic embedding vector and the first V vector, and judging whether the second user request has a similar intention or not according to a calculation result; and if yes, performing incremental reasoning on the second user request by multiplexing the Key-Value cache file. By monitoring the state of the terminal equipment in real time, dynamically adjusting the quantization level, multiplexing the cache data of the semantic related requests and merging batch request reasoning, the effects of balancing the energy consumption and performance of the terminal equipment, reducing repeated calculation and improving the service time of the equipment and the user experience are achieved.

The invention belongs to the field of machine learning, discloses an expert parallelism processing method and system for a large language model based on MoE, and realizes efficient parallel processing of the MoE model by dynamically distributing expert quantization bit width and sparse mode, predicting and prefetching to-be-activated expert parameters, grouping tokens to generate task queues and dynamically configuring hardware accelerators. Firstly, an importance score is calculated based on expert historical activation frequency, weight distribution and a model structure, so that quantization precision and a sparse proportion are adaptively allocated, and resource waste and precision loss of a unified strategy are avoided; secondly, predicting an expert to be activated by using a current layer hidden state and a historical activation sequence, loading parameters to a special cache in advance, reducing high-bandwidth memory access, and relieving bandwidth peak scrambling; moreover, tokens are grouped through a token-expert mapping table, a task queue is created, and dynamic configuration of a systolic array is combined, so that the problem of expert heterogeneity after compression is solved, and efficient and parallel hybrid precision matrix operation is ensured.

The invention belongs to the field of storage, and discloses a high-energy-efficiency mixed-precision charge domain in-storage computing architecture and a working method thereof. Comprising a single-slope analog-to-digital converter, a sparsity perception input alignment module, a multi-bit input accumulation module, a controller and an accumulation module, wherein the single-slope analog-to-digital converter consists of an index calculation array, a mantissa calculation array, a shared ramp voltage generator and a bidirectional counter. According to the invention, serial input binary coding based on capacitor voltage is suitable for floating point and integer multiply-accumulate operation with flexible bit width; according to the invention, a shared single-slope analog-to-digital converter (SS-ADC) is introduced to realize maximum index search and index difference calculation; according to the method, a sparsity perception calculation scheme is provided, low-importance input-weight pairs are filtered out through an adjustable threshold value, and invalid power consumption is reduced; according to the method, a multi-bit input accumulation method is further combined, and an ADC redundancy optimization quantization and normalization process is utilized, so that the overall energy efficiency is improved.

The invention relates to a self-adaptive mixing precision quantification method and device, equipment and a medium, and the method comprises the steps: carrying out the comprehensive sorting through the product of the cosine similarity difference value of adjacent layers and the sensitivity weight of each layer, so as to guarantee that bottleneck layers which are liable to be influenced by quantification and are crucial to the final precision can be accurately recognized; therefore, precise positioning of a protection target is realized, an iterative optimization loop is introduced, an optimal solution meeting a preset performance target can be spontaneously found finally by continuously evaluating a time-precision balance point of an overall model and automatically adjusting configuration, and a suboptimal result caused by improper primary configuration is avoided; and the input of the user is simplified into a visual final performance target, and the complicated layer sorting and selection process is automatically processed in the system, so that the use threshold of the technology is reduced.

A device for deep learning acceleration with mixed precision may include matrix-vector (MV) components that each include vector-vector (VV) components that are each configured to generate a respective VV output based on an input precision mode, an output precision mode, and an accumulation of products. The accumulation of products may be calculated by adding products based on the input precision mode. Each product may be calculated by multiplying, based on the input precision mode, a map data segment and a kernel data segment. Each MV component may include one or more components configured to concatenate VV outputs to generate a concatenated VV output. The device may include activation function components that are each configured to receive a corresponding concatenated VV output, generate an activation function output based on the corresponding concatenated VV output and the output precision mode, and output the activation function output.

The invention discloses a large model fine tuning optimization method based on multi-strategy fusion, which comprises the following steps: designing a dynamic parameter selection mechanism, adaptively determining a parameter subset needing fine tuning according to a task demand and a model structure, and reducing unnecessary parameter updating calculation; constructing a dynamic low-rank decomposition framework, dynamically adjusting the rank of a low-rank matrix according to a model training state and data characteristics, and keeping key information while compressing a parameter scale; a self-adaptive task sensing mechanism is introduced, a fine adjustment strategy is automatically adjusted according to different task characteristics, and the adaptability of the model to various tasks is improved; and a mixed precision training method is adopted, so that the calculation complexity and the memory occupation are reduced on the premise of ensuring the model precision. According to the method, a parameter efficient fine tuning technology and a dynamic low-rank decomposition strategy are innovatively combined, and an adaptive task perception mechanism and a mixed precision training technology are introduced, so that the operand and resource requirements of model training are effectively reduced, and the fine tuning efficiency and the model performance are improved.

Apparatuses, systems, and techniques to enable efficient use of mixed precision hardware architecture. In at least one embodiment, one or more first precision circuits and one or more second precision circuits are used on portions of data sets.

Systems and techniques for providing mixed-precision matrix multiplication in multi-chiplet processors recognize different precision formats of matrices to be multiplied based on, e.g., parameters provided with instructions or start and end memory locations of the matrices. A plurality of different multiplication chains are provided for different formats such that mixed-precision matrix multiplication can be performed using multiplication chains configured to handle multiplication of different precision formats. The multiplication chains are automatically selected based on the precision formats of the matrices to be multiplied, enabling programmers to utilize the chains without having to directly access the individual multiplication chains.

The invention relates to the field of power system simulation, and provides a self-adaptive mixing precision calculation method, system and device and a storage medium, applied to a multi-type programmable chip cooperation system, the method comprises the following steps: receiving runtime monitoring information returned by a multi-chip cooperation controller and a precision gateway through an intelligent precision manager, updating precision configuration information and a gateway conversion strategy according to the monitoring information during operation; decomposing the target calculation task into a plurality of sub-tasks through the multi-chip cooperative controller according to the precision configuration information, and mapping each sub-task to a target programmable chip corresponding to each precision partition; wherein when cross-precision transmission of data exists, data conversion and error control are carried out through the precision gateway according to the gateway conversion strategy, and an output result is obtained. According to the method, fine-grained identification and precision configuration can be realized, standardized conversion and error suppression are realized in a cross-precision boundary, closed-loop cooperation is carried out, and self-adaptive cooperative calculation of mixed precision is realized.

The application relates to the technical field of intelligent reasoning and safety judgment, and discloses a high-speed rail platform safety judgment method and system based on mixed-precision reasoning, which comprises the following steps: acquiring multi-source semantic observation records and generating a safety observation element set; constructing a platform safety relation graph; determining relation conflict density, closed residual error, cross-source divergence degree and reasoning difficulty level; generating a mixed-precision bit width scheduling table; performing graph relation reasoning to obtain an initial safety judgment vector and an initial judgment boundary quantity; in step 6, a final judgment vector is determined; and in step 7, a locked safety level is determined and a safety judgment package is output. The application realizes mixed-precision safety judgment and locked output driven by multi-source semantic relation of a high-speed rail platform.

A method, system, and computer program product perform machine-learning inferences with a tree-based model. The tree-based model includes a decision tree that was trained on a first system, which is configured to perform computations with a first arithmetic precision. The inferences are performed with the tree-based model on a second system, which is configured to perform computations with an arithmetic precision that is lower than the first arithmetic precision. Performing the inference includes determining that an input feature value is equal to a threshold value of a corresponding node and, in response, using a majority voting to select a left or right path of the decision tree. The majority voting is based on historical statistical data that includes tree-path statistics.

A The present invention relates to a technique for controlling input / output operation of a vector processor, which is designed to optimize vector operation in a mixed precision environment, and to a technique for maximizing data processing performance while minimizing waste of operation resources by improving the data conversion process between the memory and the vector processor.

The RWKV-based novel translation model inference method and system includes the following steps: 1) Collecting novel translations and extracting parallel corpora, and using dynamic MicroBatch concatenation technology for sequence compression; 2) Introducing a lightweight group query attention mechanism on the basis of the RWKV architecture, directly obtaining KV information from the Embedding layer to build the model; 3) Employing a sublinear complexity hybrid parallel training mode, combined with a global scalar scaling FP16 mixed precision strategy for training; 4) Applying a hierarchical distributed heterogeneous architecture, offloading the optimizer to low-performance devices and performing gradient compression transmission; 5) Outputting the translation using a joint decoder and dynamic batch inference technology. This invention, through the above method and system, effectively reduces the computational complexity and memory usage in the long text translation process, improves model training efficiency and inference throughput, and significantly improves the translation efficiency and contextual coherence of ultra-long texts.

This invention discloses a hardware acceleration method and accelerator for Transformer based on mixed-precision quantization and Huffman coding. The acceleration method includes: using a genetic algorithm to obtain several configuration schemes for mixed-precision quantization of Transformer network layers; performing mixed-precision quantization on each Transformer network layer based on each quantization configuration scheme to obtain a corresponding KL divergence; training a multilayer perceptron to obtain a quantization configuration prediction network using the quantization configuration scheme and the corresponding KL divergence as the output label and input feature, respectively; receiving a user-set target KL divergence value, using the quantization configuration prediction network to obtain the corresponding quantization configuration scheme, and performing mixed-precision quantization on each network layer based on the quantization scheme; and using Huffman coding to encode and compress all quantization weights before on-chip storage. This invention can reduce storage and computational overhead while maintaining model accuracy.

The invention discloses a federated learning acceleration method and system based on pipeline in-batch real-time data sampling and training. According to the method, in-batch data sampling is provided on the algorithm level aiming at the problems that computing resources of edge equipment are limited and importance is outdated and gradient deviation is caused by existing static sampling: the importance of samples in a fixed mini-batch is evaluated in real time by utilizing a latest model in each round of iteration, and a dynamic micro-batch is constructed; and through a gradient correction coefficient based on a sampling probability reciprocal, distribution deviation is eliminated, and unbiased training is realized. On the system level, an assembly line parallel mechanism based on a CPU-GPU heterogeneous architecture is designed, overlapping execution of sampling and training is achieved through a double-thread-double-flow concurrent model, data competition is solved through an annular buffer area and an event synchronization mechanism, and overhead is reduced in cooperation with mixing precision reasoning. According to the method, the hardware utilization rate can be remarkably improved, the model convergence precision is improved while the training time is greatly shortened, and the method is suitable for various edge computing scenes.

The present invention provides a hybrid precision MAC (multiple-and-multiple-accumulation) tree structure for maximizing memory bandwidth usage to accelerate the operation of a generative large scale language model. The present invention provides a hybrid precision MAC (multiple-and-multiple-accumulation) tree structure for maximizing memory bandwidth usage to accelerate the operation of the generative large scale language model. A MAC tree-based arithmetic unit according to one embodiment may include: a plurality of floating point multipliers connected in parallel and processing multiplication of data transferred from an external memory; a plurality of first converters for converting the output of each of the plurality of floating point multipliers from a floating point to a fixed point; a fixed-point adder tree which is connected to the plurality of first converters and processes the addition of the multiplication results of the plurality of floating-point multipliers; a fixed-point accumulator that accumulates the output of the fixed-point adder tree; and a second converter that converts the output of the fixed-point accumulator from a fixed point to a floating point.

Systems and techniques for providing mixed-precision matrix multiplication in multi-chiplet processors recognize different precision formats of matrices to be multiplied based on, e.g., parameters provided with instructions or start and end memory locations of the matrices. A plurality of different multiplication chains (208, 210, 212) are provided for different formats such that mixed-precision matrix multiplication can be performed using multiplication chains configured to handle multiplication of different precision formats. The multiplication chains are automatically selected based on the precision formats of the matrices to be multiplied, enabling programmers to utilize the chains without having to directly access the individual multiplication chains.

A technique for matching the throughput between writing into and reading from a memory can include receiving, in parallel, computational results in a high precision format for storing into the memory at a first frequency, and storing the computational results in the memory. The technique may further include rounding the computational results using round-to-the-nearest-even or stochastic rounding to down-convert the computational results from the high precision format to a low precision format in parallel, and outputting the computational results in the low precision format in parallel from the memory at a second frequency.

The invention discloses an ARM (Advanced RISC Machines) architecture-oriented CGFDM (China General Microbiological Fused Deposition Modeling) seismic wave solver for hybrid precision and vectorization collaborative optimization and a simulation method, and aims to break through the bottleneck of memory bandwidth and calculation efficiency of ARM platform seismic simulation. According to the method, firstly, an elastic wave equation is reconstructed in a dimensionless mode, key physical quantities are made to adapt to the FP16 range, the mixed precision strategy of FP16 storage and FP32 calculation is adopted, and on the premise that precision is guaranteed, about 50% of memory consumption is reduced. And secondly, for ARM scalable vector expansion (SVE), the CGFDM differential template is mapped to a vector register, SIMD parallel computing is realized, and the throughput is remarkably improved. Finally, the FP16-SVE solver realizes data rearrangement and conversion between the FP16 and the FP32, the problems of alignment and efficiency in mixed precision operation are solved, and synchronous improvement of storage and calculation efficiency is realized. Tests show that the method has the advantages that while the precision is maintained, the memory is halved, nearly three times of speed-up ratio is obtained, and an efficient scheme is provided for large-scale earthquake simulation and real-time disaster assessment of an ARM platform.

The application relates to the technical field of data compression, and discloses a multimedia data compression method and device for narrowband transmission. The method comprises the following steps: performing double-threshold contour extraction on an input image to obtain main contour data and secondary contour data; performing mixed-precision chain code coding on the main contour data and the secondary contour data to obtain compressed contour chain codes; performing parameterization decomposition and superframe coding on an input speech signal to obtain compressed speech parameters; allocating a transmission bandwidth according to the compressed speech parameters, and packing the compressed contour chain codes and the compressed speech parameters based on the transmission bandwidth to obtain a target transmission packet sequence. The application ensures the time synchronization and transmission reliability of image and speech data, solves the problems of multi-modal data asynchronous transmission and high bit error rate in a narrowband environment, and guarantees correct reconstruction and playing at a receiving end.

One embodiment provides a graphics processor comprising processing resources configured to perform a supersampling anti-aliasing operation via a mixed precision convolutional neural network. The processing resources include circuitry configured to receive, at an input block of a neural network model, a data including previous frame data, current frame data, jitter offset data, and velocity data, pre-process the data to generate pre-processed data, provide pre-processed data to a feature extraction network of the neural network model and an output block of the neural network model, process the first pre-processed data at the feature extraction network via one or more encoder stages and one or more decoder stages, output tensor data from the feature extraction network to the output block, and generate an anti-aliased output frame via the output block based on the current frame data and the tensor data output from the feature extraction network.

This invention relates to the field of integrated circuits and discloses a mixed-precision in-memory computing circuit, data processing method, and chip. The invention includes: a storage array, an index memory, and a multiply-accumulate calculation circuit. The storage array includes a first data element stored in a first precision format and / or a second data element stored in a second precision format; wherein the bit width of the first precision format is greater than that of the second precision format; the storage locations of the storage array are divided according to the bit width of the second precision format; the first data element occupies multiple consecutive storage locations; the read bit width of the storage array is an integer multiple of the bit width of the first precision format; the index memory stores the position index information of the first data element in the storage array; the multiply-accumulate calculation circuit is used to determine the precision format corresponding to the data read from the storage array based on the position index information, and performs a multiply-accumulate operation corresponding to the precision format, outputting the calculation result. This solves the compatibility problem between mixed-precision data storage and computing circuits.

The invention relates to a high-order interface capturing method for a coupling level set and a fluid volume based on mixing precision, which comprises the following steps of: firstly, dividing a calculation area into a grid comprising a plurality of control body units, initializing, then identifying interface units, and reconstructing a THINC interface function by adopting a high-order polynomial based on an LS value; thirdly, introducing a mixed precision iterative algorithm to solve an interface position nonlinear equation, and adaptively switching to multi-precision operation to obtain a robust solution when double precision cannot converge; then, flux is calculated through an accurate interface, a VOF transport equation is solved, and mass conservation is guaranteed; and finally, on the basis of the predicted interface geometry, an LS field which does not need to be reinitialized is directly reconstructed through pure geometric methods such as interface point cloud generation through projection and nearest point searching in a narrow band. According to the method, the capture precision and geometric fidelity of complex interfaces with high curvature, sharp corners and the like on the unstructured grids are remarkably improved, and the method has high precision and strong robustness.

The application discloses a pedestrian fall detection method based on mixed precision quantization and a storage medium, relates to the technical field of computer vision, and solves the technical problem that the mixed quantization precision model used in the existing pedestrian fall detection is difficult to achieve the best balance in model volume, precision and processing speed. The method comprises the following steps: S1, encoding a pedestrian neural network and quantizing the pedestrian neural network into a basic neural network; S2, initializing the basic neural network to obtain K different individuals, and performing selection and crossover operations; S3, performing game mutation operations; and S4, repeating steps S2-S3 until the iteration number reaches a preset maximum iteration number or the iteration termination condition is met, and obtaining an optimal mixed precision quantized pedestrian neural network model. The genetic algorithm improved by the game theory algorithm can make the model after mixed precision quantization achieve the best balance in model volume, precision and processing speed.

The application discloses a heterogeneous computing power cooperative scheduling system and method for mixed precision training, and belongs to the technical field of artificial intelligence calculation. The system comprises: a calculation graph analysis and operator image module, which is used for analyzing and dividing a model calculation graph and extracting operator features; a heterogeneous hardware capability sensing and matching module, which is used for managing the performance profile and real-time state of heterogeneous hardware in a cluster, and matching optimal execution hardware for each calculation partition; a data flow coordination and pipeline parallel controller, which is used for generating a global execution plan, managing cross-device data dependency and communication, and optimizing execution efficiency through communication and calculation overlap. The application solves the problem of inefficient scheduling of mixed precision training in a heterogeneous environment, realizes automatic and accurate mapping of calculation tasks to heterogeneous hardware, significantly improves training speed and reduces training cost, and improves the overall resource utilization of the cluster.

A device for deep learning acceleration with mixed precision may include a first data port configured to receive a map data segment and a second data port configured to receive a kernel data segment. The device may include a precision mode port configured to receive an indication of an input precision mode that indicates a word length for the map data segment and for the kernel data segment. The device may include a multiplier component configured to generate a multiplier component output based on the input precision mode and based on multiplying the map data segment and the kernel data segment. The device may include an adder component configured to generate an adder component output based on the input precision mode and based on the multiplier component output. The device may include an output port configured to output the adder component output.

The invention discloses a dynamic mixed precision reasoning resource scheduling method and system, and the method comprises the steps: S1, collecting the CPU / GPU utilization rate of a system, the memory occupation amount, the reasoning task queue length and the predicted calculation time consumption of each reasoning task, and generating a load state matrix; s2, grading each reasoning task, and outputting a task grading matrix containing complexity grades, accuracy demand grades and timeliness demand grades of all reasoning tasks; s3, extracting features of the reasoning task input data to obtain an input data feature matrix; s4, performing feature fusion on the input data feature matrix, the load state matrix and the task grading matrix to obtain a joint feature matrix; and S5, performing dimensionality reduction on the joint feature matrix to obtain a low-dimensional nonlinear feature vector, and inputting the low-dimensional nonlinear feature vector into a pre-trained decision model to obtain a precision adjustment decision and an execution sequence adjustment decision of each reasoning task.

Techniques and apparatus for improving machine learning model quantification are disclosed. A parameter tensor of a machine learning model is accessed, and a set of rows in the parameter tensor each including one or more outliers is identified. The parametric tensor is decomposed into a first parametric sub-tensor corresponding to the set of rows and a second parametric sub-tensor corresponding to at least one remaining row of the parametric tensor. The first parametric sub-tensor is quantized according to a first quantization scheme, and the second parametric sub-tensor is quantized according to a second quantization scheme. A quantized version of the machine learning model that includes the quantized first parametric sub-tensor and the quantized second parametric sub-tensor is generated.