252 results about "Mixed precision" patented technology

The invention relates to the field of artificial intelligence hardware acceleration, and discloses a memristor-based multilevel compression collaborative optimization neural network deployment method and device, and a storage medium. The method comprises the following steps: carrying out progressive structured pruning on a pre-training model based on a dual-drive scoring mechanism of an L1 norm and gradient sensitivity and hardware feedback, and generating a hardware-friendly sparse weight structure; the characteristics of the memristor are simulated through a micro-nonlinear conductance modeling function, and network weight and conductance parameters are synchronously optimized to reduce errors by adopting mixed precision quantification of four bits of a convolutional layer and two bits of a full-connection layer; a conductance drift and read-write noise model is injected, and the anti-interference capability of the model is improved in combination with adaptive noise enhancement and KL divergence loss; and mapping the optimized model to a memristor memory architecture to complete weight coding and reasoning. Through collaborative optimization of pruning, quantification and distillation, the problems of insufficient storage density, non-ideal characteristic interference and algorithm and hardware mismatch in memristor deployment are solved.

The invention belongs to the technical field of model training, and particularly relates to a large model increment training method and system based on dynamic sparsification. The method comprises the following steps: S1, evaluating the parameter importance of a large model by adopting a multi-dimensional evaluation method, and reflecting the contribution of parameters to the performance of the large model; s2, performing dynamic sparsification control on the large model structure by adopting a self-adaptive structure adjustment method; s3, performing incremental training management on the large model by constructing a capacity-adjustable empirical playback buffer pool and proposing a multi-objective loss function; and S4, carrying out model compression optimization on the large model by adopting a mixed precision quantification strategy. The method has the characteristics that the network structure can be dynamically adjusted by evaluating the importance of model parameters in real time, so that the efficient training is realized while the model performance is ensured.

The invention discloses mixed precision multiply-add operation hardware, which comprises an input processing unit used for obtaining standard symbols, standard indexes and standard mantissas according to a pre-configured input format and unpacked input multiply-add operation factors, and obtaining input and output special value detection results; the floating point multiplier-adder is used for calculating an operation result after the operand multiplies the scaling factor and the offset is accumulated according to the symbol, the exponent and the mantissa of each multiplication-addition operation factor; and the output processing unit is used for obtaining a target format result according to a pre-configured output format, a rounding type and a normalization and rounding operation result, packaging the target format result according to a detection result and outputting the target format result, and by multiplexing a data path of the floating-point multiplier-adder, the chip area is saved, format conversion is integrated, a data stream is simplified, and the data processing efficiency is improved. The requirements for automatic reconstruction of the multiply-add operation unit according to input and output formats and quantization and inverse quantization of floating points and integers are met, and the reuse rate of computing resources is increased.

The invention belongs to the technical field of mixing precision quantification, and discloses a mixing precision quantification method of a deep neural network and a related device. The mixing precision quantification method comprises the following steps: carrying out diversity processing on a population by using a variation strategy in a differential evolution algorithm to obtain a variation population; recombining the sub-components of the variation individuals in the variation population with the sub-components of the target individuals in the current population by using binomial crossover operation to obtain a plurality of test individuals; performing hybrid precision quantification on the deep neural network based on the test individual and the target individual, and selecting a better individual from the test individual and the target individual; and based on the selected better individual, judging whether the differential evolution algorithm converges or not, if so, obtaining the deep neural network after final mixing precision quantification, and otherwise, selecting the obtained better individual as a population of the next generation. The method can realize mixing precision quantification of the deep neural network, and has the advantages of relatively low calculation complexity and relatively high adaptability.

The invention relates to the technical field of network security, in particular to an intrusion detection system and method based on machine learning, and the system comprises a dynamic feature selection module which dynamically generates a feature mask through a reinforcement learning strategy network, and selects an optimal feature subset in real time according to an action return function # imgabs0 #; the lightweight detection module adopts a depth separable convolution structure, implements mixed precision quantization and structured pruning, and takes dynamic ReLU as an activation function; the incremental learning engine is used for restraining the weight through dynamic regularization based on a Fisher information matrix on the basis of a local cache data online fine tuning model; and the edge-cloud collaboration module is used for performing homomorphic encryption and differential privacy processing on model parameter differences, and updating a global model through a robust federated aggregation algorithm. Based on lightweight edge deployment and real-time incremental learning, the calculation overhead is reduced, and the robustness is improved.

The invention discloses a self-adaptive precision adjustment method in the NPU field. In the invention, through a mixed precision conflict resolution mechanism, global optimal balance of precision and performance in a deep learning reasoning task is realized. According to a traditional method, when the calculation precision is adjusted, independent decision making is usually carried out only based on local operator characteristics, and therefore the overall precision of an operator group dependent on series connection is collapsed due to error accumulation. According to the method, an error propagation chain is constructed through a quantitative sensing model, influence paths of different precision combinations on task output are dynamically predicted, and differential precision distribution is implemented on conflict operator groups by adopting a critical path priority algorithm. In a target detection task, a key path such as a classification head forcibly retains high-precision calculation, and a background filtering layer is reduced to a low-precision mode, so that the mechanism fundamentally solves the global sub-optimal problem caused by local optimization in a traditional method, and ensures the calculation reliability of a high-value operator.

The invention discloses a heterogeneous computing power cooperative scheduling system and method for mixed precision training, and belongs to the technical field of artificial intelligence computing. The system comprises a computational graph analysis and operator portrait module which is used for analyzing and dividing a model computational graph and extracting operator features; the heterogeneous hardware capability sensing and matching module is used for managing performance files and real-time states of heterogeneous hardware in the cluster and matching optimal execution hardware for each calculation partition; and the data flow coordination and pipeline parallel controller is used for generating a global execution plan, managing cross-device data dependence and communication and calculating overlapping optimization execution efficiency through communication. According to the method, the problem of low scheduling efficiency of mixed precision training in a heterogeneous environment is solved, automatic and accurate mapping from a calculation task to heterogeneous hardware is realized, the training speed is remarkably improved, the training cost is reduced, and the overall resource utilization rate of a cluster is improved.

In some examples, a method of performing matrix multiplication using mixed precision decomposition of operands includes converting a first matrix including a plurality of elements having a first precision to a second matrix, where the second matrix includes a plurality of elements having a second precision, the second precision being lower than the first precision; generating a first output matrix comprising a product of the second matrix and a third matrix, where the third matrix comprises a plurality of elements having the second precision, the elements of the first output matrix being stored at the first precision or converted to the first precision; calculating a residual value matrix using the first matrix and the second matrix, where the residual value matrix comprises elements representing respective differences between elements of the first matrix and elements of the second matrix; the elements of the residual matrix are scaled using a first scaling factor to generate a first scaled matrix.

The invention relates to the technical field of monolithic integration, in particular to a monolithic three-dimensional integration implementation method and device for a hybrid precision in-memory computing architecture, and the method comprises the steps: obtaining a silicon-based CMOS technology and a subsequent integration technology in a monolithic three-dimensional integration technology; manufacturing a control logic circuit on the computing architecture in the mixed precision memory by adopting a silicon-based CMOS (Complementary Metal Oxide Semiconductor) process; a plurality of functional layers are vertically stacked on a logic circuit by adopting a back-end integration process, communication among the functional layers is realized through interlayer dielectric via holes in the vertical direction, an analog in-memory array, a digital in-memory array and a cache are arranged on the functional layers, the analog in-memory array realizes matrix and vector multiplication with fixed weight, and the digital in-memory array realizes matrix and vector multiplication with fixed weight. And the array in the digital memory realizes matrix multiplication of refresh weight. Therefore, the problems that an in-memory computing architecture usually needs to occupy a relatively large part of chip area, the computing efficiency is relatively low and the like in related technologies are solved.

The invention discloses a floating point addition implementation method and device for mixed precision operation and electronic equipment. The method comprises the steps of performing mild preprocessing on two low-precision operands of floating point addition operation; if the improved near-path condition is met, leading zero prediction and mantissa addition operation are carried out on the two operands in parallel, a near-path operation result is obtained, and the number of leading zeros comprises the number of leading zeros subjected to floating-point number addition mantissa high-order offset and the number of non-normalized leading zeros; if the improved near-path condition is not met, performing far-path order shifting and mantissa addition operation on the two operands to obtain a far-path operation result; and selecting one as an operation result to carry out mantissa rounding and normalized shifting to obtain a high-precision floating point addition operation result. Through the above mode, according to the embodiment of the invention, normalized calculation with improved precision and existing near-path calculation can be combined into one, floating point additive operation resource reuse and delay elimination are realized, and the execution efficiency is improved.

The invention belongs to the technical field of storage and calculation integration, and particularly relates to a low-bit-width and high-energy-efficiency floating point storage and calculation integrated circuit based on a partial pre-alignment framework. The circuit comprises a memory array, a pre-calculation unit, an adder tree, a configurable arithmetic unit and a normalization unit, and supports mixed precision operation of FP8MACFP4 and FP8MACFP8. The method is characterized in that a partial pre-alignment strategy dominated by an activation value is adopted, the maximum index of the activation value is dynamically counted, the mantissa of the maximum index is aligned, and multiple partial pre-alignment intermediate results are pre-calculated and latched for reuse; in combination with a customized lookup table and a multiplexer, a pre-calculation result is directly selected to replace real-time multiplication and displacement; and through the reconfigurable hardware, the FP8MACFP8 high-precision operation is realized by utilizing the FP8MACFP4 unit combination. According to the method, complete online floating point multiplication and addition operation is realized, and excellent energy efficiency ratio and operation speed are obtained while high precision is kept.

The invention provides an arithmetic logic unit of a processor and a floating-point number calculation method, and belongs to the technical field of floating-point numbers. The arithmetic logic unit comprises a format conversion module and a mixed precision multiplier-adder, the format conversion module performs format conversion on a high-precision floating-point number to obtain a low-precision floating-point number, and retains an index lost in the conversion process as index auxiliary information, and when the mixed precision multiplier-adder performs multiply-add operation on the low-precision floating-point number, the mixed precision multiplier-adder performs multiply-add operation on the low-precision floating-point number. And the index auxiliary information is used for compensating the lost index, so that the precision loss of the floating-point number can be reduced when the mixed precision multiplier-adder is used.

The present application relates to the technical field of computers. Provided are a mixed-precision quantization method and apparatus, and a device. The method is applied to each quantization layer comprised in a quantization model, and comprises: acquiring a plurality of parameter files, wherein each parameter file at least comprises a preset quantization bit width and a preset number of iterations; on the basis of each parameter file, generating an initial quantization layer to obtain a plurality of initial quantization layers; on the basis of a target node comprised in each initial quantization layer, constructing an initial node link to obtain a plurality of initial node links, and on the basis of a quantization result of each initial node link, determining a quantization loss value; and on the basis of the quantization loss value corresponding to each initial node link, determining a target node link, and determining the preset quantization bit width corresponding to the target node link to be a target bit width. The method in the present application improves the automation degree and accuracy in determining the quantization bit width of a quantization model.

The invention provides a model reasoning method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: performing attention calculation on inferred tokens by adopting first numerical precision through a large language model to obtain a first attention score corresponding to each inferred token; screening a target tokens from the inferred tokens based on the first attention score; reasoning the input sequence by adopting second numerical precision through the large language model to obtain a reasoning result output by the large language model; the input sequence comprises a target token and tokens to be input corresponding to the token to be inferred; the tokens to be input are tokens pre-selected from the inferred tokens according to a preset rule; the first numerical precision is lower than the second numerical precision. According to the method, the sparse processing of the token is realized by adopting mixed precision calculation, so that the reasoning efficiency of the large language model is improved.

Techniques for performing fine-grained mixed precision quantization for an ML model are disclosed. A quantizable operation is identified. This quantizable operation is partitioned into multiple sub-operations having multiple different precision data formats. One or more kernels are generated. These kernel(s) are tasked with simultaneously executing the sub-operations. Consequently, the first sub-operation, which has the first precision data format, is executed simultaneously with the second sub-operation, which has the second, different precision data format.

The invention provides a diffusion model mixing precision quantification method and device based on dynamic sensitivity and electronic equipment, and the method comprises the steps: setting a bit width indicator and setting quantification parameters, the bit width indicator represents the combination of a weight bit width and an activation bit width in each layer, and the quantification parameters comprise a scaling factor and a zero value; based on the output difference between the minimum full-precision network and the quantization network under each preset bit width, optimizing quantization parameters under multiple bit widths; based on the scale factor in the weight quantization process and the weighted sum of the scale factor in the activation quantization process, obtaining the quantization sensitivity of any layer on each time step; under given resource constraints, the sum of sensitivities of all layers on all time steps is minimized, an adaptive bit width configuration strategy is obtained, a quantitative diffusion model with mixed precision is obtained, bit width distribution of a network layer can be dynamically adjusted according to the quantitative sensitivities of the diffusion model on denoising time steps, and the accuracy of the network layer is improved. Therefore, the method can better adapt to the sensitivity change in the denoising process.

The invention provides a quantification method for a large language model, and the method comprises the steps: for each linear layer, calculating the weight quantification sensitivity of the linear layer, and determining the high-precision bit width ratio of each linear layer according to the weight quantification sensitivity of each linear layer and a target precision mixing ratio; for a plurality of channels of each linear layer, based on the input activation of the linear layer, obtaining the channel quantization sensitivity of each channel in the linear layer; sequencing a plurality of channels in the linear layer according to the channel quantization sensitivity, clustering the sequenced channels in combination with the high-precision bit width ratio of the linear layer, and distributing quantization bit widths with different precisions for weight parameters corresponding to the channels of different clusters; and rearranging a plurality of channels of the same cluster according to distribution similarity indexes, grouping the rearranged channels, and synchronously quantizing weight parameters of each group of channels according to the allocated quantization bit width to obtain a quantized large language model.

The invention provides an iterative solver optimization method and system based on a Shenwei architecture, relates to the technical field of electronic information, and aims to solve the problems that an AztecOO suitable for an SW26010Pro architecture in a new generation of Shenwei super computer at present still adopts a double-precision algorithm, the storage and calculation cost is high, and an AztecOO solver is low in calculation efficiency and low in calculation precision. The method comprises the following steps: acquiring a solving task of a sparse linear equation set, and issuing the solving task to a main core; the main core divides a solving task and allocates a memory for the solving task; the master core calls a slave core starting function and executes calculation; and the slave core feeds back a calculation result to the master core to obtain a solution of the sparse linear equation set. According to the method, the iterative solver is optimized through a designed precision selection strategy, a matrix segmentation algorithm and a mixed precision algorithm, so that the calculation speed of the algorithm is improved and memory resources are saved under the condition that sufficient calculation precision is ensured, the whole solving process of the solver is accelerated, and more efficient development efficiency is provided for researchers in the aspect of numerical calculation.

A mixed-precision model quantization method includes loading a trained model, and quantizing the trained model with a mixed-precision setting to generate a quantized model for inference. The trained model includes a plurality of residual connections. In each residual connection, a first activation bypasses at least one operator and is added to a second activation to generate a fourth activation. The second activation is the output of the first activation after being processed by the at least one operator, The mixed-precision setting includes (a) the first activation, the second activation, and the fourth activation in at least one residual connection of the plurality of residual connections being assigned a first precision, and (b) third activations in all operators bypassed by the at least one residual connection being assigned a second precision. The third activations are generated by the bypassed operators and processed within the bypassed operators.

The invention discloses an efficient sampling adaptive data fusion method. The method comprises the following steps: S100, generating an initial low-precision sample set; s200, obtaining a high-precision sample set by an infinitesimal method; s300, mixing the low-precision and high-precision sample sets to obtain a new sample set; s400, calculating the real distance between the low-precision sample set and the nearest high-precision sample; s500, establishing an adaptive RBF neural network model, and processing the new sample set; s600, substituting the real distance into an adaptive RBF neural network model to update parameters and search an optimal function value; according to the method, a mixed precision pneumatic data fusion model of different precision data is associated, and an infinitesimal division sampling method and a self-adaptive data fusion method are provided, so that efficient initialization and high-precision prediction of an RBF variable credibility model are realized.

The invention provides a lightweight large model operation method based on end-side deployment. The lightweight large model operation method comprises the following steps: acquiring equipment operation data and storing a model file in a mixed precision format according to quantization precision supported by terminal equipment; determining a mixing precision quantification strategy and carrying out strategy analysis on the equipment operation data; a Key-Value cache file obtained through model reasoning in the historical dialogue is reserved; calculating a semantic embedding vector of the second user request, calculating a cosine similarity between the semantic embedding vector and the first V vector, and judging whether the second user request has a similar intention or not according to a calculation result; and if yes, performing incremental reasoning on the second user request by multiplexing the Key-Value cache file. By monitoring the state of the terminal equipment in real time, dynamically adjusting the quantization level, multiplexing the cache data of the semantic related requests and merging batch request reasoning, the effects of balancing the energy consumption and performance of the terminal equipment, reducing repeated calculation and improving the service time of the equipment and the user experience are achieved.

The invention belongs to the field of machine learning, discloses an expert parallelism processing method and system for a large language model based on MoE, and realizes efficient parallel processing of the MoE model by dynamically distributing expert quantization bit width and sparse mode, predicting and prefetching to-be-activated expert parameters, grouping tokens to generate task queues and dynamically configuring hardware accelerators. Firstly, an importance score is calculated based on expert historical activation frequency, weight distribution and a model structure, so that quantization precision and a sparse proportion are adaptively allocated, and resource waste and precision loss of a unified strategy are avoided; secondly, predicting an expert to be activated by using a current layer hidden state and a historical activation sequence, loading parameters to a special cache in advance, reducing high-bandwidth memory access, and relieving bandwidth peak scrambling; moreover, tokens are grouped through a token-expert mapping table, a task queue is created, and dynamic configuration of a systolic array is combined, so that the problem of expert heterogeneity after compression is solved, and efficient and parallel hybrid precision matrix operation is ensured.

The invention discloses a large model mixing precision quantification method based on dynamic tree quantification. The method comprises the following steps: step 1, dividing input feature points into outliers and non-outliers; 2, extracting an outlier from the matrix sequence of the feature points, and extracting a row where a weight corresponding to the outlier is located; or processing and quantizing the non-outlier data used for the linear operator; non-outlier data applied to a nonlinear operator is not quantized; 3, performing matrix multiplication on the outlier data and the corresponding weight of the outlier; or, performing inverse quantization on non-outlier data stored in the linear operator in a quantized manner, and performing matrix multiplication operation; directly performing matrix multiplication on non-outlier data in the nonlinear operator; and 4, combining a non-outlier matrix multiplication operation result and an outlier matrix multiplication operation result through addition, and outputting a final result. The invention further discloses application and system of the method and a storage medium.

The invention relates to the technical field of digital signal processing, and particularly discloses a mixed precision low-resource digital control oscillator implementation method and system based on an FPGA (Field Programmable Gate Array), and the method comprises the steps: dividing a 32-bit phase accumulator into a high-bit 24-bit dominant frequency control section and a low-bit 8-bit dynamic compensation section, and carrying out the phase accumulation in parallel through double carry chains, the high-order section receives high 24 bits of a frequency tuning word FTW to control a dominant frequency, and the low-order section receives low 8 bits of the FTW and overlaps an overflow remainder to dynamically compensate for a phase error. According to the implementation method of the mixed precision low-resource digital control oscillator based on the FPGA disclosed by the embodiment of the invention, through collaborative innovation of mixed precision phase accumulation, hierarchical mixed storage, high-order interpolation reconstruction and dynamic phase compensation, the core problems of a traditional NCO in resource occupation, phase continuity, waveform precision and time sequence performance are systematically solved.

The invention discloses a mixed precision error analysis method and system based on instant reduction, and the method comprises the steps: inputting a to-be-analyzed code region, constructing and initializing an error propagation matrix EPM, and aggregating any group of error sources into an error source group through a column mapping function; the method comprises the following steps: for a current instruction in operation, spreading an error of a source operand of the current instruction to an error vector of a target operand based on a forward differential error analysis method and an error source aggregation instant reduction method; calculating a result value of the current instruction under the mixed precision configuration, and calculating a truncation error of the current instruction according to the result value under the mixed precision configuration; all code instructions are traversed, and finally the error contribution of each input error source to the program output variable is obtained. According to the method, mixing precision error analysis can be carried out on a scientific calculation program lightly and efficiently, and accurate error sensitivity information is provided.

The invention provides a bit-width-adaptive mixed precision quantification method and system. The method comprises the steps that quantizers with variable bit widths are configured for all modules in a model respectively; setting a loss function and an upper bound and a lower bound of each module; according to the comparison condition of the numerical value of the loss function and the upper bound and the lower bound, the bit width of the corresponding quantizer is automatically adjusted module by module, and a mixed precision model is obtained. According to the method, a self-adaptive precision allocation technical scheme based on module characteristic difference is adopted, adaptive quantization bit widths are accurately allocated for different modules by dynamically sensing the importance, numerical distribution and quantization disturbance resistance of each sub-module in a network structure, and the adaptive quantization bit widths are accurately allocated for different modules on the premise that the overall computing resource overhead is not remarkably increased. And the compression potential of the model is fully released.

The invention belongs to the field of storage, and discloses a high-energy-efficiency mixed-precision charge domain in-storage computing architecture and a working method thereof. Comprising a single-slope analog-to-digital converter, a sparsity perception input alignment module, a multi-bit input accumulation module, a controller and an accumulation module, wherein the single-slope analog-to-digital converter consists of an index calculation array, a mantissa calculation array, a shared ramp voltage generator and a bidirectional counter. According to the invention, serial input binary coding based on capacitor voltage is suitable for floating point and integer multiply-accumulate operation with flexible bit width; according to the invention, a shared single-slope analog-to-digital converter (SS-ADC) is introduced to realize maximum index search and index difference calculation; according to the method, a sparsity perception calculation scheme is provided, low-importance input-weight pairs are filtered out through an adjustable threshold value, and invalid power consumption is reduced; according to the method, a multi-bit input accumulation method is further combined, and an ADC redundancy optimization quantization and normalization process is utilized, so that the overall energy efficiency is improved.

The invention relates to a self-adaptive mixing precision quantification method and device, equipment and a medium, and the method comprises the steps: carrying out the comprehensive sorting through the product of the cosine similarity difference value of adjacent layers and the sensitivity weight of each layer, so as to guarantee that bottleneck layers which are liable to be influenced by quantification and are crucial to the final precision can be accurately recognized; therefore, precise positioning of a protection target is realized, an iterative optimization loop is introduced, an optimal solution meeting a preset performance target can be spontaneously found finally by continuously evaluating a time-precision balance point of an overall model and automatically adjusting configuration, and a suboptimal result caused by improper primary configuration is avoided; and the input of the user is simplified into a visual final performance target, and the complicated layer sorting and selection process is automatically processed in the system, so that the use threshold of the technology is reduced.

A device for deep learning acceleration with mixed precision may include matrix-vector (MV) components that each include vector-vector (VV) components that are each configured to generate a respective VV output based on an input precision mode, an output precision mode, and an accumulation of products. The accumulation of products may be calculated by adding products based on the input precision mode. Each product may be calculated by multiplying, based on the input precision mode, a map data segment and a kernel data segment. Each MV component may include one or more components configured to concatenate VV outputs to generate a concatenated VV output. The device may include activation function components that are each configured to receive a corresponding concatenated VV output, generate an activation function output based on the corresponding concatenated VV output and the output precision mode, and output the activation function output.

Provided is a mixed-precision multiply-and-accumulation (MAC) tree structure to maximize memory bandwidth usage for computational acceleration of a generative large language model. A MAC tree-based operator may include a plurality of floating-point (FP) multipliers connected in parallel and configured to process a multiplication operation on data delivered from an external memory; a plurality of first converters configured to convert output of each of the plurality of FP multipliers from floating point to fixed point; a fixed-point (FXP) adder tree connected to the plurality of first converters and configured to process summation of multiplication results of the plurality of FP multipliers; an FXP accumulator configured to accumulate output of the FXP adder tree; and a second converter configured to convert output of the FXP accumulator from the fixed point to the floating point.