225 results about "Mixed precision" patented technology

The invention relates to the field of artificial intelligence hardware acceleration, and discloses a memristor-based multilevel compression collaborative optimization neural network deployment method and device, and a storage medium. The method comprises the following steps: carrying out progressive structured pruning on a pre-training model based on a dual-drive scoring mechanism of an L1 norm and gradient sensitivity and hardware feedback, and generating a hardware-friendly sparse weight structure; the characteristics of the memristor are simulated through a micro-nonlinear conductance modeling function, and network weight and conductance parameters are synchronously optimized to reduce errors by adopting mixed precision quantification of four bits of a convolutional layer and two bits of a full-connection layer; a conductance drift and read-write noise model is injected, and the anti-interference capability of the model is improved in combination with adaptive noise enhancement and KL divergence loss; and mapping the optimized model to a memristor memory architecture to complete weight coding and reasoning. Through collaborative optimization of pruning, quantification and distillation, the problems of insufficient storage density, non-ideal characteristic interference and algorithm and hardware mismatch in memristor deployment are solved.

The invention relates to the technical field of network security, in particular to an intrusion detection system and method based on machine learning, and the system comprises a dynamic feature selection module which dynamically generates a feature mask through a reinforcement learning strategy network, and selects an optimal feature subset in real time according to an action return function # imgabs0 #; the lightweight detection module adopts a depth separable convolution structure, implements mixed precision quantization and structured pruning, and takes dynamic ReLU as an activation function; the incremental learning engine is used for restraining the weight through dynamic regularization based on a Fisher information matrix on the basis of a local cache data online fine tuning model; and the edge-cloud collaboration module is used for performing homomorphic encryption and differential privacy processing on model parameter differences, and updating a global model through a robust federated aggregation algorithm. Based on lightweight edge deployment and real-time incremental learning, the calculation overhead is reduced, and the robustness is improved.

The invention discloses a self-adaptive precision adjustment method in the NPU field. In the invention, through a mixed precision conflict resolution mechanism, global optimal balance of precision and performance in a deep learning reasoning task is realized. According to a traditional method, when the calculation precision is adjusted, independent decision making is usually carried out only based on local operator characteristics, and therefore the overall precision of an operator group dependent on series connection is collapsed due to error accumulation. According to the method, an error propagation chain is constructed through a quantitative sensing model, influence paths of different precision combinations on task output are dynamically predicted, and differential precision distribution is implemented on conflict operator groups by adopting a critical path priority algorithm. In a target detection task, a key path such as a classification head forcibly retains high-precision calculation, and a background filtering layer is reduced to a low-precision mode, so that the mechanism fundamentally solves the global sub-optimal problem caused by local optimization in a traditional method, and ensures the calculation reliability of a high-value operator.

The invention discloses a heterogeneous computing power cooperative scheduling system and method for mixed precision training, and belongs to the technical field of artificial intelligence computing. The system comprises a computational graph analysis and operator portrait module which is used for analyzing and dividing a model computational graph and extracting operator features; the heterogeneous hardware capability sensing and matching module is used for managing performance files and real-time states of heterogeneous hardware in the cluster and matching optimal execution hardware for each calculation partition; and the data flow coordination and pipeline parallel controller is used for generating a global execution plan, managing cross-device data dependence and communication and calculating overlapping optimization execution efficiency through communication. According to the method, the problem of low scheduling efficiency of mixed precision training in a heterogeneous environment is solved, automatic and accurate mapping from a calculation task to heterogeneous hardware is realized, the training speed is remarkably improved, the training cost is reduced, and the overall resource utilization rate of a cluster is improved.

The invention relates to the technical field of monolithic integration, in particular to a monolithic three-dimensional integration implementation method and device for a hybrid precision in-memory computing architecture, and the method comprises the steps: obtaining a silicon-based CMOS technology and a subsequent integration technology in a monolithic three-dimensional integration technology; manufacturing a control logic circuit on the computing architecture in the mixed precision memory by adopting a silicon-based CMOS (Complementary Metal Oxide Semiconductor) process; a plurality of functional layers are vertically stacked on a logic circuit by adopting a back-end integration process, communication among the functional layers is realized through interlayer dielectric via holes in the vertical direction, an analog in-memory array, a digital in-memory array and a cache are arranged on the functional layers, the analog in-memory array realizes matrix and vector multiplication with fixed weight, and the digital in-memory array realizes matrix and vector multiplication with fixed weight. And the array in the digital memory realizes matrix multiplication of refresh weight. Therefore, the problems that an in-memory computing architecture usually needs to occupy a relatively large part of chip area, the computing efficiency is relatively low and the like in related technologies are solved.

The invention belongs to the technical field of storage and calculation integration, and particularly relates to a low-bit-width and high-energy-efficiency floating point storage and calculation integrated circuit based on a partial pre-alignment framework. The circuit comprises a memory array, a pre-calculation unit, an adder tree, a configurable arithmetic unit and a normalization unit, and supports mixed precision operation of FP8MACFP4 and FP8MACFP8. The method is characterized in that a partial pre-alignment strategy dominated by an activation value is adopted, the maximum index of the activation value is dynamically counted, the mantissa of the maximum index is aligned, and multiple partial pre-alignment intermediate results are pre-calculated and latched for reuse; in combination with a customized lookup table and a multiplexer, a pre-calculation result is directly selected to replace real-time multiplication and displacement; and through the reconfigurable hardware, the FP8MACFP8 high-precision operation is realized by utilizing the FP8MACFP4 unit combination. According to the method, complete online floating point multiplication and addition operation is realized, and excellent energy efficiency ratio and operation speed are obtained while high precision is kept.

The invention provides an arithmetic logic unit of a processor and a floating-point number calculation method, and belongs to the technical field of floating-point numbers. The arithmetic logic unit comprises a format conversion module and a mixed precision multiplier-adder, the format conversion module performs format conversion on a high-precision floating-point number to obtain a low-precision floating-point number, and retains an index lost in the conversion process as index auxiliary information, and when the mixed precision multiplier-adder performs multiply-add operation on the low-precision floating-point number, the mixed precision multiplier-adder performs multiply-add operation on the low-precision floating-point number. And the index auxiliary information is used for compensating the lost index, so that the precision loss of the floating-point number can be reduced when the mixed precision multiplier-adder is used.

The present application relates to the technical field of computers. Provided are a mixed-precision quantization method and apparatus, and a device. The method is applied to each quantization layer comprised in a quantization model, and comprises: acquiring a plurality of parameter files, wherein each parameter file at least comprises a preset quantization bit width and a preset number of iterations; on the basis of each parameter file, generating an initial quantization layer to obtain a plurality of initial quantization layers; on the basis of a target node comprised in each initial quantization layer, constructing an initial node link to obtain a plurality of initial node links, and on the basis of a quantization result of each initial node link, determining a quantization loss value; and on the basis of the quantization loss value corresponding to each initial node link, determining a target node link, and determining the preset quantization bit width corresponding to the target node link to be a target bit width. The method in the present application improves the automation degree and accuracy in determining the quantization bit width of a quantization model.

The invention provides a model reasoning method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: performing attention calculation on inferred tokens by adopting first numerical precision through a large language model to obtain a first attention score corresponding to each inferred token; screening a target tokens from the inferred tokens based on the first attention score; reasoning the input sequence by adopting second numerical precision through the large language model to obtain a reasoning result output by the large language model; the input sequence comprises a target token and tokens to be input corresponding to the token to be inferred; the tokens to be input are tokens pre-selected from the inferred tokens according to a preset rule; the first numerical precision is lower than the second numerical precision. According to the method, the sparse processing of the token is realized by adopting mixed precision calculation, so that the reasoning efficiency of the large language model is improved.

Techniques for performing fine-grained mixed precision quantization for an ML model are disclosed. A quantizable operation is identified. This quantizable operation is partitioned into multiple sub-operations having multiple different precision data formats. One or more kernels are generated. These kernel(s) are tasked with simultaneously executing the sub-operations. Consequently, the first sub-operation, which has the first precision data format, is executed simultaneously with the second sub-operation, which has the second, different precision data format.

The invention provides a diffusion model mixing precision quantification method and device based on dynamic sensitivity and electronic equipment, and the method comprises the steps: setting a bit width indicator and setting quantification parameters, the bit width indicator represents the combination of a weight bit width and an activation bit width in each layer, and the quantification parameters comprise a scaling factor and a zero value; based on the output difference between the minimum full-precision network and the quantization network under each preset bit width, optimizing quantization parameters under multiple bit widths; based on the scale factor in the weight quantization process and the weighted sum of the scale factor in the activation quantization process, obtaining the quantization sensitivity of any layer on each time step; under given resource constraints, the sum of sensitivities of all layers on all time steps is minimized, an adaptive bit width configuration strategy is obtained, a quantitative diffusion model with mixed precision is obtained, bit width distribution of a network layer can be dynamically adjusted according to the quantitative sensitivities of the diffusion model on denoising time steps, and the accuracy of the network layer is improved. Therefore, the method can better adapt to the sensitivity change in the denoising process.

The invention provides a quantification method for a large language model, and the method comprises the steps: for each linear layer, calculating the weight quantification sensitivity of the linear layer, and determining the high-precision bit width ratio of each linear layer according to the weight quantification sensitivity of each linear layer and a target precision mixing ratio; for a plurality of channels of each linear layer, based on the input activation of the linear layer, obtaining the channel quantization sensitivity of each channel in the linear layer; sequencing a plurality of channels in the linear layer according to the channel quantization sensitivity, clustering the sequenced channels in combination with the high-precision bit width ratio of the linear layer, and distributing quantization bit widths with different precisions for weight parameters corresponding to the channels of different clusters; and rearranging a plurality of channels of the same cluster according to distribution similarity indexes, grouping the rearranged channels, and synchronously quantizing weight parameters of each group of channels according to the allocated quantization bit width to obtain a quantized large language model.

The invention provides an iterative solver optimization method and system based on a Shenwei architecture, relates to the technical field of electronic information, and aims to solve the problems that an AztecOO suitable for an SW26010Pro architecture in a new generation of Shenwei super computer at present still adopts a double-precision algorithm, the storage and calculation cost is high, and an AztecOO solver is low in calculation efficiency and low in calculation precision. The method comprises the following steps: acquiring a solving task of a sparse linear equation set, and issuing the solving task to a main core; the main core divides a solving task and allocates a memory for the solving task; the master core calls a slave core starting function and executes calculation; and the slave core feeds back a calculation result to the master core to obtain a solution of the sparse linear equation set. According to the method, the iterative solver is optimized through a designed precision selection strategy, a matrix segmentation algorithm and a mixed precision algorithm, so that the calculation speed of the algorithm is improved and memory resources are saved under the condition that sufficient calculation precision is ensured, the whole solving process of the solver is accelerated, and more efficient development efficiency is provided for researchers in the aspect of numerical calculation.

A mixed-precision model quantization method includes loading a trained model, and quantizing the trained model with a mixed-precision setting to generate a quantized model for inference. The trained model includes a plurality of residual connections. In each residual connection, a first activation bypasses at least one operator and is added to a second activation to generate a fourth activation. The second activation is the output of the first activation after being processed by the at least one operator, The mixed-precision setting includes (a) the first activation, the second activation, and the fourth activation in at least one residual connection of the plurality of residual connections being assigned a first precision, and (b) third activations in all operators bypassed by the at least one residual connection being assigned a second precision. The third activations are generated by the bypassed operators and processed within the bypassed operators.

The invention discloses an efficient sampling adaptive data fusion method. The method comprises the following steps: S100, generating an initial low-precision sample set; s200, obtaining a high-precision sample set by an infinitesimal method; s300, mixing the low-precision and high-precision sample sets to obtain a new sample set; s400, calculating the real distance between the low-precision sample set and the nearest high-precision sample; s500, establishing an adaptive RBF neural network model, and processing the new sample set; s600, substituting the real distance into an adaptive RBF neural network model to update parameters and search an optimal function value; according to the method, a mixed precision pneumatic data fusion model of different precision data is associated, and an infinitesimal division sampling method and a self-adaptive data fusion method are provided, so that efficient initialization and high-precision prediction of an RBF variable credibility model are realized.

The invention provides a lightweight large model operation method based on end-side deployment. The lightweight large model operation method comprises the following steps: acquiring equipment operation data and storing a model file in a mixed precision format according to quantization precision supported by terminal equipment; determining a mixing precision quantification strategy and carrying out strategy analysis on the equipment operation data; a Key-Value cache file obtained through model reasoning in the historical dialogue is reserved; calculating a semantic embedding vector of the second user request, calculating a cosine similarity between the semantic embedding vector and the first V vector, and judging whether the second user request has a similar intention or not according to a calculation result; and if yes, performing incremental reasoning on the second user request by multiplexing the Key-Value cache file. By monitoring the state of the terminal equipment in real time, dynamically adjusting the quantization level, multiplexing the cache data of the semantic related requests and merging batch request reasoning, the effects of balancing the energy consumption and performance of the terminal equipment, reducing repeated calculation and improving the service time of the equipment and the user experience are achieved.

The invention belongs to the field of machine learning, discloses an expert parallelism processing method and system for a large language model based on MoE, and realizes efficient parallel processing of the MoE model by dynamically distributing expert quantization bit width and sparse mode, predicting and prefetching to-be-activated expert parameters, grouping tokens to generate task queues and dynamically configuring hardware accelerators. Firstly, an importance score is calculated based on expert historical activation frequency, weight distribution and a model structure, so that quantization precision and a sparse proportion are adaptively allocated, and resource waste and precision loss of a unified strategy are avoided; secondly, predicting an expert to be activated by using a current layer hidden state and a historical activation sequence, loading parameters to a special cache in advance, reducing high-bandwidth memory access, and relieving bandwidth peak scrambling; moreover, tokens are grouped through a token-expert mapping table, a task queue is created, and dynamic configuration of a systolic array is combined, so that the problem of expert heterogeneity after compression is solved, and efficient and parallel hybrid precision matrix operation is ensured.

The invention discloses a large model mixing precision quantification method based on dynamic tree quantification. The method comprises the following steps: step 1, dividing input feature points into outliers and non-outliers; 2, extracting an outlier from the matrix sequence of the feature points, and extracting a row where a weight corresponding to the outlier is located; or processing and quantizing the non-outlier data used for the linear operator; non-outlier data applied to a nonlinear operator is not quantized; 3, performing matrix multiplication on the outlier data and the corresponding weight of the outlier; or, performing inverse quantization on non-outlier data stored in the linear operator in a quantized manner, and performing matrix multiplication operation; directly performing matrix multiplication on non-outlier data in the nonlinear operator; and 4, combining a non-outlier matrix multiplication operation result and an outlier matrix multiplication operation result through addition, and outputting a final result. The invention further discloses application and system of the method and a storage medium.

The invention relates to the technical field of digital signal processing, and particularly discloses a mixed precision low-resource digital control oscillator implementation method and system based on an FPGA (Field Programmable Gate Array), and the method comprises the steps: dividing a 32-bit phase accumulator into a high-bit 24-bit dominant frequency control section and a low-bit 8-bit dynamic compensation section, and carrying out the phase accumulation in parallel through double carry chains, the high-order section receives high 24 bits of a frequency tuning word FTW to control a dominant frequency, and the low-order section receives low 8 bits of the FTW and overlaps an overflow remainder to dynamically compensate for a phase error. According to the implementation method of the mixed precision low-resource digital control oscillator based on the FPGA disclosed by the embodiment of the invention, through collaborative innovation of mixed precision phase accumulation, hierarchical mixed storage, high-order interpolation reconstruction and dynamic phase compensation, the core problems of a traditional NCO in resource occupation, phase continuity, waveform precision and time sequence performance are systematically solved.

The invention discloses a mixed precision error analysis method and system based on instant reduction, and the method comprises the steps: inputting a to-be-analyzed code region, constructing and initializing an error propagation matrix EPM, and aggregating any group of error sources into an error source group through a column mapping function; the method comprises the following steps: for a current instruction in operation, spreading an error of a source operand of the current instruction to an error vector of a target operand based on a forward differential error analysis method and an error source aggregation instant reduction method; calculating a result value of the current instruction under the mixed precision configuration, and calculating a truncation error of the current instruction according to the result value under the mixed precision configuration; all code instructions are traversed, and finally the error contribution of each input error source to the program output variable is obtained. According to the method, mixing precision error analysis can be carried out on a scientific calculation program lightly and efficiently, and accurate error sensitivity information is provided.

The invention provides a bit-width-adaptive mixed precision quantification method and system. The method comprises the steps that quantizers with variable bit widths are configured for all modules in a model respectively; setting a loss function and an upper bound and a lower bound of each module; according to the comparison condition of the numerical value of the loss function and the upper bound and the lower bound, the bit width of the corresponding quantizer is automatically adjusted module by module, and a mixed precision model is obtained. According to the method, a self-adaptive precision allocation technical scheme based on module characteristic difference is adopted, adaptive quantization bit widths are accurately allocated for different modules by dynamically sensing the importance, numerical distribution and quantization disturbance resistance of each sub-module in a network structure, and the adaptive quantization bit widths are accurately allocated for different modules on the premise that the overall computing resource overhead is not remarkably increased. And the compression potential of the model is fully released.

The invention belongs to the field of storage, and discloses a high-energy-efficiency mixed-precision charge domain in-storage computing architecture and a working method thereof. Comprising a single-slope analog-to-digital converter, a sparsity perception input alignment module, a multi-bit input accumulation module, a controller and an accumulation module, wherein the single-slope analog-to-digital converter consists of an index calculation array, a mantissa calculation array, a shared ramp voltage generator and a bidirectional counter. According to the invention, serial input binary coding based on capacitor voltage is suitable for floating point and integer multiply-accumulate operation with flexible bit width; according to the invention, a shared single-slope analog-to-digital converter (SS-ADC) is introduced to realize maximum index search and index difference calculation; according to the method, a sparsity perception calculation scheme is provided, low-importance input-weight pairs are filtered out through an adjustable threshold value, and invalid power consumption is reduced; according to the method, a multi-bit input accumulation method is further combined, and an ADC redundancy optimization quantization and normalization process is utilized, so that the overall energy efficiency is improved.

The invention relates to a self-adaptive mixing precision quantification method and device, equipment and a medium, and the method comprises the steps: carrying out the comprehensive sorting through the product of the cosine similarity difference value of adjacent layers and the sensitivity weight of each layer, so as to guarantee that bottleneck layers which are liable to be influenced by quantification and are crucial to the final precision can be accurately recognized; therefore, precise positioning of a protection target is realized, an iterative optimization loop is introduced, an optimal solution meeting a preset performance target can be spontaneously found finally by continuously evaluating a time-precision balance point of an overall model and automatically adjusting configuration, and a suboptimal result caused by improper primary configuration is avoided; and the input of the user is simplified into a visual final performance target, and the complicated layer sorting and selection process is automatically processed in the system, so that the use threshold of the technology is reduced.

A device for deep learning acceleration with mixed precision may include matrix-vector (MV) components that each include vector-vector (VV) components that are each configured to generate a respective VV output based on an input precision mode, an output precision mode, and an accumulation of products. The accumulation of products may be calculated by adding products based on the input precision mode. Each product may be calculated by multiplying, based on the input precision mode, a map data segment and a kernel data segment. Each MV component may include one or more components configured to concatenate VV outputs to generate a concatenated VV output. The device may include activation function components that are each configured to receive a corresponding concatenated VV output, generate an activation function output based on the corresponding concatenated VV output and the output precision mode, and output the activation function output.

The invention discloses a mixed precision parallel compression method for optimizing large model key value cache, which combines the advantages of mixed precision key value cache compression with an advanced system optimization technology. Based on the characteristic that key value pairs needing high-precision retention in mixed precision compression are the same as key value pairs used for attention calculation in a prefetching strategy, low-precision KV caches are stored in a GPU memory, and meanwhile predicted high-precision important KV caches are dynamically prefetched from a CPU memory according to needs. The technical problems that an existing multi-head attention mechanism-based method is incompatible with an existing pre-training model and cannot be directly applied to a closed source or a fine-tuned large model, and generalization of the method is reduced can be solved, and the situation that an existing pruning-based method is prone to deleting unimportant marks in the current stage can be solved. And contextual information loss is caused.

The invention discloses a data layout optimization method and device applied to NPU code compilation and a medium. The method comprises the steps that an intermediate representation IR input by an upper layer is split into an operation type OP and an operand, and the operand input by the upper layer is divided into logic data and a logic mask; converting the logic data of the upper layer into data vector representation of the abstraction layer, and converting the logic mask of the upper layer into mask vector representation of the abstraction layer; obtaining a corresponding bottom hardware instruction capability according to an operation type OP input by an upper layer, and performing legalization operation on the operation type OP; and performing legalization processing on the converted abstract data vector type representation and mask vector representation according to the target hardware capability, performing instruction mapping on the processed abstract instruction according to the underlying hardware pair operation type OP, and generating a target LLVM instruction compatible with the underlying hardware. Register layout abstraction and conversion can be carried out on the NPU code under mixed precision input, and the utilization efficiency of the NPU bottom layer register is improved.

The invention discloses a large model fine tuning optimization method based on multi-strategy fusion, which comprises the following steps: designing a dynamic parameter selection mechanism, adaptively determining a parameter subset needing fine tuning according to a task demand and a model structure, and reducing unnecessary parameter updating calculation; constructing a dynamic low-rank decomposition framework, dynamically adjusting the rank of a low-rank matrix according to a model training state and data characteristics, and keeping key information while compressing a parameter scale; a self-adaptive task sensing mechanism is introduced, a fine adjustment strategy is automatically adjusted according to different task characteristics, and the adaptability of the model to various tasks is improved; and a mixed precision training method is adopted, so that the calculation complexity and the memory occupation are reduced on the premise of ensuring the model precision. According to the method, a parameter efficient fine tuning technology and a dynamic low-rank decomposition strategy are innovatively combined, and an adaptive task perception mechanism and a mixed precision training technology are introduced, so that the operand and resource requirements of model training are effectively reduced, and the fine tuning efficiency and the model performance are improved.

The invention discloses an efficient INT6 quantification method, device and equipment for a large language model, and the method comprises the steps: carrying out the mixing precision quantification of the large language model, and obtaining a quantized large language model; performing bit-level data packaging on the weight and the activation value in the quantized large language model to obtain bit-level data; loading the bit-level data to a register of the GPU, and carrying out matrix product accumulation operation and weighted summation by utilizing BTC to obtain output data; and storing the output data back to a global memory of the GPU so as to complete the quantitative reasoning process of the large language model. According to the method, mixed precision quantification is carried out on a large language model by utilizing different precisions, the reasoning speed is improved through a scheduling strategy of the GPU while relatively high quantification precision is achieved, and the reasoning potential of the GPU is fully mined, so that all potential of 6-bit quantification is released.

The invention provides a multi-modal model lightweight deployment method based on mixed precision quantification, and the method comprises the steps: obtaining a trained quantitative multi-modal large model through a mixed precision quantification method, and deploying the model on an electronic device; the method comprises the following steps: acquiring a running environment which is established for a trained quantitative multi-modal large model and can execute the model on a general CPU (Central Processing Unit), the running environment being established through a C standard library, and the C standard library comprising a machine learning tensor library GGML compiled by a C language; a machine learning tensor library GGML is used, and a deployed trained quantitative multi-modal large model is used for reasoning on electronic equipment, so that the problem of high development cost of deploying the multi-modal large model on electronic equipment such as an edge end and a mobile terminal is solved.

The invention provides a computing device and method based on RISC-V extension instructions, the computing device supports approximate computation of mixed precision according to approximate computation instructions in an extended approximate computation instruction set, and the computing device comprises an out-of-order scheduling and register reading module used for executing instruction dependency analysis and operand preloading, scheduling the non-approximate calculation instruction and the approximate calculation instruction to different transmitting queues respectively; the first instruction transmitting queue is used for temporarily storing a to-be-transmitted non-approximate calculation instruction; the second instruction transmitting queue is used for temporarily storing approximate calculation instructions to be transmitted; the precise calculation module is used for completing precise calculation related tasks according to the instruction from the first instruction transmitting queue; and the approximate calculation module is used for completing approximate calculation related tasks according to the instructions from the second instruction transmitting queue, and supports approximate calculation of various precisions.