51 results about "Quantification methods" patented technology

The invention provides a large language model quantification method, related equipment and a computer program product, and the method comprises the steps: carrying out the reasoning of a to-be-quantized large language model through calibration data, obtaining the activation of the large language model, and carrying out the statistics of the activation distribution of a target layer according to channels; calculating a smoothing factor of each channel according to the activation distribution; compensating the weight of the target layer channel by channel according to the smoothing factor to obtain a compensated weight; performing 4-bit quantization on the compensated weight; in the model reasoning process, activation of a target layer is smoothed, and 16-bit quantization is carried out on the smoothed activation. According to the method, a W4A16 quantification scheme is adopted, compared with W8A8, the weight storage amount is compressed by half, meanwhile, the precision loss is controlled to be smaller than 3%, and deployment of a large language model on edge equipment is facilitated.

The invention relates to a self-adaptive mixing precision quantification method and device, equipment and a medium, and the method comprises the steps: carrying out the comprehensive sorting through the product of the cosine similarity difference value of adjacent layers and the sensitivity weight of each layer, so as to guarantee that bottleneck layers which are liable to be influenced by quantification and are crucial to the final precision can be accurately recognized; therefore, precise positioning of a protection target is realized, an iterative optimization loop is introduced, an optimal solution meeting a preset performance target can be spontaneously found finally by continuously evaluating a time-precision balance point of an overall model and automatically adjusting configuration, and a suboptimal result caused by improper primary configuration is avoided; and the input of the user is simplified into a visual final performance target, and the complicated layer sorting and selection process is automatically processed in the system, so that the use threshold of the technology is reduced.

Method and apparatus for quantization of base station driven multi-vendor sequential training. The apparatus generates an encoder output by inputting an input CSI to a reference encoder. The apparatus quantizes the encoder output by inputting the encoder output to a quantizer to generate a quantizer output. The apparatus trains a decoder of the network entity based at least on the quantizer output to generate a training dataset. The apparatus outputs a training dataset indication comprising the training dataset to a UE, the training dataset indication comprising at least the input CSI. The apparatus communicates with the UE using the trained decoder.

The invention relates to a processing device and a quantization method for quantization and inverse quantization of numeric data. In one or more aspects, a processing apparatus for numeric data quantization includes processing circuitry to determine a maximum exponent from a set of exponents of a set of digit representations of a set of digits, obtain a set of scaled exponents based on the maximum exponent, and quantize the scaled exponents based on the set of scaled exponents. And one of (i) obtaining a set of quantized significant numbers based on the set of digit representations and a set of mantissas of the set of scaled exponents, or (ii) obtaining a set of quantized mantissas based on the set of mantissas. The processing circuitry is configured to output a set of quantized digit representations of the set of digits based on the set of quantized significant numbers or based on the set of quantized mantissas and the set of scaled exponents; and outputting an offset exponential scaling factor based on the maximum exponent.

Embodiments of the present application disclose a model quantization method and device, computer equipment and a storage medium, belonging to the technical field of computers. The method comprises: obtaining a first model, the first model comprising a plurality of first operators, a splicing layer and a second operator; determining an input range of each operator in the plurality of first operators and the second operator; updating a quantization parameter of each operator based on the input range of each operator and a target output range, so that the quantization parameter of each operator converges to a quantization parameter corresponding to the target output range; and performing quantization processing on network parameters in the splicing layer based on the quantization parameter corresponding to the target output range, so as to ensure that the plurality of input ranges of the splicing layer and the range of the network parameters are the same, thereby completing quantization of the splicing layer.

The application provides a method and device for quantifying elastic KV cache through computing power of an intelligent computing cloud platform, and relates to the technical fields of intelligent computing centers, intelligent computing centers, computing power infrastructure and intelligent computing cloud technology.The method comprises the following steps: S1, dividing historical tokens into multiple cache blocks, quantifying KV data and writing the data into corresponding cache blocks, and selecting multiple candidate anchor points; S2, calculating block-level summary data; S3, in response to a new target token, scoring the cache blocks to generate cache block scores and screening out candidate cache blocks; S4, calculating uncertainty index data and determining a target cache block with to-be-restored precision according to the uncertainty index data; and S5, locating an upstream target anchor point and locally playing back the historical tokens based on the upstream target anchor point to generate target high-precision KV data of the target cache block.The application can greatly improve the quantification effect of KV cache of the intelligent computing cloud platform.

The application belongs to the technical field of large language model quantization, and particularly relates to a large language model quantization method based on orthogonal characteristics and an accelerator architecture. The quantization method divides the activation tensor of the large language model into multiple column blocks, and allocates an FP4 quantization format to the entire activation tensor with the column block as the granularity. The concept of the column block is defined as follows: the matrix of the activation tensor is divided into multiple segments with the same number of elements, wherein each element in the segment is arranged continuously in the same row in the first dimension of the matrix, and arranged in multiple continuous columns in the second dimension; the column block includes multiple columns in the second dimension, and the number of columns in each column block is consistent with the number of elements in the segment. The application overcomes the defects existing in the existing large language model grouping quantization technology, and solves the contradiction between the precision of the large language model and the hardware efficiency.

The present application relates to the technical field of computer software development and version control, and particularly relates to a software technical debt identification and quantification method and system based on version control logs. The method obtains version control logs containing commit records, commit parent node relationships, branch reference records, merge commit records, merge conflict records, code difference records, file renaming records and version tag records, constructs a merge propagation graph anchored by version tags and a code entity identity chain; generates a merge influence unit for a merge commit node within a version tag interval, identifies source branch introduction segments, conflict resolution segments, target branch coverage segments and pre-tag re-introduction segments on the same code entity identity chain; when the above segments are connected in the order of merge propagation edges and commit topologies, a version propagation closed loop technical debt event is generated and a technical debt quantification record is formed.

This application provides a network quantization method and apparatus, and a related device. A first device receives a first model and a first KL divergence in tth aggregation from a second device. The first device determines a second model in (t + 1)th update and quantization based on the first model and the first KL divergence in the tth aggregation, and sends the second model in the (t + 1)th update and quantization to the second device. It can be learned that the first device may update a quantized local model (namely, the second model) based on global aggregation information / a model (namely, the first model and the first KL divergence). Because global knowledge is integrated, the first device can implement faster convergence when updating the quantized local model, to improve training efficiency. In addition, even if quantization causes a specific accuracy loss, this method still implements good learning performance, can meet an accuracy requirement for a local model, and helps reduce consumption of transmission bandwidth.

A model quantization method, apparatus, device, and storage medium are disclosed. The method involves inputting acquired historical lexical units into a large language model to be quantized, obtaining a set of candidate lexical units and their probability distributions; determining a first contribution value for each candidate lexical unit based on the feature space and depth-sensing weights of each linear layer in the large language model; determining a target lexical unit from the set of candidate lexical units based on the first contribution value; iteratively executing the operation of inputting the acquired historical lexical units into the large language model to be quantized until the iteration stop condition is met, generating calibration samples; using the calibration samples, quantizing the large language model to be quantized to obtain the quantized target model, which is then deployed to a hardware device. The quantized target model can be called by the hardware device to perform corresponding tasks, realizing the generation of calibration data samples from the geometric perspective of the latent manifold of the large language model to be quantized.

In an aspect of the disclosure, a method, a computer-readable medium, and an apparatus are provided. The method may be performed by a UE. In certain configurations, the UE collects data samples for an artificial intelligence (AI) / machine learning (ML) model at the UE and a base station. The AI / ML model is trained at the UE or at the base station in a training stage. The UE performs, according to a quantization method, quantization of the data samples to obtain quantized data samples. The UE transmits the quantized data samples to the base station. The UE executes an encoder or a decoder of the trained AI / ML model in an inference stage. The quantization method may be a latent quantization method for a latent space, and the data samples are latent vectors of Channel State Information (CSI) samples measured by the UE.

The present disclosure discloses a quantization method of a neural network, a computing device and a computer readable storage medium. The computing device can be included in a combined processing device, which can further include an interface device and other processing devices. The computing device interacts with the other processing devices to jointly complete a user-specified computing operation. The combined processing device can further include a storage device connected to the computing device and the other processing devices respectively for storing data of the computing device and the other processing devices. The scheme of the present disclosure can greatly reduce the operation time required for computing quantization parameters while maintaining the required network accuracy.

The invention provides a quantization method for training quantized feature map introduced noise. The method comprises the following steps: S1, inserting a quantization node; s2, counting a quantization range; s3, quantization of the feature map is carried out; and S4, introducing a noise mechanism to obtain a final result: according to the method, the query feature is supplemented, and the noise mechanism is introduced. Compared with an existing general method, the method is higher in precision. And the training quantification model precision can be further improved. Specifically, the model identification is more accurate, and the error rate is lower.

The invention relates to the technical field of intelligent teaching, and discloses a cognitive level quantification method for a large-model dynamic adaptive teaching strategy, and the method comprises the steps: collecting the multi-modal data of a learner in real time; preprocessing the multi-modal data to obtain preprocessed multi-modal data; inputting the preprocessed multi-modal data into a pre-constructed large model to obtain a target teaching strategy corresponding to the learner; the large model is used for calculating an index score corresponding to a preset dimension based on the preprocessed multi-modal data; determining a weight coefficient corresponding to the preset dimension based on the index score corresponding to the preset dimension; and determining a cognitive conflict level corresponding to the learner according to the index score corresponding to the preset dimension and the weight coefficient, and determining a target teaching strategy according to the cognitive conflict level corresponding to the learner. The teaching strategy can be immediately adjusted according to the cognitive conflict level judged in real time, the interaction behavior is responded in time, and the problems that behavior prediction lags behind and the strategy cannot be adjusted in real time are effectively solved.

A quantization method and apparatus for a text feature extraction model, and a device and a storage medium. The method includes: in a training process of a text feature extraction model, determining, according to a target quantization parameter, a quantization interval corresponding to the target quantization parameter, where the quantization interval includes a part of floating-point values of the target quantization parameter; constructing a mapping relationship between floating-point values and fixed-point values of the target quantization parameter based on the quantization interval, where a floating-point value smaller than a left end point of the quantization interval—is mapped to a quantized minimum fixed-point value, and a floating-point values larger than a right end point of the quantization interval is mapped to a quantized maximum fixed-point value; and performing a quantization operation on the target quantization parameter based on the mapping relationship.

This disclosure provides a mixed-precision quantization method, apparatus, device, medium, and program product, relating to the field of deep learning technology. The method is applied to a large language model, where the feedforward network of the large language model includes a multiplicative structure configured with a first weight matrix and a second weight matrix. The method includes: determining the statistical characteristics of the multiplicative structure on each output channel of the feedforward network based on the first and second weight matrices; determining the numerical amplification risk index of the multiplicative structure on each output channel based on the statistical characteristics; and configuring a corresponding computational precision for each output channel according to the numerical amplification risk index; wherein output channels with different numerical amplification risk indices are configured with different computational precisions. This method eliminates the need for real-time scanning of input data, significantly reducing runtime overhead.

The invention discloses a multi-dimensional quantitative literature value evaluation method and device, and relates to the related field of literature metrology, and the method comprises the steps: carrying out the feature extraction of a preset dimension on a target literature through a large model, and obtaining multi-dimensional literature information; based on the belonging field, respectively constructing a reference library of each dimension according to preset dimensions; a two-way confrontation verification mechanism is introduced to dynamically check the mapping quantization relation between feature extraction and the reference library; after dynamic verification is carried out, dimension mapping quantification is carried out on the document information extracted from each dimension by utilizing the reference library of each dimension; and performing knowledge structure entropy change operation according to the quantitative result of each dimension, quantifying a knowledge structure entropy change value, and performing literature value fusion evaluation according to the quantitative result of each dimension and the knowledge structure entropy change value. The technical problems that existing literature value evaluation is high in subjectivity and incomplete and accurate in evaluation are solved, and the technical effect of objectively, comprehensively and accurately evaluating the literature value is achieved.

The invention relates to the technical field of communication security, in particular to a quantization method suitable for atmospheric turbulence channel physical layer key extraction, which comprises the following steps: two communication parties firstly obtain time domain direct observation sequences through atmospheric turbulence channel detection, and respectively execute fractional order Fourier transform operation on the two groups of observation sequences; mapping to different transform domains to obtain complex signals; then traversing the candidate fractional order by constructing an optimization function, determining an optimal transformation order, extracting a real part of a transformation domain signal under the order to construct a real number characteristic quantity sum, and performing quantization processing on the real number characteristic quantity sum by adopting a double-threshold quantization algorithm to generate an initial key bit sequence sum; according to the technical scheme, the shared initial key with consistency and randomness can be generated by the two communication ends based on the optimal transform domain characteristics and the quantization rule, so that the high efficiency and robustness of physical layer key extraction under the atmospheric turbulence channel are achieved, and a light-weight and high-safety key generation scheme is provided for a wireless communication system.

This invention belongs to the field of artificial intelligence and model compression technology, and discloses an additive quantization method for large language models. Without changing the additive quantization encoding format and codebook structure, a sampling strategy that jointly considers sub-vector energy density and spatial coverage is introduced in the residual K-means initialization stage. This allows high-energy-density regions to obtain more reasonable codeword coverage, significantly improving the codebook initialization quality. In the block-by-block end-to-end fine-tuning stage, a direction alignment loss based on negative log-cosine similarity is introduced in addition to the mean square error. This explicitly constrains the directional consistency between the quantized block output and the full-precision block output, making the direction constraint and amplitude constraint complementary. This method requires no quantization-aware training or large-scale fine-tuning, does not introduce additional trainable parameters, has low computational and storage overhead, is simple to implement in engineering, maintains near-full-precision inference performance under extremely low 2-bit quantization conditions, and significantly reduces memory usage and energy consumption during model deployment.

The invention provides a quantification method and system of a hybrid expert large language model, a medium and equipment, and the method comprises the steps: carrying out the forward propagation of a to-be-quantified hybrid expert large language model, and determining an activation value of each layer; according to the activation value of each layer, Fisher information of each expert weight is determined; according to the Fisher information of the weight of each expert, performing linear weighted fusion on the weight of each expert, and determining a shared basic weight; taking a difference value between the weight of each expert and the shared basic weight as a residual weight of each expert; and carrying out weight binarization processing on the shared basic weight by adopting a preset alternative refining binarization strategy, carrying out weight binarization processing on the residual weight of each expert by adopting a mode of combining a symbol matrix and a scaling factor, and determining a quantized hybrid expert large language model. By means of the method and device, effective separation and compression of expert generality and personality information are achieved, the compression precision and the compression rate are effectively balanced, and the calculation cost and the deployment cost are reduced.

The application discloses a multi-dimension quantification literature value evaluation method and device, relates to the related field of literature metrology, and comprises the following steps: extracting features of target literature in a preset dimension through a large model to obtain multi-dimension literature information; constructing a benchmark library of each dimension according to the preset dimension based on the field of expertise; introducing a bidirectional adversarial verification mechanism to dynamically check the mapping and quantification relationship between feature extraction and the benchmark library; after dynamic checking, respectively using the benchmark library of each dimension to perform dimension mapping and quantification on the literature information extracted in each dimension; performing knowledge structure entropy change operation according to the dimension quantification results, quantifying the knowledge structure entropy change value, and performing literature value fusion evaluation according to the dimension quantification results and the knowledge structure entropy change value. The application solves the technical problems of strong subjectivity and incomplete and inaccurate evaluation of the existing literature value evaluation, and achieves the technical effect of objectively, comprehensively and accurately evaluating the literature value.

The invention provides a large language model lightweight method and system based on 1bit quantization, and the method comprises the steps: carrying out the hierarchical adaptive smoothing of an original full-precision weight of a large language model through a hierarchical channel-level scaling coefficient; based on the smoothed full-precision weight, initialization of a 1-bit quantization core parameter is carried out; performing binarization parameter collaborative optimization on the 1-bit quantization core parameter and the hierarchical channel-level scaling coefficient by taking the reasoning error minimization of the quantized weight and the original full-precision weight as a target; and carrying out collaborative optimization and continuous iteration on the binarized parameters to generate a final quantized weight, and carrying out differential storage. According to the method, efficient 1bit quantization of the full-precision weight of the large language model can be realized, the model memory occupation is greatly reduced, the reasoning precision is reserved to the maximum extent, and the problems that memory compression and precision reservation are difficult to consider, the optimization efficiency is low and the deployment cost is high in the existing 1bit quantization technology are solved.

A method and system of a stochastic quantification method (StochQuant) is described that uses molecular counts obtained from a test measurement, an absolute anchor measurement of a reference molecule, and possibly other physical parameters (e.g., a quantitative measurable of the sample) to determine a probability distribution, a confidence interval, and / or a confidence level of a target molecule test measurement result, thereby improving the reliability and accuracy of a target molecule quantitative detection performed by a test measurement.

The invention provides a language recognition and intensity quantification method based on non-violent communication, and relates to the technical field of big data processing, and the method is characterized in that the method comprises the following steps: S1, a classification model or a rule set is used to determine the category of an input statement, the classification model employs a deep learning model, and training is carried out through labeling; s2, corresponding features are extracted according to the category judgment result, the violent language features include vocabulary intensity, mood intensity, context intensity and emotion intensity, and the non-violent language features include definition, emotion input, constructability and mood; s3, defining a weight and calculating an intensity function, wherein the intensity function is in a weighted summation form; s4, the intensity value is calculated according to the category and output, the output result is the intensity value of the violent language or the non-violent language, and the range is 0-10. The method has the advantages that violent languages and non-violent languages are automatically recognized through a natural language processing technology, and manual intervention is reduced; the language intensity can be accurately quantified, and analysis is more visual.

The invention discloses a 2-bit quantification method of a large language model based on residual error refinement, which is applied to a model reasoning system driven by a computer processor, and comprises the following steps: loading a pre-trained large language model to be quantized, and preparing a calibration data set or synthetic data for quantitative perception training; rough approximation is carried out on the original weight through a first 1-bit kernel, and a residual error is calculated; carrying out second 1-bit kernel refining quantization on the residual error; and finally, reconstructing the weight through the linear combination of the two scaled 1-bit kernels, approximating the gradient of a quantization function by using a straight-through estimator, and performing fine adjustment on model parameters through quantization perception training until the model is converged. According to the method, self-adaptive quantitative point locations are constructed, non-uniform distribution of weights can be flexibly fitted, and the reasoning precision, the training stability and the convergence speed of the model are remarkably improved while the extremely high compression rate is kept.

The invention provides an elastic multi-bit-width Transform architecture quantification method based on lexical-level feature fusion, and belongs to the field of neural network model compression. Block-by-block reconstruction is carried out on a Transform architecture model, trainable parameters are initialized for a current block, a bit is sampled from a high bit group, a middle bit group and a low bit group respectively, a low-rank compensation matrix is calculated for each sampled bit, and a low-rank compensation matrix is calculated for each sampled bit; calculating a joint quantization result of the full-precision weight and the low-rank compensation matrix, and updating trainable parameters through back propagation; calculating the cosine similarity between the output characteristics of the current block under a high bit and the output characteristics of the current block under a low bit, selecting tokens with a set proportion, generating a token index set, judging each token, splicing all tokens, obtaining fused characteristics, inputting the fused characteristics into the next block until reconstruction of all blocks is completed, and outputting quantized model weights; the invention further provides a quantification system. The problems that an existing Transform architecture multi-bit-width quantization method is low in robustness and large in storage and calculation overhead are solved.

The invention relates to the technical field of communication, in particular to a model determination method and device, a semantic quantification method and device, communication equipment and a storage medium, and the model determination method comprises the following steps: determining a training sample set; the first initial model and the second initial model are trained based on the training sample set, the first initial model when training is stopped serves as a semantic quantification model, and the second initial model when training is stopped serves as a perceptual model; wherein the input of the semantic quantification model comprises channel measurement data, the output of the semantic quantification model comprises prediction semantic quantification data, the input of the perception model comprises prediction semantic quantification data, and the output of the perception model comprises a prediction perception result corresponding to the channel measurement data. According to the method and the device, the prediction semantic quantification data output by the semantic quantification model can be ensured, the characteristics of the channel measurement data input into the semantic quantification model can be accurately reflected, and the accuracy of the prediction semantic quantification data used for the perception model to predict the perception result of the channel measurement data is further ensured.

The invention provides an accelerated reasoning system of a large language model, a mixing precision quantification method and a medium. The accelerated reasoning system of the large language model comprises the steps that the system is integrated in an HBM stack to form a heterogeneous system with an XPU; the system comprises a CPU (Central Processing Unit), the XPU and a plurality of groups of HBM stacks, each group of HBM stacks comprises eight DRAM (Dynamic Random Access Memory) chips which are stacked in sequence and a buffer chip which is connected through a silicon through hole; the system performs mixing precision quantification on a target large language model by using in-memory calculation, and performs reasoning calculation on an attention layer by using quantized model parameters to accelerate reasoning calculation of the target large language model. According to the method, mixing precision quantification can be carried out on the large language model, the accuracy and the compression ratio are balanced, the bandwidth and the calculation efficiency are maximized, and it is ensured that the quantification advantage is effectively converted into actual reasoning acceleration.

The invention relates to a quantization method and device for a generative model, computer equipment, a chip and a readable storage medium. The method comprises the following steps: acquiring a plurality of candidate sampling coefficients of a generative model and generated data corresponding to each operation step; calculating a data acquisition state list corresponding to each candidate sampling coefficient based on each candidate sampling coefficient, a preset initial sampling interval and a maximum step number threshold value of the operation step; performing data acquisition based on the data acquisition state list in the generated data corresponding to each operation step to obtain calibration data; and performing quantitative screening on the generative model through the calibration data to obtain a target quantitative model of the generative model. By adopting the method, the generated data corresponding to each operation step can be screened, the data with high quality can be extracted as the calibration data, and the efficiency and the accuracy of model quantification are further improved.