Large language model lightweight method and system based on 1bit quantization

Abstract

The invention provides a large language model lightweight method and system based on 1bit quantization, and the method comprises the steps: carrying out the hierarchical adaptive smoothing of an original full-precision weight of a large language model through a hierarchical channel-level scaling coefficient; based on the smoothed full-precision weight, initialization of a 1-bit quantization core parameter is carried out; performing binarization parameter collaborative optimization on the 1-bit quantization core parameter and the hierarchical channel-level scaling coefficient by taking the reasoning error minimization of the quantized weight and the original full-precision weight as a target; and carrying out collaborative optimization and continuous iteration on the binarized parameters to generate a final quantized weight, and carrying out differential storage. According to the method, efficient 1bit quantization of the full-precision weight of the large language model can be realized, the model memory occupation is greatly reduced, the reasoning precision is reserved to the maximum extent, and the problems that memory compression and precision reservation are difficult to consider, the optimization efficiency is low and the deployment cost is high in the existing 1bit quantization technology are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer software, and particularly relates to a large language model lightening method and system based on 1bit quantization. BACKGROUND

[0002] Large language models (LLM) have greatly promoted the progress of natural language processing, but their high memory and computing requirements hinder their practical deployment. As a kind of quantization technology, binarization belongs to the highest compression method in quantization. It greatly reduces the computing and memory requirements by compressing the model weights to only 1 bit. However, although binarization can effectively reduce the storage and computing burden of large language models, the existing binarization methods still have large errors compared to full-precision models, and ignore the problem of outliers in weight distribution. In the development process of quantization technology, how to further optimize binarization without sacrificing accuracy and minimize resource consumption of the model remains a challenge to be solved. SUMMARY

[0003] The present application provides a large language model lightening method and system based on 1bit quantization to solve the above problems in the prior art.

[0004] According to one aspect of the present application, a large language model lightening method based on 1bit quantization is provided, comprising: using hierarchical channel level scaling coefficients to perform hierarchical adaptive smoothing processing on the original full-precision weights of the large language model; based on the smoothed full-precision weights, initializing the 1bit quantization core parameters; wherein the core parameters include: a 1bit weight matrix B, hierarchical channel level scaling coefficients s of the weight matrix B, and a re-distribution mean m; minimizing the inference error between the quantized weights and the original full-precision weights as the target, performing binarization parameter collaborative optimization on the 1bit weight matrix B, the scaling coefficient s, the re-distribution mean m, and the hierarchical channel level scaling coefficients; continuously iterating the binarization parameter collaborative optimization to generate the final quantized weights and perform differential storage.

[0005] Preferably, the hierarchical adaptive smoothing processing on the original full-precision weights of the large language model using hierarchical channel level scaling coefficients comprises: calculating hierarchical channel level scaling coefficients; multiplying each channel element of the original full-precision weights of the large language model by the corresponding hierarchical channel level scaling coefficients to optimize the weight distribution through coefficient regulation.

[0006] Preferably, the calculation of hierarchical channel level scaling coefficients comprises: obtaining a calibration dataset matching the actual application scenario of the model; For each output channel of the full-precision weight, the maximum amplitude of all elements in the corresponding channel of the calibration dataset is calculated, and the maximum amplitude is taken as the basic scaling coefficient of the channel; Meanwhile, according to the feature importance of different layers of the model, a differentiated weighting coefficient is configured for each layer, the basic scaling coefficient is multiplied by the weighting coefficient of the corresponding layer to obtain the final layered channel-level scaling coefficient.

[0007] Preferably, the initialization of the 1bit quantization core parameter based on the smoothed full-precision weight includes: All elements of the smoothed full-precision weight are traversed, and the initial binary matrix B0 is obtained according to the positive and negative attributes of the elements; The overall amplitude distribution of the smoothed full-precision weight is counted, the absolute value mean of all non-zero elements is calculated, and the absolute value mean is taken as the initial layered channel-level scaling coefficient s0 of the weight matrix B, so as to ensure that the initial coefficient can match the amplitude range of the original weight; The global element mean of the smoothed full-precision weight is calculated, and the global element mean is taken as the initial redistribution mean m0, so that the distribution of the quantized weight is preliminarily aligned with the distribution of the original weight.

[0008] Preferably, the assignment according to the positive and negative attributes of the elements includes: When the element value is greater than 0, the corresponding element in the 1bit weight matrix B is set to 1; When the element value is less than or equal to 0, the corresponding element in the 1bit weight matrix B is set to -1.

[0009] Preferably, the binary parameter collaborative optimization of the 1bit weight matrix B, the scaling coefficient s, the redistribution mean m, and the layered channel-level scaling coefficient is performed with the objective of minimizing the inference error between the quantized weight and the original full-precision weight, including: The quantized weight expression is defined as newW = s×(B - m), and the difference between the quantized weight and the original full-precision weight in the inference result of the calibration dataset is taken as the quantization error evaluation standard. Each element in the 1bit weight matrix B is traversed, and the element value is flipped one by one. The quantization error after each flip is calculated in real time, and the 1bit weight matrix B with the lowest error is retained as the updated binary matrix B1; Fixing the updated binary matrix B1, an optimization problem with the objective of minimizing the quantization error is constructed, and the closed-form solution of the scaling coefficient s and the redistribution mean m is obtained through mathematical derivation to obtain the optimal parameters s1 and m1; Based on the obtained optimal parameters s1 and m1, with the goal of further reducing the overall quantization error, the scaling coefficients of the hierarchical channel level are adjusted in reverse to achieve the coordinated adaptation of weight smoothing and binarization parameters, thereby further compressing the quantization error.

[0010] Preferably, the iterative process of collaboratively optimizing the binarized parameters to generate the final quantized weights and storing them differentially includes: After multiple rounds of collaborative optimization of binarization parameters, the optimal 1-bit weight matrix is ​​obtained. Optimal hierarchical channel-level scaling factor for weight matrix B and the mean of optimal redistribution According to the formula: Final quantization weights ; A 1-bit quantized weight is obtained that matches the performance of the original full-precision weight; Storage solutions are designed to address the characteristics of different parameters, resulting in a significant reduction in memory usage.

[0011] Preferably, the storage scheme designed for the characteristics of different parameters includes: For a 1-bit weight matrix It uses bit compression storage, packing 8 elements into 1 byte, where 0 represents -1 and 1 represents 1; Regarding scaling factor , stored independently by channel, using 16-bit floating-point format; For the redistribution mean It is stored globally in a single format and uses a 16-bit floating-point number format.

[0012] According to a second aspect of the present invention, a lightweight system for large language models based on 1-bit quantization is provided, comprising: The full-precision weight smoothing module uses hierarchical channel-level scaling coefficients to perform hierarchical adaptive smoothing on the original full-precision weights of the large language model. The binarization parameter initialization module initializes the 1-bit quantization core parameters based on the smoothed full-precision weights. The core parameters include: a 1-bit weight matrix B, a hierarchical channel-level scaling factor s for the weight matrix B, and a redistribution mean m. The binarization parameter co-optimization module aims to minimize the inference error between the quantized weights and the original full-precision weights. It performs binarization parameter co-optimization on the 1-bit weight matrix B, scaling coefficients s, redistribution mean m, and hierarchical channel-level scaling coefficients. The quantization weight generation and storage module generates the final quantization weights through continuous iteration of collaborative optimization of the quantization parameters, and stores them in a differentiated manner.

[0013] By adopting the above technical solution, the present invention has at least one of the following beneficial effects compared with the prior art: This invention employs a full-precision weighted hierarchical adaptive smoothing processing technique, including hierarchical channel-level scaling coefficient calculation and weight smoothing adjustment. This technique can accurately suppress and optimize the distribution of outliers in the original weights, providing a stable weight foundation for subsequent binarization and solving the quantization distortion problem caused by outliers and uneven distribution of the original weights.

[0014] This invention employs a binarized parameter collaborative iterative optimization technique, including 1-bit weight matrix flipping optimization, closed-form solution of s and m, and hierarchical scaling coefficient secondary optimization. This technique can minimize the inference error between the quantized weights and the original full-precision weights, thus solving the problems of insufficient quantization accuracy and low optimization efficiency caused by single parameter optimization.

[0015] This invention employs differentiated and efficient storage technologies, including 1-bit weighted bit compressed storage, s-channel 16-bit storage, and m-global 16-bit storage, which can significantly reduce the model's memory usage without affecting inference speed, and solve the problems of high memory overhead and high deployment cost for full-precision storage of large language models.

[0016] This invention enables efficient 1-bit quantization of full-precision weights in large language models, significantly reducing model memory usage (by 90% compared to the original) while maximizing the preservation of inference accuracy. It overcomes the problems of existing 1-bit quantization technologies, such as difficulty in balancing memory compression and accuracy preservation, low optimization efficiency, and high deployment costs. Attached Figure Description

[0017] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating the workflow of a lightweight method for large language models based on 1-bit quantization in a preferred embodiment of the present invention.

[0018] Figure 2 This is a working architecture diagram of a lightweight system for large language models based on 1-bit quantization in a preferred embodiment of the present invention. Detailed Implementation

[0019] The embodiments of the present invention are described in detail below: These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention.

[0020] As large language models continue to grow in size, their storage and inference face severe challenges. On the one hand, the massive number of model parameters leads to extremely high memory consumption, significantly increasing model deployment costs and limiting their application on resource-constrained devices. On the other hand, when quantizing to reduce memory consumption, outliers and uneven distribution exist in the original full-precision weights, which can easily lead to quantization distortion and consequently a loss of model inference accuracy, making it difficult to meet the performance requirements of practical applications.

[0021] To address the aforementioned issues, one embodiment of the present invention provides a lightweight quantization method for large language models based on 1-bit quantization. This method is a novel 1-bit post-quantization (PTQ) technique specifically tailored for large language models (LLMs), aiming to reduce the storage and computational requirements of deep learning models through more efficient weight processing techniques. Compared to existing 1-bit methods, this 1-bit quantization method occupies less memory and achieves higher accuracy; simultaneously, it also surpasses full-precision methods within the same memory footprint.

[0022] Specifically, such as Figure 1 As shown, the lightweight method for large language models based on 1-bit quantization provided in this embodiment aims to achieve efficient 1-bit quantization of full-precision weights in large language models. Through hierarchical smoothing optimization and parameter co-tuning, it significantly reduces model memory usage while maximizing the preservation of model inference accuracy. This method may include: S1 utilizes hierarchical channel-level scaling coefficients to perform hierarchical adaptive smoothing on the original full-precision weights of the large language model; S2 initializes the 1-bit quantization core parameters based on the smoothed full-precision weights, providing a precise starting point for subsequent optimization; the core parameters include: 1-bit weight matrix B, hierarchical channel-level scaling coefficients s for weight matrix B, and redistribution mean m; S3 aims to minimize the inference error between the quantized weights and the original full-precision weights by performing binarization parameter co-optimization on the 1-bit weight matrix B, scaling coefficients s, redistribution mean m, and hierarchical channel-level scaling coefficients. S4 iterates through collaborative optimization of the binarized parameters to generate the final quantized weights, which are then stored differentially.

[0023] In some preferred embodiments, S1 above, which uses hierarchical channel-level scaling coefficients to perform hierarchical adaptive smoothing processing on the original full-precision weights of the large language model, may further include: S11, calculate the scaling factor of the hierarchical channel level; S12 multiplies each channel element of the original full-precision weights of the large language model with the corresponding hierarchical channel-level scaling coefficient, and smooths and optimizes the weight distribution through coefficient adjustment.

[0024] In some preferred embodiments, S11 above, calculating the scaling factor of the layered channel level, may further include: S111, Obtain a calibration dataset that matches the actual application scenario of the model; S112, For each output channel with full precision weight, calculate the maximum amplitude of all elements in the corresponding channel of the calibration dataset, and use the maximum amplitude as the basic scaling factor of the channel. S113. At the same time, based on the feature importance of different layers of the model, differentiated weighting coefficients are configured for each layer. The basic scaling coefficients are multiplied by the weighting coefficients of the corresponding layers to obtain the final layered channel-level scaling coefficients.

[0025] In some preferred embodiments, the above-mentioned S2, which initializes the 1-bit quantization core parameters based on the smoothed full-precision weights, may further include: S21, traverse all elements of the smoothed full-precision weights, assign values ​​according to the positive or negative attributes of the elements, and obtain the initial binary matrix B0; S22, Statistically smooth the overall amplitude distribution of the full-precision weights, calculate the mean of the absolute values ​​of all non-zero elements, and use this mean of absolute values ​​as the initial hierarchical channel-level scaling coefficient s0 for the weight matrix B, ensuring that the initial coefficient can match the amplitude range of the original weights. S23, calculate the global element mean of the smoothed full-precision weights, and use this global element mean as the initial redistribution mean m0 to keep the quantized weight distribution initially aligned with the original weight distribution.

[0026] In some preferred embodiments, the above-mentioned S21, which assigns values ​​based on the positive or negative attributes of elements, may further include: When the element value is greater than 0, the corresponding element in the 1-bit weight matrix B is set to 1; When the element value is less than or equal to 0, the element in the corresponding 1-bit weight matrix B is set to -1.

[0027] In some preferred embodiments, S3 above, with the goal of minimizing the inference error between the quantized weights and the original full-precision weights, performs binarization parameter co-optimization on the 1-bit weight matrix B, scaling coefficients s, redistribution mean m, and hierarchical channel-level scaling coefficients, and may further include: S31, define the quantization weight expression as newW = s×(B - m), and use the difference between the inference results of the quantized weight and the original full-precision weight on the calibration dataset as the quantization error evaluation standard; traverse each element in the 1-bit weight matrix B, flip the element value one by one, calculate the quantization error after each flip in real time, and retain the 1-bit weight matrix B that minimizes the error as the updated binary matrix B1. S32, with a fixed updated binary matrix B1, constructs an optimization problem with the goal of minimizing quantization error. Through mathematical derivation, a closed-form solution for the scaling coefficient s and the redistribution mean m is obtained. The optimal parameters s1 and m1 can be obtained in one go without iteration, improving optimization efficiency and parameter accuracy. S33, based on the obtained optimal parameters s1 and m1, aims to further reduce the overall quantization error by adjusting the hierarchical channel-level scaling coefficients in reverse, thereby achieving coordinated adaptation of weight smoothing and binarization parameters and further compressing the quantization error.

[0028] In some preferred embodiments, the above-mentioned S4, which iteratively optimizes the binarization parameters to generate the final quantized weights and stores them differentially, may further include: S41, after multiple rounds of collaborative optimization of binarization parameters, obtains the optimal 1-bit weight matrix. Optimal hierarchical channel-level scaling factor for weight matrix B and the mean of optimal redistribution According to the formula: Final quantization weights ; A 1-bit quantized weight is obtained that matches the performance of the original full-precision weight; S42 employs storage solutions designed for the characteristics of different parameters, resulting in a significant reduction in memory usage.

[0029] In some preferred embodiments, the above-mentioned S42, which designs a storage scheme for the characteristics of different parameters, may further include: S421, for a 1-bit weight matrix It uses bit-compressed storage, packing 8 elements into 1 byte, where 0 represents -1 and 1 represents 1. Compared with full-precision storage (usually 4 bytes / element), the storage overhead is compressed to 1 / 32. S422, for scaling factor It is stored independently by channel, using a 16-bit floating-point format to balance storage efficiency while ensuring accuracy. S423, for the redistributed mean It uses a global single storage and a 16-bit floating-point format to ensure that the mean accuracy meets the quantization restoration requirements.

[0030] With the above storage scheme, the overall memory usage is reduced by 90% compared to the original full-precision weights, without affecting the model inference speed.

[0031] Based on the same inventive concept, one embodiment of the present invention also provides a lightweight system for large language models based on 1-bit quantization.

[0032] Specifically, such as Figure 2As shown, the lightweight system for large language models based on 1-bit quantization provided in this embodiment may include: The full-precision weight smoothing module uses hierarchical channel-level scaling coefficients to perform hierarchical adaptive smoothing on the original full-precision weights of the large language model. The binarization parameter initialization module initializes the 1-bit quantization core parameters based on the smoothed full-precision weights, providing a precise starting point for subsequent optimization. The core parameters include: a 1-bit weight matrix B, a hierarchical channel-level scaling factor s for the weight matrix B, and a redistribution mean m. The binarization parameter co-optimization module aims to minimize the inference error between the quantized weights and the original full-precision weights. It performs binarization parameter co-optimization on the 1-bit weight matrix B, scaling coefficients s, redistribution mean m, and hierarchical channel-level scaling coefficients. The quantization weight generation and storage module generates the final quantization weights through continuous iteration of collaborative optimization of the quantization parameters, and stores them in a differentiated manner.

[0033] The implementation methods of each functional module constituting the system provided in the above embodiments of the present invention will be further described in detail below with reference to preferred embodiments.

[0034] Full-precision weight smoothing module: Used for hierarchical adaptive smoothing of full-precision weights. To address the quantization distortion caused by outliers and uneven distribution in the original full-precision weights, hierarchical adaptive smoothing is performed on the original full-precision weights of the large language model. Specifically, it includes: The hierarchical channel-level scaling coefficient calculation unit first obtains a calibration dataset that matches the actual application scenario of the model (it needs to cover the text distribution features of the model's core task to ensure data representativeness); for each output channel with full-precision weights, the maximum amplitude of all elements in the corresponding channel of the calibration dataset is calculated, and this maximum amplitude is used as the basic scaling coefficient for that channel; at the same time, according to the feature importance of different layers of the model (shallow layers focus on feature extraction, and deep layers focus on semantic modeling), differentiated weighting coefficients are configured for each layer (shallow layer weighting coefficients range from 0.8 to 1.0, and deep layer weighting coefficients range from 1.0 to 1.2), and the basic scaling coefficient is multiplied by the weighting coefficient of the corresponding layer to obtain the final hierarchical channel-level scaling coefficient; Weight smoothing adjustment unit: Multiply each channel element of the original full-precision weight with the corresponding hierarchical channel scaling coefficient. By adjusting the coefficient, the weight distribution is smoothly optimized, which not only suppresses outlier interference but also avoids excessive suppression of effective features, providing a more stable weight foundation for subsequent binarization.

[0035] Binarization parameter initialization module: Used for initializing core binarization parameters. Based on the smoothed full-precision weights, it initializes the 1-bit quantization core parameters, providing a precise starting point for subsequent optimization. Specifically, it includes: 1-bit weight matrix B initialization unit: Traverse all elements of the smoothed full-precision weights and assign values ​​according to the positive or negative attribute of the elements. That is, when the element value is greater than 0, the corresponding element in the 1-bit weight matrix B is set to 1; when the element value is less than or equal to 0, the corresponding element is set to -1, thus obtaining the initial binary matrix B0. Initialization unit for scaling factor s: Statistically smooth the overall amplitude distribution of the full-precision weights, calculate the mean of the absolute values ​​of all non-zero elements, and use this mean as the initial hierarchical channel-level scaling factor s0 for the weight matrix B, ensuring that the initial coefficient can match the amplitude range of the original weights; Initialization unit for redistribution mean m: Calculate the global element mean of the smoothed full-precision weights, and use this mean as the initial redistribution mean m0 to keep the quantized weight distribution initially aligned with the original weight distribution.

[0036] Binarization Parameter Co-optimization Module: Used for co-iterative optimization of binarization parameters. With the goal of minimizing the inference error between the quantized weights and the original full-precision weights, it co-optimizes the 1-bit weight matrix B, scaling coefficients s, redistribution mean m, and the hierarchical channel-level scaling coefficients from step 1. Specifically, it includes: The 1-bit weight matrix B optimization unit defines the quantization weight expression as newW = s×(B - m). The difference between the inference results of the quantized weight and the original full-precision weight on the calibration dataset (i.e., the deviation between newW×X and W×X, where X is the calibration data used to calibrate the quantization parameters) is used as the quantization error evaluation standard. Each element in the 1-bit matrix B is traversed, and the element value is flipped one by one (1 becomes -1 or -1 becomes 1). The quantization error after each flip is calculated in real time, and the B that minimizes the error is retained as the updated binary matrix B1. The optimal solution unit for scaling factor s and redistribution mean m is: with a fixed updated 1-bit weight matrix B1, an optimization problem is constructed with the goal of minimizing quantization error. The closed-form solution of s and m is obtained through mathematical derivation. The optimal parameters s1 and m1 can be obtained in one go without iteration, which improves optimization efficiency and parameter accuracy. The hierarchical channel-level scaling coefficient secondary optimization unit: Based on the optimal s1 and m1 obtained by the optimal solution unit of scaling coefficient s and redistribution mean m, the hierarchical channel-level scaling coefficients obtained in the hierarchical channel-level scaling coefficient calculation unit are adjusted in reverse. That is, with the goal of further reducing the overall quantization error, the scaling coefficient values ​​of each layer and each channel are finely adjusted to achieve the coordinated adaptation of weight smoothing and binarization parameters, and further compress the quantization error.

[0037] Quantization weight generation and storage module: used for generating and efficiently storing quantization results. Specifically includes: Final quantization weight generation unit: After multiple rounds of collaborative optimization, the optimal 1-bit weight matrix is ​​obtained. Optimal hierarchical channel-level scaling factor for weight matrix B and the mean of optimal redistribution According to the formula "final quantization weight" "Obtain a 1-bit quantized weight that matches the performance of the original full-precision weight; Differentiated high-efficiency storage units: Storage solutions are designed for the characteristics of different parameters, resulting in a significant reduction in memory usage. Specific solutions include: 1-bit weight matrix Bit compression storage is used, with 8 elements packed into 1 byte (0 represents -1 and 1 represents 1), which reduces storage overhead to 1 / 32 compared to full-precision storage (usually 4 bytes / element). scaling factor Stored independently by channel, using a 16-bit floating-point format, balancing storage efficiency while ensuring accuracy; redistribution mean Global single storage, using 16-bit floating-point format, ensures that the mean accuracy meets the quantization restoration requirements; With the above storage scheme, the overall memory usage is reduced by 90% compared to the original full-precision weights, without affecting the model inference speed.

[0038] It should be noted that the steps in the method provided by the present invention can be implemented using the corresponding components in the system. Those skilled in the art can refer to the technical solution of the system to implement the steps of the method, and can also refer to the technical solution of the method to implement the composition of the system. That is, the embodiments in the system and the embodiments in the method can be understood as preferred examples of each other, which will not be elaborated here.

[0039] The technical solution provided by the above embodiments of the present invention will be further described in detail below with reference to a specific application example.

[0040] This invention can be used to accelerate the deployment of large language models in AI customer service, thereby speeding up the response time of AI customer service and improving user experience. For the Llama 2-7B model commonly used in AI customer service, this invention performs 1-bit quantization, the process of which includes: Step 1: Full-precision weighted hierarchical adaptive smoothing processing A general text calibration dataset (containing news, papers, novels, etc., totaling 100,000 texts, covering the Llama2-7B core task distribution) was selected. Layered channel-level scaling coefficient calculation: For the 12-layer Transformer block of the model, the weighting coefficients were set to 0.9 for shallow layers (layers 1-4), 1.1 for deep layers (layers 9-12), and 1.0 for intermediate layers (layers 5-8). For each output channel, the maximum amplitude of the corresponding channel element in the calibration data was calculated as the basic scaling coefficient, which was multiplied by the weighting coefficient to obtain the final scaling coefficient. Weight smoothing adjustment: The original full-precision weight elements of each channel were multiplied by the corresponding scaling coefficient to complete the weight distribution optimization. Step 2, Initialization of binarization core parameters 1-bit weight matrix B0: Traverse the smoothed weights, set elements > 0 to 1, and ≤ 0 to -1, to generate the initial binary matrix; scaling factor s0: calculate the mean of the absolute values ​​of the non-zero elements of the smoothed weights, resulting in s0 = 0.72; redistribution mean m0: calculate the global mean of the smoothed weights, resulting in m0 = 0.03. Step 3, Binarized parameter cooperative iterative optimization B optimization: Using the "inference deviation between newW×X and W×X" as the error standard, the B0 element is flipped element by element, and the B1 with the lowest error is retained. After 3 iterations, the error is reduced by 42%. Optimal solutions for s and m: With B1 fixed, s1=0.75 and m1=0.028 are calculated through closed-form solutions. Secondary optimization of layer scaling coefficients: Based on s1 and m1, the scaling coefficients of shallow layers are fine-tuned to 0.88 and deep layers to 1.12, further reducing the quantization error by 8%. Step 4: Generation and storage of quantification results Generate final quantization weights: Press " " calculate, =0.75、 =0.028、 To optimize the binary matrix; differentiated storage: B-bit compressed storage (1 byte stores 8 elements), s-bit 16-bit floating-point storage per channel. Global 16-bit floating-point storage.

[0041] The lightweight method and system for large language models based on 1-bit quantization provided in the above embodiments of the present invention employs a full-precision weight hierarchical adaptive smoothing processing technique, including hierarchical channel-level scaling coefficient calculation and weight smoothing adjustment. This enables precise suppression and distribution optimization of outliers in the original weights, providing a stable weight foundation for subsequent binarization and solving the quantization distortion problem caused by outliers and uneven distribution of the original weights. It also employs a binarization parameter collaborative iterative optimization technique, including 1-bit weight matrix flipping optimization, closed-form solution of s and m, and hierarchical scaling coefficient secondary optimization. This minimizes the inference error between the quantized weights and the original full-precision weights, solving the problems of insufficient quantization accuracy and low optimization efficiency caused by single-parameter optimization. Finally, it employs differentiated high-efficiency storage technology, including 1-bit weight bit compressed storage, s channel-by-channel 16-bit storage, and m global 16-bit storage. This significantly reduces model memory usage without affecting inference speed, solving the problems of high memory overhead and high deployment cost for full-precision storage of large language models. It can achieve efficient 1-bit quantization of full-precision weights in large language models, significantly reducing model memory usage (90% less than the original) while maximizing the preservation of inference accuracy. This overcomes the problems of existing 1-bit quantization technologies, such as difficulty in balancing memory compression and accuracy preservation, low optimization efficiency, and high deployment costs.

[0042] Any matters not covered in the above embodiments of the present invention are well-known in the art.

[0043] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A lightweight method for large language models based on 1-bit quantization, characterized in that, include: By using hierarchical channel-level scaling coefficients, hierarchical adaptive smoothing is performed on the original full-precision weights of the large language model; Based on the smoothed full-precision weights, the core parameters of 1-bit quantization are initialized; wherein, the core parameters include: 1-bit weight matrix B, hierarchical channel-level scaling coefficients s of weight matrix B, and redistribution mean m; With the goal of minimizing the inference error between the quantized weights and the original full-precision weights, the binarization parameters of the 1-bit weight matrix B, scaling coefficients s, redistribution mean m, and hierarchical channel-level scaling coefficients are optimized in a coordinated manner. The binarized parameters are continuously optimized iteratively to generate the final quantized weights, which are then stored in a differentiated manner.

2. The lightweight method for large language models based on 1-bit quantization according to claim 1, characterized in that, The method of using hierarchical channel-level scaling coefficients to perform hierarchical adaptive smoothing on the original full-precision weights of a large language model includes: Calculate the scaling factor for the hierarchical channel level; Each channel element of the original full-precision weights of the large language model is multiplied by the corresponding hierarchical channel-level scaling coefficient, and the weight distribution is smoothly optimized by adjusting the coefficients.

3. The lightweight method for large language models based on 1-bit quantization according to claim 2, characterized in that, The calculation of the hierarchical channel-level scaling factor includes: Obtain a calibration dataset that matches the actual application scenario of the model; For each output channel with full precision weights, calculate the maximum amplitude of all elements in the corresponding channel of the calibration dataset, and use the maximum amplitude as the basic scaling factor for that channel. Meanwhile, based on the feature importance of different layers of the model, differentiated weighting coefficients are configured for each layer. The base scaling coefficient is multiplied by the weighting coefficient of the corresponding layer to obtain the final hierarchical channel-level scaling coefficient.

4. The lightweight method for large language models based on 1-bit quantization according to claim 1, characterized in that, The initialization of the 1-bit quantization core parameters based on the smoothed full-precision weights includes: Traverse all elements of the smoothed full-precision weights and assign values ​​according to the positive or negative attributes of the elements to obtain the initial binary matrix B0; The overall amplitude distribution of the full-precision weights after statistical smoothing is calculated, and the mean of the absolute values ​​of all non-zero elements is used as the initial channel-level scaling factor s0 for the weight matrix B, ensuring that the initial factor can match the amplitude range of the original weights. Calculate the global mean of the smoothed full-precision weights, and use this global mean as the initial redistribution mean m0 to keep the quantized weight distribution initially aligned with the original weight distribution.

5. The lightweight method for large language models based on 1-bit quantization according to claim 4, characterized in that, The assignment based on the positive or negative attribute of an element includes: When the element value is greater than 0, the corresponding element in the 1-bit weight matrix B is set to 1; When the element value is less than or equal to 0, the element in the corresponding 1-bit weight matrix B is set to -1.

6. The lightweight method for large language models based on 1-bit quantization according to claim 1, characterized in that, The objective is to minimize the inference error between the quantized weights and the original full-precision weights. This involves the collaborative optimization of binarization parameters for the 1-bit weight matrix B, scaling coefficients s, redistribution mean m, and hierarchical channel-level scaling coefficients, including: The quantization weight expression is defined as newW = s×(B - m). The difference between the inference results of the quantized weight and the original full-precision weight on the calibration dataset is used as the quantization error evaluation standard. Each element in the 1-bit weight matrix B is traversed, and the element value is flipped one by one. The quantization error after each flip is calculated in real time. The 1-bit weight matrix B that minimizes the error is retained as the updated binary matrix B1. After fixing the updated binary matrix B1, we construct an optimization problem with the goal of minimizing the quantization error. Through mathematical derivation, we obtain a closed-form solution for the scaling coefficient s and the redistribution mean m, and obtain the optimal parameters s1 and m1. Based on the obtained optimal parameters s1 and m1, with the goal of further reducing the overall quantization error, the scaling coefficients of the hierarchical channel level are adjusted in reverse to achieve the coordinated adaptation of weight smoothing and binarization parameters, thereby further compressing the quantization error.

7. The lightweight method for large language models based on 1-bit quantization according to claim 1, characterized in that, The iterative process of collaboratively optimizing the binarized parameters to generate the final quantized weights and then storing them differentially includes: After multiple rounds of collaborative optimization of binarization parameters, the optimal 1-bit weight matrix is ​​obtained. Optimal hierarchical channel-level scaling factor for weight matrix B and the mean of optimal redistribution According to the formula: Final quantization weights ; A 1-bit quantized weight is obtained that matches the performance of the original full-precision weight; Storage solutions are designed to address the characteristics of different parameters, resulting in a significant reduction in memory usage.

8. The lightweight method for large language models based on 1-bit quantization according to claim 7, characterized in that, The storage scheme designed for the characteristics of different parameters includes: For a 1-bit weight matrix It uses bit compression storage, packing 8 elements into 1 byte, where 0 represents -1 and 1 represents 1; Regarding scaling factor , stored independently by channel, using 16-bit floating-point format; For the redistribution mean It is stored globally in a single format and uses a 16-bit floating-point number format.

9. A lightweight system for large language models based on 1-bit quantization, characterized in that, include: The full-precision weight smoothing module uses hierarchical channel-level scaling coefficients to perform hierarchical adaptive smoothing on the original full-precision weights of the large language model. The binarization parameter initialization module initializes the 1-bit quantization core parameters based on the smoothed full-precision weights. The core parameters include: a 1-bit weight matrix B, a hierarchical channel-level scaling factor s for the weight matrix B, and a redistribution mean m. The binarization parameter co-optimization module aims to minimize the inference error between the quantized weights and the original full-precision weights. It performs binarization parameter co-optimization on the 1-bit weight matrix B, scaling coefficients s, redistribution mean m, and hierarchical channel-level scaling coefficients. The quantization weight generation and storage module generates the final quantization weights through continuous iteration of collaborative optimization of the quantization parameters, and stores them in a differentiated manner.

Images