Quaternary weight compression method and device for large language model and storage medium

Abstract

The invention provides a large language model quaternary weight compression method and device and a storage medium. The large language model quaternary weight compression method comprises the steps that a model and data are prepared, specifically, a pre-trained large language model is selected, and a Wikitext2 data set is adopted as calibration data; executing a first-stage PTQ process, and carrying out weight transformation: activating quantization, defining a quaternary quantization function, carrying out multi-parameter grid search, and obtaining an optimal weight scaling factor and double thresholds; and executing a second-stage PTQ process, and carrying out weight optimization: defining block reconstruction loss, carrying out greedy iterative quantization, and generating a quaternary weight compression model. The technical problems of large model performance loss and low high-precision quantization compression ratio caused by insufficient information retention of a ternary quantization scheme in the prior art can be solved.

Description

Technical Field

[0001] This invention belongs to the field of large model deployment technology, and in particular relates to a method, device and storage medium for quaternary weight compression of large language models. Background Technology

[0002] In recent years, large language models (LLMs) such as GPT-4 and the LLaMA series, with their billions to hundreds of billions of parameters, have demonstrated outstanding performance in tasks such as text generation, question answering, and translation, and have been widely used in fields such as intelligent customer service and content creation. However, the huge parameter scale of LLMs leads to three core problems in their deployment: First, high hardware resource requirements, such as a 70-parameter model requiring 132GB of memory for its FP16 weights and needing multiple high-performance GPUs; second, long inference latency, as traditional full-precision models cannot meet the needs of real-time interactive scenarios; and third, high energy consumption, as large-scale matrix operations result in significant energy consumption during operation, increasing enterprise operating costs.

[0003] To address the aforementioned problems, quantization techniques have become a key approach, with low-precision quantization (e.g., 1-2 bits) becoming a research hotspot due to its significant compression efficiency. This is mainly divided into two categories:

[0004] 1. Quantization-Aware Training (QAT): A representative approach is BitNet, which uses ternary weights (±1,0) quantization to reduce energy consumption and latency by eliminating multiplication operations in matrix multiplication. However, BitNet requires training the model from scratch based on the ternary quantization scheme, which has significant drawbacks: during training, it is necessary to store full-precision weights (e.g., FP32 weights in a 70B model occupy 280GB), gradients (280GB), and optimizer momentum (560GB), which cannot be supported by a single GPU. This results in a model size limit of only 3.9B parameters, making it unsuitable for ultra-large models such as 70B.

[0005] 2. Post-Training Quantization (PTQ): This method directly quantizes the pre-trained model without requiring retraining from scratch, reducing resource demands. However, existing PTQ schemes suffer from significant performance degradation due to severe information loss caused by extreme precision compression when implementing ternary weight quantization. For example, when using GPTQ to perform 1.58-bit ternary quantization on the LLaMA-2-7B model, the perplexity on the Wikitext2 dataset reaches as high as 94.8, far exceeding the 5.47 of the full-precision model; rounding quantization even produces meaningless results, failing to meet the performance requirements of practical applications.

[0006] In summary, existing technologies suffer from two major contradictions: first, the insufficient information retention of ternary quantization (QAT / PTQ) contradicts performance requirements; second, the insufficient compression ratio of high-precision quantization contradicts hardware resource limitations. There is an urgent need for a low-precision quantization scheme that balances high compression ratio, adaptability to ultra-large models, and high performance retention. Quadruple quantization (containing four discrete weight values) can compensate for the information loss of ternary quantization by increasing the number of quantization levels, while maintaining the compression advantages of low precision, making it a key direction for resolving these contradictions. Summary of the Invention

[0007] The present invention aims to solve at least one of the technical problems existing in the prior art.

[0008] According to one aspect of the present invention, a method for compressing quadruple weights in a large language model is provided, the method comprising:

[0009] Model and data preparation: Select a pre-trained large language model and use the Wikitext2 dataset as calibration data;

[0010] The first stage PTQ process is executed to perform weight transformation: activate quantization, define a quaternary quantization function, perform multi-parameter grid search, and obtain the optimal weight scaling factor and dual thresholds.

[0011] The second-stage PTQ process is executed to optimize the weights: the block reconstruction loss is defined, greedy iterative quantization is performed, and a quaternary weighted compression model is generated.

[0012] Furthermore, the pre-trained large language models use LLaMA-2, OPT, or Mistral series, with parameter sizes ranging from 1.3B to 70B.

[0013] Furthermore, in the weight transformation stage, 128 samples containing 512 tokens each are used. The weight transformation specifically includes:

[0014] (1.1) Activation Quantization: The activation X input to the model is quantized using 8-bit integers. The quantization formula is as follows:

[0015]

[0016] in, 8-bit quantization activation, where X is the activation value; β is the activation quantization scaling factor, calculated based on the activation value range of each token: β = (max(X) - min(X)) / 255; round(·) represents the approximation function, approximating the decimal to the integer; clip(,-128,127) represents the truncation function, replacing numbers less than -128 with -128 and numbers greater than 127 with 127;

[0017] (1.2) Define the quaternion quantization function: Define the quaternion quantization mapping relationship Q(·), and divide the weights into 4 discrete values ​​by double thresholds, with +1, +0.5, -1, and -0.5 used as the quaternion values:

[0018]

[0019] Where W represents the full-precision weights of the large language model, Δ1 is the high threshold used to distinguish between +1 and +0.5, and between -1 and -0.5; Δ2 is the low threshold used to distinguish between ±0.5 and the weight zero region.

[0020] (1.3) Perform a multi-parameter grid search to obtain the optimal weight scaling factor and double threshold:

[0021] Based on the principle of "weight-activation scaling equivalence", the optimal weight scaling factor s, dual thresholds Δ1 and Δ2, and optimal weight projection factor α are searched simultaneously through hierarchical optimization objectives. The optimization objective is:

[0022]

[0023] Where W is the full-precision weight of the large language model; α is the weight projection factor, used to match the numerical range of the quaternary quantization value and the original weight, with an initial value of 0.7×mean(|W|); F is the norm operation, and the above formula is the 2-norm; mean(·) represents the average function.

[0024] Further, in step (1.3), a 20-level grid is used to search for the optimal weight scaling factor s, the two thresholds Δ1 and Δ2, and the optimal weight projection factor α. The search range of the optimal weight scaling factor s is [0.8, 1.2], the search range of threshold Δ1 is [0.6×mean(|W|), 0.9×mean(|W|)], and the search range of threshold Δ2 is [0.3×mean(|W|), 0.5×mean(|W|)]. The parameter combination with the smallest layer output error is selected as the optimal result.

[0025] Furthermore, in the weight optimization phase, 512–1024 samples containing 2048 tokens each are used. The weight optimization specifically includes:

[0026] (2.1) Define block reconstruction loss:

[0027] Taking a Transformer block containing a self-attention layer and an MLP layer as the unit, the optimization objective is defined as minimizing the block output error before and after quantization. Adapting to quaternary quantization weights, the block reconstruction loss is defined as:

[0028]

[0029] in, This is the output function of the Transformer block; ε is the quantization perturbation. For the reconstructed quantized quaternion weights, These are the scaled full-precision weights.

[0030] (2.2) Perform greedy iterative quantization:

[0031] A large model network is used to process each Transformer block from front to back. For each Transformer block, the weight indexing and iterative quantization operations are repeated.

[0032] (2.3) Generate quantization model:

[0033] After the iterative quantization is completed, the model weights are quaternion values, the activation is in INT8 quantization format, forming the final compressed model. At the same time, the optimized weight scaling factor s, threshold Δ1, threshold Δ2 and weight projection factor α are stored for parameter recovery in the inference stage.

[0034] Further, in step (2.2), for each Transformer block, the weights that are 1 / 20 of the total weights in the block closest to the quantization distance are selected by weight index, and the selected weights are quantized to the nearest quaternion value according to the distance, and then the remaining weights are optimized; the above iterative quantization operation is repeated 20 times for each Transformer block.

[0035] Furthermore, the weight index specifically includes: calculating the Euclidean distance between each weight and the four quaternion values, and selecting the weight index with the smallest Euclidean distance. To ensure that the weight error is minimized in each quantization, where W i W is the i-th value in the full-precision weights; q It is a quaternion value, which is +1, +0.5, -0.5 or -1.

[0036] Furthermore, the iterative quantization specifically includes: dividing the total number D of weights within the Transformer block into 20 iterations of quantization, quantizing D / 20 weights each time, converting them from floating-point to +1, +0.5, -1, or -0.5; the remaining floating-point weights are optimized by the Adam optimizer with a learning rate of 0.002, while α is optimized with a learning rate of 1e-5 to minimize the block reconstruction loss.

[0037] According to another aspect of the present invention, a computer device is provided, the computer device including a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the large language model quaternary weight compression method as described above.

[0038] According to another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the large language model quaternion weight compression method as described above.

[0039] This invention provides a method, device, and storage medium for compressing quaternary weights in large language models. This method achieves quaternary weight compression of large models based on a two-stage post-training quantization process involving weight transformation and optimization. It can reduce the hardware cost of deploying large models, inference latency, and energy consumption, while ensuring model performance. Compared with existing technologies, this invention solves the technical problems of insufficient information retention in ternary quantization schemes, leading to significant performance loss in large models and low compression ratios in high-precision quantization. Attached Figure Description

[0040] The accompanying drawings, which form part of this specification, are provided to further illustrate embodiments of the invention and, together with the textual description, explain the principles of the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0041] Figure 1 A flowchart illustrating a large language model quaternary weight compression method according to a specific embodiment of the present invention is shown. Detailed Implementation

[0042] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present invention or its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0044] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention. It should also be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0045] like Figure 1 As shown in the figure, a method for compressing quaternary weights in a large language model is provided according to a specific embodiment of the present invention. The method includes:

[0046] Model and data preparation: Select a pre-trained large language model and use the Wikitext2 dataset as calibration data;

[0047] The first stage PTQ process is executed to perform weight transformation: activate quantization, define a quaternary quantization function, perform multi-parameter grid search, and obtain the optimal weight scaling factor and dual thresholds.

[0048] The second-stage PTQ process is executed to optimize the weights: the block reconstruction loss is defined, greedy iterative quantization is performed, and a quaternary weighted compression model is generated.

[0049] This configuration provides a method for compressing the four-gram weights of large language models. This method achieves the compression of the four-gram weights of large models based on a two-stage post-training quantization of weight transformation and weight optimization. It can be used to reduce the deployment hardware cost, inference latency and energy consumption of large models, while ensuring model performance.

[0050] First, the model and data are prepared: a pre-trained large language model is selected, and the Wikitext2 dataset is used as calibration data. The specific data used in this invention is language text.

[0051] In this invention, the pre-trained large language model can be LLaMA-2, OPT, or the Mistral series, with a parameter scale of 1.3B to 70B.

[0052] The hardware and framework configuration of this invention is as follows: based on the PyTorch framework, a single NVIDIA A100 GPU with 80GB of video memory can complete the quantization process.

[0053] Furthermore, in this invention, the first-stage PTQ process is performed: weight transformation is carried out. Weight transformation optimizes the weight distribution and reduces quantization loss.

[0054] In the weight transformation phase, 128 samples containing 512 tokens each are used. The specific weight transformation steps include:

[0055] (1.1) Activation Quantization (INT8): The activation X of the model input is quantized using an 8-bit integer (INT8). The quantization formula is as follows:

[0056]

[0057] in, Activation is 8-bit quantization, where X is the activation value; β is the activation quantization scaling factor, calculated based on the activation value range of each token: β = (max(X) - min(X)) / 255; round(·) represents the approximation function, which approximates the decimal to an integer; clip(,-128,127) represents the truncation function, which replaces numbers less than -128 with -128 and numbers greater than 127 with 127.

[0058] (1.2) Define the quaternion quantization function: Define the quaternion quantization mapping relationship Q(·), and divide the weights into 4 discrete values ​​by double thresholds, with +1, +0.5, -1, and -0.5 used as the quaternion values:

[0059]

[0060] Where W represents the full-precision weights of the large language model, Δ1 is the high threshold (distinguishing between ±1 and ±0.5), and Δ2 is the low threshold (distinguishing between ±0.5 and the zero weight region; in practical applications, scaling is used to make the weights mainly distributed in ±1 and ±0.5, reducing the proportion of zero values ​​to retain information).

[0061] (1.3) Perform a multi-parameter grid search to obtain the optimal weight scaling factor and double threshold:

[0062] Based on the principle of "weight-activation scaling equivalence" ( (where s is the weight scaling factor). The optimal weight scaling factor s, the two thresholds Δ1 and Δ2, and the optimal weight projection factor α are searched simultaneously through hierarchical optimization objectives. The optimization objective is:

[0063]

[0064] Where W is the full-precision weight of the large language model, α is the weight projection factor (used to match the numerical range of the quaternary quantization value and the original weight), and the initial value is set to 0.7×mean(|W|); F is the norm operation, which is the 2 norm here, and mean(·) represents the average function.

[0065] This invention employs a 20-level grid (dividing the search range into 20 equal parts) to search for the optimal weight scaling factor s (search range [0.8, 1.2]), threshold Δ1 (search range [0.6×mean(|W|), 0.9×mean(|W|)]), and threshold Δ2 (search range [0.3×mean(|W|), 0.5×mean(|W|)]), selecting the parameter combination with the smallest layer output error.

[0066] By adjusting the above parameters, the weight distribution can be concentrated around the four values ​​of ±1 and ±0.5, reducing the impact of extreme values ​​and noise on quantization and laying the foundation for subsequent weight optimization.

[0067] Furthermore, in this invention, a second-stage PTQ process is performed: weight optimization. Through iterative quantization and loss minimization operations, the performance of the large model is further guaranteed.

[0068] In the weight optimization phase, 512 to 1024 samples containing 2048 tokens each were used. The specific weight optimization steps included:

[0069] (2.1) Define the block reconstruction loss: Taking a Transformer block (including the self-attention layer and the MLP layer) as the unit, the optimization objective is defined as minimizing the block output error before and after quantization. Adapting to quaternary quantization weights, the block reconstruction loss is defined as:

[0070]

[0071] in, This is the output function of the Transformer block; ε is the quantization perturbation. For the reconstructed quantized quaternion weights, These are the scaled full-precision weights.

[0072] (2.2) Perform greedy iterative quantization:

[0073] A large model network is used to process each Transformer block sequentially from beginning to end. For each Transformer block, the weight indexing and quantization operations are repeated. In this invention, each block undergoes 20 iterations of quantization, specifically:

[0074] First, select 1 / 20 of the total weights within the block closest to the nearest quantization value using the weight index. Then, quantize the selected weights to the nearest quaternion value according to the distance. Finally, optimize the remaining weights. Repeat the above process 20 times.

[0075] Specifically, for weight index selection: calculate the Euclidean distance between each weight and the four quaternion values ​​(±1, ±0.5), and select the weight index with the smallest Euclidean distance. (W i W is the i-th value in the full-precision weights; q The values ​​are quaternions (+1, +0.5, -0.5, or -1) to ensure that the weight error is minimized in each quantization.

[0076] For batch quantization: the total number of weights D in the block is divided into 20 iterations of quantization, each time quantizing D / 20 weights (converting from floating point to ±1 / ±0.5), the remaining floating point weights are optimized by the Adam optimizer (learning rate 0.002), and α (learning rate 1e-5) is optimized to minimize the block reconstruction loss.

[0077] (2.3) Generate quantization model: After the iteration, the model weights are quaternion values ​​(±1,±0.5), activated in INT8 quantization format, forming the final compressed model. At the same time, the optimized weight scaling factor s, threshold Δ1, threshold Δ2 and weight projection factor α are stored for parameter recovery in the inference stage.

[0078] The large language model quadruple weight compression method of this invention innovatively introduces a quadruple weight (±1, ±0.5) quantization strategy through a two-stage PTQ process of "weight transformation-weight optimization". Combined with 8-bit activation quantization, it can achieve a high compression ratio (close to ternary quantization) and high performance retention (better than ternary quantization) for large models of up to 70B, thus achieving a balance between efficient compression and performance retention of large models. At the same time, it reduces deployment costs while meeting the accuracy requirements of complex tasks.

[0079] Compared with existing technologies, the beneficial effects of the large language model quadruple weight compression method of this invention are as follows:

[0080] (1) Adaptability to ultra-large models:

[0081] No need to train from scratch; a single 80GB GPU can complete the quantization of a 70B parameter model, breaking through the BitNet 3.9B parameter size limit.

[0082] (2) Balancing hardware efficiency:

[0083] Memory compression: The memory of the 70B model was reduced from 132GB (FP16) to 22GB (quadruple weights require 2 bits of storage, slightly higher than the 1.58 bits of ternary weights), with a compression ratio of 5.91 times, close to the 6.28 times of ternary BitNet; the 7B model was reduced from 12.6GB to 2.3GB, with a compression ratio of 5.48 times, and the hardware deployment cost was still significantly lower than that of the full-precision model.

[0084] Reduced energy consumption: Because quaternary quantization is still low precision, the energy consumption of linear layer computation is only 11% of that of the full-precision model (based on simulation of 45nm CMOS technology, the energy consumption of full-precision MAC is 4.6pJ, while this invention requires 0.51pJ).

[0085] According to another aspect of the present invention, a computer device is provided, the computer device including a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the large language model quaternary weight compression method as described above.

[0086] According to another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the large language model quaternion weight compression method as described above.

[0087] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for four-gram weighted compression of large language models, characterized in that, The large language model quadruple weight compression method includes: Model and data preparation: Select a pre-trained large language model and use the Wikitext2 dataset as calibration data; The first-stage PTQ process is executed, and weight transformation is performed: activation quantization, definition of quaternary quantization function, multi-parameter grid search is performed, and the optimal weight scaling factor and dual threshold are obtained. The second-stage PTQ process is executed to optimize the weights: the block reconstruction loss is defined, greedy iterative quantization is performed, and a quaternary weight compression model is generated.

2. The large language model quadruple weight compression method according to claim 1, characterized in that, Pre-trained large language models use LLaMA-2, OPT, or Mistral series, with parameter sizes ranging from 1.3B to 70B.

3. The method for compressing large language models using quadruple weights according to claim 1, characterized in that, The weight transformation phase uses 128 samples, each containing 512 tokens. The weight transformation specifically includes: (1.1) Activation Quantization: The activation X input to the model is quantized using 8-bit integers. The quantization formula is as follows: in, 8-bit quantization activation, where X is the activation value; β is the activation quantization scaling factor, calculated based on the activation value range of each token: β = (max(X) - min(X)) / 255; round(·) represents the approximation function, approximating the decimal to the integer; clip(,-128,127) represents the truncation function, replacing numbers less than -128 with -128 and numbers greater than 127 with 127; (1.2) Define the quaternion quantization function: Define the quaternion quantization mapping relationship Q(·), and divide the weights into 4 discrete values ​​by double thresholds, with +1, +0.5, -1, and -0.5 used as the quaternion values: Where W represents the full-precision weights of the large language model, Δ1 is the high threshold used to distinguish between +1 and +0.5, and between -1 and -0.5; Δ2 is the low threshold used to distinguish between ±0.5 and the weight zero region. (1.3) Perform a multi-parameter grid search to obtain the optimal weight scaling factor and dual thresholds: Based on the principle of "weight-activation scaling equivalence", the optimal weight scaling factor s, dual thresholds Δ1 and Δ2, and optimal weight projection factor α are searched simultaneously through hierarchical optimization objectives. The optimization objective is: Where W is the full-precision weight of the large language model; α is the weight projection factor, used to match the numerical range of the quaternary quantization value and the original weight, with an initial value of 0.7×mean(|W|); F is the norm operation, and the above formula is the 2-norm; mean(·) represents the average function.

4. The large language model quadruple weight compression method according to claim 3, characterized in that, In step (1.3), a 20-level grid is used to search for the optimal weight scaling factor s, the two thresholds Δ1 and Δ2, and the optimal weight projection factor α. The search range of the optimal weight scaling factor s is [0.8, 1.2], the search range of threshold Δ1 is [0.6×mean(|W|), 0.9×mean(|W|)], and the search range of threshold Δ2 is [0.3×mean(|W|), 0.5×mean(|W|)]. The parameter combination with the smallest layer output error is selected as the optimal result.

5. The large language model quadruple weight compression method according to claim 3, characterized in that, During the weight optimization phase, 512 to 1024 samples containing 2048 tokens each were used. The weight optimization specifically included: (2.1) Define block reconstruction loss: Taking a Transformer block containing a self-attention layer and an MLP layer as the unit, the optimization objective is defined as minimizing the block output error before and after quantization. Adapting to quaternary quantization weights, the block reconstruction loss is defined as: in, This is the output function of the Transformer block; ε is the quantization perturbation. For the reconstructed quantized quaternion weights, These are the scaled full-precision weights. (2.2) Perform greedy iterative quantization: A large model network is used to process each Transformer block from front to back. For each Transformer block, the weight indexing and iterative quantization operations are repeated. (2.3) Generate quantization model: After the iterative quantization is completed, the model weights are quaternion values, the activation is in INT8 quantization format, forming the final compressed model. At the same time, the optimized weight scaling factor s, threshold Δ1, threshold Δ2 and weight projection factor α are stored for parameter recovery in the inference stage.

6. The method for compressing large language models using quadruple weights according to claim 5, characterized in that, In step (2.2), for each Transformer block, the weights that are 1 / 20 of the total weights in the block closest to the quantization distance are selected by weight index, and the selected weights are quantized to the nearest quaternion value according to the distance. Then the remaining weights are optimized. The above iterative quantization operation is repeated 20 times for each Transformer block.

7. The method for compressing large language models using quadruple weights according to claim 6, characterized in that, The weight indexing process specifically involves: calculating the Euclidean distance between each weight and its four quaternions, and selecting the weight index with the smallest Euclidean distance. To ensure that the weight error is minimized in each quantization, where W i W is the i-th value in the full-precision weights; q It is a quaternion value, which is +1, +0.5, -0.5 or -1.

8. The method for compressing large language models using quadruple weights according to claim 7, characterized in that, The iterative quantization specifically includes: dividing the total number of weights D in the Transformer block into 20 iterations of quantization, each time quantizing D / 20 weights, converting them from floating-point to +1, +0.5, -1, or -0.5; the remaining floating-point weights are optimized by the Adam optimizer with a learning rate of 0.002, while α is optimized with a learning rate of 1e-5 to minimize the block reconstruction loss.

9. A computer device, the computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the large language model quadruple weight compression method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the large language model quadruple weight compression method as described in any one of claims 1 to 8.

Images