A large model smoothing quantization method and system based on learnable rotation matrix kronecker decomposition

Abstract

This invention discloses a method and system for smoothing quantization of large models based on learnable rotation matrix Kronecker integral solutions, relating to the field of artificial intelligence model compression technology. The method includes: constructing a search space containing Kronecker integral solutions of different numbers of matrices, with the tensor-matrix i-th modal product as the computational implementation method for each solution; adaptively selecting the optimal rotation matrix hybrid decomposition scheme for the attention modules and feedforward networks of each decoder layer of the large model through a preset scoring function and layer-by-layer greedy heuristic search; training the complete parameters of the decomposition matrix, learnable weights, activation value truncation threshold, and diagonal scaling matrix parameters of the scheme; fusing the extra parameter matrix on the weight side of the linear layer, retaining the extra parameter matrix on the activation value side, and quantizing the model based on the trained quantization parameters to generate a deployable model. This effectively solves the problems of accuracy loss and computational overhead under low-bit quantization of large models, significantly improving computational and storage efficiency while maintaining high accuracy.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence model compression technology, and more specifically to a method and system for smoothing quantization of large models based on the Kronecker integral solution of a learnable rotation matrix. Background Technology

[0002] With the rapid development of artificial intelligence technology, Large Language Models (LLMs) have made groundbreaking progress in natural language processing tasks. However, the huge parameter size and computational demands of LLMs during inference limit their deployment on resource-constrained devices. Model quantization techniques, which compress the model by converting high-precision floating-point numbers to low-precision integers such as INT4, are key to solving this problem. However, in LLMs, outliers are prevalent in activation values, meaning that the values ​​of a few channels are much larger than those of other channels, leading to a significant drop in accuracy during low-bit quantization.

[0003] Existing smoothing quantization methods are mainly divided into two categories: one is scaling methods, represented by SmoothQuant and OmniQuant, which only use diagonal matrices to adjust the relative amplitudes between channels. This method has low computational overhead, but at low bit depths, such as 4-bit quantization, its limited mathematical transformation capabilities make it unable to effectively smooth complex outlier distributions. The other category is rotation methods, represented by QuaRot and SpinQuant, which introduce orthogonal rotation matrices to distribute the energy of outliers across all dimensions. However, existing rotation methods have significant drawbacks: introducing a full-size learnable rotation matrix leads to a huge number of parameters and training overhead, making convergence difficult; using a fixed decomposition scheme, such as using only the Hadamard matrix or using a single decomposition structure across the entire model, ignores the significant differences in data distribution between different levels of the model, such as shallow and deep layers, attention modules and feedforward networks, lacking adaptability to local model characteristics, resulting in the quantized model's accuracy not reaching its optimal level.

[0004] Therefore, there is an urgent need for a large-model smoothing quantization technique that balances quantization accuracy, storage efficiency, and computational efficiency to address the shortcomings of existing technologies. Summary of the Invention

[0005] In view of the above problems, this invention is proposed to provide a large model smoothing quantization method and system based on the Kronecker integral solution of a learnable rotation matrix to overcome or at least partially solve the above problems, so as to solve the problems of insufficient accuracy, large number of parameters or poor adaptability of existing quantization methods in low bit scenarios, and to achieve efficient compression and deployment of large language models.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, embodiments of the present invention provide a large model smoothing quantization method based on the Kronecker integral solution of a learnable rotation matrix, comprising the following steps: Model construction: A search space containing multiple decomposition schemes is constructed using Kronecker products of different numbers of matrices. For each decomposition scheme in the search space, the tensor-matrix i-th modal product is used as its corresponding operation implementation. Model Training: Based on a preset scoring function, a layer-by-layer greedy heuristic search strategy is used to adaptively select the highest-scoring rotation matrix hybrid decomposition scheme for the attention module and feedforward network of each decoder layer of the large model; the complete parameters, learnable weight parameters, activation value truncation threshold parameters, and diagonal scaling matrix parameters of the decomposed matrix after the selected optimal decomposition scheme are trained. Quantization phase: For the trained matrix parameters, the extra parameter matrix of the linear layer weight side is fused into the weight, while the extra parameter matrix of the activation value side is retained normally. Then, based on the quantization parameters obtained from training, the model with the weight fusion is quantized to generate the final deployable model.

[0008] Furthermore, the search space is: the rotation matrix Decomposed into Smaller-sized matrices The Kronecker product, i.e.:

[0009]

[0010] in, The dimension is submatrix The dimension is And satisfy .

[0011] Furthermore, the operation of the tensor-matrix i-th modal product specifically includes the following steps: S121, Input Shaping: Shape the input shape as... Two-dimensional input matrix Logic shaping dimensional tensor , shape ; the shape is weight matrix Logic shaping dimensional tensor , shape ; S122, Sequence Modal Product Operation: Sequentially performing modal product operations on the tensor... To the In terms of dimension, using the corresponding submatrix The modal product is calculated using the following formula:

[0012]

[0013] in, Indicates the inverse of a matrix; Represents the tensor-matrix of the first Modal product, the first (1, N); ; and Both represent tensors with the same dimension as the original input tensor obtained after performing the corresponding modal product operation on the original input tensor. S123, Result Shaping: The shape after all modal product calculations are completed is... The tensor shaping return shape is From a two-dimensional matrix, we obtain the rotated input matrix. ; the shape is The tensor shaping return shape is From the two-dimensional matrix, we obtain the rotated weight matrix. .

[0014] Furthermore, the tensor-matrix modal product operation employs a general N-stage calculation method, including the following steps: Scalability of unified computing logic: through logical batch size dimension and additional dimensions The design logically transforms the input high-dimensional tensor into a shape of... Modal product calculation is performed on the three-dimensional tensor; Staged kernel calls: Based on unified computation logic, this is achieved by changing the logical batch size dimension and the input of additional dimensions. The process of tensor-matrix product calculation in different dimensions; Step-based logical slicing: The kernel addresses the input tensor by using the memory step size information of each dimension. The kernel dynamically calculates the access offset based on the input dimension and directly slices and reads and calculates the multidimensional tensor in the logical view.

[0015] Furthermore, the heuristic search strategy includes the following steps: S211, Layer-by-layer traversal: Process each decoder layer sequentially according to the forward propagation order of the model; S212. Candidate Solution Evaluation: For the current decoder layer, traverse all possible decomposition combinations in the search space. ,in The number of decompositions for the attention module. Let be the number of decompositions in the feedforward network, and The search space Defined as the number of decompositions The candidate set; S213. Rapid Iterative Evaluation: For each combination scheme, construct the corresponding number of initial rotation matrices. And perform a small amount of rapid iterative training; a scoring function based on reconstruction error and time delay factors. The hybrid decomposition scheme that yields the highest score for the current layer is selected. During the evaluation process, tensor-matrix modal product operations are used to perform efficient forward computation and parameter updates on the input data. S214. Optimal Decision Making: Utilizing Scoring Functions Calculate the score, compare the scores of all candidate combinations, and select the highest-scoring combination. This is the optimal hybrid decomposition scheme for the current layer.

[0016] Furthermore, the complete parameters, learnable weight parameters, activation truncation threshold parameters, and diagonal scaling matrix parameters of the selected optimal decomposition scheme are trained, including the following steps: S221. Optimization Objective Modeling: Construct an optimization objective function for each layer:

[0017] in, The parameter matrix set formed after rotation matrix decomposition This is the introduced diagonal scaling factor. These are the learnable weights and activation value truncation thresholds, respectively. and They represent the first X is the input matrix; S222, Forward Propagation and Parameter Update: Forward computation is performed using the substitution operation form of the tensor matrix product based on the decomposed matrix. For the input matrix... and weight Calculate the results after rotation:

[0018]

[0019] in, Indicates along the first Tensor-matrix product performed in dimensionality; S223, Inter-layer progression: After the current layer is trained, the quantized output is used as the input of the next layer, and the above process is repeated until the entire model is trained.

[0020] Furthermore, the quantification stage specifically includes the following steps: (1) Integrate the rotation matrix parameters of the trained linear layer weights into the original model weights. Let the tensor of the original weights after reshaping be... Then the weight after fusion The specific calculation method is as follows: ; (2) The rotation matrix parameters located on the activation side are retained as online computation operators; (3) Based on the weight and activation value truncation threshold parameters obtained during training, uniform quantization is performed on the fused weights and activation values. The uniform quantization process is defined by the following formula: First, for floating-point values Truncation:

[0021] in, The cutoff threshold corresponding to a pair of weights or activation values, i.e. or ; Then, quantization-dequantization with a truncation threshold is performed, the process defined as follows:

[0022]

[0023] in, As a scaling factor, Zero point and These are the lower and upper bounds of the range of quantized integers, respectively:

[0024]

[0025] Finally, a large-scale model with quantification is obtained.

[0026] Secondly, embodiments of the present invention provide a large model smoothing quantization system based on a learnable rotation matrix Kronecker integral solution. Using the above-described method, the system includes: Model building module: used to construct a search space containing multiple decomposition schemes using Kronecker products of different numbers of matrices. For each decomposition scheme in the search space, the tensor-matrix i-th modal product is used as its corresponding operation implementation. Model training module: Based on a preset scoring function, it uses a layer-by-layer greedy heuristic search strategy to adaptively select the highest-scoring rotation matrix hybrid decomposition scheme for the attention module and feedforward network of each decoder layer of the large model; and trains the complete parameters, learnable weight parameters, activation value truncation threshold parameters, and diagonal scaling matrix parameters of the decomposed matrix after the selected optimal decomposition scheme. The quantization phase module is used to fuse the extra parameter matrix of the linear layer weights into the weights after training, while retaining the extra parameter matrix of the activation value side as normal. Then, based on the quantization parameters obtained during training, the model with completed weight fusion is quantized to generate the final deployable model.

[0027] Thirdly, embodiments of the present invention provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing, implements the above-mentioned spatiotemporal feature fusion ultra-short-term photovoltaic power generation prediction method.

[0028] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program thereon, characterized in that the program, when executed by a processor, implements the above-described spatiotemporal feature fusion ultra-short-term photovoltaic power generation prediction method.

[0029] The beneficial effects of the above-described technical solutions provided in the embodiments of the present invention include at least the following: 1. High precision: By introducing a learnable truncation threshold and rotation matrix for joint optimization, the ability to suppress outliers in long-tailed distributions is further enhanced, making the method robust even in low-bit quantization scenarios such as bit.

[0030] 2. High Adaptability: This invention breaks away from the limitation of traditional methods that use a single transformation structure for all layers. Through a hybrid decomposition strategy, it can adaptively allocate transformation complexity, i.e., the number of decompositions, based on the differences in the sensitivity of the attention module and the feedforward network to quantization errors. With the same quantization bit width, compared to scaling transformation methods and rotation transformation methods with full parameters, it significantly reduces quantization errors and improves the model's performance in downstream tasks such as commonsense reasoning.

[0031] 3. High storage efficiency: By using the Kronecker product, the large number of parameters of the rotation matrix is ​​transformed into multiple small matrix parameters, and the weights and activation values ​​are further quantized to low bits, which reduces the storage requirements of the model. In addition, compared with other quantization methods that directly introduce large-size rotation matrices, it significantly accelerates the parameter convergence speed in the calibration stage.

[0032] 4. Computationally efficient: The decomposed matrix proposed in this invention is based on the alternative operation form of tensor matrix product, which avoids the explicit construction of huge rotation matrices and performs operations directly on the tensor modes. Compared with the quantization method that directly introduces large-size rotation matrices, it can significantly reduce the amount of computation in the smoothing process. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0034] Figure 1 This is an overall flowchart provided in an embodiment of the present invention.

[0035] Figure 2 This is a schematic diagram of the overall architecture for smooth quantization of the LLaMA3 model embedded with a rotation matrix, as provided in an embodiment of the present invention.

[0036] Figure 3 This is a logical diagram illustrating the alternative operation form based on tensor matrix product when N=4, provided in an embodiment of the present invention.

[0037] Figure 4 This is a schematic diagram of the preferred search space for the heuristic hybrid scheme provided in the embodiments of the present invention.

[0038] Figure 5 This is a schematic diagram of the preferred search process for the heuristic hybrid scheme provided in the embodiments of the present invention.

[0039] Figure 6 This is a schematic diagram illustrating how the truncation threshold provided in this embodiment of the invention is used to improve the accuracy of the numerical representation after quantization.

[0040] Figure 7 This is a structural diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] Example 1: This invention discloses a large-model smoothing quantization method based on the Kronecker integral solution of a learnable rotation matrix. The core idea is that for a rotation matrix introduced into a large model, a heuristic search based on a hybrid decomposition scheme using the Kronecker product, along with learning relevant matrix parameters and quantization parameters, smooths the data distribution of weights and activation values, thereby reducing the overall quantization error during weight-activated quantization. The overall process of the invention is as follows: Figure 1 As shown, this method consists of three stages: model building, model training, and quantization.

[0043] Step S100, model building, also known as search space construction, includes two main steps: Step S110: Construct the Kronecker integral form of the rotation matrix: Construct a rotation matrix for smoothing the parameter distribution inside a large model using Kronecker products of different numbers of matrices. Step S120: Construct an alternative operation form of the decomposed matrix based on the tensor matrix product: For the decomposed matrix in S110, the operation of the original Kronecker product form is replaced by the tensor-matrix i-th modal product, thereby accelerating the operation.

[0044] In the two steps described above, step S110 models rotation matrix decomposition schemes with different smoothing capabilities for the internal data distribution of the model, and step S120 models acceleration schemes with different computational efficiency for each decomposition scheme. Together, they constitute the search space for the heuristic search training phase.

[0045] Step S200, model training, also known as the heuristic search and training phase, includes two main steps: Step S210, Heuristic Hybrid Scheme Optimization Search: Based on the scoring function, a layer-by-layer greedy heuristic search strategy is used to adaptively select the rotation matrix hybrid decomposition scheme with the highest score for the attention module and feedforward network of each decoder layer of the large model. Step S220: Complete training of the matrix parameters of the optimal hybrid decomposition scheme: Training the complete parameters of the matrix after decomposition of the optimal decomposition scheme obtained in S210, as well as the training of learnable weights, activation value truncation threshold parameters, and diagonal scaling matrix parameters.

[0046] The two steps described above evaluate and train different rotation matrix decomposition schemes for the target large model, and combine them with learnable quantization parameters such as truncation thresholds. This can significantly suppress the long-tail distribution problem of activation values ​​in large models. Under the premise of balancing the two requirements of reducing computational and storage overhead and smoothing outliers in activation values, relevant parameters for high-precision low-bit quantization are obtained.

[0047] Step S300, the quantization stage, also known as the parameter fusion and quantization stage, includes a main step: Step S310, Weight Parameter Fusion and Model Quantization: For the matrix parameters trained in S220, the extra parameter matrix of the linear layer weight side is fused into the weight, thereby reducing some matrix multiplication operations, while the extra parameter matrix of the activation value side is retained normally; then, based on the quantization parameter information obtained in S220, the large model after weight fusion is quantized using the nearest neighbor rounding method.

[0048] This invention constructs a search space containing multiple decomposition schemes by introducing a rotation matrix based on learnable Kronecker integral solutions, and employs tensor-matrix modal product for efficient computation. Building upon this, a layer-by-layer greedy heuristic search strategy adaptively selects the optimal hybrid decomposition scheme for different modules of each decoder layer, and jointly optimizes the rotation matrix, truncation threshold, and scaling factor to suppress outliers. Finally, a deployable model is generated through weight-side parameter fusion and low-bit quantization. This method maintains high model accuracy even with low-precision quantization such as 4-bit, significantly reduces computational and storage overhead during training and inference, and possesses strong adaptability and deployment friendliness, effectively solving the core challenge of deploying large language models on resource-constrained devices.

[0049] The following is a detailed description of the large model smoothing quantization method based on the Kronecker integral solution of a learnable rotation matrix of the present invention: Step S100, Model Construction, includes two main steps: Step S110: Construct the Kronecker integral form of the rotation matrix for linear layer operations within the decoder layer of the large language model. ,in Represents the original two-dimensional input matrix. The weight matrix represents the linear layer; paired invertible parameter matrices are introduced. The rotated input matrix is ​​obtained. and the rotated weight matrix Thus, a rotation matrix decomposition model based on the Kronecker product is constructed. .in, .

[0050] Existing methods perform orthogonal transformations on activation values ​​and weights, and high-dimensional rotations on the input matrix, thereby achieving a more uniform distribution of values ​​across the input matrix's channels. The parameter matrix used here... Building upon this foundation, the matrix orthogonality constraint is broken. By utilizing invertible transformations and specifically learning the distribution characteristics of model weights and activation values, quantization errors can be further reduced, achieving high-precision quantization.

[0051] Let the original large-scale rotation matrix be... For the set of real numbers, This indicates that the matrix has n rows and n columns. To reduce the number of parameters introduced by the additional rotation matrix and thus reduce training costs, this invention decomposes it into... Smaller-sized matrices The Kronecker product, i.e.:

[0052]

[0053] in, The dimension is submatrix The dimension is And satisfy:

[0054] To address the uncertainty in the number of decomposition matrices required to achieve the lowest quantization error when decomposing and replacing rotation matrices in different large models, each decoder layer in the model is divided into two sub-modules: an attention module and a feedforward network. The number of decomposition matrices is then selected at this granularity.

[0055] Search space Defined as the number of decompositions Due to different decomposition quantities The manifold dimensions of the corresponding transformation matrices are different. The more decompositions there are, the fewer parameters there are, but the more limited the representational power. This step provides independent decomposition scheme selection rights for the attention module and feedforward network of each decoder layer, thereby forming a search space for hybrid decomposition schemes of the rotation matrix.

[0056] Preferably, the search space It can usually balance the data smoothing effect of algorithm implementation with the difficulty of computational acceleration.

[0057] This decomposition scheme retains the effectiveness of rotational smoothing in improving quantization accuracy while avoiding the introduction of a large number of trainable parameters, which would lead to long training times and slow convergence.

[0058] Step S120: Construct an alternative operation form of the decomposed matrix based on the tensor matrix product, in order to avoid explicitly constructing a full-size rotation matrix with a large number of parameters during the calculation process. This step replaces matrix multiplication. The Kronecker product is further transformed into a series of modal product operations of small matrices, thereby reducing memory or GPU memory usage and facilitating faster computation after the substitution. The substitution operation includes the following logical steps: Step S121, Input shaping: Shape the object as... Two-dimensional input matrix Logic shaping dimensional tensor , shape Similarly, the shape is weight matrix Logic shaping dimensional tensor , shape .

[0059] Step S122, Sequence Modal Product Operation: Perform the following operations sequentially on the tensor's... To the In terms of dimension, using the corresponding submatrix Perform modal product calculation. The formula is:

[0060]

[0061] in: (1) This represents the inverse of a matrix; it is used for efficient computation of the inverse of a rotation matrix. For each decomposition small matrix Perform singular value decomposition, i.e. ,but This avoids the numerical instability of direct inversion.

[0062] (2) Symbols Represents the tensor-matrix of the first This operation does not change the number of dimensions of a tensor. In the case of the tensor-matrix first modal product, let the tensor be... ,matrix The first modal product of the two is denoted as Then, at this time, the corresponding input tensor The operation is performed on the second dimension of the tensor. The rules for operations on each element are as follows:

[0063] in These represent the elements at the corresponding indices of a tensor or matrix. It can be observed that a tensor... It's still a three-dimensional tensor, and tensors same.

[0064] (3) and Both represent tensors with the same dimension as the original input tensor obtained after performing the corresponding modal product operation on the original input tensor.

[0065] Step S123, Result Shaping: The shape after calculating all modal products is as follows: The tensor shaping return shape is From a two-dimensional matrix, we obtain the rotated input matrix. ; the shape is The tensor shaping return shape is From the two-dimensional matrix, we obtain the rotated weight matrix. .

[0066] To address the inefficiency caused by multiple kernel calls and GPU memory read / write operations in traditional deep learning frameworks, this invention designs a general-purpose... The phased calculation method, whose design features include: (1) Scalability of unified computation logic: When performing tensor-matrix product calculations, the scalability is achieved through the logical batch size dimension. and additional dimensions The design logically transforms the input high-dimensional tensor into a shape of... The three-dimensional tensor ensures logical consistency in the computation mode for each kernel call, supporting different numbers of... The decomposed matrix is ​​used for tensor-matrix product calculation, which has high scalability.

[0067] (2) Staged kernel call: By leveraging the uniformity of computational logic, the input of the logical batch size dimension and the extra dimension can be changed during tensor-matrix product calculation. The process of calculating the tensor matrix product in different dimensions.

[0068] (3) Step-based logical slicing: The kernel addresses the input tensor by using the memory step information of each dimension. The kernel dynamically calculates the access offset based on the input dimension and directly slices and reads and calculates the multidimensional tensor in the logical view. There is no need to perform actual reshaping or data rearrangement operations in physical video memory, thus eliminating the video memory copy overhead.

[0069] Step S200, model training, includes two main steps: Step S210, Heuristic Hybrid Scheme Optimization Search: This step employs a layer-by-layer heuristic optimization strategy to select the optimal combination of decomposition quantities for each module in the search space, thereby achieving high-precision quantization. The specific process is as follows: Step S211, Layer-by-layer traversal: Process each decoder layer sequentially according to the forward propagation order of the model.

[0070] Step S212, Candidate Solution Evaluation: For the current decoder layer, traverse all possible decomposition combinations in the search space. ,in It is the additional rotation matrix added to all linear layers in the attention module within the current decoder layer. The number of decompositions, It is the additional rotation matrix added to all linear layers in the feedforward network within the current decoder layer. The number of decompositions, and The search space Defined as the number of decompositions The candidate set, for example .

[0071] Step S213, Rapid Iterative Evaluation: For each combination scheme, construct the corresponding number of initial rotation matrices. And perform a small number of rapid iterative training iterations. The evaluation objective is to select the layer whose scoring function can achieve the best results during rapid iterative training. The highest-scoring hybrid decomposition scheme, where the scoring function is... A comprehensive evaluation is conducted, considering factors such as the quantified reconstruction error and latency. During the evaluation process, the decomposition results from step S120 are utilized. The matrix is ​​based on an alternative operation form of tensor matrix product, which takes the input... or The model is shaped into a high-dimensional tensor, and then based on the decomposition matrix group... The tensor is then subjected to tensor-matrix modal product operations sequentially along the corresponding dimensions to train the parameter matrix.

[0072] Step S214, Optimal Decision: Using a scoring function Calculate the score, compare the scores of all candidate combinations, and select the highest-scoring combination. This is the optimal hybrid decomposition scheme for the current layer.

[0073] Step S220: Complete training of the optimal hybrid decomposition scheme matrix parameters, based on the highest-scoring attention module and the number of rotation matrix decompositions within the feedforward network in step S210. We perform complete parameter learning on the rotation matrix under the optimal hybrid decomposition scheme to obtain the final quantization parameters. Hybrid schemes with other decomposition numbers are discarded, and the parameters under these schemes are no longer learned.

[0074] Step S221, Optimize objective modeling: Construct an optimization objective function for each layer:

[0075] in, The parameter matrix set formed after rotation matrix decomposition This is the introduced diagonal scaling factor. These are the learnable weights and activation value truncation thresholds, respectively. and They represent the first The outputs of each decoder layer before and after quantization are analyzed. These parameters are then jointly updated using a gradient descent algorithm until the model converges. This process suppresses long-tailed data distributions, improves model quantization accuracy, and ultimately yields the quantized model weights and parameters.

[0076] Step S222, Forward Propagation and Parameter Update: Using the decomposition from step S120... The matrix performs forward computation based on the substitutional form of the tensor matrix product. For the input matrix... and weight Calculate the results after rotation:

[0077]

[0078] in Indicates along the first The tensor-matrix product is performed in dimensionality. The quantization error is calculated based on the optimization objective, and the matrix parameters are updated accordingly.

[0079] Step S223, Inter-layer progression: After the current layer is trained, the quantized output is used as the input of the next layer. The above process is repeated until the entire model is trained.

[0080] Step S300, the quantification stage, includes one main step: Step S310, Weight Parameter Fusion and Model Quantization: To maintain mathematical equivalence during the inference phase and eliminate the additional computational overhead caused by some rotation transformations, this step performs offline weight parameter fusion. Based on the position of the rotation matrix in the model structure, it is divided into "online rotation matrices" and "fusionable rotation matrices." For linear layers... ,in A portion becomes the new linear layer weights, and this portion of the rotation matrix can be incorporated into the linear layer weights; Since the parts and weights are quantized independently, it's impossible to fuse the rotation matrix of one part with the others to form an online rotation matrix. For fusionable rotation matrices, the original model weights are updated using the corresponding inverse transpose matrix:

[0081] By incorporating the rotation matrix parameters into the model weights, the overall numerical distribution of the model's weights and activation values ​​generated during inference can be ensured to be more uniform, resulting in higher accuracy and smaller errors at low-bit quantization.

[0082] After completing the rotation matrix parameter learning and weight fusion, this step performs low-bit quantization, such as 4-bit quantization, on the smoothed activation values ​​and weights. To further suppress residual outliers and minimize quantization error, this step uses the learned weight and activation value truncation threshold parameters obtained in step 4 to truncate the weights and activation values ​​respectively. The truncation operation is specifically as follows:

[0083] in, The cutoff threshold corresponding to a pair of weights or activation values, i.e. or .

[0084] This step employs a uniform quantization strategy for a given floating-point value. The quantization-dequantization process using the applied truncation threshold is defined as follows:

[0085]

[0086] in, As a scaling factor, Zero point and These are the lower and upper bounds of the range of quantized integers, respectively:

[0087]

[0088] Finally, a large-scale model with quantification is obtained.

[0089] The following example illustrates the construction and training process of this invention: This invention provides a large-scale model smoothing quantization method based on the Kronecker integral solution of a learnable rotation matrix. This method is not dependent on a specific large language model architecture and is applicable to various deep neural network models based on the Transformer architecture. The following, in conjunction with the accompanying drawings, provides a detailed explanation of the computational details, parameter optimization logic, and specific execution flow of this invention on the LLaMA3 model.

[0090] Step S110: Construct the Kronecker integral form of the rotation matrix: To address the issue of excessively large parameters in a full-size rotation matrix, this invention first constructs a model incorporating a rotation matrix based on a structured decomposition using the Kronecker product. For any dimension of the model... The linear layer, and the corresponding rotation matrix introduced. Characterized as Low-dimensional factor matrix The direct product form, take :

[0091] in, And satisfy This decomposition reduces parameter complexity from... Reduce to .

[0092] for Condition, ;for Condition, ;for Condition, .

[0093] The overall architecture diagram of the LLaMA3 model after embedding the above method of transforming and smoothing activation values ​​and weights using rotation matrices is shown below. Figure 2 As shown, where All are decomposable rotation matrices, corresponding to the ones in the above formula. .

[0094] Step S120: Construct an alternative operation form for the decomposed matrix based on the tensor matrix product: In actual calculations, to avoid explicitly constructing huge matrices This leads to memory overflow. This invention employs an alternative computational form based on tensor matrix product to perform the calculation. Specifically, for the input activation value matrix... Reshape it into a dimension of High-dimensional tensors ,in To determine the batch size, a linear transformation is performed sequentially along each mode using tensor-matrix product:

[0095] This method of calculation is mathematically equivalent to However, it significantly reduces the number of floating-point operations and intermediate storage requirements.

[0096] Similarly, for the input weight matrix Shape it into High-dimensional tensors (in Batch size; and (Irrelevant) Perform linear transformations along each mode sequentially using tensor-matrix product:

[0097] for Condition, , , , ; for Condition, , , , ; for Condition, , , , .

[0098] in: (1) This represents the inverse of a matrix; it is used for efficient computation of the inverse of a rotation matrix. For each decomposition small matrix Perform singular value decomposition, i.e. ,but This avoids the numerical instability of direct inversion.

[0099] (2) Symbols Represents the tensor-matrix of the first Modal product, convention The first corresponding input tensor This operation does not change the number of dimensions of a tensor. In the case of the tensor-matrix first modal product, let the tensor be... ,matrix The first modal product of the two is denoted as Then, at this time, the corresponding input tensor The operation is performed on the second dimension of the tensor. The rules for operations on each element are as follows:

[0100] in These represent the elements at the corresponding indices of a tensor or matrix. It can be observed that a tensor... It's still a three-dimensional tensor, and tensors same.

[0101] (3) and Both represent tensors with the same dimension as the original input tensor obtained after performing the corresponding modal product operation on the original input tensor.

[0102] by For example, the first two steps of the operation logic based on the alternative operation form of tensor matrix product are as follows: Figure 3As shown in the figure, the first two steps of the stepwise modal product calculation of the algorithm are illustrated. The boxes above and below the equation represent the second and third dimensions, respectively. and The diagram shows the operations performed in the current step, with the yellow area representing the operands of the broadcast multiplication. After each modal product, the matrix is ​​logically reorganized. Different shaping processes were performed in different steps, among which...

[0103]

[0104] Through the above logical representation, each step of the 4-step tensor-matrix product can be represented as a single modal product:

[0105] in .

[0106] Step S210, Heuristic Hybrid Solution Optimization Search: Overall, the optimization process employs a layer-by-layer optimization strategy, where the selection of the hybrid scheme is based on the sub-decoder layer granularity, i.e., the number of rotation matrix decompositions applied to the attention modules and feedforward networks within each layer. Since the quantization error reflecting accuracy and the computational delay reflecting efficiency are usually mutually constrained, the optimization strategy needs to find the Pareto optimal solution layer by layer among multiple objectives.

[0107] This step employs a layer-by-layer heuristic optimization strategy to select the optimal combination of decomposition quantities for each module of the LLaMA3 model within the search space. The specific process is as follows: Step S211, Layer-by-layer traversal: Following the forward propagation order of the LLaMA3 model, process each decoder layer sequentially, based on the scoring function. The result is that the score of each layer is maximized layer by layer, and the combination scheme with the largest score in the current layer is greedily expanded, and a recursive search is performed. A schematic diagram of the search space constructed by this method is shown below. Figure 4 As shown, starting from the top, the corresponding modules are selected layer by layer. Each layer has K² possible solutions, of which those marked with an X (…). A node is defined as an extension of its parent node and is evaluated, but it fails in peer comparisons and is not extended further. For each decoder layer module in the model, this step first backs up the original floating-point weight parameters of the current layer to memory to ensure that subsequent evaluations of different decomposition schemes are independent and do not interfere with each other.

[0108] Step S212, Candidate Solution Evaluation: For each combination solution, the policy first restores the original weights of the layer and applies the current decomposition configuration. For the current decoder layer, iterates through all possible decomposition combinations in the search space. ,in Among them, the search space for setting the number of decompositions. Therefore, for each decoder layer, the search space for the hybrid decomposition scheme is the set of pairs (2,2), (2,3), (2,4), (3,2), (3,3), (3,4), (4,2), (4,3), (4,4). This corresponds to... Figure 4 Medium parameters .

[0109] Step S213, Rapid Iterative Evaluation: For each combination scheme, construct the corresponding number of initial rotation matrices. ( The training process involves a small number of rapid iterations. The training objective is to maximize the score of the current layer's scoring function.

[0110] The scoring function It can comprehensively evaluate factors such as quantization error and latency, in the form of

[0111] in, The model describes the impact of the current decomposition scheme on training and inference latency. The term models the current hybrid decomposition scheme within a finite number of heuristic iterations. The quantization error reached within the range. The parameter is used to balance the importance of the two. Specifically, when... When optimizing a hybrid scheme, only quantization loss is used as heuristic information. In this case, if the number of iterations is set... If the number of training iterations is the same as the total number of iterations, the heuristic algorithm degenerates into a fine-grained exhaustive search of training and evaluating quantization errors on a per-scheme basis; when In this case, the hybrid scheme is preferably evaluated solely based on calibration speed. However, this hybrid scheme degenerates into a single scheme, and its effectiveness depends only on the Kronecker product substitution acceleration kernel implementation for each decomposition quantity. By solving this optimization problem using a search method, a better overall decomposition optimization scheme for the rotation matrix can be obtained layer by layer, improving the learning efficiency of the rotation matrix parameters while reducing quantization error.

[0112] During the evaluation process, the decomposition in step S120 is utilized. The matrix is ​​based on an alternative operation form of tensor matrix product, which takes the input... or The model is shaped into a high-dimensional tensor, and then based on the decomposition matrix group... The tensor is then subjected to tensor-matrix modal product operations sequentially along the corresponding dimensions to train the parameter matrix.

[0113] Step S214, Optimal Decision: Compare the scores of all candidate combinations and select the scoring function. Calculate the highest-scoring combination This is the optimal hybrid decomposition scheme for the current layer.

[0114] The heuristic hybrid scheme optimization search process involved in this step is illustrated in the diagram below. Figure 5 As shown.

[0115] Step S220: Complete training of the optimal hybrid decomposition scheme matrix parameters: During the full training phase, the rotation matrix parameters will be trained and quantized based on the optimal decomposition scheme selected by the layer in the heuristic evaluation phase, and the rotation matrix parameters of the layer will be obtained in the end.

[0116] Based on the optimal decomposition scheme determined in step S210, this invention performs complete parameter training on the model. The goal of the training is to minimize the inter-layer quantization reconstruction error at each layer. The optimization process involves the following key technical details: (1) Inverse rotation calculation of the weight matrix: In order to maintain mathematical equivalence, when the activation value Left multiplication At that time, the weights must be adjusted. Left multiplication Under the Kronecker product, transforming the weight matrix requires calculating the inverse rest. To improve numerical stability and reduce computational complexity, this invention does not perform numerical analysis on each factor matrix. Instead of directly calculating the inverse, we use singular value decomposition to achieve a stable and faster solution:

[0117] In this way, the inverse rotation of weights can also be efficiently accomplished through tensor-matrix product operations.

[0118] (2) Multi-parameter joint learning: In addition to the rotation matrix In addition, this invention introduces a learnable diagonal scaling matrix. and dynamic truncation threshold During training, these parameters and the rotation matrix are jointly updated using the gradient descent algorithm to adapt to the long-tailed distribution of the activation values. The objective function of the optimization problem is defined as:

[0119] in This indicates quantization and dequantization operations. It is the set of all learnable parameters. The parameters are the rotation matrix after decomposition. This is the introduced diagonal scaling factor. These represent learnable weights and activation value truncation thresholds, respectively. By learning the weight and activation value truncation thresholds, quantization error can be further reduced. A schematic diagram illustrating how truncation thresholds improve the accuracy of quantized numerical representations is shown below. Figure 6 As shown, by concentrating the numerical ranges within data intervals that occur more frequently or have a greater impact on quantization loss, the accuracy of the quantized numerical representation can be improved. The above parameters are then jointly updated using the gradient descent algorithm until the model converges, ultimately yielding the quantized model weights and quantization parameter table.

[0120] (3) Forward propagation and parameter update: using the decomposition in step S120 The matrix performs forward computation based on the substitutional form of the tensor matrix product. For the input matrix... and weight Calculate the results after rotation:

[0121]

[0122] in Indicates along the first The tensor-matrix product is performed in dimensionality. The quantization error is calculated based on the optimization objective, and the matrix parameters are updated accordingly.

[0123] (4) Inter-layer progression: After the current layer is trained, the quantized output is used as the input of the next layer, and the above process is repeated until the entire model is trained.

[0124] Step S310, Weight Parameter Fusion and Model Quantization: After completing parameter training, this invention divides the processing flow into two parts based on whether the rotation matrices can be fused: (1) Offline parameter fusion: For rotation transformations between adjacent linear layers, such as the output projection of the previous layer and the input projection of the next layer, the learned decomposed rotation matrix is ​​used. The weights are directly absorbed into the model. In the process, new weights are generated. It is then quantized and stored. During the inference phase, this part does not introduce additional computational overhead.

[0125] Based on the structural characteristics of the LLaMA3 model, the specific fusion strategy is as follows: Attention module: Input rotation matrix inverse matrix Projected weights that are integrated into the query, key, and value. In the middle. The rotation matrix after the value vector. inverse matrix and the rotation matrix of the context vector after Softmax inverse matrix Combined into output projection weights middle.

[0126] Feedforward network: Input rotation matrix (with attention module) (Different) inverse matrix Integrating upward projection and gated projection weights In the middle, activate the rotation matrix. inverse matrix Integrating into the projection weights middle.

[0127] Fusion calculation based on decomposition matrix, in order to efficiently calculate the fused weights. This invention does not explicitly construct a full-size inverse matrix, but directly uses the decomposed submatrices for tensor operations. Let the tensor after the original weight reshaping be... Then the weight after fusion The specific calculation method is as follows:

[0128] In practice, firstly, for each submatrix Perform singular value decomposition, i.e. Then its inverse matrix can be quickly calculated as Then, the weights The logic is restructured into a high-dimensional tensor, and then sequentially connected with... Modal product operations are performed to complete parameter fusion.

[0129] (2) Online activation value rotation: For input activation value transformations that cannot be fused, a rotation operator based on modal product is implanted in the inference engine. During the quantization phase, the activation values ​​are rotated in real time. The calculation is then performed, followed by quantization. Thanks to the decomposition scheme, the latency of this online computation is kept to an extremely low level.

[0130] After weight parameter fusion, this step performs low-bit quantization, such as 4-bit quantization, on the smoothed activation values ​​and weights. To further suppress residual outliers and minimize quantization error, this step uses the learned weight and activation value truncation threshold parameters obtained in step 4 to truncate the weights and activation values ​​respectively. The truncation operation is specifically as follows:

[0131] in, The cutoff threshold corresponding to a pair of weights or activation values, i.e. or .

[0132] This step employs a uniform quantization strategy for a given floating-point value. The quantization-dequantization process using the applied truncation threshold is defined as follows:

[0133]

[0134] Where, Δ Zero point and These are the lower and upper bounds of the range of quantized integers, respectively:

[0135]

[0136] Finally, a large-scale model with quantification is obtained.

[0137] Example 2: Based on the same inventive concept, embodiments of the present invention also provide a large-model smoothing quantization system based on the Kronecker integral solution of a learnable rotation matrix, the system comprising: Model building module: used to construct a search space containing multiple decomposition schemes using Kronecker products of different numbers of matrices. For each decomposition scheme in the search space, the tensor-matrix i-th modal product is used as its corresponding operation implementation. Model training module: Based on a preset scoring function, it uses a layer-by-layer greedy heuristic search strategy to adaptively select the highest-scoring rotation matrix hybrid decomposition scheme for the attention module and feedforward network of each decoder layer of the large model; and trains the complete parameters, learnable weight parameters, activation value truncation threshold parameters, and diagonal scaling matrix parameters of the decomposed matrix after the selected optimal decomposition scheme. The quantization phase module is used to fuse the extra parameter matrix of the linear layer weights into the weights after training, while retaining the extra parameter matrix of the activation value side as normal. Then, based on the quantization parameters obtained during training, the model with completed weight fusion is quantized to generate the final deployable model.

[0138] Example 3: Based on the same inventive concept, the present invention also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When the processor executes a program stored in memory, it is able to implement a large model smoothing quantization method based on the Kronecker integral solution of a learnable rotation matrix, as described in any one of Embodiments 1.

[0139] like Figure 7 As shown, the electronic device may include: a processor 10, a communication interface 20, a memory 30, and a communication bus 40, wherein the processor 10, the communication interface 20, and the memory 30 communicate with each other via the communication bus 40. The processor 10 can call logical instructions in the memory 30 to execute a large model smoothing quantization method based on the Kronecker integral solution of a learnable rotation matrix. This method includes: Model construction: A search space containing multiple decomposition schemes is constructed using Kronecker products of different numbers of matrices. For each decomposition scheme in the search space, the tensor-matrix i-th modal product is used as its corresponding operation implementation. Model Training: Based on a preset scoring function, a layer-by-layer greedy heuristic search strategy is used to adaptively select the highest-scoring rotation matrix hybrid decomposition scheme for the attention module and feedforward network of each decoder layer of the large model; the complete parameters, learnable weight parameters, activation value truncation threshold parameters, and diagonal scaling matrix parameters of the decomposed matrix after the selected optimal decomposition scheme are trained. Quantization phase: For the trained matrix parameters, the extra parameter matrix of the linear layer weight side is fused into the weight, while the extra parameter matrix of the activation value side is retained normally. Then, based on the quantization parameters obtained from training, the model with the weight fusion is quantized to generate the final deployable model.

[0140] Example 4: This invention also provides a computer-readable storage medium, in which a program is stored for executing a spatiotemporal feature fusion ultra-short-term photovoltaic power generation prediction method according to Embodiment 1 above. The program can be executed on a processor.

[0141] Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0142] The program stored on this medium is loaded into the processor's memory and executed to perform various functions. This storage medium, connected to hardware devices, enables the computer to perform the steps of Embodiment 1 described above.

[0143] Since these devices and the principles underlying the problems they solve are similar to the aforementioned large model smoothing quantization method based on the Kronecker integral solution of a learnable rotation matrix, the implementation of this device can be found in the implementation of the aforementioned method, and the repetitions will not be repeated.

[0144] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0145] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A large-scale model smoothing quantization method based on the Kronecker integral solution of a learnable rotation matrix, characterized in that, Includes the following steps: Model construction: A search space containing multiple decomposition schemes is constructed using Kronecker products of different numbers of matrices. For each decomposition scheme in the search space, the tensor-matrix i-th modal product is used as its corresponding operation implementation. Model Training: Based on a preset scoring function, a layer-by-layer greedy heuristic search strategy is used to adaptively select the highest-scoring rotation matrix hybrid decomposition scheme for the attention module and feedforward network of each decoder layer of the large model; the complete parameters, learnable weight parameters, activation value truncation threshold parameters, and diagonal scaling matrix parameters of the decomposed matrix after the selected optimal decomposition scheme are trained. Quantization phase: For the trained matrix parameters, the extra parameter matrix of the linear layer weight side is fused into the weight, while the extra parameter matrix of the activation value side is retained normally. Then, based on the quantization parameters obtained from training, the model with the weight fusion is quantized to generate the final deployable model.

2. The method as described in claim 1, characterized in that, The search space is: the rotation matrix Decomposed into Smaller-sized matrices The Kronecker product, i.e.: in, The dimension is submatrix The dimension is And satisfy .

3. The method as described in claim 1, characterized in that, The operation of the tensor-matrix modal product specifically includes the following steps: S121, Input Shaping: Shape the input shape as... Two-dimensional input matrix Logic shaping dimensional tensor , shape ; the shape is weight matrix Logic shaping dimensional tensor , shape ; S122, Sequence Modal Product Operation: Sequentially performing modal product operations on the tensor... To the In terms of dimension, using the corresponding submatrix The modal product is calculated using the following formula: in, Indicates the inverse of a matrix; Represents the tensor-matrix of the first Modal product, the first (1, N); and Both represent tensors with the same dimension as the original input tensor obtained after performing the corresponding modal product operation on the original input tensor. S123, Result Shaping: The shape after all modal product calculations are completed is... The tensor shaping return shape is From a two-dimensional matrix, we obtain the rotated input matrix. ; the shape is The tensor shaping return shape is From the two-dimensional matrix, we obtain the rotated weight matrix. .

4. The method as described in claim 3, characterized in that, The tensor-matrix modal product operation employs a general N-stage computation method, including the following steps: Scalability of unified computing logic: through logical batch size dimension and additional dimensions The design logically transforms the input high-dimensional tensor into a shape of... Modal product calculation is performed on the three-dimensional tensor; Staged kernel calls: Based on unified computation logic, this is achieved by changing the logical batch size dimension and the input of additional dimensions. The process of tensor-matrix product calculation in different dimensions; Step-based logical slicing: The kernel addresses the input tensor by using the memory step size information of each dimension. The kernel dynamically calculates the access offset based on the input dimension and directly slices and reads and calculates the multidimensional tensor in the logical view.

5. The method as described in claim 1, characterized in that, The heuristic search strategy includes the following steps: S211, Layer-by-layer traversal: Process each decoder layer sequentially according to the forward propagation order of the model; S212. Candidate Solution Evaluation: For the current decoder layer, traverse all possible decomposition combinations in the search space. ,in The number of decompositions for the attention module. Let be the number of decompositions in the feedforward network, and The search space Defined as the number of decompositions The candidate set; S213. Rapid Iterative Evaluation: For each combination scheme, construct the corresponding number of initial rotation matrices. Iterative training is performed; a scoring function based on reconstruction error and time delay factors is used. The hybrid decomposition scheme that yields the highest score for the current layer is selected. During the evaluation process, tensor-matrix modal product operations are used to perform efficient forward computation and parameter updates on the input data. S214. Optimal Decision Making: Utilizing Scoring Functions Calculate the score, compare the scores of all candidate combinations, and select the highest-scoring combination. This is the optimal hybrid decomposition scheme for the current layer.

6. The method as described in claim 5, characterized in that, The complete parameters, learnable weight parameters, activation truncation threshold parameters, and diagonal scaling matrix parameters of the matrix after decomposition of the selected optimal decomposition scheme are trained, including the following steps: S221. Optimization Objective Modeling: Construct an optimization objective function for each layer: in, The parameter matrix set formed after rotation matrix decomposition This is the introduced diagonal scaling factor. These are the learnable weights and activation value truncation thresholds, respectively. and They represent the first X is the input matrix; S222, Forward Propagation and Parameter Update: Forward computation is performed using the substitution operation form of the tensor matrix product based on the decomposed matrix. For the input matrix... and weight Calculate the results after rotation: in, Indicates along the first Tensor-matrix product performed in dimensionality; S223, Inter-layer progression: After the current layer is trained, the quantized output is used as the input of the next layer, and the above process is repeated until the entire model is trained.

7. The method as described in claim 1, characterized in that, The quantification stage specifically includes the following steps: (1) Integrate the rotation matrix parameters of the trained linear layer weights into the original model weights. Let the tensor of the original weights after reshaping be... Then the weight after fusion The specific calculation method is as follows: ； (2) The rotation matrix parameters located on the activation side are retained as online computation operators; (3) Based on the weight and activation value truncation threshold parameters obtained during training, uniform quantization is performed on the fused weights and activation values. The uniform quantization process is defined by the following formula: First, for floating-point values Truncation: in, The cutoff threshold corresponding to a pair of weights or activation values, i.e. or ; Then, quantization-dequantization with a truncation threshold is performed, the process defined as follows: in, As a scaling factor, Zero point and These are the lower and upper bounds of the range of quantized integers, respectively: Finally, a large-scale model with quantification is obtained.

8. A large-scale model smoothing quantization system based on the Kronecker integral solution of a learnable rotation matrix, characterized in that, The system comprises: Model building module: used to construct a search space containing multiple decomposition schemes using Kronecker products of different numbers of matrices. For each decomposition scheme in the search space, the tensor-matrix i-th modal product is used as its corresponding operation implementation. Model training module: Based on a preset scoring function, it uses a layer-by-layer greedy heuristic search strategy to adaptively select the highest-scoring rotation matrix hybrid decomposition scheme for the attention module and feedforward network of each decoder layer of the large model; and trains the complete parameters, learnable weight parameters, activation value truncation threshold parameters, and diagonal scaling matrix parameters of the decomposed matrix after the selected optimal decomposition scheme. The quantization phase module is used to fuse the extra parameter matrix of the linear layer weights into the weights after training, while retaining the extra parameter matrix of the activation value side as normal. Then, based on the quantization parameters obtained during training, the model with completed weight fusion is quantized to generate the final deployable model.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements a large model smoothing quantization method based on the Kronecker integral solution of a learnable rotation matrix as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements a large model smoothing quantization method based on the Kronecker integral solution of a learnable rotation matrix as described in any one of claims 1 to 7.

Images