Method, device and equipment for model training, medium and program product

Abstract

According to the embodiment of the invention, a method, a device, equipment, a medium and a program product for model training are provided. The method comprises the steps of obtaining a to-be-trained target model, wherein the target model comprises a plurality of processing blocks; in the process of training the target model by using the training data set, for each processing block in the at least one processing block, executing a processing process corresponding to the processing block on the input of the processing block at least based on a weight set in a parameter value set of the processing block to obtain non-quantization output of the processing block; performing quantization operation on a first product of the weight set and an affine transformation matrix for the processing block by using a quantization function to obtain a quantization weight set; executing a processing process corresponding to the processing block on a second product of the input and the inverse matrix of the affine transformation matrix at least based on the quantization weight set to obtain quantization output of the processing block; and iteratively updating the parameter value set of the target model and the affine transformation matrix corresponding to the at least one processing block according to the optimization target.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computer technology, and in particular, to methods, apparatuses, devices, computer-readable storage media, and computer program products for model training. Background Art

[0002] Machine learning and deep learning technologies have been widely applied in many fields. Machine learning models can be configured and trained to generate corresponding outputs based on given model inputs. With the development of large-scale language models, these models have found increasing application in natural language processing (NLP). Their powerful general conversational capabilities enable them to provide fluent and coherent responses across a wide range of topics. However, these models still have limitations and require further improvement. Summary of the Invention

[0003] In a first aspect of the present disclosure, a method for model training is provided. The method comprises: obtaining a target model to be trained, the target model comprising a plurality of processing blocks, a parameter value set of the target model comprising a parameter value set of the plurality of processing blocks, and at least one of the plurality of processing blocks being configured with a corresponding affine transformation matrix; in the process of training the target model using a training data set, for each processing block in at least one processing block, performing a processing process corresponding to the processing block on the input of the processing block based on at least a weight set in the parameter value set of the processing block, and obtaining a non-quantized output of the processing block; performing a quantization operation on a first product of the weight set and the affine transformation matrix for the processing block using a quantization function, and obtaining a quantized weight set; performing a processing process corresponding to the processing block on a second product of the input and the inverse matrix of the affine transformation matrix based on at least the quantized weight set, and obtaining a quantized output of the processing block; and iteratively updating the parameter value set of the target model and the affine transformation matrix corresponding to each of the at least one processing block according to an optimization objective, the optimization objective being at least configured to reduce or minimize the error between the non-quantized output and the quantized output of at least one processing block.

[0004] In a second aspect of the present disclosure, a device for model training is provided. The device includes: a model acquisition module configured to obtain a target model to be trained, the target model including a plurality of processing blocks, a parameter value set of the target model including parameter value sets of the plurality of processing blocks, and at least one processing block in the plurality of processing blocks being configured with a corresponding affine transformation matrix; a block processing module configured to, in the process of training the target model using a training data set, for each processing block in at least one processing block, perform a processing process corresponding to the processing block on the input of the processing block based at least on a weight set in the parameter value set of the processing block, thereby obtaining a non-quantized output of the processing block; perform a quantization operation on a first product of the weight set and an affine transformation matrix for the processing block using a quantization function, thereby obtaining a quantized weight set; perform a processing process corresponding to the processing block on a second product of the input and an inverse matrix of the affine transformation matrix based at least on the quantized weight set, thereby obtaining a quantized output of the processing block; and an iterative update module configured to iteratively update the parameter value set of the target model and the affine transformation matrix corresponding to each of the at least one processing block according to an optimization objective, wherein the optimization objective is at least configured to reduce or minimize the error between the non-quantized output and the quantized output of at least one processing block.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the medium, and when the computer program is executed by a processor, the method of the first aspect is implemented.

[0007] In a fifth aspect of the present disclosure, a computer program product is provided, which includes a computer program, and when the computer program is executed by a processor, the method of the first aspect is implemented.

[0008] It should be understood that the content described in this section is not intended to limit the key features or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0010] Figure 1A schematic diagram illustrating an example environment in which embodiments of the present disclosure can be implemented;

[0011] Figure 2 A flowchart illustrating a process for model training according to some embodiments of the present disclosure is shown;

[0012] Figure 3 A schematic diagram illustrating scaling transformation quantization, translation transformation quantization, and affine transformation quantization according to some embodiments of the present disclosure is shown;

[0013] Figure 4 An example process of applying a progressive mask to an affine transformation matrix according to some embodiments of the present disclosure is shown;

[0014] Figure 5 A block diagram illustrating an apparatus for model training according to some embodiments of the present disclosure is shown; and

[0015] Figure 6 A block diagram of an electronic device is shown in which one or more embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION

[0016] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0017] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, i.e., "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may be included below.

[0018] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) must comply with the requirements of relevant laws, regulations and relevant provisions.

[0019] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0020] For example, in response to receiving a user's active request, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information, so that the user can independently choose whether to provide personal information to the electronic device, application, server or storage medium and other software or hardware that performs the operation of the technical solution of the present disclosure based on the prompt message.

[0021] As an optional but non-limiting implementation, in response to receiving a user's active request, a prompt message may be sent to the user, for example, in the form of a pop-up window, in which the prompt message may be presented in text form. Furthermore, the pop-up window may also include a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0022] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0023] As used herein, the term "model" can learn the association between corresponding inputs and outputs from training data, so that after training is completed, corresponding outputs can be generated for given inputs. The generation of the model can be based on machine learning technology. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs by using multiple layers of processing units. A neural network model is an example of a model based on deep learning. In this article, "model" may also be referred to as "machine learning model", "learning model", "machine learning network" or "learning network", and these terms are used interchangeably in this article.

[0024] A "neural network" is a machine learning network based on deep learning. A neural network is capable of processing inputs and providing corresponding outputs. It typically includes an input layer, an output layer, and one or more hidden layers between the input and output layers. Neural networks used in deep learning applications typically include many hidden layers, thereby increasing the depth of the network. The layers of a neural network are connected in sequence so that the output of the previous layer is provided as input to the next layer, where the input layer receives the input of the neural network and the output of the output layer serves as the final output of the neural network. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), each of which processes the input from the previous layer.

[0025] Generally speaking, machine learning can be roughly divided into three stages, namely the training stage, the testing stage, and the application stage (also called the inference stage). In the training stage, a given model can be trained using a large amount of training data, and the parameter values ​​are continuously updated iteratively until the model can obtain consistent inferences that meet the expected goals from the training data. Through training, the model can be considered to be able to learn the association between input and output (also called input-to-output mapping) from the training data. The parameter values ​​of the trained model are determined. In the testing stage, the test input is applied to the trained model to test whether the model can provide the correct output, thereby determining the performance of the model. The testing stage can sometimes be integrated into the training stage. In the application or inference stage, the trained model can be used to process the actual model input based on the parameter values ​​obtained through training to determine the corresponding model output.

[0026] Figure 1 1 shows a schematic diagram of an environment 100 in which embodiments of the present disclosure can be implemented. Figure 1 The three different stages of the model are shown in the environment 100, including a pre-training stage 102, a fine-tuning stage 104, and an application stage 106. After the pre-training or fine-tuning stage is completed, there may also be a testing stage, which is not shown in the figure.

[0027] In the training phase 102, the model training system 110 is configured to perform training of the model 105 using the training data set 112. At the beginning of the training, the model may have initial parameter values. The training process is to update the parameter values ​​of the model 105 to the desired values ​​based on the training data.

[0028] In the application phase 106 , the obtained model 105 has trained parameter values ​​and can be provided to the model application system 130 for use. In the application phase 106 , the model 105 can be used to process corresponding target inputs 132 in actual scenarios and provide corresponding target outputs 134 .

[0029] exist Figure 1 In the present invention, the model training system 110 and the model application system 130 may include any computing system with computing capabilities, such as various computing devices / systems, terminal devices, servers, etc. Terminal devices may involve any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. Servers include but are not limited to mainframes, edge computing nodes, computing devices in cloud environments, etc.

[0030] It should be understood that Figure 1The components and arrangements in the illustrated environment 100 are merely examples, and a computing system suitable for implementing the exemplary implementations described herein may include one or more different components, other components, and / or different arrangements. For example, although shown as separate, the model training system 110 and the model application system 130 may be integrated into the same system or device. Implementations of the present disclosure are not limited in this respect.

[0031] It should be understood that the structure and function of the various elements in the environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of the present disclosure.

[0032] Currently, large and complex models with increasingly powerful reasoning capabilities are being developed. These large models typically have very large parameter sets. This requires significant storage and computational overhead during the model inference phase, limiting their applicability and preventing them from being deployed on resource-constrained devices. Improving model inference efficiency and reducing resource overhead has become a hot topic.

[0033] For example, large language models (LLMs) have attracted increasing attention due to their impressive performance. However, the emergent logical reasoning capabilities only exist in models above a certain size threshold. Therefore, the training and inference efficiency of large language models requires careful consideration. Specifically, the potential use of large language models for inference on mobile and edge devices has motivated us to focus on accelerating model inference. Quantization is considered one of the most promising of these compression methods. In particular, quantization methods map the weights or activations of a model to lower-bit representations, effectively reducing the model's memory usage. In addition, operator compilation that optimizes low-bit operations can significantly improve their efficiency and accelerate model inference.

[0034] At the same time, since large language models require extensive computational resources and high-quality data, achieving quantization through model fine-tuning is challenging. Therefore, the research community is increasingly emphasizing training-free algorithms, known as post-training quantization (PTQ). Post-training quantization can effectively optimize with less calibration data. However, this process can lead to significant performance degradation, especially in small-scale models or low-bit scenarios.

[0035] Activation-aware weight quantization (AWQ) enhances scale calculation by optimizing statistics and introduces the mean squared error loss between the feature maps before and after quantization as an optimization metric in LLM. Recently, there are also schemes that introduce block-wise learnable scaling and shift parameters to enhance optimization. In higher dimensions, the concept of affine quantization is also gaining attention. The reordered post-training quantization method (RPTQ) achieves activation quantization of each cluster by sorting the activation value columns. Reordering can be mathematically represented by transforming the scale factor from a vector to a matrix form, where each row and each column has one and only one scale factor. This transformation effectively rearranges the activation columns and weight rows in an equivalent way.

[0036] In summary, the evolution of affine transformation quantization has progressed from manual design to gradient optimization, from low-dimensional transformations to high-dimensional transformations, and from single-scale merging to a combination of multiple operations, including translation and reordering. Affine quantization offers advantages in two key aspects. First, by ensuring consistency between the output before and after quantization, the introduced quantization noise can be effectively mitigated by optimizing the affine transformation parameters. This is consistent with the concept of post-training quantization, where the affine transformation acts as an intermediate proxy for noise reduction. Second, different types of affine transformations are orthogonal to each other. Intuitively, the introduction of each new affine transformation expands the parameter optimization space, thereby improving performance. However, affine transformation operations based on scaling and translation still limit the optimization space for quantization. Therefore, more generalized affine transformation operations may improve the performance of quantized models.

[0037] In order to improve the model processing efficiency, especially the processing efficiency in the inference stage, in an embodiment of the present disclosure, an improved model training scheme is proposed for realizing post-training quantization. During the model training process, a corresponding affine transformation matrix is ​​configured for one or more processing blocks of the target model. For each processing block, a quantization operation is performed on the first product of the weight set and the affine transformation matrix for the processing block using a quantization function to obtain a quantized weight set. Based at least on the quantized weight set, the processing process corresponding to the processing block is performed on the second product of the input and the inverse matrix of the affine transformation matrix to obtain the quantized output of the processing block. The error between the quantized output and the non-quantized output of the processing block is used to construct the optimization target of the target model. The parameter value set of the target model and the affine transformation matrix corresponding to each of the at least one processing block are iteratively updated according to the optimization target.

[0038] This scheme expands the optimization scope of the target model and can significantly reduce or even minimize the quantization error.

[0039] Figure 2 FIG2 is a schematic block diagram of a process 200 for model training according to some embodiments of the present disclosure. The process 200 may be implemented at the model training system 110 .

[0040] In box 210, the model training system 110 obtains a target model to be trained, where the target model includes multiple processing blocks, the parameter value set of the target model includes the parameter value set of the multiple processing blocks, and at least one processing block among the multiple processing blocks is configured with a corresponding affine transformation matrix.

[0041] The target model can be any type of machine learning model or neural network model. The target model can have multiple layers, each layer including a corresponding processing block for performing response processing. The target model receives model input and processes the model input layer by layer. The processing block of the latter layer receives the output result of the processing block of the previous layer as its own input. The model input that the target model can process can be a text modality, a visual modality, or a combination of multimodal data, etc. In some embodiments, the target model can be based on a language model (LM), in particular a large language model (LLM). In some embodiments, the target model can include one or more Transformer blocks. Of course, other types of processing blocks are also feasible.

[0042] Without being too general, the processing in the i-th processing block of the target model can be expressed as f i (X, W), where X represents the input of the i-th processing block, W represents the weight set (or weight matrix) of the i-th processing block, and f i () represents the processing algorithm of the processing block. In some embodiments, the parameter value set of a processing block includes a weight set and may also include a bias value. The corresponding processing can be represented as f i (X,W,b), where b is the bias value.

[0043] During model training, the target model's parameter value set (including the parameter value sets for each processing block) is iteratively updated based on the training dataset until convergence or optimization is achieved. In each iteration, the sample model inputs from the training dataset are fed into the target model, which then determines the predicted model output based on the current parameter value set. By designing an optimization objective, the parameter value set is continuously updated to bring the target model's predicted model output closer to the desired model output.

[0044] In an embodiment of the present disclosure, it is expected that while training a target model, corresponding affine transformation matrices are determined for some processing blocks to be used for implementing affine transformation quantization after training.

[0045] When considering the concept of affine transformations from a physical perspective, some operations can be compared. For example, in some schemes, scaling can be compared to scaling a vector, while in other schemes, translation can be compared to translating a vector. Similarly, rotation of a vector can also be classified as an equivalent transformation.

[0046] In the process of training the target model using the training dataset, the quantization function can be expressed as follows:

[0047]

[0048] Where Δ, zp and n are quantization step size, zero point and number of bits respectively. is a rounding operation.

[0049] In the process of training the target model using the training data set, for each processing block in at least one processing block, the model training system 110 performs operations corresponding to boxes 220, 222 and 224.

[0050] At block 220 , the model training system 110 performs the processing corresponding to the processing block on the input X (also known as the activation value) of the processing block based on at least the weight set W in the parameter value set of the current processing block, to obtain the unquantized output of the processing block. In other words, the processing block uses the current weight set to determine the output of the processing block in a conventional unquantized manner.

[0051] At block 222, the model training system 110 uses a quantization function to perform a quantization operation on a first product of a weight set (also referred to as a weight matrix) and an affine transformation matrix for the processing block, thereby obtaining a quantized weight set. At block 224, the model training system 110 performs a processing procedure corresponding to the processing block on a second product of the input and an inverse matrix of the affine transformation matrix based on at least the quantized weight set, thereby obtaining a quantized output for the processing block.

[0052] In the embodiment of the present disclosure, the affine transformation matrix A is an affine matrix and is reversible, and the inverse matrix can be calculated. For a single processing block, the affine transformation matrix A is multiplied by the weight matrix W on the left to better match the weight distribution with the quantization function. Alignment. Expanding the optimization space can make the quantization error in the transformed weights smaller, thereby reducing the perplexity. At the same time, the inverse matrix of the affine transformation matrix A is right-multiplied by the input X to maintain the invariance of the matrix multiplication output between the input and the weight.

[0053] At block 230 , the model training system 110 iteratively updates the parameter value set of the target model and the affine transformation matrix corresponding to each of the at least one processing block according to an optimization objective, wherein the optimization objective is configured to at least reduce or minimize an error between an unquantized output and a quantized output of the at least one processing block.

[0054] For a single processing block, this optimization objective can be expressed as follows:

[0055]

[0056] Where XW represents the non-quantized output of the processing block, represents the quantized output of the processing block, Represents the mean square error between the unquantized output and the quantized output.

[0057] In some embodiments, the processing block configured with the affine transformation matrix can be a linear processing block, wherein the processing process is based on a linear function. For such a processing block, equivalent calculations after the introduction of the affine transformation matrix can be supported. In some embodiments, the target model can also include one or more additional nonlinear processing blocks. For nonlinear processing blocks, the optimization target of their parameter values ​​can be based on a general model optimization target, such as constructed based on cross entropy loss.

[0058] As an affine transformation matrix, the main diagonal elements of the matrix A are calculated based on the weights and activation statistics. In addition, when each row and each column of the matrix A has only one element 1, the subset of the affine transformation matrix A is equivalent to the reordering matrix used in RPTQ. Therefore, affine transformation-based quantization covers a variety of previously equivalent quantization algorithms, thereby expanding the optimization possibilities of the weight distribution W.

[0059] Figure 3 Quantization of the weight set W by scaling transformation, quantization of translation transformation and quantization of affine transformation according to some embodiments of the present disclosure are shown. Figure 3 In the example, the “fixed point” refers to the (2nd) n -1) quantization level, s, b and A are the scaling factor in scaling transformation, the translation factor in translation transformation, and the affine transformation matrix in affine transformation. Figure 3 In the example, assume that the input channel and output channel of W are both 2, and each output channel is a two-dimensional vector, so W is represented as

[0060] The weight matrix is ​​divided into 2 vectors {v1, v2} according to the output channels. The scaling transformation si*vi uniformly scales each element of vi. The translation transformation vi+bi translates vi along different axes. The affine transformation Avi allows arbitrary transformations of vi. However, the scaling and translation transformations have limited ability to map the dimensions in vi to adjacent quantization fixed points. In contrast, the affine transformation guarantees that all dimensions in the vector converge to the quantization fixed point. In other words, the affine transformation transforms the weight distribution with the quantization function above The introduced noise is aligned, thereby reducing the quantization error. It is worth noting that the affine transformation matrix is ​​normalized along the rows (A→s'A), where the norm of each row of the matrix A' is 1, transforming A' into a standard rotation matrix. This rotation matrix rotates the output channels of the weights while preserving the magnitude of the vector. The scaling factor s' scales the rotation vector. Therefore, the affine transformation matrix A combines the equivalent transformations of scaling and rotation and is orthogonal to the translation transformation.

[0061] During model training, the affine transformation matrix A is iteratively updated, and the parameter values ​​of the entire model are also optimized. Furthermore, the perplexity (PPL) and cross entropy (CE) loss show an exponential relationship, and the cross entropy loss is positively correlated with the mean square error of the output activation before and after quantization. Therefore, the perplexity can be optimized by optimizing the mean square error before and after quantization. Specifically, it can satisfy:

[0062]

[0063] In some embodiments, if the parameter value set of a processing block includes a weight set and a bias value, that is, the processing of the processing block is represented as f i (X, W, b). The processing process of the processing block is expressed as: based on the weight set W and bias value b of the processing block, the processing process corresponding to the processing block is performed on the input of the processing block to obtain the non-quantized result f of the processing block. i (X, W, b). When performing affine transformation quantization to calculate the quantization result of the processing block, the bias input X-δ after applying the first bias value δ to the input is calculated, and the second product between the inverse matrix of the affine transformation matrix and the bias input is calculated, that is, (X-δ)A -1 The second bias value is added to the product of the first bias value and the weight set to obtain the bias result b+δW.

[0064] Based on the quantized weight set And the bias result b+δW, the bias input X-δ and the inverse matrix A of the affine transformation matrix -1 The second product of performs the processing corresponding to the processing module, which can be expressed as

[0065] Considering the weight set and bias value, the optimization objective of the target model can be expressed as:

[0066]

[0067] where (X-δ)A -1 , and b+δW are the equivalent activation values, weights, and biases after the equivalent transformation, respectively. The above formula (4) still expresses the optimization goal of minimizing the mean square error between the non-quantized output and the quantized output of each processing block.

[0068] During the optimization process, the affine transformation matrix A needs to be inverted. However, the objective function corresponding to the optimization target (Equation (4)) does not contain any constraints to ensure that the matrix remains full rank or well-conditioned. In order to keep the matrix A affine during the optimization process, at the beginning of the training process, the affine transformation matrix of at least one processing unit is initialized to a diagonally dominant matrix, and during the iterative update, the affine transformation matrix corresponding to at least one processing unit is maintained as a diagonally dominant matrix.

[0069] Here, we first give the definition of strict diagonal dominance. A matrix A is considered strictly diagonally dominant if the absolute value of each diagonal element is greater than the sum of the absolute values ​​of the remaining elements in the corresponding row. Specifically, for each element a in the matrix A (for all i)

[0070]

[0071] The Levy-Desplanques theorem states that all strictly diagonally dominant matrices are affine. Initializing the affine transformation matrix A with diagonal elements ensures that it is initially strictly diagonally dominant. Although optimizers may utilize second-order momentum and lower learning rates during iterative updates to help meet the requirements of the Levy-Desplanques theorem, optimizing large affine transformation matrices still faces instability challenges as model size increases.

[0072] In some embodiments, to ensure that the affine transformation matrix strictly maintains diagonal dominance during the optimization process, in one round of iterative updates, multiple elements around the diagonal of the affine transformation matrix A corresponding to each of the at least one processing block are updated, and other elements in the affine transformation matrix remain unchanged. To support such diagonal updates, a progressive masking method is introduced. In some embodiments, a mask matrix GM corresponding to the current round of updates is applied to the affine transformation matrix corresponding to each of the at least one processing block to obtain a masked affine transformation matrix corresponding to each of the at least one processing block. Then, in the process of training the target model using the training data set, for each of the at least one processing block, the masked affine transformation matrix is ​​used to determine the quantized output. When updating multiple elements around the diagonal of the affine transformation matrix corresponding to each of the at least one processing block, the masked affine transformation matrix is ​​updated by back-gradient propagation of the masked affine transformation matrix corresponding to each of the at least one processing block.

[0073] In the iterative update, the mask matrix GM is initialized so that the elements on the diagonal are 1 and the elements other than the diagonal are 0. In subsequent rounds of updates in the iterative update, the mask matrix GM is updated so that the elements on the diagonal are 0, and the elements adjacent to the diagonal are set to a predetermined value (denoted as α) greater than 0 and less than or equal to 1, while the other elements remain 0. In the iterative update, more elements adjacent to the diagonal in the mask matrix GM are continuously set to predetermined values ​​greater than 0 and less than or equal to 1 until, in the last round of updates in the iterative update, the mask matrix has no elements equal to 0.

[0074] After applying such a mask matrix GM, the quantized output is calculated in each processing block based on the masked affine transformation matrix. In this way, when the affine transformation matrix update is performed based on the aforementioned optimization objective, the element values ​​of the affine transformation matrix masked by element 0 will not be updated, while the elements around the diagonal of the affine transformation matrix will be updated. At the beginning of the optimization of each processing block, all elements of the corresponding affine transformation matrix A except the elements on the main diagonal will be frozen. As the optimization proceeds, the elements near the main diagonal are gradually unfrozen. Eventually, all matrix elements become learnable for optimization. This freezing mechanism is called a progressive mask matrix (GM) and is defined as follows:

[0075]

[0076] GM ij is the element of the i-th row and j-th column of the mask matrix, t is the total number of iterative updates, and e∈[1,t] is the current update round. Each round of iterative updates includes updating the parameter value set of the target model and the affine transformation matrix corresponding to at least one processing block using the entire training data set, also known as epoch. In addition, in the above formula (6), "hiddensize" is the dimension of the affine transformation matrix. The predetermined value α can be a hyperparameter, which can be adjusted according to the actual model as a stability coefficient. In some embodiments, if the target model includes an attention module, a progressive mask is applied in each attention head. The progressive mask matrix GM can be used as a learning rate regulator to achieve its purpose by performing an element-by-element dot product with the affine transformation matrix A.

[0077] Figure 4An example process for applying progressive masking to an affine transformation matrix according to some embodiments of the present disclosure is shown. During each epoch's iterative update, affine transformation matrix A 410 is masked by the mask matrix corresponding to each epoch (e.g., mask matrices 420, 422, ..., through 430). Mask matrix 420 is the initial mask matrix, with elements on its main diagonal set to 1 and the remaining elements set to 0. Thus, during the initial epoch iteration, the elements on the main diagonal of affine transformation matrix A are updated, while the remaining elements are frozen. In the next epoch iteration, the elements around the main diagonal of mask matrix 422 are updated to a predetermined value α, while the remaining elements remain 0. Then, in the second epoch iteration, the elements masked by non-zero elements in affine transformation matrix A are updated, while the remaining elements are frozen. By the final epoch iteration, all elements in mask matrix 430 are non-zero, so all elements in affine transformation matrix A are updated.

[0078] The update of the mask matrix 420 can be updated once in one or more epochs, which may be related to the size of the mask matrix and the total number of iterative updates to ensure that all elements in the mask matrix 430 are non-zero elements after the progressive update to the last one or more epochs.

[0079] In the forward propagation of the model, the progressive mask is represented as:

[0080]

[0081] The backward gradient propagation based on progressive mask is expressed as:

[0082]

[0083] Where ○ is the Hadamard product; Ae and GMe are the affine transformation matrix A and the progressive mask (GM) matrix corresponding to the e-th epoch, respectively; η is the learning rate of matrix A; L is the optimization loss.

[0084] When the stability factor α is less than 1, the GM matrix effectively reduces the size of the non-diagonal elements in the affine transformation matrix A during the forward pass. This ensures that a stable inverse matrix A* exists during the optimization process at the e-th epoch, conforming to the Levy-Desplanques theorem. During backward pass, the GM influences the learning rate η, thereby suppressing the update rate of the non-diagonal elements in the affine transformation matrix A. Therefore, the GM's influence on η ensures that the affine transformation matrix A remains strictly diagonally dominant at the e+1th epoch, satisfying the Levy-Desplanques theorem.

[0085] In the model inference stage after the training of the target model is completed, post-training quantization can be performed based on the affine transformation matrix determined for each processing unit. Model inference can be implemented by the model application system 130, for example. After obtaining the target model input to be processed, the target model input is processed using the trained target model. During the processing of the target model input, for each processing block in at least one processing block, a quantization function is used to perform a quantization operation on the first product of the trained weight set of the processing block and the trained affine transformation matrix for the processing block to obtain a quantized weight set. Then, based at least on the quantized weight set, the processing process corresponding to the processing block is performed on the second product of the target input of the processing block and the inverse matrix of the affine transformation matrix to obtain the quantized output of the processing block. The target input of the processing block is based on the target model input to be processed currently, and is obtained after processing by one or more previous processing blocks. Of course, if the target model input is directly provided to the current processing block, then its target input is the target model input. The quantized output of the i-th processing block can be represented as However, at this time, W and A are the trained and optimized weight value set and affine transformation matrix of the i-th processing block.

[0086] Accordingly, from the perspective of overall model processing, the target model output corresponding to the target model input is determined based on the quantized output of each processing block configured with the affine transformation matrix. Of course, if some processing blocks in the target model are not configured to perform affine transformation quantization, then those processing blocks will use the unquantized parameter value set to process their respective inputs, ultimately obtaining the output of the entire model.

[0087] According to the embodiments of the present disclosure, an improved affine transform quantization is proposed, which retains the advantages of PTQ, ensures efficiency and generalization, significantly minimizes quantization error (especially under low-bit quantization), and supports the deployment of LLM on edge devices. In addition, in the embodiments of the present disclosure, an innovative optimization algorithm is proposed, which uses the Levy-Desplanques theorem to ensure the affinity of the entire process while reducing the computational cost. This scheme can achieve state-of-the-art performance in the quantization of complex large models (e.g., large language models), especially on low-bit models.

[0088] Figure 5 FIG2 shows a block diagram of an apparatus 500 for model training according to some embodiments of the present disclosure. The apparatus 500 may be implemented as or included in Figure 1 The model training system 110 is located at the apparatus 500. Each module / component in the apparatus 500 may be implemented by hardware, software, firmware, or any combination thereof.

[0089] As shown in the figure, the device 500 includes a model acquisition module 510, which is configured to obtain a target model to be trained, wherein the target model includes multiple processing blocks, the parameter value set of the target model includes the parameter value sets of multiple processing blocks, and at least one processing block among the multiple processing blocks is configured with a corresponding affine transformation matrix. The device 500 also includes a block processing module 520, which is configured to, in the process of training the target model using a training data set, for each processing block in at least one processing block, perform a processing process corresponding to the processing block on the input of the processing block based on at least a weight set in the parameter value set of the processing block to obtain a non-quantized output of the processing block; perform a quantization operation on the first product of the weight set and the affine transformation matrix for the processing block using a quantization function to obtain a quantized weight set; and perform a processing process corresponding to the processing block on the second product of the input and the inverse matrix of the affine transformation matrix based on at least the quantized weight set to obtain a quantized output of the processing block. The device 500 also includes an iterative update module 530, which is configured to iteratively update the parameter value set of the target model and the affine transformation matrix corresponding to at least one processing block according to the optimization objective, and the optimization objective is at least configured to reduce or minimize the error between the non-quantized output and the quantized output of at least one processing block.

[0090] In some embodiments, the parameter value set of the first processing block in at least one processing block includes a weight set and a bias value, and the block processing module includes: a first execution module, configured to perform a processing process corresponding to the processing block on the input of the processing block based on the weight set and the bias value of the first processing block; and the block processing module also includes: a bias input determination module, configured to determine the bias input after applying the first bias value to the input, a product calculation module, configured to calculate the second product between the inverse matrix of the affine transformation matrix and the bias input, a bias result determination module, configured to add the second bias value to the product of the first bias value and the weight set to obtain a bias result, and a second execution module, configured to perform the processing process corresponding to the processing module on the second product of the bias input and the inverse matrix of the affine transformation matrix based on the quantized weight set and the bias result.

[0091] In some embodiments, at the beginning of the training process, the affine transformation matrix of each of at least one processing unit is initialized as a diagonally dominant matrix, and during iterative updates, the affine transformation matrix corresponding to each of at least one processing unit is maintained as a diagonally dominant matrix.

[0092] In some embodiments, the iterative update module 530 is further configured to: in a round of iterative update, update multiple elements around the diagonal of the affine transformation matrix corresponding to at least one processing block, and keep other elements in the affine transformation matrix unchanged.

[0093] In some embodiments, the device 500 also includes: a mask matrix application module, configured to apply a mask matrix corresponding to the current round of update on the affine transformation matrix corresponding to each of the at least one processing blocks to obtain a masked affine transformation matrix corresponding to each of the at least one processing blocks; wherein in the process of using the training data set to train the target model, for each processing block in the at least one processing block, the masked affine transformation matrix is ​​used to determine the quantized output, and the iterative update module 530 is further configured to: update the masked affine transformation matrix by back gradient propagation of the masked affine transformation matrix corresponding to each of the at least one processing block.

[0094] In some embodiments, in an iterative update, the mask matrix is ​​initialized so that the elements on the diagonal are 1 and the elements other than the diagonal are 0. In subsequent rounds of updates in the iterative update, the mask matrix is ​​updated so that the elements on the diagonal are 0, and the elements adjacent to the diagonal are continuously set to predetermined values ​​greater than 0 and less than or equal to 1, until in the last round of updates in the iterative update, the mask matrix has no elements equal to 0.

[0095] In some embodiments, each round of iterative updating includes updating the parameter value set of the target model and the affine transformation matrix corresponding to each of the at least one processing block using the entire training dataset.

[0096] In some embodiments, the processing of at least one processing block is based on a linear function.

[0097] In some embodiments, the device 500 also includes: a processing module, configured to use the trained target model to process the target model input after the training of the target model is completed; during the processing of the target model input, for each processing block in at least one processing block, use a quantization function to perform a quantization operation on the trained weight set of the processing block and the first product of the trained affine transformation matrix for the processing block to obtain a quantized weight set; at least based on the quantization weight set, perform a processing process corresponding to the processing block on the target input of the processing block and the second product of the inverse matrix of the affine transformation matrix to obtain the quantized output of the processing block, and the target input of the processing block is based on the target model input; and an output determination module, configured to determine the target model output corresponding to the target model input based on the quantized output of at least one processing block.

[0098] In some embodiments, the target model comprises a language model.

[0099] Figure 6 1 shows a block diagram of an electronic device 600 in which one or more embodiments of the present disclosure may be implemented. It should be understood that Figure 6The illustrated electronic device 600 is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 6 The electronic device 600 shown can be used to implement Figure 1 Model training system 110 and / or model application system 130, or Figure 6 device 600.

[0100] like Figure 6 As shown, electronic device 600 is in the form of a general-purpose computing device. Components of electronic device 600 may include, but are not limited to, one or more processors or processing units 610, memory 620, storage device 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. Processing unit 610 may be a real or virtual processor and is capable of performing various processes according to programs stored in memory 620. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to increase the parallel processing capabilities of electronic device 600.

[0101] The electronic device 600 typically includes a plurality of computer storage media. Such media can be any available media accessible to the electronic device 600, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 620 can be a volatile memory (e.g., registers, cache, random access memory (RAM)), a non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 630 can be a removable or non-removable medium and can include a machine-readable medium, such as a flash drive, a disk, or any other medium that can be used to store information and / or data and can be accessed within the electronic device 600.

[0102] The electronic device 600 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 6 As shown in FIG, a magnetic disk drive for reading from or writing to a removable, non-volatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to a bus (not shown) by one or more data media interfaces. Memory 620 may include a computer program product 626 having one or more program modules configured to perform various methods or actions of various embodiments of the present disclosure.

[0103] The communication unit 640 enables communication with other electronic devices via a communication medium. Additionally, the functions of the components of the electronic device 600 can be implemented in a single computing cluster or multiple computing machines that can communicate via a communication connection. Thus, the electronic device 600 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0104] The input device 650 may be one or more input devices, such as a mouse, keyboard, or trackball. The output device 660 may be one or more output devices, such as a display, a speaker, or a printer. The electronic device 600 may also communicate with one or more external devices (not shown) through the communication unit 640 as needed, such as a storage device, a display device, or the like, with one or more devices that allow a user to interact with the electronic device 600, or with any device that allows the electronic device 600 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0105] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0106] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0107] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, such that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0108] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0109] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple implementations of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part for a module, program segment or instruction, and a part for a module, program segment or instruction comprises one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0110] While various implementations of the present disclosure have been described above, the foregoing description is intended to be illustrative, not exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is selected to best explain the principles of the implementations, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for model training, comprising: Obtaining a target model to be trained, the target model comprising a plurality of processing blocks, the parameter value set of the target model comprising the parameter value sets of the plurality of processing blocks, and at least one processing block of the plurality of processing blocks being configured with a corresponding affine transformation matrix; In the process of training the target model using the training data set, for each processing block in the at least one processing block, Based at least on a weight set in a parameter value set of the processing block, performing a processing process corresponding to the processing block on an input of the processing block to obtain a non-quantized output of the processing block; Using a quantization function, performing a quantization operation on a first product of the weight set and an affine transformation matrix for the processing block to obtain a quantized weight set; performing, based at least on the set of quantization weights, a processing procedure corresponding to the processing block on a second product of the input and the inverse matrix of the affine transformation matrix to obtain a quantized output of the processing block; as well as The parameter value set of the target model and the affine transformation matrix corresponding to each of the at least one processing block are iteratively updated according to an optimization objective, and the optimization objective is at least configured to reduce or minimize the error between the non-quantized output and the quantized output of the at least one processing block.

2. The method according to claim 1 , wherein the parameter value set of a first processing block in the at least one processing block comprises a weight set and a bias value, and wherein performing the processing corresponding to the processing block on the input of the processing block comprises: Based on the weight set and bias value of the first processing block, perform a processing process corresponding to the processing block on the input of the processing block; and The processing of performing the processing corresponding to the processing block on the second product of the input and the inverse matrix of the affine transformation matrix comprises: determining a biased input after applying a first bias value to the input, computing a second product between the inverse of the affine transformation matrix and the bias input, adding a second bias value to the product of the first bias value and the weight set to obtain a bias result, and Based on the quantization weight set and the bias result, a processing process corresponding to the processing module is performed on a second product of the bias input and the inverse matrix of the affine transformation matrix.

3. The method according to claim 1, wherein at the beginning of the training process, the affine transformation matrix of each of the at least one processing unit is initialized as a diagonally dominant matrix, and in the iterative update, the affine transformation matrix corresponding to each of the at least one processing unit is maintained as a diagonally dominant matrix.

4. The method according to claim 3, wherein iteratively updating the parameter value set of the target model and the affine transformation matrix corresponding to each of the at least one processing block according to the optimization objective comprises: In one round of the iterative update, multiple elements around the diagonal of the affine transformation matrix corresponding to each of the at least one processing block are updated, and other elements in the affine transformation matrix remain unchanged.

5. The method according to claim 4, further comprising: Applying a mask matrix corresponding to the current round of update to the affine transformation matrix corresponding to each of the at least one processing blocks to obtain a masked affine transformation matrix corresponding to each of the at least one processing blocks; wherein in the process of training the target model using the training data set, for each processing block in the at least one processing block, the quantized output is determined using a masked affine transformation matrix, and Wherein updating a plurality of elements around a diagonal line of an affine transformation matrix corresponding to each of the at least one processing block comprises: The masked affine transformation matrix is ​​updated by back-gradient propagation of the masked affine transformation matrix corresponding to each of the at least one processing block.

6. The method according to claim 4, wherein in the iterative updating, the mask matrix is ​​initialized so that the elements on the diagonal are 1 and the elements other than the diagonal are 0; and In subsequent rounds of updates in the iterative update, the mask matrix is ​​updated so that the elements on the diagonal are 0, and the elements adjacent to the diagonal are continuously set to predetermined values ​​greater than 0 and less than or equal to 1, until in the last round of updates in the iterative update, the mask matrix has no elements equal to 0.

7. The method according to any one of claims 4 to 6, wherein each round of update in the iterative update includes updating the parameter value set of the target model and the affine transformation matrix corresponding to each of the at least one processing blocks using the training data set as a whole. The method according to claim 1 , wherein the processing of the at least one processing block is based on a linear function.

9. The method according to claim 1, further comprising: After the training of the target model is completed, the trained target model is used to process the target model input. During the processing of the target model input, for each processing block in the at least one processing block, Using the quantization function, perform a quantization operation on a first product of the trained weight set of the processing block and the trained affine transformation matrix for the processing block to obtain a quantized weight set; performing, based at least on the set of quantization weights, a processing procedure corresponding to the processing block on a second product of a target input of the processing block and an inverse matrix of the affine transformation matrix to obtain a quantized output of the processing block, wherein the target input of the processing block is based on the target model input; as well as A target model output corresponding to the target model input is determined based on the quantized output of the at least one processing block.

10. The method according to any one of claims 1 to 9, wherein the target model comprises a language model.

11. A device for model training, comprising: a model acquisition module configured to obtain a target model to be trained, wherein the target model includes a plurality of processing blocks, a parameter value set of the target model includes parameter value sets of the plurality of processing blocks, and at least one processing block of the plurality of processing blocks is configured with a corresponding affine transformation matrix; A block processing module is configured to, in the process of training the target model using the training data set, for each processing block in the at least one processing block, Based at least on a weight set in a parameter value set of the processing block, performing a processing process corresponding to the processing block on an input of the processing block to obtain a non-quantized output of the processing block; Using a quantization function, performing a quantization operation on a first product of the weight set and an affine transformation matrix for the processing block to obtain a quantized weight set; performing, based at least on the set of quantization weights, a processing procedure corresponding to the processing block on a second product of the input and the inverse matrix of the affine transformation matrix to obtain a quantized output of the processing block; as well as An iterative update module is configured to iteratively update the parameter value set of the target model and the affine transformation matrix corresponding to each of the at least one processing blocks according to an optimization objective, wherein the optimization objective is at least configured to reduce or minimize the error between the non-quantized output and the quantized output of the at least one processing block.

12. An electronic device comprising: at least one processing unit; as well as At least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the apparatus to perform the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method according to any one of claims 1 to 10 when executed by a processor.

14. A computer program product comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images