Fine-tuning method for large model based on hybrid quantization and related equipment

Abstract

The application provides a large model fine-tuning method based on hybrid quantization and related equipment, including: determining a plurality of parameter matrices of a to-be-trained model, a plurality of quantization methods for training the to-be-trained model, and a constraint condition; iteratively quantizing any parameter matrix according to the constraint condition and the plurality of quantization methods, determining a quantization loss value set and a video memory occupation value set of any parameter matrix; determining an optimal quantization algorithm of any parameter matrix through an optimization method according to the quantization loss value set and the video memory occupation value set; iteratively hybrid-quantizing a plurality of parameter matrices in the to-be-trained model according to a plurality of optimal quantization algorithms, and determining a large model; and adjusting parameters of a low-rank component of the large model through a fine-tuning algorithm to complete model fine-tuning.

Description

Technical Field

[0001] The present application relates to the technical field of large model training, and in particular to a large model fine-tuning method based on hybrid quantization and related equipment. Background Art

[0002] With advances in computing hardware and the availability of massive amounts of data, large models have achieved breakthroughs in fields such as natural language processing, computer vision, and speech recognition. These models often contain billions or more parameters and are capable of capturing complex data patterns, enabling high-precision prediction and generation tasks. However, the training and deployment of large models also face numerous challenges, such as high computing resource consumption, increased storage requirements, and prolonged training time.

[0003] When fine-tuning large models, traditional methods often directly apply full-precision training, which is not only inefficient but also prone to resource bottlenecks. To reduce the resources required for fine-tuning large models, researchers have begun exploring quantization methods, quantizing the non-training parameters of large models. While this method can reduce the graphics memory required for fine-tuning large models to a certain extent, it does not fully utilize device resources and also suffers from quantization losses. Summary of the Invention

[0004] In view of this, the purpose of this application is to propose a large model fine-tuning method based on hybrid quantization and related equipment.

[0005] Based on the above objectives, this application provides a large model fine-tuning method based on hybrid quantization, including:

[0006] Determine several parameter matrices of the model to be trained, several quantization methods and constraints for training the model to be trained;

[0007] According to the restriction conditions and the plurality of quantization methods, iterative hybrid quantization is performed on any of the parameter matrices to determine a quantization loss value set and a video memory occupancy value set of any of the parameter matrices;

[0008] Determining an optimal quantization algorithm for any of the parameter matrices by an optimization method according to the quantization loss value set and the video memory occupancy value set;

[0009] Performing iterative hybrid quantization on a plurality of parameter matrices in the to-be-trained model according to the plurality of the optimal quantization algorithms to determine a large model;

[0010] The parameters of the low-rank components of the large model are adjusted through a fine-tuning algorithm to complete model fine-tuning.

[0011] Optionally, the constraint conditions include: a preset quantization loss value and a preset video memory value; the iterative hybrid quantization includes iteratively training a parameter matrix through multiple quantization algorithms;

[0012] The iterative hybrid quantization is performed on any of the parameter matrices according to the restriction conditions and the plurality of quantization methods to determine a quantization loss value set and a video memory occupancy value set of any of the parameter matrices, including:

[0013] Determining an initial quantization loss and an initial quantization component, decomposing a difference between the initial quantization loss and the initial quantization component, and determining a low-rank component;

[0014] quantizing the parameter matrix and the low-rank component by a quantization algorithm to determine a quantized component;

[0015] Calculating the norm of any of the parameter matrices and the sum of the quantization component and the low-rank component to determine the quantization loss of the current iteration round;

[0016] Determine a quantization loss value and a video memory occupancy value of a current quantization algorithm of any parameter matrix according to a current iteration round and the quantization loss of the current iteration round;

[0017] Determine quantization loss values ​​and video memory occupancy values ​​of several quantization algorithms, and determine a quantization loss value set and a video memory occupancy value set.

[0018] Optionally, determining the quantization loss and video memory occupancy value of the current quantization algorithm of any parameter matrix according to the current iteration round and the quantization loss of the current iteration round includes:

[0019] Determining a current iteration round and a quantization loss value of the current iteration round;

[0020] In response to determining that the current iteration round is greater than the preset iteration round, and / or the quantization loss of the current iteration round is greater than the preset quantization loss value, the quantization loss value of the current iteration round is used as the quantization loss value of the current quantization algorithm, and the video memory occupancy size of the current iteration round is used as the video memory occupancy value of the current quantization algorithm.

[0021] Optionally, the quantization algorithm is a high-precision quantization algorithm, the norm is a Frobenius norm, and the decomposition method is an SVD decomposition method.

[0022] Optionally, determining the optimal quantization algorithm for any of the parameter matrices according to the quantization loss value set and the video memory occupancy value set includes:

[0023] Determining a quantization loss value and a video memory occupancy value of any of the quantization algorithms according to the quantization loss value set and the video memory occupancy value set;

[0024] The optimal quantization algorithm is determined by an integer linear programming method according to the preset quantization loss value, the preset video memory value, the quantization loss value and the video memory occupancy value of any of the quantization algorithms.

[0025] Optionally, performing iterative hybrid quantization on a plurality of parameter matrices in the to-be-trained model according to the plurality of the optimal quantization algorithms to determine a large model includes:

[0026] The parameter matrix is ​​iteratively quantized according to an optimal quantization algorithm corresponding to any of the parameter matrices, and in response to determining that a preset number of iterations is reached, the large model is determined.

[0027] Optionally, adjusting the parameters of the low-rank components of the large model by a fine-tuning algorithm to complete model fine-tuning includes:

[0028] The parameters of the low-rank components of the large model are adjusted through a low-rank fine-tuning algorithm to complete model fine-tuning.

[0029] Based on the same inventive concept, an embodiment of the present application further provides a large model fine-tuning device based on hybrid quantization, comprising:

[0030] A determination module is configured to determine a number of parameter matrices of a model to be trained, a number of quantization methods and constraints for training the model to be trained;

[0031] A first hybrid quantization module is configured to perform iterative hybrid quantization on any of the parameter matrices according to the constraint conditions and the plurality of quantization methods, and determine a quantization loss value set and a video memory occupancy value set of any of the parameter matrices;

[0032] an optimization module configured to determine an optimal quantization algorithm for any of the parameter matrices through an optimization method based on the quantization loss value set and the video memory occupancy value set;

[0033] A second hybrid quantization module is configured to perform iterative hybrid quantization on a plurality of parameter matrices in the to-be-trained model according to the plurality of the optimal quantization algorithms to determine a large model;

[0034] The fine-tuning module is configured to adjust the parameters of the low-rank components of the large model through a fine-tuning algorithm to complete model fine-tuning.

[0035] Based on the same inventive concept, an embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, it implements the large model fine-tuning method based on hybrid quantization as described in any one of the above items.

[0036] Based on the same inventive concept, an embodiment of the present application also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute any of the above-mentioned large model fine-tuning methods based on hybrid quantization.

[0037] From the above, it can be seen that the large model fine-tuning method, device, electronic device and storage medium based on hybrid quantization provided by this application solve the optimal quantization algorithm for each parameter matrix of the large model through an optimization algorithm, so that the video memory resources can be fully utilized while the total quantization loss of the large model is minimized. The parameter matrix of the large model is quantized through mixed iteration, further reducing the quantization loss of the large model. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in this application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are merely embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0039] Figure 1 Schematic diagram of the process of a large model fine-tuning method based on hybrid quantization according to an embodiment of the present application;

[0040] Figure 2 Schematic diagram of a large model fine-tuning device based on hybrid quantization according to an embodiment of the present application;

[0041] Figure 3 This is a schematic structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the objectives, technical solutions and advantages of this application more clear, this application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0043] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should have the usual meanings understood by people with ordinary skills in the field to which this application belongs. The "first", "second" and similar words used in the embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0044] In order to facilitate understanding of the technical solutions of the present disclosure, some technical terms involved in the present disclosure are introduced below.

[0045] The Frobenius norm, abbreviated as F-norm, is a matrix norm used to measure the "size" or "energy" of a matrix.

[0046] Hyperparameter, short for hyperparameter.

[0047] In mathematical or engineering software like MATLAB, a parameter matrix is ​​a multidimensional array used to store and process multidimensional data. Each element can be a scalar, vector, matrix, or other data type. The dimensionality of a parameter matrix is ​​determined by its size and can be one-dimensional, two-dimensional, or higher. In large-scale machine learning and deep learning models, parameter matrices store the model's weights and biases. These parameters are learned and adjusted during the training process to minimize a loss function, enabling the model to accurately make predictions or classifications. The dimensionality and complexity of these parameter matrices depend on the specific model architecture and task.

[0048] Video memory, or "GPU memory," is a dedicated high-speed cache memory used to store graphics data in a computer. It's a specialized type of memory on a graphics card, primarily used to store the data the graphics processing unit (GPU) needs to perform rendering tasks.

[0049] Quantization: The process of converting non-training parameters (such as weights and activation values) in a large model from one precision (usually higher floating-point precision, such as FP32) to another lower precision (such as INT8, INT4, or lower). This conversion helps reduce the model's computing and storage resource requirements while maintaining the model's performance as much as possible.

[0050] In order to make the technical solution of the present disclosure clearer and easier to understand, the large model fine-tuning method based on hybrid quantization provided by the embodiment of the present disclosure is described in detail below with reference to the accompanying drawings.

[0051] As mentioned in the technical background section, advances in computing hardware and the availability of massive amounts of data have enabled breakthroughs in fields such as natural language processing, computer vision, and speech recognition. These models, which often contain billions or more parameters, are capable of capturing complex data patterns, enabling high-precision prediction and generation tasks. However, the training and deployment of large models also face numerous challenges, such as high computing resource consumption, increased storage requirements, and prolonged training time.

[0052] When fine-tuning large models, traditional methods often directly apply full-precision training, which is not only inefficient but also prone to resource bottlenecks. To reduce the resources required for fine-tuning large models, researchers have begun exploring quantization methods, quantizing the non-training parameters of large models. While this method can reduce the graphics memory required for fine-tuning large models to a certain extent, it does not fully utilize device resources and also suffers from quantization losses.

[0053] In view of this, the embodiments of the present application provide a large model fine-tuning method, device, electronic device and storage medium based on hybrid quantization. It includes: determining several parameter matrices of the model to be trained, several quantization methods and constraints for training the model to be trained; according to the constraints and several quantization methods, iteratively hybrid quantizing any of the parameter matrices to determine the quantization loss value set and the video memory occupancy value set of any of the parameter matrices; according to the quantization loss value set and the video memory occupancy value set, determining the optimal quantization algorithm of any of the parameter matrices through an optimization method; according to several of the optimal quantization algorithms, iteratively hybrid quantizing several parameter matrices in the model to be trained to determine the large model; adjusting the parameters of the low-rank components of the large model through a fine-tuning algorithm to complete the model fine-tuning. The optimal quantization algorithm for each parameter matrix of the large model is solved by the optimization algorithm, so that the video memory resources can be fully utilized while the total quantization loss of the large model is minimized. The parameter matrix of the large model is quantized through mixed iteration to further reduce the quantization loss of the large model.

[0054] like Figure 1 As shown, the large model fine-tuning method based on hybrid quantization includes:

[0055] Step S102: determining several parameter matrices of the model to be trained, and several quantization methods and constraints for training the model to be trained;

[0056] Step S104: performing iterative quantization on any of the parameter matrices according to the restriction conditions and the plurality of quantization methods, and determining a quantization loss value set and a video memory occupancy value set of any of the parameter matrices;

[0057] Step S106: determining an optimal quantization algorithm for any of the parameter matrices through an optimization method according to the quantization loss value set and the video memory occupancy value set;

[0058] Step S108: performing iterative hybrid quantization on a plurality of parameter matrices in the model to be trained according to the plurality of the optimal quantization algorithms to determine a large model;

[0059] Step S110: Adjust the parameters of the low-rank components of the large model through a fine-tuning algorithm to complete model fine-tuning.

[0060] In step S102, the model to be trained is a large model to be trained, which can be a computer model used in the fields of natural language processing, computer vision, speech recognition, etc. For example, it can be a language large model (NLP), a visual large model (CV), a multimodal large model, a Transformer-based model, a convolutional neural network (CNN)-based model, a recurrent neural network (RNN)-based model, or a hybrid model of one or more types of models.

[0061] Furthermore, in large-scale machine learning and deep learning models, parameter matrices are matrices that store model weights and biases. These parameters are learned and adjusted during the training process to minimize a loss function, enabling the model to accurately predict or classify. The dimensionality and complexity of these parameter matrices depend on the specific model architecture and task.

[0062] Large models typically have complex network structures and a large number of neurons. These neurons are connected by weights, forming complex parameter matrices. These parameter matrices play a crucial role in the training process of large models, determining how the model learns features and patterns from input data. Large models often contain multiple layers, such as input layers, hidden layers, and output layers. Each layer may have multiple neurons and weight connections. In particular, in architectures such as deep neural networks (DNNs) and Transformers, each layer may contain a large number of parameter matrices. For example, in the Transformer architecture, multi-head self-attention layers and feedforward network layers introduce a large number of parameter matrices.

[0063] Among them, the parameter matrix has the functions of feature extraction, weight distribution, and pattern recognition;

[0064] (1) Feature extraction: The parameter matrix can help the model extract useful features from the input data. These features are crucial for the model's subsequent processing and decision-making.

[0065] (2) Weight distribution: In a large model, different parameter matrices represent different weight distributions. These weights determine the relative importance of each part of the model when processing input data.

[0066] (3) Pattern Recognition: Through training, the parameter matrix can learn the potential patterns and regularities in the input data. These patterns and regularities are the basis for the model to make predictions and decisions.

[0067] In some embodiments, the constraints include the actual available video memory size, i.e., a preset video memory value, and a preset quantization loss value. Before implementing the iteration, optimization, quantization, and fine-tuning steps of the large model, it is necessary to determine a target budget, i.e., the available video memory size. Furthermore, it is necessary to determine which quantization methods to use for hybrid quantization. The quantization method must satisfy the requirement that, after quantizing the large model using the lowest bit width quantization method, the video memory size occupied by the large model must be less than the preset target budget, i.e., the available video memory size.

[0068] It's important to note that during the deployment of neural network models, model quantization is often employed to reduce model storage requirements and computational complexity. Model quantization can be roughly categorized into two main types: online quantization (Quantization Aware Training, QAT) and offline quantization (Post Training Quantization, PTQ). Offline quantization is more commonly used during model deployment due to its lower development cost and lower barriers to entry. Neural network quantization essentially maps data from a continuous space to a discrete space, and as such, this process can introduce quantization loss.

[0069] In some embodiments, the large model is quantized by iterative hybrid quantization according to the restriction conditions and several of the quantization methods. Specifically, for each parameter matrix in the large model, iterative quantization is performed by multiple quantization methods, and the iteration is stopped when the preset iteration conditions are met, so as to determine the video memory occupancy (video memory occupancy value size) and quantization loss value corresponding to the multiple quantization methods corresponding to a parameter matrix. For example, for parameter matrix 1, quantization can be performed by 5 methods such as quantization methods A, B, C, D, and E, and finally the video memory occupancy and quantization loss value corresponding to quantization methods A, B, C, D, and E are recorded. Among them, the data set storing the video memory occupancy sizes of the 5 quantization methods is the video memory occupancy value set, and the data set storing the quantization loss values ​​of the 5 quantization methods is the quantization loss value set.

[0070] In some embodiments, iteratively quantizing a parameter matrix using any quantization method includes: determining an initial quantization loss and an initial quantization component, decomposing a difference between the initial quantization loss and the initial quantization component to determine a low-rank component; wherein the difference decomposition is performed using an SVD decomposition method;

[0071] quantizing the parameter matrix and the low-rank component by a quantization algorithm to determine a quantized component;

[0072] Calculating the norm of any of the parameter matrices and the sum of the quantization component and the low-rank component to determine the quantization loss of the current iteration round;

[0073] According to the current iteration round and the quantization loss of the current iteration round, the quantization loss value and the video memory occupancy value of the current quantization algorithm of any parameter matrix are determined.

[0074] In some embodiments, the difference between the parameter matrix W and the quantization component Q is decomposed to obtain low-rank components L1 and L2, as follows:

[0075] L1, L2 = Factorize(WQ, r)

[0076] Where r is the rank of the matrix used for decomposition.

[0077] Use the quantization algorithm to quantize the difference between the parameter matrix W and the low-rank components L1 and L2, and update the quantization component Q. The formula is as follows:

[0078] Q=Quantize(W-L1L2,c)

[0079] Where c is the quantization algorithm used.

[0080] Calculate the Frobenius norm of the sum of the parameter matrix W and the quantization component Q, low-rank components L1, L2, which is the quantization loss of the current iteration round. The formula is as follows:

[0081] ∈ t =||W-(Q+L1L2)|| F

[0082] Determine the quantization loss ∈ of the current iteration t Compared with the quantization loss before the current iteration round ∈ t-1 Whether it increases, or whether it reaches the maximum number of iterations T, if so, stop the iteration and use the quantized loss ∈ t is the final quantization loss, otherwise continue the iterative process.

[0083] In some embodiments, the input of SVD decomposition is matrix A and the rank r of the matrix. The SVD decomposition formula is as follows:

[0084] U,∑,V T =SVD(A, r)

[0085] Among them, U, ∑, V T is the result of SVD decomposition of matrix A, which is 3 matrices; when the shape of matrix A is m*n, the shape of matrix U is m*min(m,n); Σ is a diagonal matrix with a size of min(m,n)*min(m,n); V T The shape is min(m,n)*n.

[0086] The low-rank component L1L2 is obtained by SVD decomposition. The formula is as follows:

[0087]

[0088] In some embodiments, determining the quantization loss and video memory occupancy value of the current quantization algorithm of any parameter matrix based on the current iteration round and the quantization loss of the current iteration round includes:

[0089] Determining a current iteration round and a quantization loss value of the current iteration round;

[0090] In response to determining that the current iteration round is greater than a preset iteration round, and / or the quantization loss of the current iteration round is greater than a preset quantization loss value, the quantization loss value of the current iteration round is used as the quantization loss value of the current quantization algorithm, and the video memory occupied size of the current iteration round is used as the video memory occupied value of the current quantization algorithm. In the iterative process, the parameter matrix is ​​first decomposed into a low-rank component and a quantization component. While iterative quantization is being performed, the low-rank component and the quantization component are updated until a preset iteration end condition is met, that is, the quantization loss no longer increases or the maximum number of iterations is reached.

[0091] In some embodiments, the quantization algorithm is a high-precision quantization algorithm, and the norm is a Frobenius norm.

[0092] In some embodiments, the optimization method inputs the quantization loss, video memory usage, and constraints for each parameter matrix under different quantization algorithms, and outputs the optimal quantization algorithm for each parameter matrix of the large model under the constraints. The optimization algorithm aims to minimize the quantization loss while minimizing the available video memory size after quantization of the large model using the quantization algorithm corresponding to each parameter matrix being solved, given a certain available video memory size.

[0093] The constraint is the maximum memory size that all parameter matrices can occupy. The quantization loss and memory size of each parameter matrix under different quantization algorithms are specific to each parameter matrix. The "quantization loss" in the constraint means that the quantization loss must be minimized while ensuring that "all parameter matrices, after quantization, are smaller than the maximum memory size."

[0094] If the quantization algorithms are [Quantization Method 1, Quantization Method 2, Quantization Method 3], the memory usage of each parameter matrix under different quantization algorithms will also be input in the order [Memory Occupancy 1, Memory Occupancy 2, Memory Occupancy 3] and [Quantization Loss 1, Quantization Loss 2, Quantization Loss 3] in the optimization method input. For each parameter matrix, the output of the optimization method is a value in the range of [0 to (number of quantization methods - 1)]. Based on this value, the optimal quantization method for the parameter matrix can be determined.

[0095] Therefore, determining the optimal quantization algorithm of any of the parameter matrices according to the quantization loss value set and the video memory occupancy value set includes:

[0096] Determining a quantization loss value and a video memory occupancy value of any of the quantization algorithms according to the quantization loss value set and the video memory occupancy value set;

[0097] The optimal quantization algorithm is determined by an integer linear programming method according to the preset quantization loss value, the preset video memory value, the quantization loss value and the video memory occupancy value of any of the quantization algorithms.

[0098] In some optional implementations, the parameter matrices of the large model are not quantized using the same quantization method, but rather an optimization algorithm is used to find the optimal quantization method for each parameter matrix of the large model under constraints. The iterative hybrid quantization of the parameter matrices in the to-be-trained model according to the optimal quantization algorithms to determine the large model includes:

[0099] The parameter matrix is ​​iteratively quantized according to an optimal quantization algorithm corresponding to any of the parameter matrices, and in response to determining that a preset number of iterations is reached, the large model is determined.

[0100] In some optional embodiments, the method of adjusting the parameters of the low-rank components of the large model through a fine-tuning algorithm to complete model fine-tuning includes: adjusting the parameters of the low-rank components of the large model through a low-rank fine-tuning algorithm to complete model fine-tuning.

[0101] In some implementations, fine-tuning of a large model is accomplished by setting hyperparameters such as a learning rate and training rounds.

[0102] From the above, it can be seen that this application uses an optimization algorithm to solve the optimal quantization algorithm for each parameter matrix of the large model, so that the video memory resources can be fully utilized while the total quantization loss of the large model is minimized. The parameter matrix of the large model is quantized through mixed iteration, which further reduces the quantization loss of the large model.

[0103] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied in a distributed scenario and performed by multiple devices working together. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the method.

[0104] It should be noted that the above description is limited to some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that described in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0105] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a large model fine-tuning device based on hybrid quantization.

[0106] refer to Figure 2 The large model fine-tuning device based on hybrid quantization includes:

[0107] A determination module 202 is configured to determine a number of parameter matrices of a model to be trained, a number of quantization methods and constraints for training the model to be trained;

[0108] The first quantization module 204 is configured to perform iterative hybrid quantization on any of the parameter matrices according to the constraint conditions and the plurality of quantization methods, and determine a quantization loss value set and a video memory occupancy value set of any of the parameter matrices;

[0109] An optimization module 206 is configured to determine an optimal quantization algorithm for any of the parameter matrices by an optimization method based on the quantization loss value set and the video memory occupancy value set;

[0110] The second quantization module 208 is configured to perform iterative hybrid quantization on a plurality of parameter matrices in the to-be-trained model according to the plurality of the optimal quantization algorithms to determine a large model;

[0111] The fine-tuning module 210 is configured to adjust the parameters of the low-rank components of the large model through a fine-tuning algorithm to complete model fine-tuning.

[0112] For the convenience of description, the above devices are described as being divided into various modules according to their functions. Of course, when implementing this application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0113] The device of the above embodiment is used to implement the corresponding large model fine-tuning method based on hybrid quantization in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0114] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the program, the large model fine-tuning method based on hybrid quantization described in any of the above embodiments is implemented.

[0115] Figure 3 10 is a schematic diagram showing a more specific hardware structure of an electronic device provided in this embodiment. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are communicatively connected to each other within the device via the bus 1050.

[0116] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0117] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 1020 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0118] The input / output interface 1030 is used to connect input / output modules to implement information input and output. The input / output modules can be configured as components within the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0119] The communication interface 1040 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WiFi, Bluetooth, etc.).

[0120] The bus 1050 comprises a path for transmitting information between the various components of the device (eg, the processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 ).

[0121] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may only include the components necessary to implement the embodiments of this specification, and does not necessarily include all the components shown in the figure.

[0122] The electronic device of the above embodiment is used to implement the corresponding large model fine-tuning method based on hybrid quantization in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0123] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the large model fine-tuning method based on hybrid quantization as described in any of the above embodiments.

[0124] The computer-readable media of this embodiment include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0125] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the large model fine-tuning method based on hybrid quantization as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0126] Based on the same inventive concept, corresponding to the large model fine-tuning method based on hybrid quantization described in any of the above embodiments, the present disclosure further provides a computer program product comprising computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer to cause the computer and / or the processor to perform the large model fine-tuning method based on hybrid quantization. Corresponding to the execution subject corresponding to each step in each embodiment of the large model fine-tuning method based on hybrid quantization, the processor executing the corresponding step can belong to the corresponding execution subject.

[0127] The computer program product of the above embodiment is used to enable the computer and / or the processor to execute the large model fine-tuning method based on hybrid quantization as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0128] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. Within the scope of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0129] In addition, for simplicity of description and discussion, and in order not to make the embodiment of the application difficult to understand, the known power supply / ground connection with integrated circuit (IC) chip and other components may or may not be shown in the accompanying drawings provided. In addition, the device can be shown in the form of a block diagram to avoid making the embodiment of the application difficult to understand, and this also takes into account the following fact, that is, the details of the embodiment of these block diagram devices are highly dependent on the platform to be implemented in the embodiment of the application (that is, these details should be fully within the scope of understanding of those skilled in the art). When specific details (for example, circuit) are set forth to describe exemplary embodiments of the application, it will be apparent to those skilled in the art that the embodiment of the application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0130] Although the present invention has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may utilize the embodiments discussed.

[0131] The embodiments of the present application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of this application.

Claims

1. A large model fine-tuning method based on hybrid quantization, characterized in that: include: Determining several parameter matrices of the model to be trained, several quantization methods and constraints for training the model to be trained; wherein the constraints include: a preset quantization loss value and a preset video memory value; According to the restriction conditions and the plurality of quantization methods, iterative hybrid quantization is performed on any of the parameter matrices to determine a quantization loss value set and a video memory occupancy value set of any of the parameter matrices; wherein the iterative hybrid quantization includes iteratively training a parameter matrix using multiple quantization algorithms; Based on the quantization loss value set and the video memory occupancy value set, an optimal quantization algorithm for any of the parameter matrices is determined by an optimization method; wherein the input of the optimization method is the quantization loss and video memory occupancy size and constraints of each parameter matrix under different quantization algorithms, and the output of the optimization method is the optimal quantization algorithm for each parameter matrix of the large model under the constraints. The goal of the optimization method is to ensure that, under the premise of a certain available video memory size, the large model, after quantization using the quantization algorithm corresponding to each solved parameter matrix, is smaller than the available video memory size while minimizing the quantization loss; Iteratively quantize several parameter matrices in the to-be-trained model according to several of the optimal quantization algorithms to determine a large model; The parameters of the low-rank components of the large model are adjusted through a fine-tuning algorithm to complete model fine-tuning.

2. The method according to claim 1, characterized in that The iterative hybrid quantization is performed on any of the parameter matrices according to the restriction conditions and the plurality of quantization methods to determine a quantization loss value set and a video memory occupancy value set of any of the parameter matrices, including: Determining an initial quantization loss and an initial quantization component, decomposing a difference between the initial quantization loss and the initial quantization component, and determining a low-rank component; quantizing the parameter matrix and the low-rank component by a quantization algorithm to determine a quantized component; Calculating the norm of any of the parameter matrices and the sum of the quantization component and the low-rank component to determine the quantization loss of the current iteration round; Determine a quantization loss value and a video memory occupancy value of a current quantization algorithm of any parameter matrix according to a current iteration round and the quantization loss of the current iteration round; Determine quantization loss values ​​and video memory occupancy values ​​of several quantization algorithms, and determine a quantization loss value set and a video memory occupancy value set.

3. The method according to claim 2, characterized in that The determining, based on the current iteration round and the quantization loss of the current iteration round, the quantization loss and the video memory occupancy value of the current quantization algorithm of any parameter matrix includes: Determining a current iteration round and a quantization loss value of the current iteration round; In response to determining that the current iteration round is greater than the preset iteration round, and / or the quantization loss of the current iteration round is greater than the preset quantization loss value, the quantization loss value of the current iteration round is used as the quantization loss value of the current quantization algorithm, and the video memory occupancy size of the current iteration round is used as the video memory occupancy value of the current quantization algorithm.

4. The method according to claim 2, characterized in that The quantization algorithm is a high-precision quantization algorithm, the norm is the Frobenius norm, and the decomposition method is the SVD decomposition method.

5. The method according to claim 2, characterized in that Determining the optimal quantization algorithm for any of the parameter matrices according to the quantization loss value set and the video memory occupancy value set includes: Determining a quantization loss value and a video memory occupancy value of any of the quantization algorithms according to the quantization loss value set and the video memory occupancy value set; The optimal quantization algorithm is determined by an integer linear programming method according to the preset quantization loss value, the preset video memory value, the quantization loss value and the video memory occupancy value of any of the quantization algorithms.

6. The method according to claim 1, characterized in that The iterative quantization of the parameter matrices in the to-be-trained model according to the optimal quantization algorithms to determine the large model includes: The parameter matrix is ​​iteratively quantized according to an optimal quantization algorithm corresponding to any of the parameter matrices, and in response to determining that a preset number of iterations is reached, the large model is determined.

7. The method according to claim 1, characterized in that The fine-tuning algorithm is used to adjust the parameters of the low-rank components of the large model to complete the model fine-tuning, including: The parameters of the low-rank components of the large model are adjusted through a low-rank fine-tuning algorithm to complete model fine-tuning.

8. A large model fine-tuning device based on hybrid quantization, characterized in that: include: A determination module is configured to determine a plurality of parameter matrices of a model to be trained, a plurality of quantization methods and constraints for training the model to be trained; wherein the constraints include: a preset quantization loss value and a preset video memory value; a first hybrid quantization module configured to perform iterative hybrid quantization on any of the parameter matrices according to the constraint conditions and the plurality of quantization methods, and determine a quantization loss value set and a video memory occupancy value set of any of the parameter matrices; wherein the iterative hybrid quantization includes iteratively training a parameter matrix using multiple quantization algorithms; An optimization module is configured to determine the optimal quantization algorithm for any of the parameter matrices through an optimization method based on the quantization loss value set and the video memory occupancy value set; wherein the input of the optimization method is the quantization loss and video memory occupancy size and constraints of each parameter matrix under different quantization algorithms; the output of the optimization method is the optimal quantization algorithm for each parameter matrix of the large model under the constraints; the goal of the optimization method is to ensure that, under the premise of a certain available video memory size, the large model, after quantization using the quantization algorithm corresponding to each solved parameter matrix, is smaller than the available video memory size while minimizing the quantization loss; A second hybrid quantization module is configured to iteratively quantize a plurality of parameter matrices in the to-be-trained model according to the plurality of the optimal quantization algorithms to determine a large model; The fine-tuning module is configured to adjust the parameters of the low-rank components of the large model through a fine-tuning algorithm to complete model fine-tuning.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the computer program. 10 . A non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method according to claim 1 .

Images