Model training method and device and computer storage medium

Abstract

The invention provides a model training method and device and a computer storage medium. The method comprises the steps of obtaining a pre-trained model; the pre-trained model comprises a first parameter value (stored by a first bit width) of a first model parameter; quantizing the pre-trained model to obtain a quantized model; the quantization model comprises a second parameter value (stored through a second bit width smaller than the first bit width) of the first model parameter and a quantization parameter, and the second parameter value is obtained by quantizing the first parameter value through a quantization scale (the number of parameter values sharing the quantization parameter); taking the samples as input of a quantization model, taking a result obtained by aggregating the samples based on the quantization scale as input of a target model, and training the target model based on a prediction result obtained by performing label prediction based on output of the quantization model and the target model and labels of the samples; and correcting quantization parameters based on the trained target model. And the corrected quantitative model can be directly subjected to low-bit reasoning, so that the deployment efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model training method, device and computer storage medium. Background Art

[0002] In recent years, more and more large models have been proposed, such as Large Language Models (LLMs). LLMs have demonstrated amazing capabilities in multiple tasks (such as dialogue, intelligent question and answer, code generation, translation, etc.). However, the number of parameters of LLMs is very large, generally billions or even tens of billions, which means that model fine-tuning and reasoning deployment of LLMs require a huge amount of resources. For example, the Generative Pre-Trained Transformer (GPT)-3 model is 350GB in size at half-precision FP16 (half-precision floating point). However, the current highest-end graphics processing unit (GPU) has 95G of video memory.

[0003] Therefore, there is an urgent need for a model training method to train an inference model that can be efficiently deployed based on fine-tuning.

[0004] The information disclosed in this background technology section is only intended to increase the understanding of the overall background of the application and should not be regarded as an admission or any form of suggestion that the information constitutes the prior art already known to a person skilled in the art. Summary of the invention

[0005] The embodiments of the present application provide a model training method, device and computer storage medium. In the fine-tuning stage after model pre-training, the parameter values ​​of the pre-trained model are quantized through a quantization scale to obtain a quantized model. On the basis of the quantized model, an additional target model is added, the quantized model is fixed, and the features input to the target model are aggregated through the quantization scale, so that the target model can learn the changes in quantization parameters on different tasks. Subsequently, the quantization parameters are directly updated through the model parameters of the trained target model without any processing of the quantized model parameters in the quantized model. In addition, the fused quantized model can directly perform low-bit inference, so that an inference model that can be efficiently deployed can be obtained while ensuring fine-tuning performance.

[0006] In a first aspect, an embodiment of the present application provides a model training method, including: obtaining a pre-trained model; the pre-trained model includes a first parameter value of a first model parameter, and the first parameter value is stored using a first bit width; the pre-trained model is quantized to obtain a quantized model; the quantized model includes a second parameter value of the first model parameter and a quantization parameter, the second parameter value is a parameter value after quantizing the first parameter value using a quantization scale, the quantization scale indicates the number of parameter values ​​of a shared quantization parameter, and the second parameter value is stored using a second bit width, and the second bit width is smaller than the first bit width; determining a sample corresponding to a business task, the sample having a label; using the sample as an input to the quantization model, aggregating the sample based on the quantization scale as an input to a target model, and performing label prediction based on the output of the quantization model and the target model to obtain a prediction result, and a label training target model; based on the second model parameter of the trained target model, correcting the quantization parameter in the quantization model, and using the corrected quantization model as an inference model.

[0007] In this solution, in the fine-tuning stage after model pre-training, the parameter values ​​of the pre-trained model are quantized through the quantization scale to obtain a quantized model. On the basis of the quantized model, an additional target model is added, and the quantized model is fixed. The features input to the target model are aggregated through the quantization scale, so that the target model can learn the changes in quantization parameters on different tasks. Subsequently, the quantization parameters are directly updated through the model parameters of the trained target model without any processing of the quantized model parameters in the quantized model. In addition, the fused quantized model can directly perform low-bit inference, so that an inference model that can be efficiently deployed can be obtained while ensuring fine-tuning performance.

[0008] In one possible implementation, the pre-trained model is quantized, including:

[0009] Based on the quantization scale, the first parameter value is divided to determine a parameter value set; the number of parameter values ​​in the parameter value set is the number of parameter values ​​indicated by the quantization scale; for each parameter value set, a quantization parameter corresponding to the parameter value set is determined; based on the quantization parameter corresponding to the parameter value set, each parameter value in the parameter value set is quantized to obtain a quantized parameter value.

[0010] In an example of this implementation, the first parameter value is expressed as at least one weight matrix, and each weight matrix is ​​configured with a quantization scale; based on the quantization scale, the first parameter value is divided, including: for each weight matrix, based on the quantization scale configured by the weight matrix, the weight of the weight matrix is ​​divided.

[0011] In an example of this implementation, the quantization parameter includes a first parameter and a second parameter, the first parameter indicating the scaling of the parameter value set for the weight range of the second bit width, and the second parameter indicating the offset of the parameter value in the parameter value set.

[0012] Exemplarily, the first parameter is the minimum value of the parameter value in the parameter value set; the second parameter is the ratio of the difference between the maximum value and the minimum value of the parameter value in the parameter value set to the weight range of the second bit width.

[0013] Exemplarily, based on the quantization parameter corresponding to the parameter value set, quantizing each parameter value in the parameter value set to obtain the quantized weight includes: for each parameter value in the parameter value set, determining the difference between the parameter value and the first parameter; determining the result of dividing the difference by the second parameter; and rounding the division result to obtain the quantized parameter value.

[0014] In one possible implementation, the space for storing the target model is smaller than the space for storing the quantized model.

[0015] In one possible implementation, the pre-trained model includes N first network layers, and the quantized model includes N first network layers; the first parameter value indicates the parameter value of the parameter that needs to be learned in the N first network layers; and the quantization scale indicates the quantization scale configured for each first network layer in the N first network layers.

[0016] In an example of the implementation, the target model includes N second network layers; the N second network layers correspond one-to-one to the N first network layers respectively;

[0017] For the first first network layer, the input of the first network layer is the sample; for the i-th first network layer greater than or equal to 2 and less than or equal to N, the input of the i-th first network layer is the target output, and the target output is the result of the fusion of the output of the first network layer of the i-1th layer and the output of the second network layer of the i-1th layer;

[0018] For the first second network layer, the input of the second network layer is the feature after the samples are aggregated by the quantization scale configured by the first network layer; for the i-th first network layer greater than or equal to 2 and less than or equal to N, the input of the i-th first network layer is the result of aggregating the target output by the quantization scale configured by the i-1-th network layer.

[0019] Exemplarily, for each of the N first network layers, the quantization scale configured by the first network layer is the quantization scale of the convolution kernel of each channel in multiple channels, then the input of the i-th first network layer is the minimum scale of the quantization scales configured by each of the multiple channels configured by the i-1-th network layer, and the result of aggregating the target output.

[0020] Exemplarily, for the i-th first network layer, the modified quantization parameter of the first network layer is a result of fusing the quantization parameter with the model parameter of the corresponding second network layer.

[0021] Exemplarily, for the i-th first network layer, the modified quantization parameter of the first network layer is the sum of the result of multiplying the dimension increase matrix, the dimension reduction matrix and the scale constant of the corresponding second network layer and the quantization parameter. Exemplarily, the modified quantization parameter is the second parameter.

[0022] In one possible implementation, a prediction result of label prediction is performed based on the output of a quantization model and a target model, and a target model is trained with labels, including: determining a prediction result for label prediction based on the output of the quantization model and the target model; determining a loss value based on the prediction result and the label, and training the target model based on the loss value.

[0023] In the second aspect, the embodiment of the present application provides a model training device, which includes several modules, each of which is used to execute each step in the model training method provided in the first aspect of the embodiment of the present application, and the division of modules is not limited here. For the specific functions performed by each module of the model training device and the beneficial effects achieved, please refer to the functions of each step of the model training method provided in the first aspect of the embodiment of the present application, which will not be repeated here.

[0024] For example, the model training device includes:

[0025] An acquisition module, used for acquiring a pre-trained model; the pre-trained model includes a first parameter value of a first model parameter, and the first parameter value is stored by a first bit width;

[0026] A quantization module, used to quantize the pre-trained model to obtain a quantized model; the quantized model includes a second parameter value of the first model parameter and a quantization parameter, the second parameter value is a parameter value obtained by quantizing the first parameter value through a quantization scale, the quantization scale indicates the number of parameter values ​​of the shared quantization parameter, and the second parameter value is stored through a second bit width, and the second bit width is smaller than the first bit width;

[0027] A sample determination module is used to determine samples corresponding to business tasks, where the samples have labels;

[0028] A training module is used to use samples as input of a quantization model, aggregate the samples based on the quantization scale as input of a target model, and perform label prediction based on the output of the quantization model and the target model, and label train the target model;

[0029] The correction module is used to correct the quantization parameters in the quantization model based on the second model parameters of the trained target model, and use the corrected quantization model as the inference model.

[0030] In this solution, in the fine-tuning stage after model pre-training, the parameter values ​​of the pre-trained model are quantized through the quantization scale to obtain a quantized model. On the basis of the quantized model, an additional target model is added, and the quantized model is fixed. The features input to the target model are aggregated through the quantization scale, so that the target model can learn the changes in quantization parameters on different tasks. Subsequently, the quantization parameters are directly updated through the model parameters of the trained target model without any processing of the quantized model parameters in the quantized model. In addition, the fused quantized model can directly perform low-bit inference, so that an inference model that can be efficiently deployed can be obtained while ensuring fine-tuning performance.

[0031] In a possible implementation, the quantization module includes:

[0032] A dividing unit, configured to divide the first parameter value based on a quantization scale to determine a parameter value set; the number of parameter values ​​in the parameter value set is the number of parameter values ​​indicated by the quantization scale;

[0033] The quantization unit is used to determine the quantization parameter corresponding to each parameter value set; based on the quantization parameter corresponding to the parameter value set, quantize each parameter value in the parameter value set to obtain a quantized parameter value.

[0034] In an example of this implementation, the first model parameter is at least one weight matrix, each weight matrix is ​​configured with a quantization scale; the parameter value set is a weight set; and the partitioning unit is used to partition the weight matrix for each weight matrix based on the quantization scale configured by the weight matrix to obtain a parameter value set.

[0035] In an example of this implementation, the quantization parameter includes a first parameter and a second parameter, the first parameter indicating the scaling of the parameter value set for the weight range of the second bit width, and the second parameter indicating the offset of the parameter value in the parameter value set.

[0036] Exemplarily, the first parameter is the minimum value of the parameter value in the parameter value set;

[0037] The second parameter is the ratio of the difference between the maximum value and the minimum value of the parameter value in the parameter value set to the weight range of the second bit width.

[0038] Exemplarily, the quantization unit is used to determine, for each parameter value in the parameter value set, a difference between the parameter value and a first parameter; determine a division result of the difference divided by a second parameter; and round the division result to obtain a quantized parameter value.

[0039] In one possible implementation, the space for storing the target model is smaller than the space for storing the quantized model.

[0040] In one possible implementation, the pre-trained model includes N first network layers, and the quantized model includes N first network layers; the first parameter value indicates the parameter value of the parameter that needs to be learned in the N first network layers; and the quantization scale indicates the quantization scale configured for each first network layer in the N first network layers.

[0041] In an example of the implementation, the target model includes N second network layers; the N second network layers correspond one-to-one to the N first network layers respectively;

[0042] For the first first network layer, the input of the first network layer is the sample; for the i-th first network layer greater than or equal to 2 and less than or equal to N, the input of the i-th first network layer is the target output, and the target output is the result of the fusion of the output of the first network layer of the i-1th layer and the output of the second network layer of the i-1th layer;

[0043] For the first second network layer, the input of the second network layer is the feature after the samples are aggregated by the quantization scale configured by the first network layer; for the i-th first network layer greater than or equal to 2 and less than or equal to N, the input of the i-th first network layer is the result of aggregating the target output by the quantization scale configured by the i-1-th network layer.

[0044] Exemplarily, for each of the N first network layers, the quantization scale configured by the first network layer is the quantization scale of the convolution kernel of each channel in multiple channels, then the input of the i-th first network layer is the minimum scale of the quantization scales configured by each of the multiple channels configured by the i-1-th network layer, and the result of aggregating the target output.

[0045] Exemplarily, for the i-th first network layer, the modified quantization parameter of the first network layer is a result of fusing the quantization parameter with the model parameter of the corresponding second network layer.

[0046] Exemplarily, for the i-th first network layer, the modified quantization parameter of the first network layer is the sum of the result of multiplying the dimension increase matrix, the dimension reduction matrix and the scale constant of the corresponding second network layer and the quantization parameter. Exemplarily, the modified quantization parameter is the second parameter.

[0047] In a possible implementation, the training module is used to determine a prediction result for a label based on the output of the quantization model and the target model; determine a loss value based on the prediction result and the label, and train the target model based on the loss value.

[0048] In a third aspect, an embodiment of the present application provides a model training device, comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method provided in the first aspect.

[0049] In a fourth aspect, an embodiment of the present application provides a model training device, characterized in that the device runs computer program instructions to execute the method provided in the first aspect. Exemplarily, the device can be a chip or a processor.

[0050] In one example, the device may include a processor, which may be coupled to a memory, read instructions in the memory and execute the method provided in the first aspect according to the instructions. The memory may be integrated in a chip or a processor, or may be independent of the chip or the processor.

[0051] In a fifth aspect, an embodiment of the present application provides a computer storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer executes the method provided in the first aspect.

[0052] In a sixth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method provided in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a schematic diagram of an artificial intelligence main framework provided by an embodiment of the present application;

[0054] Figure 2 It is a schematic diagram of a model training scenario provided in the related art;

[0055] Figure 3 It is a schematic diagram of a training method using the Lora method provided in the related art;

[0056] Figure 4 It is a scenario schematic diagram of the model training method provided in the embodiment of the present application;

[0057] Figure 5 It is a schematic diagram of the architecture of a model training system provided in an embodiment of the present application;

[0058] Figure 6 It is a flow chart of a model training method provided in an embodiment of the present application;

[0059] Figure 7 is a schematic diagram of a model training scenario provided in an embodiment of the present application;

[0060] Figure 8a yes Figure 7 Schematic diagram of medium quantification;

[0061] Figure 8b yes Figure 7 Schematic diagram of sample aggregation in ;

[0062] Figure 9a This is a schematic diagram of a system architecture provided by an embodiment of the present application. Figure 1 ;

[0063] Figure 9b This is a schematic diagram of a system architecture provided by an embodiment of the present application. Figure 2 ;

[0064] Fig.10 It is a structural schematic diagram of a model training device provided in an embodiment of the present application;

[0065] Fig.11 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0067] In the description of the embodiments of the present application, words such as "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a concrete way.

[0068] In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the term "multiple" means two or more. For example, multiple systems refers to two or more systems, and multiple terminals refers to two or more terminals.

[0069] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprises", "has" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0070] In the embodiments shown below, various terms and English abbreviations, such as weight matrix parameter, freeze, fine-tune, etc., are illustrative examples given for the convenience of description and should not constitute any limitation to this application. This application does not exclude the possibility of defining other terms that can achieve the same or similar functions in existing or future protocols.

[0071] Below, some terms in this embodiment are explained. It should be noted that these explanations are for the convenience of understanding by those skilled in the art and do not constitute a limitation on the scope of protection claimed in this application.

[0072] Artificial Intelligence (AI) is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems.

[0073] Large model: refers to a machine learning model with a large number of parameters and complex structure.

[0074] Model quantization: It is a technology that converts floating-point calculations into low-bit fixed-point calculations, which can effectively reduce the model's computing intensity, parameter size, and memory consumption, but often results in huge precision loss. For example, the parameter type in the AI ​​model is quantized from full precision or half precision (FP32 / FP16) to low precision (Int8 / Int4).

[0075] Dequantization: Quantized floating-point numbers are dequantized back to floating-point numbers.

[0076] Pre-training (pre-training / trained): refers to a pre-trained model or the process of pre-training a model. Generally, pre-training is obtained by training with a large data set, which facilitates subsequent fine-tuning.

[0077] Fine-tuning: refers to the process of applying a pre-trained model to one’s own dataset and adapting the parameters to one’s own dataset. Generally, the dataset of a real task is small, and better results can be achieved by fine-tuning the pre-trained model.

[0078] Weight Sharing: Reduce the size and storage requirements of the model by sharing some weight matrix parameters.

[0079] Figure 1 A schematic diagram of an artificial intelligence main framework is shown, which describes the overall workflow of the artificial intelligence system and is applicable to general artificial intelligence field requirements.

[0080] The following is an explanation of the above artificial intelligence theme framework from two dimensions: "intelligent information chain" (horizontal axis) and "IT value chain" (vertical axis).

[0081] The "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it can be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, data undergoes a condensed process of "data-information-knowledge-wisdom".

[0082] The "IT value chain" reflects the value that artificial intelligence brings to the information technology industry, from the underlying infrastructure of artificial intelligence, information (providing and processing technology implementation) to the system's industrial ecological process.

[0083] (1) Infrastructure:

[0084] The infrastructure provides computing power support for the AI ​​system, enables communication with the outside world, and supports it through the basic platform. It communicates with the outside world through sensors; computing power is provided by smart chips (CPU, NPU, GPU, ASIC, FPGA and other hardware acceleration chips); the basic platform includes distributed computing frameworks and networks and other related platform guarantees and support, which can include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to obtain data, and these data are provided to the smart chips in the distributed computing system provided by the basic platform for calculation.

[0085] (2) Data

[0086] The data on the upper layer of the infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, text, and IoT data of traditional devices, including business data of existing systems and perception data such as force, displacement, liquid level, temperature, and humidity.

[0087] (3) Data processing

[0088] Data processing usually includes data training, machine learning, deep learning, search, reasoning, decision-making and other methods.

[0089] Among them, machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, and training.

[0090] Reasoning refers to the process of simulating human intelligent reasoning in computers or intelligent systems, using formalized information to perform machine thinking and solve problems based on reasoning control strategies. Typical functions are search and matching.

[0091] Decision-making refers to the process of making decisions after intelligent information is reasoned, usually providing functions such as classification, sorting, and prediction.

[0092] (4) General ability

[0093] After the data has undergone the data processing mentioned above, some general capabilities can be further formed based on the results of the data processing, such as an algorithm or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0094] (5) Smart products and industry applications

[0095] Smart products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, which productizes intelligent information decision-making and realizes practical application. Its application areas mainly include: smart manufacturing, smart transportation, smart home, smart medical care, smart security, autonomous driving, safe cities, smart terminals, etc.

[0096] Next, the application scenarios of the embodiments of the present application are introduced.

[0097] Under the above artificial intelligence framework, Figure 1 The data can be voice, text and other data. Correspondingly, the data processing can be wizIran language processing, which can realize various functions such as translation, text analysis, speech recognition, question and answer, voice control, etc.

[0098] At present, many scenarios of artificial intelligence follow the following paradigm: Figure 2 As shown in the figure, firstly, a large model is pre-trained with large-scale data, and then fine-tuned towards multiple, for example, N downstream tasks, to obtain task models for different downstream tasks. However, when fine-tuning downstream data sets or in the inference stage, large language models (LLMs) with millions or even billions of parameters require a large amount of memory and computing power (for example, when using heterogeneous processors such as GPUs for calculations, it consumes video memory and computing power), which greatly limits the application of large models in many tasks.

[0099] At present, large models are mainly used in natural language processing scenarios, and correspondingly, large models are large language models (LLMs).

[0100] It should be known that the large model can be an AI model, which needs to be trained with a data set before it can perform correct reasoning. The training process of the large model does not change the structure of the model, but updates the parameter values ​​of the model parameters. Model parameters are the basic units of the AI ​​model, and many parameters work together to control the results of the model output. The model parameters can be adjusted to increase or decrease the accuracy and precision of the model. Exemplarily, when the large model is a neural network model, the model parameters can be the parameters of each layer in the neural network model, such as weight matrices and biases. Of course, there is usually an initialization process before the parameter values ​​of the model parameters are updated for the first time, that is, the parameter values ​​are pre-configured for the large model.

[0101] The process of training a large model is a supervised learning process, which is specifically: learning (determining) the ideal model parameter values ​​through labeled samples. These determined model parameter values ​​can perform high-accuracy inference on the input feature values ​​during the inference phase of the large model.

[0102] In the process of supervised learning, the parameter values ​​of the model parameters are learned by examining multiple samples and trying to find the parameter values ​​that can minimize the loss. It should be noted that in the process of training the large model, because it is hoped that the output of the large model is as close to the label of the sample as possible (that is, the target value that is really wanted), the parameter values ​​of the model parameters of the large model can be updated according to the difference between the predicted value of the large model and the target value that is really wanted. For example, if the predicted value of the large model is high, the parameter values ​​of the model parameters are adjusted to make it predict lower, and the adjustment is continued until the large model can predict the target value that is really wanted or a value that is very close to the target value that is really wanted. Generally, the "difference between the comparison of the predicted value and the target value" is defined through the loss function or the objective function, which is used to measure the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, so the training of the large model becomes the process of minimizing this loss as much as possible. Therefore, the process of training the large model is also the process of minimizing the output value of the loss function. For example, if the large model is a neural network model, the purpose of training the large network can be transformed into a process of minimizing the loss function of the weight matrix and the bias. Usually, a gradient descent algorithm (a back propagation algorithm is used in multi-layer neural network training) is used to implement the above minimization process.

[0103] In the back propagation algorithm, the repeated iterative process of forward propagation and back propagation is involved. The forward propagation process is the process in which the inter-layer neurons are connected through the weight matrix so that the stimulus (eigenvalue) is continuously transmitted from the previous layer to the next layer through the excitation function of each layer. In the back propagation, the error of this layer needs to be reversely deduced from the error of the next layer. Therefore, through the iterative process of forward and back propagation mentioned above, the weight matrix and bias are continuously adjusted so that the loss function gradually approaches the minimum value, thereby completing the training of the neural network.

[0104] At present, in order to reduce the consumption of pre-trained large models in reasoning, the parameter values ​​of the model parameters of the pre-trained large models can be quantized. Model quantization quantizes the parameter values ​​in the pre-trained large model into smaller fixed-point parameter values ​​and has a faster reasoning speed. It is suitable for the vast majority of models and usage scenarios. Specifically, model quantization linearly maps the floating-point parameters with continuous or discrete values ​​in the pre-trained large model (for example, the weight matrix or tensor in the neural network model) to discrete values ​​of fixed-point approximations (int2, int3, int4, int8) at the cost of losing reasoning accuracy, replacing the original float32 or float16 format data, while keeping the input and output as floating point, thereby achieving the goals of reducing model size, reducing model memory consumption and speeding up model reasoning.

[0105] In order to solve the consumption of pre-trained large models in fine-tuning, the Low-Rank Adaptation of Large Language Models (Lora) was proposed, which is a parameter-efficient fine-tuning method (PEFT). The basic principle of Lora is to freeze the pre-trained model parameters. While freezing the original model parameters, additional network layers are added to the model and only the parameters of these newly added network layers are trained. Since the number of these newly added parameters is small, not only the cost of fine-tuning is significantly reduced, but also similar effects to full model fine-tuning can be obtained. The core idea of ​​Lora is to add a bypass next to the original pre-trained language model, perform a dimensionality reduction and then dimensionality increase operation to simulate the so-called intrinsic rank (the process of generalizing the pre-trained model on various downstream tasks is actually to optimize a very small number of free parameters in the common low-dimensional intrinsic subspace of various tasks). Figure 3As shown in the figure, during training, the parameters of the pre-trained language model are fixed, and only the dimension reduction matrix A and dimension increase matrix B of the bypass are trained. The input and output dimensions of the trained language model and the bypass remain unchanged. When outputting, the output of the bypass is added to the output of the pre-trained language model. A is initialized with a random Gaussian distribution, and B is initialized with a 0 matrix. This ensures that at the beginning of training, the newly added path B*A=0, which has no effect on the model results. Lora mainly improves the performance of the model by learning the weight matrix between each layer in the neural network. Specifically, the Lora model automatically adjusts the weight matrix of the current layer by learning the correlation between the previous layer and the next layer, thereby improving the performance of the model.

[0106] However, the above quantization method does not solve the resource consumption of fine-tuning, and Lora's fine-tuning method only saves the computational overhead during fine-tuning, and cannot ensure that the inference model consumes less hardware resources. Therefore, there is an urgent need for a model training method to train an inference model that can be efficiently deployed based on fine-tuning.

[0107] Based on this, the embodiment of the present application proposes the following model training method.

[0108] like Figure 4As shown, the method uses a quantization technique to quantize the first parameter value (storage bit width is larger) of the first model parameter in the pre-trained model to a second parameter value (storage bit width is smaller) of a lower bit (e.g., int2, int3, int4) according to a quantization scale (used to illustrate the number of parameter values ​​in the quantized set), and obtains a quantized model, wherein the quantized model includes the second parameter value of the first model parameter and the parameter value of the quantized parameter. Then, according to the quantization scale, the input features are aggregated according to the quantization scale to obtain aggregated features; the input features are input into the quantization model, the aggregated features are input into the target model, the output of the quantization model and the output of the target model are fused, such as adding, to obtain output features, and then, label prediction is performed based on the output features to obtain a prediction result, and the target model is trained based on the error between the prediction result and the label. After the target model is trained, the quantization parameter is updated by the second model parameter of the trained target model to update the parameter value of the quantization parameter, and the updated quantization model is used as the final inference model. In this solution, the input features are aggregated according to the quantization scale, so that the process of training the target model is equivalent to aggregating the model parameters of the quantization model, and the changes in the quantization parameters can be learned. Therefore, the model parameters of the trained target model can update the quantization parameters and be directly integrated into the quantization model. The fused quantization model can directly perform low-bit reasoning without dequantizing the quantized model parameters in the quantization model and then fusing them into the target model, so that an inference model that can be efficiently deployed can be obtained while ensuring fine-tuning performance. This is just a brief description of the method. For details about the method, see the description below.

[0109] Next, the model training system that may be applied to the model training method provided in the embodiment of the present application is introduced. Figure 5 The present invention provides a model training system architecture example diagram. The present invention provides a model training method that can be applied to Figure 5 As shown in the system architecture diagram. Figure 5 As shown, the model training includes a terminal device 510 and one or more electronic devices 520.

[0110] Among them, the terminal device 510 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices. Exemplary embodiments of the terminal devices involved in this solution include, but are not limited to, electronic devices equipped with iOS, Android, Windows, Harmony OS, or other operating systems. The embodiment of this application does not specifically limit the type of electronic device.

[0111] Among them, each electronic device 520 can be configured as an independent physical server, and multiple electronic devices 520 can also be configured as a server cluster or distributed system composed of multiple physical servers. They can also be configured as cloud servers or cloud server clusters that provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN) and basic cloud computing services such as big data and artificial intelligence platforms. The cloud server clusters are deployed in several cloud data centers, etc., but are not limited to the above forms.

[0112] Among them, the terminal device 510 communicates with the electronic device 520 through a network. The network can be a wired network or a wireless network. For example, the wired network can be a cable network, an optical fiber network, a digital data network (Digital Data Network, DDN), etc., and the wireless network can be a telecommunication network, an internal network, the Internet, a local area network (Local Area Network, LAN), a wide area network (Wide Area Network, WAN), a wireless local area network (Wireless Local Area Network, WLAN), a metropolitan area network (Metropolitan Area Network, MAN), a public switched telephone network (Public Service Telephone Network, PSTN), a Bluetooth network, a ZigBee network (ZigBee), a mobile phone (Global System for Mobile Communications, GSM), a CDMA (Code Division Multiple Access) network, a CPRS (General Packet Radio Service) network, etc. or any combination thereof. It is understandable that the network can use any known network communication protocol to achieve communication between different client layers and gateways. The above-mentioned network communication protocol can be various wired or wireless communication protocols, such as Ethernet, universal serial bus (USB), Firewire, global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time division code division multiple access (TD-SCDMA), long term evolution (LTE), new radio (NR), Bluetooth, wireless fidelity (Wi-Fi) and other communication protocols.

[0113] In one possible scenario, the electronic device 520 can act as a cloud (a software platform that uses application virtualization technology, integrating multiple functions such as software search, download, use, management, and backup); in specific use, the electronic device 520 deploys a cloud management platform and a data center, and the terminal device 510 and the cloud interact through the cloud management platform 311; in addition, the data center can deploy nodes, where the nodes in the data center can be virtual machine instances, container instances, physical servers, etc.; in the data center, the number of nodes is generally massive.

[0114] In the embodiments of the present application, on the one hand, the node can be used for pre-training or fine-tuning of the model; in one possible case, when the scale of the model is large, the model can be distributed and deployed on multiple nodes for parallel pre-training or fine-tuning based on the idea of ​​model parallelism. In another possible case, the model can be pre-trained or fine-tuned on one node.

[0115] On the other hand, the node can be used to deploy inference models (generally a model that is fine-tuned from a pre-trained model). The deployed inference model can become an AI application, or a part of an AI application. Users can access the AI ​​application online through a web page, or through a client app. When the AI ​​application is used, it can provide a response by calling the inference model deployed on the node through an online call. Thus. The cloud management platform and the inference model can realize inference of online request data and return the inference result. In the process of providing online services using the trained inference model, the cloud management platform can charge according to the number of calls to the inference model, or according to the resource consumption of the inference model.

[0116] In some possible scenarios, if the scale of the inference model is small, it can also be directly deployed in the terminal device 510 or some edge devices for inference.

[0117] The model training method provided in the embodiment of the present application can be applied to the above-mentioned terminal device 510, and can also be applied to the electronic device 520. When it is specifically implemented, it can be run on the terminal device 510 or the electronic device 520 by software. For example, the software can be a service or an application. In addition, the model training method provided in the embodiment of the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network personal computers (Personal Computer, PC), minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The embodiment of the present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The embodiment of the present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0118] In addition, for the development of models that require pre-training and fine-tuning, developers can install the AI ​​development framework on the terminal device 510 and then develop the AI ​​model locally, or use the AI ​​development framework on an online platform (e.g., an online open source framework platform, a public cloud AI basic development platform, etc.) to develop the AI ​​model. The AI ​​development framework in the industry is usually open source. Typical AI development frameworks for developing deep learning models, also known as deep learning frameworks, include PaddlePaddle, Tensorflow, Caffe, Theano, MXNet, Torch, and PyTorch.

[0119] Next, in combination with the model training system provided above, a model training method provided by an embodiment of the present application is introduced in detail. The present application will take the electronic device 520 as an example to explain the solution of the present application.

[0120] Figure 6 Schematic diagram of the flow of the model training method provided in the embodiment of the present application. This embodiment can be applied to electronic devices, and specifically can be applied to electronic devices 520.

[0121] like Figure 6 As shown, the model training method provided in the embodiment of the present application includes at least the following steps:

[0122] Step 601: The electronic device 520 obtains a pre-trained model; the pre-trained model includes a first parameter value of a first model parameter, and the first parameter value is stored using a first bit width.

[0123] Exemplarily, the pre-trained model may be a large model, such as a large language model (LLMs). It should be noted that the pre-trained model is generally obtained by training with a massive data set.

[0124] Exemplarily, the first bit width may be 32 bits or 16 bits. As mentioned above, it may be full precision or half precision (FP32 / FP16).

[0125] It should be noted that model parameters can be understood as parameters that need to be determined through learning.

[0126] It should be understood that the first parameter value can be understood as a set of multiple parameter values.

[0127] Step 602: The electronic device 520 quantizes the pre-trained model to obtain a quantized model; the quantized model includes a second parameter value of a first model parameter and a quantization parameter, the second parameter value is a parameter value after quantizing the first parameter value through a quantization scale, the quantization scale indicates the number of parameter values ​​of a shared quantization parameter, and the second parameter value is stored through a second bit width, and the second bit width is smaller than the first bit width.

[0128] It should be noted that the model structures of the quantized model and the pre-trained model are the same, the difference is that the data types of the parameter values ​​of the model parameters are stored in different data. Since the quantized model does not change the structure of the pre-trained model, it only changes the data type of the parameter value stored in the first model parameter, thereby reducing the storage space to a certain extent. Specifically, the quantized model includes the model structure of the pre-trained model, the second parameter value of the first model parameter, and the parameter value of the second parameter value for the quantized parameter.

[0129] Exemplarily, the second bit width is smaller than the first bit width, for example, it may be 8 bits, 4 bits or 2 bits. As mentioned above, it may be low precision (Int8 / Int4).

[0130] In a feasible implementation, step 602 may specifically include: the electronic device 520 divides the first parameter value (a set of multiple parameter values) of the first model parameter based on a quantization scale (which may be recorded as groupsize for ease of description and distinction), that is, the parameter values ​​of the number of parameter values ​​indicated by the quantization scale groupsize are taken as a group (which may be referred to as a parameter value set and represented by group for ease of description and distinction), and determines the parameter value set group; the number of parameter values ​​in the parameter value set group is the number of parameter values ​​indicated by the quantization scale groupsize; for each parameter value set group, determining the quantization parameter corresponding to the parameter value set group; based on the quantization parameter corresponding to the parameter value set group, for each parameter value in the parameter value set group (which may be recorded as i=1, 2, ..., groupsize) to obtain the quantized parameter value (for the convenience of description and distinction, it can be recorded as ). It should be noted that for the parameter value set group, the quantized weights in the parameter value set group It corresponds to the quantization parameter. The quantization parameter can be understood as a parameter related to quantization. The second parameter value can be understood as all quantized parameter values.

[0131] It should be noted that for each quantized When actually fine-tuning and reasoning the model, processing is performed as needed, which can be dequantization or quantization of parameter values. The correction of ; Assume that the parameter value after processing is recorded as Exemplarily, the processing is performed by the following formula (1).

[0132]

[0133] Here, α and β are The parameter value set group is calculated.

[0134] In one example, the quantization parameter may be a first parameter (denoted as α) and a second parameter (denoted as β). The first parameter α indicates the scaling of the parameter value set group for the parameter value range of the second bit width; the second parameter β indicates the offset of the parameter value in the parameter value set group; correspondingly, for each parameter value in the parameter value set group Quantify, including: for each parameter value in the parameter value set group The parameter value is quantized by the parameter value of the first parameter α and the second parameter β to obtain the quantized parameter value

[0135] For example, the first parameter Among them, min(W R ) represents the minimum value in the parameter value set group; max(W R ) represents the maximum value in the parameter value set group; N represents the second bit width, Indicates the parameter value range.

[0136] For example, the second parameter β=min(W R ).

[0137] For example, round(·) indicates the result of rounding to the specified number of decimal places.

[0138] In a feasible implementation, the first parameter value can be expressed as M (greater than or equal to 1) weight matrices (which can be called the first weight matrix, denoted as W). For the convenience of description and distinction, for the i-th (i=1, 2, ..., M) first weight matrix, the first weight matrix W can be denoted as W i ; Wherein, the quantization scale groupsize indicates the number of weights corresponding to each first weight matrix W. The number of weights corresponding to each first weight matrix W can be the same or different, which can be determined in combination with actual needs. For the convenience of description and distinction, the i-th first weight matrix W i The corresponding number of weights can be recorded as groupsize i ; Each parameter value in the parameter value set group can be understood as a weight; then the specific process of determining the parameter value set may include:

[0139] The electronic device 520 calculates the first weight matrix W for the i-th i , according to the i-th first weight matrix W i The corresponding weight number groupsize i , for the first weight matrix W i The weights in are divided, that is, the number of weights groupsize i The weight of is taken as a parameter value set group to obtain one or more parameter value set groups.

[0140] It should be noted that according to the i-th first weight matrix W i The corresponding weight number groupsize i , for the first weight matrix W i The specific way to divide the weights in is as follows: for the first weight matrix W iFor each column of , the weights of the column are divided according to the number of weights groupsize and the order of the weights of the column to obtain several weight set groups. Assuming that a column in the weight matrix W has 4096 weights and groupsize = 8, then for each column in the weight matrix W, the 8 weights from the 1st to the 8th row are used as a weight set group, the 8 weights from the 9th to the 16th row are used as another weight set group, and so on.

[0141] It is worth noting that the quantization model includes the second parameter value of the first model parameter and the parameter value of the second parameter value for the quantization parameter, and the second parameter value can be expressed as a weight matrix after quantization of each first weight matrix W (for the convenience of description and distinction, it can be called the second weight matrix, denoted as W′); wherein W′ and W have the same scale, and W′ is composed of each weight in W Quantized weights Composition, and weight According to each weight in W Arrangement of the arrangement relationship; each quantized weight The corresponding quantization parameter values. Among them, the i-th first weight matrix W i The second weight matrix can be recorded as W i ′.

[0142] In the embodiment of the present application, exemplarily, the pre-trained model may include N network layers (for ease of description and distinction, it may be referred to as the first network layer), and the first network layer may be a neural network layer, such as a convolutional network; wherein the input of the i-th (greater than or equal to 2) first network layer is the output of the i-1-th first network layer, and when i is equal to 1, the input of the first first network layer is an external sample. Considering that the model structure of the quantization model and the pre-training model is the same, the difference is that the parameter values ​​of the model parameters are stored in different data types, therefore, the quantization model also includes N first network layers, each of the N first network layers has a second weight matrix W′, and each second weight matrix W′ corresponds to a quantization parameter (such as a first parameter α and a second parameter β).

[0143] In the embodiment of the present application, the quantization scale can be understood as the quantization granularity, which can be of three types: layer-by-layer quantization, group-by-group quantization, and channel-by-channel quantization. When group=1, group-by-group quantization is equivalent to layer-by-layer quantization; when group=num_filters (i.e., convolution kernel), group-by-group quantization is equivalent to channel-by-channel quantization.

[0144] In the process of layer-by-layer quantization, each first network layer shares the same quantization parameter, and the selection of the second bit width needs to be determined by considering the range of all weights of the current layer. Correspondingly, the first weight matrix W can be understood as a set of weights of a first network layer.

[0145] Among them, channel-by-channel quantization is a more fine-grained quantization method. In this method, the parameter value range (second bit width) and quantization parameter to be represented are calculated separately for each channel of the convolution kernel of each first network layer. This method can better retain the information of each convolution kernel and produce a better quantization effect. Correspondingly, the first weight matrix W can be understood as a convolution kernel.

[0146] Among them, group-by-group quantization can be understood as taking one group as a unit, and the group shares the same quantization parameter. The selection of the second bit width needs to be determined by considering the range of all weights of the current group. Correspondingly, the first weight matrix W can be understood as a set of weights of a group. A group can be one first network layer or multiple first network layers, and can also be a convolution kernel of one channel.

[0147] Step 603: The electronic device 520 determines a sample, and the sample has a label.

[0148] For example, in the scenarios of translation and text question answering, the sample may be a text feature in a certain language, and the label may be a text in another language.

[0149] For example, in the scenarios of speech recognition, speech control, and speech answering, the sample may be a speech feature, and the label may be a text or a speech signal.

[0150] For example, in an image recognition scenario, the sample may be an image and the label may be a category.

[0151] It should be noted that the above scenarios are only examples and do not constitute specific limitations. Samples and labels can be flexibly designed in combination with actual conditions. There are usually multiple samples. Each sample is processed in the same way.

[0152] Step 604: The electronic device 520 uses the sample as the input of the quantization model, aggregates the sample based on the quantization scale as the input of the target model, and performs label prediction based on the output of the quantization model and the target model, and labels the target model.

[0153] It should be noted that the target model is a bypass model, which is a newly added model. Specifically, the training process of the target model is as follows:

[0154] For each sample, the sample is input into the quantization model, and the samples are aggregated according to the quantization scale to obtain the first output of the quantization model and the second output of the target model. The label prediction is performed based on the first output and the second output to obtain the prediction result; then, the loss value is determined based on the prediction results and labels of all samples, and the target model is trained based on the loss value.

[0155] It is worth noting that, considering that the output of the pre-trained model is only a representation and does not have specific semantics, it is impossible to construct a loss function; in actual applications, it is also necessary to use the task model corresponding to the business task to perform label prediction, and determine the loss value based on the difference between the prediction result of the task model for label prediction and the label. Based on the first output and the second output, the specific process of performing label prediction to obtain the prediction result may include: fusing the first output and the second output, such as adding them, and inputting the fusion result into the task model to obtain the prediction result of the task model for label prediction.

[0156] In an embodiment of the present application, the first parameter value includes a parameter value for processing an external sample (for ease of description and distinction, it may be referred to as an input processing parameter value). In a scenario where the quantization model includes N first network layers, the input processing value may be the second weight matrix W1′ of the first first network layer; the sample (denoted as F) is composed of a vector of multiple features. For each feature, a plurality of continuous element values ​​in the vector of the feature are aggregated according to the quantization scale groupsize corresponding to the input processing parameter value. After processing each feature in the sample, an aggregated sample (denoted as F′) may be obtained. For example, assuming that each feature in sample F is a row with 4096 element values, and groupsize=8, then for each row in sample F, the 8 element values ​​from the 1st column to the 8th column are merged into one element value, the 8 element values ​​from the 9th column to the 16th column are merged into one element value, and so on. It is worth noting that if the quantization granularity adopts channel-by-channel quantization, the input processing parameter value can be a multi-channel convolution kernel, and the quantization scale groupsize corresponding to the input processing parameter value is the minimum value of the groupsize of each of the multiple channels, so that the target model can learn the changes in the quantization parameters under the business task.

[0157] Exemplarily, the quantization model includes N first network layers, and each first network layer includes a second weight matrix W′ after quantizing the first weight matrix W and a parameter value of a quantization parameter corresponding to W′. Correspondingly, the target model includes N second network layers, and the N second network layers correspond one-to-one to the N first network layers. The parameter value of the model parameter of each second network layer (for the convenience of description and distinction, it can be called the second model parameter) is denoted as P. Assuming that the i-th second network layer corresponds to the i-th first network layer, the parameter value of the model parameter is denoted as P. i; In some possible implementations, P consists of a dimension reduction matrix A, a dimension increase matrix B, and a scale constant S. For the i-th second network layer, the dimension reduction matrix A is denoted as A i , the dimension-enhanced matrix B is denoted as B i , the scale constant S is denoted as S i , P i =S i *A i *B i .

[0158] For the i-th first network layer, the input of the i-th first network layer is recorded as x1i in , the output is recorded as x1i out =x1i in *W i ′, where W i Each weight in ′ By processing the formula get.

[0159] For the i-th second network layer, the input of the i-th second network layer is recorded as x2i in , the output is recorded as x2i out . For example, x2i out =x2i in *P i .

[0160] For the ith first network layer and the ith second network layer, x1i out and x2i out The result after fusion (such as addition) is recorded as X(i) out ; Then for the i-th first network layer, x1i in =X(i-1) out ; For the i-th second network layer, x1i in groupsize is the number of weights passing through the i-th first network layer i , for X(i-1) out The result after aggregation.

[0161] It is worth noting that if the quantization granularity adopts the scenario of channel-by-channel quantization, for the i-th first network layer, the first network layer includes the second weight matrix W′ of each of the multiple channels and the parameter value of the quantization parameter corresponding to W′. For the i-th second network layer, x1i in is the minimum value of the weights of the multiple channels passing through the first network layer of the i-th network. outThe result after aggregation; since the minimum number of weights groupsize can be spliced ​​to obtain the number of weights groupsize of various scales, the target model can learn the changes in quantization parameters under business tasks.

[0162] Step 605: The electronic device 520 corrects the quantization parameters in the quantization model based on the second model parameters of the trained target model, and uses the corrected quantization model as the inference model.

[0163] Specifically, based on the second model parameter of the trained target model, at least part of the quantization parameters in the quantization model can be updated to obtain an updated quantization model; the updated quantization model is used as the inference model. In a specific implementation, based on the parameter value of the second model parameter of the trained target model, the parameter value of at least part of the quantization parameters in the quantization model can be updated to obtain an updated quantization model.

[0164] In the embodiment of the present application, the inventors have found through observation that, due to the linear characteristics of matrix multiplication, aggregating the input features is actually equivalent to aggregating the first model parameters in the quantization model, such as the second weight matrix W′. There are parameters for parameter aggregation in the quantization parameters. By modifying the parameter values ​​of the quantization parameters, the distribution of the parameter values ​​of the first model parameters can be modified. Therefore, the model parameters of the target model can be fused into the quantization parameters, so that the fusion can be directly performed without dequantizing the first model parameters of the quantization model to obtain the final inference model. This can obtain an inference model that can be efficiently deployed while ensuring fine-tuning performance.

[0165] Exemplarily, in a scenario where the quantization parameters are the aforementioned first parameter α and second parameter β, the second model parameter of the target model can be integrated into the second parameter β, and the offset is updated to change the distribution of the parameter value for the first model parameter to achieve model fusion. In a more specific scenario of this example, the quantization model includes N first network layers, each first network layer has a second weight matrix W′ and a parameter value of the quantization parameter (first parameter α, second parameter β) corresponding to W′, the target model includes N second network layers, the i-th second network layer corresponds to the i-th first network layer, and the parameter value of the second model parameter is P o , P i =S i *A i *B i ; then for the first network layer of the ith layer in the quantized model, the updated β i For the previous β i and P iCorrespondingly, the first model parameter of the modified quantization model includes the second weight matrix W′ and the parameter value of the first parameter α and the updated parameter value of the second parameter β corresponding to W′.

[0166] It should be noted that the method of updating the second parameter β is only an example and does not constitute a specific limitation. In practical applications, the quantization parameter to be modified can be flexibly designed in combination with actual conditions.

[0167] In this solution, the input of the target model is aggregated according to the quantization scale, so that the process of training the target model is equivalent to aggregating the first model parameters of the quantization model. Therefore, the model parameters of the trained target model can update the quantization parameters and be directly integrated into the quantization model. The fused quantization model can directly perform low-bit inference without dequantizing the first model parameters in the quantization model and then fusing them into the target model. In this way, an inference model that can be efficiently deployed can be obtained while ensuring fine-tuning performance.

[0168] Based on the model training method provided above, the specific application of the model training method is explained. Figure 7 A flowchart of a specific application of a model training method provided for the implementation of this application. Figure 7 The specific contents include:

[0169] The pre-trained model includes N (greater than or equal to 2) first network layers, each of which has a first weight matrix W. The first weight matrix W can be understood as a two-dimensional matrix.

[0170] The number of weights of each of the N first network layers, groupsize, is preset. The number of weights is also the number of parameter values.

[0171] Next, for the i-th first network layer among the N first network layers, based on the number of weights groupsize corresponding to the first network layer i , the first weight matrix W of the first network layer i According to the number of weights groupsize i Divide to obtain one or more weight set groups; for each weight in each weight set group Determine the parameter values ​​and weights of the quantization parameters (α, β) After quantification After completing the quantization of each weight set group, the quantized second weight matrix W corresponding to the i-th first network layer is obtained. i ′、α i , β i .in, In the actual calculation process, it needs to be processed as weights

[0172] For example, Figure 8a As shown, the first weight matrix W1 of the first network layer is divided according to the weight number groupsize1 to obtain multiple groups; wherein W1 consists of multiple columns, which are denoted as W 11 , W 12 , W 13 , ...; multiple groups including W 11 After the division, the four groups, W 12 After the division, the four groups, W 13 After the division, the four groups, ... are divided; each group is quantized to obtain the second weight matrix W1′, α1, β1; wherein W1′ includes W 11 The result W after quantization of each weight in the four groups after division 11 ′、W 12 The result W after quantization of each weight in the four groups after division 12 ′、W 13 The result W after quantization of each weight in the four groups after division 13 ', ...; α1 includes W 11 The parameter values ​​of α and W for each group in the four groups after division 12 The parameter value of α for each of the 4 groups after division, the parameter value of α for each of the 4 groups after division of W1, ...; β1 and α1 are similar and will not be repeated here.

[0173] After the quantization is completed, a quantization model is obtained; wherein the quantization model includes N first network layers, and the i-th first network layer has a second weight matrix W i ′、α i , β i For the i-th first network layer, the input of the i-th first network layer is recorded as x1i in , the output is recorded as x1i out =x1i in *W i ′.

[0174] Next, a new target model can be constructed. The target model includes N second network layers (also referred to as adapters), each of which corresponds to each of the N first network layers. The parameter value of the model parameter of the i-th second network layer is recorded as P i , the dimension reduction matrix A is recorded as A i , the dimension-enhanced matrix B is denoted as B i , the scale constant S is denoted as S i , then Pi =S i *A i *B i ; The input of the i-th second network layer is recorded as x2i in , the output is recorded as x2i out =x2i in *S i *A i *B i .

[0175] Assume X(i) out = output x2i of the ith second network layer out + output x1i of the i-th first network layer out .

[0176] In the training phase, for the i-th first network layer, the input x1i of the i-th first network layer in =X(i-1) out For the i-th second network layer, the input x2i of the i-th second network layer in is the number of weights groupsize passing through the i-th first network layer i , for X(i-1) out The result of aggregation of each feature in X′(i-1) out .

[0177] Next, a business task is determined. The business task includes a data set and a task model. The data set includes multiple samples F, and each sample F has a label.

[0178] Based on the business tasks, the process of updating the model parameters of the target model is as follows:

[0179] For each sample, the sample F is input into the first network layer in the quantization model to obtain the first output x11 of the first network layer out ; Based on the weight number groupsize1 of the first network layer, the sample F is processed as follows Figure 8b The polymerization shown in FIG. 1 obtains the polymerized feature F′; illustratively, as Figure 8bAs shown, assume that sample F includes multiple features denoted as f1, f2, f3, ...; divide sample F according to the weight number groupsize1 to obtain multiple group's; multiple group's; including 4 group's after f1 division, 4 group's after f2 division, 4 group's after f2 division, ...; aggregate each group to obtain the aggregated feature F', the aggregated feature F' includes the value of each group' after f1 division, such as the weighted average, the value of each group' after f2 division, such as the weighted average, the value of each group' after f3 division, ...; after the aggregated feature F is input into the first second network layer in the target model, the second output x21 of the first second network layer is obtained out , the first output x11 ou and second output x21 out After fusion (usually addition), the first output X(1) is obtained out ; X(1) out Input to the second network layer in the quantization model to get the first output x12 of the second network layer out ; Based on the weight number groupsize2 of the second first network layer, for this X(1) ou Proceed as Figure 8b The aggregation shown above gives the aggregated feature X′(1) out , the aggregated feature X′(1) out After inputting into the second second network layer in the target model, the second output x22 of the second second network layer is obtained. out , the first output x12 out and second output x22 out After fusion (usually addition), the second output X(2) is obtained out ; And so on, until we get the Nth output X(N) out ; X(N) out Input into the task model to obtain the prediction results of the task model for the business label; then, based on the prediction results and labels of all samples, determine the loss value; and update the model parameters of the target model based on the loss value.

[0180] Determine whether the iteration stop condition is met (for example, whether the number of iterations reaches a threshold, or whether the model accuracy of the target model meets a preset threshold). If not, after the model parameters of the target model are updated, the update process of the model parameters of the target model is re-executed.

[0181] Next, the quantization parameters in the quantization model are updated based on the model parameters of the trained modulus target model. The specific process is as follows:

[0182] For the i-th first network layer in the quantized model, the parameter value P of the model parameter of the i-th second network layer corresponding to the first network layer i =S i *A i *B i , update β i , the updated β i For the previous β i +P i After completing the update of β of the N first network layers, an updated quantization model is obtained, the quantization model includes N first network layers, and the model parameter of the i-th first network layer is the second weight matrix W i ′、α i , updated β i It should be understood that the updated quantization model incorporates the target model and can be directly used as an inference model.

[0183] Subsequently, the updated quantization model can be used as an inference model. In the inference stage, the input of the first network layer of the ith quantization model can be recorded as Xi in , the output is recorded as Xi out , Xi in is the output X(i-1) of the i-1th neural network out , Xi out =Xi in *W i ′, where W i Each weight in ′ Through the inverse quantization formula such as It is obtained that β is the updated quantization parameter.

[0184] The inventor of the present application has verified through experiments that the above solution can reduce the consumption of hardware resources.

[0185] Exemplarily, the electronic device 520 includes an execution device 522 and a training device 521; see the attached Figure 9a , an embodiment of the present application provides a system architecture.

[0186] The data acquisition device 910 is used to collect task data (generally raw data) under the business task and store it in the database 920. The terminal device 510 can process the task data in the database 920 to obtain multiple samples under the business task and store them in the database 920. The training device 521 executes steps 601 to 605. In step 603, the samples under the business task are obtained from the database 920.

[0187] The inference model 501 obtained by the training device 521 can be applied to different systems or devices, such as Figure 9a As shown, it can be deployed in the execution device 522 or in the terminal device 510.

[0188] The execution device 522 is configured with an I / O interface 5222 for data exchange with external devices. The “user” can input data into the I / O interface 5222 through the terminal device 510 .

[0189] The calculation module 5221 uses the reasoning model 501 to infer the input data and obtain a prediction result, and specifically the prediction result is related to the business task.

[0190] Finally, the I / O interface 5222 returns the processing result to the terminal device 510 and provides it to the user.

[0191] More deeply, the training device 521 can generate corresponding reasoning models 501 for different business tasks based on samples under different business tasks to provide users with better results.

[0192] In the attached Figure 9a In the case shown in , the user can manually specify the data to be input into the execution device 522, for example, by operating in the interface provided by the I / O interface 5222. In another case, the terminal device 510 can automatically input data into the I / O interface 5222 and obtain the result. If the terminal device 510 needs to obtain the user's authorization to automatically input data, the user can set the corresponding authority in the terminal device 510. The user can view the results output by the execution device 522 on the terminal device 510, and the specific presentation form can be a specific method such as display, sound, action, etc. The terminal device 510 can also serve as a data collection terminal to store the collected task data into the database 230.

[0193] It is worth noting that Figure 9a This is only a schematic diagram of a system architecture provided by an embodiment of the present application. The positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. Figure 9a In the embodiment, the data storage system 930 is an external memory relative to the execution device 522. In other cases, the data storage system 930 can also be placed in the execution device 522.

[0194] Exemplarily, the electronic device 520 includes an execution device 522, and the terminal device 510 includes a terminal device 511 and a terminal device 512; see the attached Figure 9b , an embodiment of the present application provides a system architecture.

[0195] The execution device 522 is implemented by one or more servers, and optionally cooperates with other computing devices, such as data storage, routers, load balancers, etc. The execution device 522 can be arranged on one physical site or distributed on multiple physical sites. In the embodiment of the present application, each server in the execution device 522 can be deployed with an inference model.

[0196] Users can operate their respective local devices (such as terminal device 511 and terminal device 512) to interact with execution device 522. Each local device can represent any computing device, such as a personal computer, a computer workstation, a smart phone, a tablet computer, a smart camera, a smart car or other types of cellular phones, a media consumption device, a wearable device, a set-top box, a game console, etc.

[0197] Each user's local device can interact with the execution device 522 through a communication network of any communication mechanism / communication standard. The communication network can be a wide area network, a local area network, a point-to-point connection, etc., or any combination thereof.

[0198] In another implementation, one or more aspects of the execution device 522 may be implemented by each local device. For example, the terminal device 511 may provide local data for the execution device 522, or the terminal device 511 may deploy an inference model and feed back the calculation results of the inference model to the execution device 522; the execution device 522 may update the parameter values ​​of the model parameters of the inference model based on the calculation results.

[0199] It should be noted that all functions of the execution device 522 can also be implemented by a local device. For example, the terminal device 511 can deploy an inference model and provide inference services for its own users, or provide inference services for users of the terminal device 512.

[0200] Based on the same concept as the embodiment of the method of the present application, the embodiment of the present application also provides a model training device. The model training device includes several modules, each of which is used to execute each step in the personal information identification method provided in the embodiment of the present application, and the division of the modules is not limited here. The technicians in the relevant field can clearly understand that in practical applications, the various steps in the model training method provided in the embodiment of the present application can be assigned to different modules as needed, that is, the internal structure of the device is divided into different modules to complete all or part of the functions described above. Each module in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more modules can be integrated in one unit, and the above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of each module are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present application. The specific working process of the module in the above-mentioned device can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0201] For example, the model training device is used to execute the model training method provided in the embodiment of the present application. Fig.10 Schematic diagram of the structure of the model training device provided in the embodiment of the present application. Fig.10 As shown, the model training device provided in the embodiment of the present application includes:

[0202] An acquisition module 1001 is used to acquire a pre-trained model; the pre-trained model includes a first parameter value of a first model parameter, and the first parameter value is stored using a first bit width;

[0203] A quantization module 1002 is used to quantize the pre-trained model to obtain a quantized model; the quantized model includes a second parameter value of the first model parameter and a quantization parameter, the second parameter value is a parameter value obtained by quantizing the first parameter value through a quantization scale, the quantization scale indicates the number of parameter values ​​of the shared quantization parameter, and the second parameter value is stored through a second bit width, and the second bit width is smaller than the first bit width;

[0204] The sample determination module 1003 is used to determine the sample corresponding to the business task, and the sample has a label;

[0205] The training module 1004 is used to use the sample as the input of the quantization model, aggregate the sample based on the quantization scale as the input of the target model, and perform label prediction based on the output of the quantization model and the target model, and label train the target model;

[0206] The correction module 1005 is used to correct the quantization parameters in the quantization model based on the second model parameters of the trained target model, and use the corrected quantization model as the inference model.

[0207] Based on the same concept as the method embodiment of the present application, the embodiment of the present application also provides an electronic device 520. Fig.11 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application.

[0208] like Fig.11 As shown, the electronic device 520 includes a processor 1101 , a memory 1102 , and a network interface 1103 .

[0209] The processor 1101 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0210] Among them, the non-volatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. This is described by way of example but not limitation.

[0211] For example, the memory 1102 may store a computer program, and the processor 1101 executes the computer program to implement the steps in the above-mentioned model training method embodiment, such as Figure 6 Steps 601 to 605 shown. Alternatively, the processor 1101 implements the functions of each module in the above-mentioned device embodiment when executing the computer program. Exemplarily, the computer program can be divided into one or more modules / units, one or more modules / units are stored in the memory 1102, and executed by the processor 1101 to complete the present application. For example, the computer program can be divided into a model acquisition module 1001, a quantization module 1002, a sample determination module 1003, a training module 1004, and a correction module 1005. The specific functions of each module are described above.

[0212] The network interface 1103 is used to send and receive data, for example, to send data processed by the processor 1101 to other electronic devices, or to receive data sent by other electronic devices.

[0213] Of course, to simplify, Fig.11 Only some of the components related to the present application in the electronic device 520 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application conditions, the electronic device 520 may also include any other appropriate components. The structure of the terminal device 510 is similar to that of the electronic device 520, and may include more components.

[0214] In addition, the embodiment of the present application may also provide a computer program product, which includes computer program instructions, and when the computer program instructions are executed by the processor, the processor executes the steps in the model training method of various embodiments of the present application described in the above "Method" section of this specification. Wherein, the computer program product can be written in any combination of one or more programming languages ​​to perform computer program code for the operation of the embodiment of the present application, and the programming language includes an object-oriented programming language, such as Java, C++, etc., and also includes a conventional procedural programming language, such as "C" language or similar programming language.

[0215] In addition, an embodiment of the present application may also provide a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, causes the processor to execute the steps in the model training method according to various embodiments of the present disclosure described in the above “Method” section of this specification. The computer-readable storage medium may adopt any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may, for example, include but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above.

[0216] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0217] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0218] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, rather than limitation, and the above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0219] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0220] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0221] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

[0222] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

Claims

1. A model training method, characterized in that: include: Acquire a pre-trained model; the pre-trained model includes a first parameter value of a first model parameter, and the first parameter value is stored using a first bit width; quantizing the pre-trained model to obtain a quantized model; the quantized model includes a second parameter value of the first model parameter and a quantization parameter, the second parameter value is a parameter value obtained by quantizing the first parameter value through a quantization scale, the quantization scale indicates the number of parameter values ​​sharing the quantization parameter, and the second parameter value is stored through a second bit width, and the second bit width is smaller than the first bit width; Determine a sample corresponding to the business task, wherein the sample has a label; The sample is used as the input of the quantization model, the result of aggregating the sample based on the quantization scale is used as the input of the target model, and the prediction result of label prediction is performed based on the output of the quantization model and the target model, and the label training target model is performed; Based on the trained second model parameters of the target model, the quantization parameters in the quantization model are corrected, and the corrected quantization model is used as the inference model.

2. The method according to claim 1, characterized in that The quantizing the pre-trained model includes: Based on the quantization scale, the first parameter value is divided to determine a parameter value set; the number of parameter values ​​in the parameter value set is the number of parameter values ​​indicated by the quantization scale; For each of the parameter value sets, a quantization parameter corresponding to the parameter value set is determined; based on the quantization parameter corresponding to the parameter value set, each parameter value in the parameter value set is quantized to obtain a quantized parameter value.

3. The method according to claim 2, characterized in that The quantization parameter includes a first parameter and a second parameter, wherein the first parameter indicates a scaling condition of the parameter value set for a weight range of the second bit width, and the second parameter indicates an offset of a parameter value in the parameter value set.

4. The method according to claim 3, characterized in that The first parameter is the minimum value of the parameter value in the parameter value set; The second parameter is a ratio of a difference between a maximum value and a minimum value of a parameter value in the parameter value set to a weight range of the second bit width.

5. The method according to claim 3, characterized in that: The step of quantizing each parameter value in the parameter value set based on the quantization parameter corresponding to the parameter value set to obtain a quantized weight includes: For each parameter value in the parameter value set, determine the difference between the parameter value and the first parameter; determine the division result of the difference divided by the second parameter; and round the division result to obtain a quantized parameter value.

6. The method according to any one of claims 1 to 5, characterized in that: The pre-trained model includes N first network layers, and the quantized model includes N first network layers; the quantization scale indicates the quantization scale configured for each of the N first network layers.

7. The method according to claim 6, characterized in that The target model includes N second network layers; the N second network layers correspond one-to-one to the N first network layers respectively; For the first first network layer, the input of the first network layer is the sample; for the i-th first network layer that is greater than or equal to 2 and less than or equal to N, the input of the i-th first network layer is the target output, and the target output is the result of the fusion of the output of the first network layer of the i-1th layer and the output of the second network layer of the i-1th layer; For the first second network layer, the input of the second network layer is the features of the samples aggregated by the quantization scale configured by the first network layer; For the i-th first network layer which is greater than or equal to 2 and less than or equal to N, the input of the i-th first network layer is a result of aggregating the target output through the quantization scale configured by the i-1-th network layer.

8. The method according to claim 6, characterized in that For the i-th first network layer, the modified quantization parameter of the first network layer is a result of fusing the quantization parameter with a model parameter of the corresponding second network layer.

9. The method according to claim 6, characterized in that The model parameters of the second network layer are a dimension-raising matrix, a dimension-reducing matrix and a scale constant; For the i-th first network layer, the modified quantization parameter of the first network layer is the sum of the result of multiplying the dimension-increasing matrix, the dimension-reducing matrix and the scale constant of the corresponding second network layer and the quantization parameter.

10. A model training device, characterized in that: include: An acquisition module, used for acquiring a pre-trained model; the pre-trained model includes a first parameter value of a first model parameter, and the first parameter value is stored by a first bit width; a quantization module, configured to quantize the pre-trained model to obtain a quantized model; the quantized model includes a second parameter value of the first model parameter and a quantization parameter, the second parameter value is a parameter value obtained by quantizing the first parameter value through a quantization scale, the quantization scale indicates the number of parameter values ​​sharing the quantization parameter, and the second parameter value is stored through a second bit width, and the second bit width is smaller than the first bit width; A sample determination module, used to determine a sample corresponding to a business task, wherein the sample has a label; A training module, configured to use the sample as an input of the quantization model, aggregate the sample based on the quantization scale as an input of the target model, and perform label prediction based on the output of the quantization model and the target model, and the label training target model; The correction module is used to correct the quantization parameters in the quantization model based on the second model parameters of the trained target model, and use the corrected quantization model as the inference model.

11. A model training device, characterized in that: include: at least one memory for storing a program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method according to any one of claims 1 to 9.

12. A model training device, characterized in that: The device runs computer program instructions to perform the method according to any one of claims 1-10.

13. A computer storage medium, characterized in that: The computer storage medium stores instructions, and when the instructions are executed on a computer, the computer executes the method according to any one of claims 1 to 9.

14. A computer program product comprising instructions, characterized in that When the instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 9.

Images