Language task processing method and electronic device

Abstract

The application discloses a language task processing method and electronic equipment, and relates to the technical field of artificial intelligence. The method comprises the following steps: selecting target weight parameters according to task calibration data and an activation function type of an original task processing model; determining a fitting processing result of a quantization parameter according to each target weight parameter, a maximum weight parameter and the task calibration data; determining a first quantization error function and a second quantization error function carrying a scaling factor according to the fitting processing result, and obtaining a value of the scaling factor by minimizing the ratio of the two; performing scaling processing on each target weight parameter by using the scaling factor, and quantizing each scaling weight parameter based on a quantization precision parameter to obtain a task processing model used for executing a to-be-processed language task. The application can solve the problem that related technologies cannot guarantee high accuracy of a language task processing result on the basis of reducing resources used in a language task execution process, and effectively improve language task processing precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a language task processing method and an electronic device. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, the scale of language models is getting larger and larger, which cannot support deployment in resource-constrained scenarios such as mobile terminals or edge computing nodes.

[0003] The related technology directly converts the model parameters of the trained language model into a low-precision format without retraining or fine-tuning the model. Although this can significantly reduce the demand for storage resources and computing resources, it is still difficult to meet the high-precision needs of users in the context of ultra-high parameter quantity. SUMMARY

[0004] The present application provides a language task processing method and an electronic device, which can not only reduce the demand for storage resources and computing resources, but also solve the problem of low accuracy of language task processing results caused by quantization bit width, effectively improving the accuracy of language task processing.

[0005] To solve the above technical problems, the present application provides the following technical solutions:

[0006] In one aspect, the present application provides a language task processing method, comprising:

[0007] According to the task calibration data matched with the to-be-processed language task and the activation function type of the original task processing model, the target weight parameters affecting the task processing result are selected from the weight parameters of the original task processing model. According to each target weight parameter, the maximum weight parameter of the original task processing model and the task calibration data, the fitting processing result of the quantization parameter is determined. According to the fitting processing result, the first quantization error function and the second quantization error function carrying the scaling factor are determined, and by minimizing the ratio of the first quantization error function to the second quantization error function, the value of the scaling factor is obtained. The scaling factor is used to scale each target weight parameter, and each scaled weight parameter is quantized based on the quantization precision parameter to obtain a task processing model for executing the to-be-processed language task.

[0008] The present application also provides an electronic device comprising a memory and a processor, wherein the processor is configured to execute the computer program stored in the memory to implement the steps of any of the above language task processing methods.

[0009] The advantages of the technical solution provided by this invention lie in its ability to analyze the output of task calibration data matching the language task to be processed, determine the target weight parameters that significantly impact the accuracy of the task processing results, and scale these target weight parameters. This reduces the computational and storage resources required during language task execution, effectively minimizing errors caused by weight parameter quantization and improving the accuracy of language task processing results. Furthermore, by fitting the quantization parameters to obtain the gradient and optimizing the scaling factor, a systematic and automated weight scaling process is achieved, compressing the quantization error range and improving the task execution performance of the quantized language task processing model. This not only reduces the demand for storage and computational resources but also solves the problem of low accuracy in language task processing results due to quantization bit width, effectively improving language task processing precision. In addition, this invention provides corresponding electronic equipment for the language task processing method, further enhancing its practicality. The electronic equipment offers corresponding advantages. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the present invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A schematic diagram of the hardware framework applicable to the language task processing method provided by the present invention;

[0012] Figure 2 A flowchart illustrating a language task processing method provided by the present invention;

[0013] Figure 3 A schematic diagram of the neural network provided by this invention;

[0014] Figure 4 A schematic diagram of the quantization results provided by this invention;

[0015] Figure 5 A schematic diagram of fitting the quantization factor provided by the present invention in an exemplary scenario;

[0016] Figure 6 A schematic diagram illustrating the fitting of the quantization function provided by the present invention in an exemplary scenario;

[0017] Figure 7 A flowchart illustrating another language task processing method provided by the present invention;

[0018] Figure 8A structural framework diagram of an exemplary embodiment of the language task processing apparatus provided by the present application;

[0019] Figure 9 A structural diagram of an exemplary embodiment of the electronic device provided by the present application. DETAILED DESCRIPTION

[0020] In order to make the personnel in the technical field better understand the technical solutions of the present application, the present application is further described in detail below in combination with the drawings and specific embodiments. In the specification and the above-mentioned drawings, the terms "first", "second", "third", "fourth" and the like are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. The term "exemplary" means "as an example, embodiment or illustrative". Any embodiment described herein as "exemplary" is not necessarily to be construed as superior to or better than other embodiments.

[0021] In recent years, the parameter size of deep learning models has grown exponentially, and the parameter quantity of some advanced language models has reached hundreds of billions or even thousands of billions. Such models rely on computing resources and massive storage resources provided by clusters of high-performance GPUs (Graphics Processing Units), TPUs (Tensor Processing Units) during training and inference, resulting in high hardware deployment costs. Moreover, due to limitations of data transmission bandwidth and memory access latency, it is difficult to run such models on mobile terminals or edge computing nodes. To break through the resource bottleneck, model quantization compression technology is proposed, which quantizes 32-bit floating-point numbers to 8-bit integer representations, significantly reducing storage space requirements and computational complexity while maintaining model performance to a certain extent. Although the storage cost and computational cost of the model will be lower as the bit width decreases, the quantization bit width (such as 8 bits, 4 bits, etc.) will introduce information loss, leading to a decrease in model output accuracy. For large models with hundreds of billions of parameters, converting high-precision floating-point parameters to low-precision values can cause a significant decrease in model accuracy. To solve this problem, PQT (Post-Quantization Training) and PTQ (Post-Training Quantization) are currently used to achieve model compression. PQT is a method that optimizes model performance through further training after quantization, which can restore model performance to a certain extent. However, for large models, PQT not only increases costs but also easily causes overfitting. PTQ is a method that quantizes the model after training, directly converting model parameters from high-precision (such as 32-bit floating-point numbers) to low-precision (such as 8-bit integers) without the need for retraining or fine-tuning, saving a lot of time and resources, and being more suitable for large-scale language model compression. However, related technologies such as GPTQ (Gradient-based Post-training Quantization) and AWQ (Activation-aware Weight Quantization) use PQT to compress large models, which can alleviate the precision decay to a certain extent but will introduce errors during the quantization process, leading to a decrease in model performance and still being difficult to meet the high-precision requirements of users in the context of ultra-high parameter quantity.

[0022] In order to further improve the language task processing accuracy of the task processing model after PTQ under the premise of extremely low computing resource consumption, the output result of the task calibration data matched with the language task to be processed is analyzed, the target weight parameter which has great influence on the accuracy of the task processing result is determined, the gradient is obtained by fitting the quantization parameter, the optimal scaling factor is obtained by optimization, the target weight parameter is scaled, the weight parameter of the original task processing model of the hundred billion / one hundred billion level parameter after training is quantized with low bits, the storage volume and operation cost are reduced, and the precision loss caused by the reduction of quantization bit width is relieved.

[0023] In combination with the specific application environment architecture or specific hardware architecture on which the execution of the language task processing method depends, the specific application environment architecture or specific hardware architecture is described herein. The following describes various non-limiting embodiments of the present application in combination with the accompanying drawings and specific embodiments. Figure 1 Some possible application scenarios related to the technical solutions of the present application are exemplarily introduced, which can include the following content:

[0024] The hardware composition framework can include a high-performance server 11, an edge computing device 12 and a user terminal 13, and the high-performance server 11 and the edge computing device 12 are connected through a network. The high-performance server 11 can pre-train a large-scale task processing model for executing a language task to be processed, and send the task processing model to the edge computing device 12. The edge computing device 12 receives the task processing model as an original task processing model, and deploys a processor for executing the language task processing method described in any of the above embodiments. The user terminal 13 deploys a user terminal for providing a human-computer interaction interface, and is also connected to the edge computing device 12 through a network. The edge computing device 12 receives the language task to be processed sent by the user terminal 13 through the network, obtains task calibration data matched with the language task to be processed from a public database through an external interface, obtains a task processing model by executing the language task processing method described in any of the above embodiments, processes the language task to be processed by using the task processing model, and sends the final task processing result to the user terminal through the network.

[0025] It should be noted that the above application scenarios are only shown for the purpose of facilitating the understanding of the ideas and principles of the present application, and the embodiments of the present application are not limited in this respect. On the contrary, the embodiments of the present application can be applied to any applicable scenario. After introducing the technical solutions of the present application, the various non-limiting embodiments of the present application will be described in detail in combination with the accompanying drawings and specific embodiments.

[0026] First, please refer to Figure 2 , Figure 2 A flowchart of a language task processing method provided for the present embodiment can include the following content:

[0027] S201: Select target weight parameters affecting the task processing result from the weight parameters of the original task processing model according to the task calibration data matched with the language task to be processed and the activation function type of the original task processing model.

[0028] Wherein, the language task to be processed can be any natural language task, including but not limited to text classification task (assigning one or more predefined categories or labels to a piece of text, such as topic classification, spam identification), sentiment analysis task (identifying and extracting subjective information such as opinions, sentiments and emotions from text), entity recognition task (identifying named entities with specific meanings in text and classifying them into predefined categories), relation extraction task (identifying semantic relationships between entities from text), semantic role labeling (analyzing the relationship between predicates and related components in a sentence), natural language generation task (machine translation, text summarization, dialogue generation, human-computer question and answer, code generation and explanation), etc. The original task processing model refers to a large-scale language model that has been trained and can directly execute the task to be processed. It can be any large language model or multi-modal language model. The task calibration data can be a small part of the training sample data or fine-tuning sample data used to train the original task processing model to have the ability to process the language task to be processed, or historical data of using other language models to process the same task as the language task to be processed. It can be obtained through a public database. For example, for text classification, sentiment analysis and other tasks, the task calibration data is text data, audio and video data, image data with sentiment type labels or text type labels. For natural language generation tasks, the task calibration data is a large amount of pure text. The model learns by itself through self-supervised learning, i.e. constructs input (above) and output (next word). For question and answer tasks, the task calibration data is a set of text data, audio and video data, and image data with question-answer-reasoning process.

[0029] In this embodiment, the original task processing model includes multiple network layers, the output of the previous layer is taken as the input of the next layer, and is transmitted to the last layer. Generally, the vector of the input layer is called input vector, the middle layer is called hidden layer, which also corresponds to a hidden layer vector, and the vector of the output layer is called output vector. Each layer has multiple neurons (i.e. nodes), such as Figure 3As shown, each line corresponds to a weight parameter. Generally, multiple nodes in the same layer can correspond to a vector, and after the task calibration data is input, the calculation is completed through multiple weighted lines, and is passed to the next layer. The output of the previous layer is used as the input of the next layer, and the process is repeated until the last layer. The more nodes and the deeper the neural network, the stronger the expression ability. The parameter amount of the original task processing model in the embodiment is at least on the order of hundreds of billions, so the computing power requirement is extremely high. In this step, the importance of each weight parameter in the original task processing model is evaluated. The type of activation function is considered in the importance evaluation process, and the importance of the weight is measured by analyzing the contribution of the weight to the model output, i.e., the task processing result. For example, the gradient information of the weight, the absolute value of the weight, or the sensitivity of the weight can be used as an indicator to measure the importance of the weight. The target weight parameter is a weight parameter that has a large impact on the accuracy of the task processing result. The more important the weight, the more careful the quantization process needs to be to avoid a large negative impact on the performance of the model.

[0030] S202: Determine the fitting processing result of the quantization parameter according to each target weight parameter, the maximum weight parameter of the original task processing model, and the task calibration data. According to the fitting processing result, determine the first quantization error function and the second quantization error function carrying the scaling factor, and obtain the value of the scaling factor by minimizing the ratio of the first quantization error function to the second quantization error function.

[0031] wherein the quantization parameter includes a quantization factor and a quantization function, the quantization factor and the quantization function can be determined using any model quantization method, and the quantization error function can be determined using the corresponding quantization error function determination method of the corresponding quantization method. Any optimization algorithm can be used to find the optimal value of the function ratio relationship. The present application does not make any limitation on this. The value of the scaling factor obtained by optimization calculation is the optimal scaling factor.

[0032] For example, a general quantization function can be determined by the following relationship:

[0033] ;

[0034] wherein w is a weight parameter, N is the number of quantization bits (such as INT8, which is quantized to a range of 8 bits), Δ is a quantization factor, Round is a rounding function (generally, it is rounding), and |w| represents the absolute value of w. The quantization error function caused by the general quantization method can be represented as: , x is an input element value, and RoundErr represents the error function rounding.

[0035] The quantization method based on the scaling factor, such as the AWQ (Activation-aware Weight Quantization, post-training quantization method), uses a quantization function as shown below:

[0036] ;

[0037] wherein x is an input element value, is a quantization factor corresponding to the method, , is a parameter after introducing scaling, and the quantization error function of the method after introducing the scaling factor can be represented as .

[0038] Only when the quantization factors before and after scaling are the same, that is, , and RoundErr is irrelevant to the internal variable (always in the interval [0, 0.5]), the quantization method based on the scaling factor will reduce the error to 1 / s of the original, that is, However, in the scaling process of the related art, the column w with the maximum activation value is extracted and multiplied by s, which may cause the maximum w to change, thereby affecting The quantization factors before and after scaling are not the same. In addition, when the parameter is purely random, the error of the Round function is 0.25, which can be regarded as a constant. However, the parameter is often not random, but regularly distributed, and RoundErr is usually not 0.25, so RoundErr is often related to the internal variable. In order to further reduce the error introduced by quantization through the scaling method, it is necessary to find the optimal value of the scaling factor. Since the quantization function and the quantization factor are both step functions, it is difficult to optimize them by gradient. In this step, the quantization parameter is fitted, that is, the purpose of fitting the step function is to approximate the quantization factor and the quantization function, and then to realize the gradient optimization process of the loss function to obtain the optimized scaling factor. The fitting result includes whether to approximate or fit the quantization factor, and whether to fit the quantization function.

[0039] S203: scaling each target weight parameter using the scaling factor, and quantizing each scaled weight parameter based on the quantization precision parameter to obtain a task processing model for executing the to-be-processed language task.

[0040] In the scaling process using the scaling factor determined in the previous step, in order to take into account different hardware environments, a scaling scale parameter can also be set based on the actual deployment environment resource configuration and user task accuracy requirements. The scaling scale parameter can be set manually or automatically generated by fine-tuning any large language model capable of dialogue through instructions. When scaling, the scaling scale parameter can be obtained. From the target weight parameters, select a number of weight parameters to be scaled that matches the scaling scale parameter, multiply each weight parameter to be scaled by the scaling factor, and obtain each scaled weight parameter. For example, S201 selects K columns of target weight parameters, and the scaling scale parameter can be M, where M is less than K. For cases where computing resources are abundant, multiple M values can be set to select the best M value for the quantized task processing model. For cases where computing resources are limited, M can generally be specified manually.

[0041] The quantization precision parameter is a pre-set quantization precision, and the precision corresponds to the data precision of the large language model or multi-modal language model parameters in the computer, which is generally in the format of FP32, FP16, BF16, FP8, INT8, FP4, etc. FP32 (single-precision floating-point number) is a floating-point number format using 32 bits of binary representation, and the specific structure is as follows: (1) sign bit: 1 bit, used to represent the positive or negative of the value. (2) exponent bit: 8 bits, used to represent the range of the value. (3) decimal part: 23 bits, used to represent the precision of the value. The value range of FP32 is approximately 1.18e-38 to 3.40e-38, and the precision is approximately 6 to 9 significant digits. A good balance is provided between the value range and the precision, and good hardware support is also obtained. Generally, the quantization precision can be set to W8A16, W4A16, W8A8, W4A4, etc. W corresponds to the precision of the weight parameters of the large language model or multi-modal language model, and A corresponds to the precision of the activation vector of the large language model or multi-modal language model. The quantization precision parameter may, for example, include the number of quantization bits, the quantization range (i.e., the minimum and maximum values, which can be determined by analyzing task calibration data and user requirements), and the quantization step. For important weights, a higher number of quantization bits (such as 8 bits or higher) can be selected to retain more precision information; while for less important weights, a lower number of quantization bits (such as 4 bits or lower) can be used to further reduce storage and computing overhead. The quantization step maps high-precision values to low-precision values, and is calculated by dividing the quantization range by the range represented by low-precision. For example, in W8A8 quantization, if the original range of the weights is from -1.0 to 1.0, the quantization step is 2.0 divided by 255 (for unsigned integers) or divided by 256 (for signed integers). This step is used to convert the weight values from floating-point numbers to 8-bit integers.

[0042] After scaling the target weight parameters selected in S201, this step can quantize the original task processing model using any quantization method described in the relevant documentation. This involves converting the original task processing model and activation values ​​from high precision (e.g., 32-bit floating-point numbers) to low precision (e.g., 8-bit integers) to reduce the storage space and computational complexity of the task processing model when performing the language task, while maintaining the accuracy of the task processing results output by the model. The original image texture is clearly visible, but storing such data occupies a large amount of memory. Therefore, it can be quantized to only have black and white colors. Figure 4 As shown, this significantly reduces the image storage space without affecting target recognition. After quantizing the original task processing model, the resulting model is defined as the task processing model in this step. The task processing model is used to execute the language task to be processed and output the corresponding task processing result. If a similar language task to be processed is received subsequently, the task processing model obtained in S203 can be used directly to execute the corresponding task.

[0043] In the technical solution provided in this embodiment, by analyzing the output results of task calibration data matching the language task to be processed, the target weight parameters that have a significant impact on the accuracy of the task processing results are determined. These target weight parameters are then scaled, which effectively reduces the errors caused by weight parameter quantization while decreasing the computational and storage resources required during language task execution, thereby improving the accuracy of the language task processing results. Furthermore, by fitting the quantization parameters to obtain the gradient and optimizing the scaling factor, a systematic and automated weight scaling is achieved, compressing the quantization error range and improving the task execution performance of the quantized language task processing model. This not only reduces the demand for storage and computational resources but also solves the problem of low accuracy in language task processing results due to quantization bit width, effectively improving the precision of language task processing.

[0044] In the above embodiments, no limitation is made on how to obtain the task calibration data of the language task to be processed. Based on the above embodiments, the present invention also provides an exemplary implementation method, which may include the following:

[0045] If the task type of the language task to be processed has a public dataset, a predetermined number of language sample data are selected from the public dataset to construct a task calibration dataset. If the task type of the language task to be processed does not have a public dataset, a corresponding number of language sample data are obtained from multiple relevant databases according to a predetermined ratio based on the task characteristics of the language task to be processed, to serve as the task calibration dataset.

[0046] In the embodiment, the data set generally corresponds to a collection of sample data, the sample data is composed of samples and labels, for a large language model, the sample can be a combination of code, formula, book, article, webpage and the like, and the label can be a classification label, or code, formula, book, article, webpage and the like. For a multi-modal language model, the sample is generally a combination of multi-modal data + text data, such as image + description of the corresponding image. For a model of a public data set, it is generally assumed that the public data set is the pre-training data set, and a small amount of data can be randomly extracted from the data set according to the classification as the calibration set X (for example, the data set contains A, B and C three sub-data sets, and then 0.01% of data can be extracted from each of them). For a model of an unpublished data set, according to the task characteristics, the task characteristics are the task types, and the types of the task processing model, such as language model type or multi-modal model type. For example, the language task to be processed is a visual language task, and the corresponding task processing model belongs to a visual language model, and the corresponding task calibration data is a combination of image + text data. The required data can be divided into three categories: College-level Problems (college-level (difficulty) problems), Math (mathematics), and General Visual question Answering (general visual question answering). Then, multiple open source data sets are selected in each category, such as Math Vista (Math Vista data set), MATH-Vision (Math Vision data set), and Math Verse (Math Verse data set). A certain amount of data (such as 0.01%) is randomly extracted from each data set to form a task calibration data set.

[0047] As can be seen from the above, the embodiment sets different generation methods of task calibration data according to different task types, and by improving the relevance of the task calibration data and the language task to be processed, the target weight parameter that has a great influence on the task processing result can be accurately selected, and the quantitative loss is reduced, and the accuracy of the task processing result is improved.

[0048] The above embodiment does not make any limitation on how to select the target weight parameter that influences the task processing result, and based on the above embodiment, the present application also provides an exemplary implementation manner, which can include the following contents:

[0049] If the activation function type is a parameterless activation function, the task calibration data is input into the original task processing model. During one forward propagation of the original task processing model, the matrix multiplication results of the weight parameters of each network layer and the corresponding input task data are obtained. A preset number of target matrix multiplication results are selected from the matrix multiplication results in descending order, and the weight parameters corresponding to each target matrix multiplication result are used as target weight parameters. If the activation function type is a parameterized activation function, the task calibration data is input into the original task processing model. During one forward propagation of the original task processing model, the weight parameters and gating parameters of each network layer are multiplied element-wise with the product of the corresponding input task data. A preset number of target matrix multiplication results are selected from the matrix multiplication results in descending order, and the weight parameters corresponding to each target matrix multiplication result are used as target weight parameters.

[0050] In this embodiment, the activation function is the type of function used by the task processing model to determine the task processing result. The type of activation function can be directly determined based on the formula or graph of the activation function. Activation function types include parameterless activation functions and parameterized activation functions. Parameterless activation functions include Sigmoid, Tanh, and ReLU (Rectified Linear Unit). Parameterized activation functions include GLU (Gated Linear Unit) and SwiGLU (Swish-Gated Linear Unit).

[0051] For nonparametric activation functions x corresponds to the input data to the model; in this example, it is task calibration data. Act represents the activation function. For large language models, it is text data; for multimodal language models, it is image + text data. w is the model weight, and h is the output value of the activation function of the large or multimodal language model, generally used for subsequent processing, or directly corresponding to the output text (or image + text). Under a specific input x, the magnitude of wx can reflect the response strength of w to the current input, thus indirectly reflecting the importance of w under that input. It can be directly used... The magnitude of w represents the importance of the weight parameter w. The value of wx is the vector result obtained after multiplying w and x in one forward propagation. That is, it is the tensor obtained after multiplying the current layer weight tensor and the input tensor according to the specified dimensions in the forward propagation. It can be obtained using the framework's built-in one forward propagation without additional training or optimization. As long as the model structure, weights w, and input x are determined, the value of wx is uniquely determined. The maximum m (flexibly determined according to actual conditions, and can be 0.1%-3% of total parameters) parameter columns are taken out as the target weight columns. In the case where no other parameters are introduced, the calculation and analysis process can be saved, thereby saving the calculation resources and storage resources. In the case where the activation function with parameters is adopted , wherein represents element-level multiplication, is a gating parameter. In this case, because the activation is also affected by the gating parameter, only is considered. This will cause the found weight parameters to be not critical, so the embodiment adopts the maximum m parameter columns in the value h after the analysis as the target weight columns for such a scenario.

[0052] As can be seen from the above, the embodiment selects the weight parameters that have a large impact on the task processing result according to the type of the activation function, and can further reduce the required calculation resources and storage resources on the basis of ensuring the accuracy of the selection of the weight parameters.

[0053] The above embodiment does not make any limitation on how to determine the fitting processing result of the quantization function, and the present application further provides an exemplary implementation manner, which can include the following contents:

[0054] For the quantization factor, if the maximum weight parameter of the original task processing model is one of the target weight parameters, the quantization factor is determined according to the maximum weight parameter, the scaling factor and the quantization bit number; if the maximum weight parameter of the original task processing model is not any of the target weight parameters, and is greater than the product of the maximum target weight parameter in the target weight parameters and the scaling factor, the maximum weight parameter is taken as the quantization factor; if the maximum weight parameter of the original task processing model is not in the target weight parameters, and is less than the product of the maximum target weight parameter in the target weight parameters and the scaling factor, the quantization factor is determined according to the maximum target weight parameter, the scaling factor and the quantization bit number, and a one-time function with learnable parameters, smoothness and non-monotonicity is used as the fitting processing result corresponding to the quantization factor.

[0055] For the quantization function, the quantization gradient is determined according to the gap between the quantization interval and the task calibration data; the target polynomial is determined according to the quantization gradient and the quantization interval; and the target polynomial is used to perform the fitting processing on the quantization function, to obtain the fitting processing result corresponding to the quantization function.

[0056] In the embodiment, for the quantization factor, if the maximum weight parameter of the original task processing model is in the target weight parameters screened out in S201, no approximation or fitting processing is performed on it. That is, the quantization factor , if the maximum weight parameter of the original task processing model , if the maximum weight parameter of the original task processing model is not in the target weight parameter, then the fitting processing is performed in different cases, when , the fitting processing is not needed, and when , the fitting processing can be represented as: , wherein is a constant value. , the fitting processing is not needed, and when , the fitting processing can be represented as: , wherein refers to the maximum element in the key weight column. , which is a non-derivative function, can be approximated by a smooth and non-monotonic linear function with learnable parameters. The linear function makes the quantization factor a continuous derivative function. That is, in the fitting process of the quantization factor, the function approximation method is used for the fitting of the quantization factor for the second case. For example, the scaling factor can be used as the independent variable, and an exponential function is determined based on the negative number of the product of the natural constant e as the base, the learnable parameter and the scaling factor. The fitting result of the quantization factor is generated according to the scaling factor as the numerator and the sum of the exponential function and the target constant value as the denominator. For example, the base is the natural constant e, and the relationship can be used to approximate the quantization factor, as shown in Figure 5 , the relationship is approximated to the quantization factor by the learnable parameter β, and a can be any target constant value, such as 1.

[0057] After the approximation of is completed, the quantization function can be fitted, and a polynomial (i.e., a target polynomial) can be used for fitting to obtain a continuous derivative fitting result. The quantization interval can be a pre-set constant, and for integer quantization, the value can be 0.5±fluctuation value. For example, the target polynomial can be: , wherein represents the polynomial, represents the quantization interval, sign represents the sign function, and x represents the task calibration data. When , the value of sign is +1, , the value of sign is -1, and k is the power number flexibly selected according to the actual scene, and the commonly used range is [3, 100]. The higher the k number, the better the fitting effect, but the corresponding gradient is also steeper, as shown in Figure 6 , which may further be unstable in the optimization process.

[0058] From the above, the embodiment can obtain the quantization error function about the scaling factor by selectively approximating the quantization factor and fitting the quantization function using a polynomial, and can calculate the optimal scaling factor using the gradient update method, and the implementation process is simpler.

[0059] Further, the embodiment can use the ratio of the first quantization error function and the second quantization error function as the loss function of the original task processing model, input the task calibration data into the original task processing model, iteratively train the original task processing model based on the loss function, and determine the value of the scaling factor when the loss function is minimum.

[0060] When the fitting of the quantization factor Δ and the quantization function Q is completed, the loss function L is a continuous derivable function related only to the scaling factor s, and s can be updated by gradient calculation to make L minimum, and the optimal s can be determined at this time.

[0061] The above embodiment does not make any limitation on how to quantize the original task processing model, and based on the above embodiment, the present application further provides an exemplary implementation, which can include the following contents:

[0062] The maximum value of the positive values corresponding to each original weight parameter and each scaling weight parameter can be used as the maximum positive weight parameter, and the new quantization factor can be determined according to the maximum positive weight parameter and the quantization bit number; the integer value of the ratio of the new quantization factor to the current weight parameter can be calculated for each original weight parameter and each scaling weight parameter, and the quantized weight parameter corresponding to the current weight parameter can be determined according to the integer value and the new quantization factor.

[0063] In the embodiment, the original weight parameter is a weight parameter in the original task processing model that is not the target weight parameter, and the scaling weight parameter is the result of scaling processing of the target weight parameter. When determining the quantization factor, the absolute values of each original weight parameter and each scaling weight parameter can be processed first, and then sorted in descending order to select the maximum value. In order to distinguish and describe, the selected maximum value can be defined as the maximum positive weight parameter. The maximum positive weight parameter can be determined according to the quantization range and the quantization step, and the quantization factor can be determined according to the maximum positive weight parameter and the quantization bit number. All weight parameters are quantized. Further, the activation function can also be quantized, such as quantizing the weight and the activation value to 8-bit integers, that is, the range of the weight and the activation value is mapped to the range of 8-bit integers, considering the case of signed integers, that is, from -128 to 127 or from 0 to 255.

[0064] In order to make the technical solution of the present application more clear to those skilled in the art, the present application further provides an exemplary implementation, as shown in Figure 7 which can include the following contents:

[0065] S1: Obtain the original task processing model and the quantization precision parameter. The original task processing model may be represented as O, for example.

[0066] S2: According to the original task processing model, determine that the language task sample matched with the language task to be processed has a public data set, and obtain a small amount of language task samples from the public data set to construct a task calibration data set X.

[0067] S3: The language task sample matched with the language task to be processed does not have a public data set, and a small amount of language task samples are obtained from the related open source data set to construct a task calibration data set X.

[0068] S4: Obtain the activation function of the original task processing model, determine the activation function type according to the activation function characteristics (whether containing parameters), if it is a non-parameter activation function, execute S5; if it is a parameter activation function, execute S6.

[0069] S5: Directly use to represent the importance of the weight parameter w, take the first m parameter columns with the maximum wx as the target weight parameters, and construct the key weight column.

[0070] S6: Take the first m parameter columns with the maximum value h after analysis as the key weight column, and execute S7.

[0071] S7: Determine the scaling scale parameter as the scaling column number m.

[0072] S8: Determine the scaling factor s by optimizing the loss function L through the gradient.

[0073] For the quantization factor , if the maximum element is in the key weight column screened out in S7, then , no approximation is needed, and the original can be directly used to calculate the gradient. For the quantization factor , if the maximum element is not in the key weight column, then , a specified continuous differentiable function can be used to fit and approximate this function, and is used to approximate . Use to fit . Update s through gradient calculation to make minimum, and at this time, the best s can be determined.

[0074] S9: Multiply each target weight parameter in the m key weight column by the scaling factor s to obtain a new weight parameter matrix w`.

[0075] S10: performing a standard quantization process on w` to complete quantization of the original task processing model.

[0076] From the above, according to the output analysis result of the task calibration data, the key weight column of the scaling processing is determined, the gradient is obtained by fitting the quantization function through the unary multiple function, and the scaling factor is optimized through the back propagation, so as to realize the systematic and automatic weight scaling, realize the compression of the quantization error range, and improve the accuracy of the task processing result of the task processing model in the case of reducing the resource use through the fine-grained weight quantization compression method provided in the embodiment.

[0077] It should be noted that there is no strict execution order between the steps in the present application, as long as the logical order is met, the steps can be executed simultaneously, or executed in a certain preset order, Figure 2 It is only an illustrative way, and does not mean that only such an execution order.

[0078] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment.

[0079] The present application also provides a corresponding device for the language task processing method, further making the method more practical. Among them, the device can be explained from the perspective of functional modules and hardware. The language task processing device provided by the present application will be introduced below. The device is used to realize the language task processing method provided by the present application. In the present embodiment, the language task processing device can include or be divided into one or more program modules, which are stored in a storage medium and executed by one or more processors to complete the language task processing method disclosed in embodiment one. The program module referred to in the present embodiment refers to a series of computer program instruction segments that can complete a specific function. It is more suitable to describe the execution process of the language task processing device in the storage medium than the program itself. The following description will specifically introduce the functions of each program module in the present embodiment. The language task processing device described in the following description can be mutually corresponding to the language task processing method described in the above description.

[0080] Based on the perspective of functional modules, referring to Figure 8 , Figure 8 The structure diagram of the language task processing device provided in the present embodiment, the device can include:

[0081] The key weight identification module 801 is used to select target weight parameters that affect the task processing results from the weight parameters of the original task processing model based on the task calibration data that matches the language task to be processed and the activation function type of the original task processing model.

[0082] The parameter fitting module 802 is used to determine the fitting result of the quantization parameters based on the target weight parameters, the maximum weight parameter of the original task processing model, and the task calibration data.

[0083] The scaling processing module 803 is used to determine the first quantization error function and the second quantization error function carrying the scaling factor based on the fitting processing result, obtain the value of the scaling factor by minimizing the ratio of the first quantization error function to the second quantization error function, and perform scaling processing on each target weight parameter using the scaling factor.

[0084] The quantization module 804 is used to quantize each scaling weight parameter based on the quantization precision parameter to obtain a task processing model for performing the language task to be processed.

[0085] For example, in some embodiments of this example, the parameter fitting module 802 can also be used to: if the maximum weight parameter of the original task processing model is one of the target weight parameters, then determine the quantization factor based on the maximum weight parameter, the scaling factor, and the number of quantization bits; if the maximum weight parameter of the original task processing model is not any of the target weight parameters, and is greater than the product of the maximum target weight parameter and the scaling factor among the target weight parameters, then use the maximum weight parameter as the quantization factor; if the maximum weight parameter of the original task processing model is not among the target weight parameters, and is less than the product of the maximum target weight parameter and the scaling factor among the target weight parameters, then determine the quantization factor based on the maximum target weight parameter, the scaling factor, and the number of quantization bits, and use a linear function with learnable parameters, smoothness, and non-monotonicity as the fitting result corresponding to the quantization factor.

[0086] As an exemplary implementation of the above embodiments, the parameter fitting module 802 can be further used to: determine an exponential function with the scaling factor as the independent variable and the negative number of the product of the learnable parameter and the scaling factor as the exponent, using a positive integer as the base; and generate a fitting result of the quantization factor by using the sum of the exponential function and the target constant value as the denominator and the scaling factor as the numerator.

[0087] For example, in some other embodiments of this embodiment, the parameter fitting module 802 can also be used to: determine the quantization gradient based on the difference between the quantization interval and the task calibration data; determine the target polynomial based on the quantization gradient and the quantization interval; and use the target polynomial to fit the quantization function to obtain the fitting result corresponding to the quantization function.

[0088] As an exemplary implementation of the above embodiment, the parameter fitting module 802 can be further configured to: call a target polynomial to fit the quantization function; the target polynomial is: ; wherein, represents the polynomial, represents the quantization interval, sign represents the sign function, x represents the task calibration data, and k is the power number.

[0089] Exemplarily, in some other embodiments of the present embodiment, the quantization module 804 can be further configured to: take the maximum value of the positive values corresponding to each original weight parameter and each scaled weight parameter as the maximum positive weight parameter, and determine a new quantization factor according to the maximum positive weight parameter and the quantization bit number; for each original weight parameter and each scaled weight parameter, calculate the integer value of the ratio of the new quantization factor to the current weight parameter, and determine the quantized weight parameter corresponding to the current weight parameter according to the integer value and the new quantization factor.

[0090] Exemplarily, in some other embodiments of the present embodiment, the scaling processing module 803 can be further configured to: obtain a scaling scale parameter; select a number of to-be-scaled weight parameters matched with the scaling scale parameter from the target weight parameters, and obtain each scaled weight parameter by multiplying each to-be-scaled weight parameter with the scaling factor.

[0091] Exemplarily, in some other embodiments of the present embodiment, the key weight identification module 801 can be further configured to: if the activation function type belongs to the non-parameter activation function, input the task calibration data into the original task processing model, obtain the matrix multiplication results of the weight parameters of each network layer and the corresponding input task data in a one-time forward propagation process of the original task processing model, and select a preset number of target matrix multiplication results from the matrix multiplication results in descending order, and take the weight parameters corresponding to each target matrix multiplication result as the target weight parameters; if the activation function type belongs to the parameter activation function, input the task calibration data into the original task processing model, and in a one-time forward propagation process of the original task processing model, multiply the weight parameters and the gating parameters of each network layer with the product of the corresponding input task data element by element, and select a preset number of target matrix multiplication results from the matrix multiplication results in descending order, and take the weight parameters corresponding to each target matrix multiplication result as the target weight parameters.

[0092] Exemplarily, in some other embodiments of the present embodiment, the scaling processing module 803 can be further configured to: take the ratio of the first quantization error function and the second quantization error function as the loss function of the original task processing model; input the task calibration data into the original task processing model, perform iterative training on the original task processing model based on the loss function, and determine the value of the scaling factor when the loss function is minimum.

[0093] The features of the embodiments of the language task processing apparatus can be referred to the related descriptions of the embodiments of the language task processing method, which will not be repeated here.

[0094] The language task processing apparatus mentioned above is described from the perspective of functional modules, and further, the present application also provides an electronic device, which is described from the perspective of hardware. Figure 9 The structure of the electronic device provided by the embodiments of the present application in an implementation manner is shown in the figure. The electronic device includes a memory 91 and a processor 92, the memory 91 stores a computer program, and the processor 92 is configured to run the computer program to perform the steps in any of the above language task processing method embodiments.

[0095] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program, wherein the computer program is configured to perform the steps in any of the above language task processing method embodiments when running.

[0096] In an example embodiment, the above computer readable storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0097] The embodiments of the present application also provide a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the steps in any of the above language task processing method embodiments.

[0098] The embodiments of the present application also provide another computer program product, which includes a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps in any of the above language task processing method embodiments.

[0099] The language task processing method and the electronic device provided by the present application are described in detail above. Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to by each other. The units and algorithm steps of each example described in each disclosed embodiment are executed in an electronic hardware or computer software manner, which depends on the specific application and design constraints of the technical solution. The professional technicians can use different methods to implement the described functions for each specific application, and such implementation should not be considered beyond the scope of the present application. Without departing from the principles of the present application, the present application can be improved and modified, and these improvements and modifications also fall within the scope of the present application.

Claims

1. A language task processing method, characterized in that, include: Based on task calibration data matching the language task to be processed and the activation function type of the original task processing model, target weight parameters that affect the task processing results are selected from the weight parameters of the original task processing model; the language task to be processed includes at least a text classification task. Based on the target weight parameters, the maximum weight parameter of the original task processing model, and the task calibration data, the fitting result of the quantization parameters is determined; Based on the fitting results, a first quantization error function and a second quantization error function carrying a scaling factor are determined. The value of the scaling factor is obtained by minimizing the ratio of the first quantization error function to the second quantization error function. The scaling factor is used to scale each target weight parameter, and the scaling weight parameter is quantized based on the quantization accuracy parameter to obtain a task processing model for performing the language task to be processed. Among them, selecting target weight parameters that influence the task processing result from the weight parameters of the original task processing model includes: If the activation function type is a parameterless activation function, the task calibration data is input into the original task processing model. During one forward propagation of the original task processing model, the matrix multiplication results of the weight parameters of each network layer and the corresponding input task data are obtained. A preset number of target matrix multiplication results are selected from the matrix multiplication results in descending order, and the weight parameters corresponding to each target matrix multiplication result are used as target weight parameters. If the activation function type is a parameterized activation function, the task calibration data is input into the original task processing model. During one forward propagation of the original task processing model, the weight parameters and gating parameters of each network layer are multiplied element-wise by the product of the corresponding input task data. A preset number of target matrix multiplication results are selected from the matrix multiplication results in descending order, and the weight parameters corresponding to each target matrix multiplication result are used as target weight parameters.

2. The language task processing method according to claim 1, characterized in that, The quantization parameters include quantization factors. Based on the target weight parameters, the maximum weight parameter of the original task processing model, and the task calibration data, the fitting result of the quantization parameters is determined, including: If the maximum weight parameter of the original task processing model is one of the target weight parameters, then the quantization factor is determined based on the maximum weight parameter, the scaling factor, and the number of quantization bits. If the maximum weight parameter of the original task processing model is not any target weight parameter, and is greater than the product of the maximum target weight parameter and the scaling factor among all target weight parameters, then the maximum weight parameter is used as the quantization factor. If the maximum weight parameter of the original task processing model is not among the target weight parameters and is less than the product of the maximum target weight parameter and the scaling factor among the target weight parameters, then the quantization factor is determined according to the maximum target weight parameter, the scaling factor and the number of quantization bits, and a smooth and non-monotonic linear function with learnable parameters is used as the fitting result corresponding to the quantization factor.

3. The language task processing method according to claim 2, characterized in that, The fitting results using a smooth, non-monotonic linear function with learnable parameters as the quantization factor include: Using the scaling factor as the independent variable, an exponential function is determined based on the negative of the product of the positive integer as the base, the learnable parameter, and the scaling factor as the exponent. The fitting result of the quantization factor is generated by using the sum of the exponential function and the target constant value as the denominator and the scaling factor as the numerator.

4. The language task processing method according to claim 1, characterized in that, The quantization parameters include a quantization function. Based on each target weight parameter, the maximum weight parameter of the original task processing model, and the task calibration data, the fitting result of the quantization parameters is determined, including: The quantization gradient is determined based on the difference between the quantization interval and the task calibration data. The target polynomial is determined based on the quantization gradient and the quantization interval; The quantization function is fitted using the target polynomial to obtain the fitting result corresponding to the quantization function.

5. The language task processing method according to claim 4, characterized in that, Using the target polynomial, the quantization function is fitted, including: The quantization function is fitted using a target polynomial; the target polynomial is: ; in, Represents a polynomial. The quantization interval is represented by , sign represents the sign function, x represents the task calibration data, and k is the power of exponentiation.

6. The language task processing method according to claim 1, characterized in that, The scaling weight parameters are quantized based on the quantization precision parameter, including: The maximum value among the positive values ​​corresponding to each original weight parameter and each scaling weight parameter is taken as the maximum positive weight parameter, and a new quantization factor is determined based on the maximum positive weight parameter and the number of quantization bits. For each original weight parameter and each scaling weight parameter, calculate the integer value of the ratio of the new quantization factor to the current weight parameter, and determine the quantization weight parameter corresponding to the current weight parameter based on the integer value and the new quantization factor.

7. The language task processing method according to claim 1, characterized in that, The scaling process for each target weight parameter using the scaling factor includes: Get the scaling parameters; Select a number of target weight parameters that match the scaling scale parameter from each target weight parameter, and multiply each target weight parameter with the scaling factor to obtain each scaling weight parameter.

8. The language task processing method according to any one of claims 1 to 7, characterized in that, The scaling factor is obtained by minimizing the ratio of the first quantization error function to the second quantization error function, including: The ratio of the first quantization error function to the second quantization error function is used as the loss function of the original task processing model. The task calibration data is input into the original task processing model, and the original task processing model is iteratively trained based on the loss function to determine the value of the scaling factor when the loss function is minimized.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the language task processing method as described in any one of claims 1 to 8.

Images