Task processing method and device
By determining the fusion process of high-precision target bias weight parameters and low-precision target quantization weight parameters in electronic devices, the accuracy reduction problem caused by weight parameter quantization is solved, and efficient and accurate task processing is achieved.
Patent Information
- Application Number
- CN202411998245.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2025-04-18
AI Technical Summary
In electronic devices, due to limited resources, weight parameters are usually required to quantify, thereby reducing the accuracy of the processing results.
By determining the target quantization weight parameters and the target bias weight parameters, ensure that the accuracy of the target bias weight parameters is higher than that of the target quantization weight parameters, and fuse them to obtain the processing results.
While reducing resource consumption, the accuracy of processing results is improved, and high-precision processing effect can be obtained under the quantized weight parameters.
Smart Images
Figure CN119847709A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of task processing, and in particular to a task processing method and device. Background Art
[0002] Electronic devices can process a variety of tasks based on pre-configured weight parameters, such as text analysis, image search, human-computer dialogue, etc. Since the resources available to electronic devices are limited, the weight parameters configured in the electronic devices can usually be quantized to reduce the accuracy of the weight parameters. For example, by quantizing a 16-bit floating-point type parameter to a 4-bit integer parameter, the resource consumption when processing tasks is reduced and the processing speed is improved.
[0003] However, in this processing method, the decrease in the precision of the weight parameters can easily lead to a decrease in the accuracy of the processing results. Summary of the invention
[0004] To this end, this application discloses the following technical solutions:
[0005] The first aspect of the present application provides a task processing method, comprising:
[0006] Get pending tasks;
[0007] At least determining a target quantization weight parameter and a target bias weight parameter for processing the task to be processed, wherein the expressions of the target quantization weight parameter and the target bias weight parameter meet the target condition, and the precision of the target bias weight parameter is higher than the precision of the target quantization weight parameter;
[0008] The task to be processed is processed based on the target quantization weight parameter and the target bias weight parameter to obtain a processing result.
[0009] Optionally, the target quantization weight parameters include a target quantization weight, a target scaling factor and a target offset, and the target scaling factor and the target offset are used to adjust the target quantization weight; the expression forms of the target quantization weight parameters and the target bias weight parameters satisfy the target conditions, including: at least the matrix expression forms of the target offset and the target bias weight parameters are the same.
[0010] Optionally, processing the task to be processed based on the target quantization weight parameter and the target bias weight parameter includes:
[0011] Superimposing the target offset and the target bias weight parameter to obtain a superimposed offset parameter;
[0012] Combining a scaling weight parameter and the stacking offset parameter to obtain a fusion weight parameter, wherein the scaling weight parameter is obtained based on the target scaling coefficient and the target quantization weight;
[0013] The task to be processed is processed based on the fusion weight parameter.
[0014] Optionally, the method for obtaining the target bias weight parameter includes:
[0015] Obtaining initial quantization weight parameters of the large model, wherein the initial quantization weight parameters include an initial quantization weight, an initial scaling factor, and an initial offset;
[0016] Dequantizing the initial quantization weight parameter to obtain an initial dequantization weight parameter, wherein the precision of the initial dequantization weight parameter is higher than the precision of the initial quantization weight parameter, and the precision of the initial dequantization weight parameter is the same as that of the target bias weight parameter;
[0017] The initial bias weight parameter is updated according to the initial dequantization weight parameter and the sample data to obtain the target bias weight parameter.
[0018] Optionally, the dequantizing the initial quantization weight parameter to obtain the initial dequantization weight parameter includes:
[0019] Determining, in the initial quantization weight parameter, an associated portion associated with the initial bias weight parameter;
[0020] Dequantizing the associated part of the initial quantization weight parameter to obtain the initial dequantization weight parameter of the associated part.
[0021] Optionally, the matrix expression form of the initial bias weight parameter is different from the matrix expression form of the initial offset;
[0022] The updating of the initial bias weight parameter according to the initial dequantization weight parameter and the sample data to obtain the target bias weight parameter includes:
[0023] Performing pooling processing on the initial bias weight parameter to obtain a processed bias weight parameter having the same matrix expression as the initial offset;
[0024] The value of the processed bias weight parameter is updated according to the initial dequantization weight parameter and the sample data to obtain the target bias weight parameter.
[0025] Optionally, updating the value of the processed bias weight parameter according to the initial inverse quantization weight parameter and the sample data to obtain the target bias weight parameter includes:
[0026] The values of the processed bias weight parameters are updated according to the initial inverse quantization weight parameters and sample data of multiple task types to obtain multiple task bias weight parameters corresponding to the multiple task types, wherein the task bias weight parameters corresponding to the task type of the task to be processed are used as the target bias weight parameters.
[0027] Optionally, the target bias weight parameters include:
[0028] At least one of a general bias weight parameter corresponding to a plurality of task types and a task bias weight parameter corresponding to the task type of the task to be processed.
[0029] Optionally, the target quantization weight parameter includes:
[0030] The initial quantization weight parameter;
[0031] Alternatively, the initial inverse quantization weight parameter is updated according to the sample data, and the obtained updated inverse quantization weight parameter is quantized to obtain a quantization weight parameter.
[0032] A second aspect of the present application provides a task processing device, comprising:
[0033] An acquisition unit, used to obtain tasks to be processed;
[0034] A determination unit, configured to determine at least a target quantization weight parameter and a target bias weight parameter for processing the task to be processed, wherein the expressions of the target quantization weight parameter and the target bias weight parameter meet a target condition, and the precision of the target bias weight parameter is higher than the precision of the target quantization weight parameter;
[0035] A processing unit is used to process the task to be processed based on the target quantization weight parameter and the target bias weight parameter to obtain a processing result.
[0036] A third aspect of the present application provides an electronic device, including a memory and a processor;
[0037] The memory is used to store computer programs;
[0038] The processor is used to execute the computer program to implement:
[0039] Get pending tasks;
[0040] At least determining a target quantization weight parameter and a target bias weight parameter for processing the task to be processed, wherein the expressions of the target quantization weight parameter and the target bias weight parameter meet the target condition, and the precision of the target bias weight parameter is higher than the precision of the target quantization weight parameter;
[0041] The task to be processed is processed based on the target quantization weight parameter and the target bias weight parameter to obtain a processing result. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0043] Figure 1 is a flowchart of a task processing method provided by an embodiment of the present application;
[0044] Figure 2 is a flow chart of obtaining target bias weight parameters provided by an embodiment of the present application;
[0045] Figure 3 is a structural diagram of a task processing device provided in an embodiment of the present application;
[0046] Figure 4 is a structural schematic diagram of another task processing device provided in an embodiment of the present application;
[0047] Figure 5 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0048] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0049] This application embodiment provides a task processing method, see Figure 1 , is a flow chart of the method, and the method may include the following steps.
[0050] S101, obtaining tasks to be processed.
[0051] The task to be processed can be any task that can be processed using a large model, such as an image generation task that generates an image based on input information, a document analysis task that analyzes and organizes input documents, a human-computer dialogue task that generates corresponding answers based on input questions, etc. A large model usually refers to a machine learning model with large-scale parameters and complex computing structures, which is a generative model.
[0052] The execution subject of the method of this embodiment can be any terminal device or cloud device that is configured with a large model and can handle the above-mentioned various tasks, such as a smart phone, a personal computer, a server and other devices.
[0053] S102, at least determine a target quantization weight parameter and a target bias weight parameter for processing the task to be processed, the expression forms of the target quantization weight parameter and the target bias weight parameter meet the target conditions, and the accuracy of the target bias weight parameter is higher than the accuracy of the target quantization weight parameter.
[0054] Both the target quantization weight parameter and the target bias weight parameter can be parameters in matrix form, that is, the target quantization weight parameter can include one or more matrices, each matrix includes several rows and several columns of weight data, and the target bias weight parameter can also include one or more matrices, each matrix includes several rows and several columns of weight data.
[0055] The expression form of the target quantization weight parameter can be understood as the matrix expression form of the matrix contained in the target quantization weight parameter, that is, the size of the matrix, and the expression form of the target bias weight parameter can be understood as the matrix expression form of the matrix contained in the target bias weight parameter. The size of the matrix can be expressed by the number of rows and columns of the matrix.
[0056] In this embodiment, the precision of the parameter can be understood as the number of bits occupied by the data in the parameter (also referred to as the bit width of the parameter). The higher the precision of the parameter, the larger the bit width of the weight data contained therein and the more bits occupied. The lower the precision of the parameter, the smaller the bit width of the weight data contained therein and the fewer bits occupied.
[0057] As an example, the weight data included in the target bias weight parameter may be floating point type (ie, fp16) data occupying 16 bits, and the weight data included in the target quantization weight parameter may include integer type (ie, int4) data occupying 4 bits.
[0058] The above target condition may be any condition that enables the matrix of the target quantization weight parameter and the target bias weight parameter to be able to be fused and calculated. In some embodiments, the target quantization weight parameter may include a target quantization weight, a target scaling factor, and a target offset, and the target scaling factor and the target offset are used to adjust the target quantization weight.
[0059] The target quantization weights can be divided into weight data corresponding to multiple layers of the network, the target scaling factor can be used to adjust the numerical differences between different layers, and the target offset can be used to fine-tune the output of each layer when processing tasks.
[0060] Correspondingly, the expressions of the target quantization weight parameter and the target bias weight parameter satisfying the target condition may include: at least the matrix expressions of the target offset and the target bias weight parameter are the same.
[0061] In this embodiment, the target quantization weight parameter can be expressed as aw+b, where a is the target scaling factor, w is the target quantization weight, and b is the target offset. To support block storage or different quantization granularities, a, w, and b can all be composed of several matrices. The matrix sizes of a and b are usually the same, while the matrix size of w can be different from the two. For example, if w contains a matrix of N×128, then a and b may only need to contain a matrix of size N×1, that is, the size of w is 128 times that of a and b.
[0062] In terms of precision, the data types of the target scaling factor, target quantization weight, and target offset can be the same or different. For example, all three can use the int4 data type, or only the target quantization weight w can be quantized to int4, while a and b can be placed in higher precision data types such as fp16.
[0063] S103, processing the task to be processed based on the target quantization weight parameter and the target bias weight parameter to obtain a processing result.
[0064] In step S103, the electronic device may fuse the target quantization weight parameter and the target bias weight parameter, and process the task to be processed based on the fused parameters to obtain a processing result.
[0065] Depending on the task to be processed, the obtained processing results may be different. For example, for an image generation task, the obtained processing results may include one or more images. For a document analysis task, the obtained processing results may include a document summary of the document to be analyzed. For a human-computer dialogue task, the obtained processing results may include answer information corresponding to the input question.
[0066] The beneficial effects of this embodiment are:
[0067] After obtaining the task, this scheme can combine the lower-precision target quantization weight parameters and the higher-precision target bias weight parameters to process the task, thereby reducing resource consumption by quantizing the weight parameters while combining them with the higher-precision target bias weight parameters to obtain a processing result with higher accuracy.
[0068] In this embodiment, the electronic device may obtain and store a variety of bias weight parameters in advance, and when executing S102, a required target bias weight parameter is selected from these bias weight parameters.
[0069] In some embodiments, the above-mentioned multiple bias weight parameters may include a general bias weight parameter and multiple task bias weight parameters. One task bias weight parameter may correspond to a task type, which contains weight data suitable for processing tasks of the task type; the general bias weight parameter may correspond to multiple task types, and the weight data contained therein can be used to process tasks of various different task types.
[0070] The electronic device may store n task bias weight parameters, which are sequentially recorded as task bias weight parameter 1 to task bias weight parameter n, and denoted by t1 to t n Indicates that the universal bias weight parameter can be represented by Δw.
[0071] Thus, the target bias weight parameters may include:
[0072] At least one of a general bias weight parameter corresponding to a plurality of task types and a task bias weight parameter corresponding to a task type of the task to be processed.
[0073] The task bias weight parameter corresponding to the task type of the task to be processed can be expressed as t k Indicates that k is an integer from 1 to n, indicating the task type corresponding to the task to be processed.
[0074] That is, in S102, Δw can be selected as the target bias weight parameter, or t k As target bias weight parameters, we can also choose Δw and t k As the target bias weight parameter.
[0075] In S103, the method of processing the task to be processed based on the target quantization weight parameter and the target bias weight parameter may be:
[0076] The target offset and the target bias weight parameter are superimposed to obtain a superimposed offset parameter;
[0077] The scaling weight parameter and the superposition offset parameter are combined to obtain a fusion weight parameter, where the scaling weight parameter is obtained based on a target scaling factor and a target quantization weight;
[0078] Process pending tasks based on fusion weight parameters.
[0079] Among them, since the target offset and the target bias weight parameter have the same matrix expression, based on this, the sum of the target bias weight parameter and the target offset can be used as the superposition offset parameter. The matrix expression of the superposition offset parameter is the same as the matrix expression of the target offset. The target bias weight parameter includes Δw and t k For example, the method of obtaining the superposition offset parameter can be expressed as: c=b+Δw+t k, c represents the superposition offset parameter. When the target bias weight parameter includes two or more items, for example, Δw and tk, each item included in the target bias weight parameter can be added to the target offset, and the sum is used as the superposition offset parameter.
[0080] For example, in the above example, Δw and t k The weight data of a are added to the weight data of b, and the sum obtained is used as the weight data of c.
[0081] The scaling weight parameter may be represented by aw, and the matrix expression of the scaling weight parameter may be the same as the matrix expression of the target quantization weight.
[0082] In some embodiments, when the electronic device does not need to process a task, it may store the target scaling factor and the target quantization weight separately, and when the task needs to be processed, the electronic device may obtain the scaling weight parameter based on the target scaling factor and the target quantization weight.
[0083] When obtaining the scaling weight parameter, the electronic device may first determine the correspondence between the weight data contained in the target scaling factor and the weight data contained in the target quantization weight, and then multiply the corresponding weight data to obtain the result as the weight data of the scaling weight parameter.
[0084] For example, when the matrix size of the target quantized weight is 128 times the matrix size of the target scaling factor, each group of 128 int4-type weight data may correspond to a scaling factor in the matrix of the target scaling factor. The electronic device may multiply the 128 int4-type weight data by the corresponding scaling factor to generate 128 scaled weight data.
[0085] The process of obtaining the fusion weight parameter can be expressed as W'=aw+c, where W' represents the fusion weight parameter.
[0086] When merging the scaling weight parameter and the superposition offset parameter, the corresponding relationship between the data of the scaling weight parameter and the data of the superposition offset parameter can be determined first, and then the result of adding the corresponding data is used as the data of the fusion weight parameter.
[0087] Exemplarily, when the matrix size of the target quantized weight is 128 times the matrix size of the target offset, the matrix size of aw is also 128 times the matrix size of c. At this time, every 128 weight data in aw corresponds to one weight data in c. The electronic device can add these 128 weight data to the corresponding weight data of c, and the obtained 128 sums constitute the corresponding 128 weight data in the fusion weight parameter.
[0088] The method of processing the task to be processed based on the fusion weight parameter can refer to the relevant technology in the field of large models, which will not be described in detail. The result of the task to be processed obtained in the above manner can be expressed as S0=W'x, where S0 represents the processing result of the task to be processed, and x represents the vector corresponding to the input information in the task to be processed. For example, in the task to be processed of generating an image based on text, x represents the vector of the input text.
[0089] In some embodiments, the scaling weight parameter and the stacking offset parameter may not be combined, and the task to be processed may be processed in the following manner:
[0090] The task to be processed is processed based on the scaling weight parameter to obtain a first result; the task to be processed is processed based on the stacking offset parameter to obtain a second result; and the first result and the second result are combined to obtain a processing result of the task to be processed.
[0091] The above process can be expressed as S0=awx+cx, where S0 represents the processing result of the task to be processed, x represents the task data to be processed in the task to be processed, awx represents the first result, and cx represents the second result.
[0092] See also Figure 2 In some embodiments, the method of obtaining the target bias weight parameter may include:
[0093] Obtaining initial quantization weight parameters of the large model, the initial quantization weight parameters including initial quantization weight, initial scaling factor and initial offset;
[0094] Dequantizing the initial quantized weight parameter to obtain an initial dequantized weight parameter, wherein the precision of the initial dequantized weight parameter is higher than the precision of the initial quantized weight parameter, and the precision of the initial dequantized weight parameter is the same as that of the target bias weight parameter;
[0095] The initial bias weight parameters are updated according to the initial dequantization weight parameters and the sample data to obtain the target bias weight parameters.
[0096] The above-mentioned large model can be any large language model obtained through pre-training, or other neural network models obtained through pre-training.
[0097] The initial quantized weight parameters can be obtained by quantizing the weight parameters contained in the large model.
[0098] The matrix expressions of the initial quantization weights and the target quantization weights are the same, and the values of the data contained therein may be the same or different; the matrix expressions of the initial scaling factors and the target scaling factors are the same, and the values of the data contained therein may be the same or different; the matrix expressions of the initial offsets and the target offsets are the same, and the values of the data contained therein may be the same or different.
[0099] Dequantization can be understood as the reverse process of quantization. For example, if quantization is a process of converting high-precision fp16 type data into low-precision int4 type data, dequantization is equivalent to a process of converting low-precision int4 type data into high-precision fp16 type data. The method for implementing this conversion can be referred to in the relevant technology and will not be described in detail.
[0100] The initial bias weight parameters can be generated randomly or obtained from some pre-trained models.
[0101] The matrix expression of the initial bias weight parameter can be the same as the initial offset or different from the initial offset.
[0102] Among them, when the matrix expression form of the initial bias weight parameter is different from the matrix expression form of the initial offset, the method of updating the initial bias weight parameter in the above embodiment can be:
[0103] The initial bias weight parameters are pooled to obtain the processed bias weight parameters having the same matrix expression as the initial offset;
[0104] The target bias weight parameter is obtained according to the initial dequantized weight parameter and the value of the bias weight parameter after the sample data update processing.
[0105] When the initial bias weight parameters are pooled, the initial bias weights can be divided into several segments, each segment contains L weight data, and the weight data are pooled to generate corresponding pooling results, and then these results are combined into the bias weight parameters after pooling. L is a preset parameter that can be set as needed, for example, it can be 32, 64, 128, etc.
[0106] Pooling can be divided into multiple processing methods, such as average pooling, maximum pooling, etc. The way to obtain the pooling result is different according to the different processing methods. For example, if average pooling is used, the average value of L data can be calculated as the pooling result, and if maximum pooling is used, the maximum value of L data can be selected as the pooling result.
[0107] See also Figure 2 After obtaining the processed bias weight parameters, the sample data can be processed with the initial dequantization weight parameters and the processed bias weight parameters respectively to obtain the processing results of the sample data, and then the loss value is determined based on the processing results of the sample data relative to the labeling results of the sample data, and the value of the processed bias weight parameters is updated based on the loss value.
[0108] When processing sample data using the initial inverse quantization weight parameter and the processed bias weight parameter, the correspondence between the processed bias weight parameter and the initial inverse quantization weight parameter can be determined, and the two can be added based on the correspondence to obtain an initial fusion weight parameter, and the sample data can be processed based on the initial fusion weight parameter to obtain a processing result of the sample data. In this case, the processing result of the sample data can be expressed as (W+ Δw')X, where W represents the initial inverse quantization weight parameter, Δw' represents the processed bias weight parameter, and X represents the sample data.
[0109] Alternatively, when the sample data is processed using the initial inverse quantization weight parameter and the processed bias weight parameter, the sample data may be processed using the initial inverse quantization weight parameter to obtain a first result of the sample data, and the sample data may be processed using the processed bias weight parameter to obtain a second result of the sample data, and then the first result and the second result may be superimposed to obtain a processing result of the sample data. In this case, the processing result of the sample data may be expressed as WX+Δw'X, where WX represents the first result and Δw'X represents the second result.
[0110] In some embodiments, different sample data may be selected to update the processed bias weight parameters to obtain a variety of bias weight parameters including a target bias weight parameter.
[0111] For example, the values of the processed bias weight parameters can be updated as follows:
[0112] The values of the bias weight parameters after the processing are updated according to the initial inverse quantization weight parameters and the sample data of the various task types, so as to obtain a plurality of task bias weight parameters corresponding to the various task types.
[0113] by Figure 2 For example, the sample data corresponding to task type 1 can be used to update the value of the bias weight parameter after processing, and the task bias weight parameter 1 corresponding to task type 1 can be obtained. The sample data corresponding to task type 2 can be used to update the value of the bias weight parameter after processing, and the task bias weight parameter 2 corresponding to task type 2 can be obtained. Similarly, the sample data corresponding to task type n can be used to update the value of the bias weight parameter after processing, and the task bias weight parameter n corresponding to task type n can be obtained.
[0114] After obtaining multiple task bias weight parameters in the above manner, when determining the target bias weight parameter, the task bias weight parameter corresponding to the task type of the task to be processed can be determined as the target bias weight parameter.
[0115] Furthermore, the value of the bias weight parameter after processing can be updated with sample data mixed with various task types to obtain the above-mentioned universal bias weight parameter.
[0116] In some optional embodiments, the value of the initial bias weight parameter may be updated according to the initial inverse quantization weight parameter and the sample data to obtain the bias weight parameter after the updated value, and then the bias weight parameter after the updated value may be pooled to obtain the target bias weight parameter.
[0117] When the matrix expression form of the initial bias weight parameter is the same as the matrix expression form of the initial offset, the method for updating the initial bias weight parameter in the above embodiment may be:
[0118] The value of the initial bias weight parameter is updated according to the initial dequantization weight parameter and the sample data to obtain the target bias weight parameter.
[0119] The updating method can refer to the method of updating the value of the bias weight parameter after processing, which will not be described in detail.
[0120] Optionally, when the matrix expression of the initial bias weight parameter is the same as the matrix expression of the initial offset, when the sample data is processed with the initial bias weight parameter, the sample data can be pooled to obtain pooled sample data that matches the matrix expression of the initial bias weight parameter, so that the initial bias weight parameter can directly process the pooled sample data.
[0121] Optionally, the target quantization weight parameter may include an initial quantization weight parameter;
[0122] Or it could include:
[0123] The initial inverse quantization weight parameter is updated according to the sample data, and the updated inverse quantization weight parameter obtained is quantized to obtain the quantization weight parameter.
[0124] That is to say, the initial quantization weight parameters of the pre-trained large model can be directly used as the target quantization weight parameters for processing the task to be processed.
[0125] Alternatively, the value of the initial inverse quantization weight parameter may be updated according to the sample data to obtain an updated inverse quantization weight parameter, which is then quantized and the quantized weight parameter finally obtained is used as the target weight parameter.
[0126] Wherein, updating the value of the initial inverse quantization weight parameter according to the sample data can be performed simultaneously with updating the value of the processed bias weight parameter. While updating the value of the processed bias weight parameter, the value of the initial inverse quantization weight parameter is updated based on the above loss value.
[0127] In any embodiment, the process of updating the value of the weight parameter may be iterated multiple times until the updated weight parameter satisfies the convergence condition. The setting of the convergence condition and the method of updating the value of the weight parameter may refer to the related art and will not be described in detail.
[0128] In some embodiments, when the initial quantization weight parameters are dequantized, all the initial quantization weight parameters may be dequantized, or only a portion of the initial dequantization weight parameters may be dequantized. The latter may be implemented as follows:
[0129] Determining an associated portion in the initial quantization weight parameter that is associated with the initial bias weight parameter;
[0130] The associated part of the initial quantization weight parameter is dequantized to obtain the initial dequantization weight parameter of the associated part.
[0131] In this embodiment, the target bias weight parameter can be used as an adapter and fused with the target quantization weight parameter to process the task to be processed. The target quantization weight parameter can generally be divided into weight data corresponding to multiple layers of the network, and only some layers need to be fused with the target bias weight parameter when processing the task to be processed. These layers that need to be fused are the associated parts of the target quantization weight parameter.
[0132] The network layer structure corresponding to the initial quantization weight parameter may be the same as the target quantization weight parameter, and the weight of the corresponding layer in the initial quantization weight parameter is the associated part of the initial quantization weight parameter.
[0133] Exemplarily, the initial quantization weight parameters and the target quantization weight parameters both correspond to the 1st to 20th layers of the network. When processing the task to be processed, only the 15th to 20th layers need to be fused with the target bias weight parameters. Therefore, it can be determined that the weight data corresponding to the 15th to 20th layers are the associated part of the initial quantization weight parameters, or the associated part of the target quantization weight parameters.
[0134] Which layers in the target quantization weight parameters need to be fused with the target bias weight parameters can be set as needed by the user responsible for updating the weight parameters.
[0135] After the associated part is determined, only the associated part of the initial quantization weight parameter may be dequantized to obtain the initial dequantization weight parameter of the associated part.
[0136] When dequantization is performed in the above manner, when updating the value of the bias weight parameter after processing, the sample data can be processed together with the initial dequantization weight parameter of the associated part, the initial quantization weight parameter other than the associated part, and the bias weight parameter after processing to obtain the processing result of the sample data. The bias weight parameter after processing is still represented by Δw', and W can represent the combination of the initial dequantization weight parameter of the associated part and the initial quantization weight parameter other than the associated part.
[0137] In combination with the foregoing example, W may be a combination of initial quantization weight parameters corresponding to the 1st to 14th layers and initial inverse quantization weight parameters corresponding to the 15th to 20th layers.
[0138] When only the associated part of the target quantization weight parameters needs to be fused with the target configuration weight parameters, only the associated part of the target quantization weight parameters and the target configuration weight parameters can be fused in the manner of obtaining the fused weight parameters as described above to obtain the fused weight parameters of the associated part, and then the target quantization weight parameters other than the associated part and the fused weight parameters of the associated part are used to process the task to be processed to obtain the processing result of the task to be processed.
[0139] In combination with the previous example, the target quantization weight parameters and target configuration weight parameters corresponding to the 15th to 20th layers can be fused to obtain the fused weight parameters corresponding to the 15th to 20th layers, and the tasks to be processed can be processed based on the target quantization weight parameters corresponding to the 1st to 14th layers and the fused weight parameters corresponding to the 15th to 20th layers to obtain the processing results of the tasks to be processed.
[0140] By performing inverse quantization in the above manner, the amount of calculation in each inverse quantization process can be reduced, thereby saving consumed resources.
[0141] The present application also provides a task processing device, see Figure 3 , the device may include the following units.
[0142] An obtaining unit 301 is used to obtain a task to be processed;
[0143] A determination unit 302 is used to determine at least a target quantization weight parameter and a target bias weight parameter for processing the task to be processed, wherein the expressions of the target quantization weight parameter and the target bias weight parameter meet the target condition, and the precision of the target bias weight parameter is higher than the precision of the target quantization weight parameter;
[0144] The processing unit 303 is used to process the task to be processed based on the target quantization weight parameter and the target bias weight parameter to obtain a processing result.
[0145] Optionally, the target quantization weight parameters include target quantization weight, target scaling factor and target offset, and the target scaling factor and target offset are used to adjust the target quantization weight; the expression forms of the target quantization weight parameters and the target bias weight parameters satisfy the target conditions including: at least the matrix expression forms of the target offset and the target bias weight parameters are the same.
[0146] Optionally, when the processing unit 303 processes the task to be processed based on the target quantization weight parameter and the target bias weight parameter, it is used to:
[0147] The target offset and the target bias weight parameter are superimposed to obtain a superimposed offset parameter;
[0148] The scaling weight parameter and the superposition offset parameter are combined to obtain a fusion weight parameter, where the scaling weight parameter is obtained based on a target scaling factor and a target quantization weight;
[0149] Process pending tasks based on fusion weight parameters.
[0150] Optional, see Figure 4 , is a schematic diagram of a task processing device provided in another embodiment, the device may include the various units of the aforementioned embodiments, and may also include an updating unit 401 for obtaining a target bias weight parameter.
[0151] When the updating unit 401 obtains the target bias weight parameter, it is used to:
[0152] Obtaining initial quantization weight parameters of the large model, the initial quantization weight parameters including initial quantization weight, initial scaling factor and initial offset;
[0153] Dequantizing the initial quantized weight parameter to obtain an initial dequantized weight parameter, wherein the precision of the initial dequantized weight parameter is higher than the precision of the initial quantized weight parameter, and the precision of the initial dequantized weight parameter is the same as that of the target bias weight parameter;
[0154] The initial bias weight parameters are updated according to the initial dequantization weight parameters and the sample data to obtain the target bias weight parameters.
[0155] Optionally, when the updating unit 401 dequantizes the initial quantization weight parameter to obtain the initial dequantization weight parameter, it is used to:
[0156] Determining an associated portion in the initial quantization weight parameter that is associated with the initial bias weight parameter;
[0157] The associated part of the initial quantization weight parameter is dequantized to obtain the initial dequantization weight parameter of the associated part.
[0158] Optionally, the matrix representation of the initial bias weight parameters is different from the matrix representation of the initial offsets;
[0159] The updating unit 401 updates the initial bias weight parameter according to the initial inverse quantization weight parameter and the sample data, and when obtaining the target bias weight parameter, is used to:
[0160] The initial bias weight parameters are pooled to obtain the processed bias weight parameters having the same matrix expression as the initial offset;
[0161] The target bias weight parameter is obtained according to the initial dequantized weight parameter and the value of the bias weight parameter after the sample data update processing.
[0162] Optionally, when the updating unit 401 updates the value of the bias weight parameter after processing according to the initial inverse quantization weight parameter and the sample data to obtain the target bias weight parameter, it is used to:
[0163] The values of the bias weight parameters after processing are updated according to the initial inverse quantization weight parameters and sample data of various task types, so as to obtain multiple task bias weight parameters corresponding to the various task types, wherein the task bias weight parameters of the task type corresponding to the task to be processed are used as the target bias weight parameters.
[0164] Optional target bias weight parameters include:
[0165] At least one of a general bias weight parameter corresponding to a plurality of task types and a task bias weight parameter corresponding to a task type of the task to be processed.
[0166] Optionally, the target quantization weight parameters include:
[0167] Initial quantization weight parameters;
[0168] Alternatively, the initial inverse quantization weight parameter is updated according to the sample data, and the updated inverse quantization weight parameter obtained is quantized to obtain the quantization weight parameter.
[0169] The present application also provides an electronic device. Figure 5 , the electronic device may include a memory 501 and a processor 502.
[0170] The memory 501 is used to store computer programs;
[0171] The processor 502 is used to execute the computer program to implement:
[0172] Get pending tasks;
[0173] At least a target quantization weight parameter and a target bias weight parameter for processing the task to be processed are determined, the expressions of the target quantization weight parameter and the target bias weight parameter meet the target condition, and the precision of the target bias weight parameter is higher than the precision of the target quantization weight parameter;
[0174] The task to be processed is processed based on the target quantization weight parameter and the target bias weight parameter to obtain a processing result.
[0175] The working principles of the task processing device and the electronic device provided in the above embodiments can refer to the relevant steps of the task processing method provided in the above embodiments, which will not be described in detail.
[0176] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0177] For the convenience of description, the above system or device is described by dividing it into various modules or units according to its functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.
[0178] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.
[0179] Finally, it should be noted that, in this article, relational terms such as first, second, third and fourth are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0180] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A task processing method, comprising: Get pending tasks; At least determining a target quantization weight parameter and a target bias weight parameter for processing the task to be processed, wherein the expressions of the target quantization weight parameter and the target bias weight parameter meet the target condition, and the precision of the target bias weight parameter is higher than the precision of the target quantization weight parameter; The task to be processed is processed based on the target quantization weight parameter and the target bias weight parameter to obtain a processing result.
2. According to the method of claim 1, the target quantization weight parameter includes a target quantization weight, a target scaling factor and a target offset, and the target scaling factor and the target offset are used to adjust the target quantization weight; the expression form of the target quantization weight parameter and the target bias weight parameter that meets the target condition includes: At least the target offset and the target bias weight parameter have the same matrix expression.
3. The method according to claim 2, wherein processing the task to be processed based on the target quantization weight parameter and the target bias weight parameter comprises: Superimposing the target offset and the target bias weight parameter to obtain a superimposed offset parameter; Combining a scaling weight parameter and the stacking offset parameter to obtain a fusion weight parameter, wherein the scaling weight parameter is obtained based on the target scaling coefficient and the target quantization weight; The task to be processed is processed based on the fusion weight parameter.
4. The method according to claim 2, wherein the method for obtaining the target bias weight parameter comprises: Obtaining initial quantization weight parameters of the large model, wherein the initial quantization weight parameters include an initial quantization weight, an initial scaling factor, and an initial offset; Dequantizing the initial quantization weight parameter to obtain an initial dequantization weight parameter, wherein the precision of the initial dequantization weight parameter is higher than the precision of the initial quantization weight parameter, and the precision of the initial dequantization weight parameter is the same as that of the target bias weight parameter; The initial bias weight parameter is updated according to the initial dequantization weight parameter and the sample data to obtain the target bias weight parameter.
5. The method according to claim 4, wherein the dequantizing the initial quantization weight parameter to obtain the initial dequantization weight parameter comprises: Determining, in the initial quantization weight parameter, an associated portion associated with the initial bias weight parameter; Dequantizing the associated part of the initial quantization weight parameter to obtain the initial dequantization weight parameter of the associated part.
6. The method according to claim 4, wherein the matrix expression of the initial bias weight parameter is different from the matrix expression of the initial offset; The updating of the initial bias weight parameter according to the initial dequantization weight parameter and the sample data to obtain the target bias weight parameter includes: Performing pooling processing on the initial bias weight parameter to obtain a processed bias weight parameter having the same matrix expression as the initial offset; The value of the processed bias weight parameter is updated according to the initial dequantization weight parameter and the sample data to obtain the target bias weight parameter.
7. The method according to claim 6, wherein the updating the value of the processed bias weight parameter according to the initial dequantization weight parameter and the sample data to obtain the target bias weight parameter comprises: The values of the processed bias weight parameters are updated according to the initial inverse quantization weight parameters and sample data of multiple task types to obtain multiple task bias weight parameters corresponding to the multiple task types, wherein the task bias weight parameters corresponding to the task type of the task to be processed are used as the target bias weight parameters.
8. The method according to claim 1, wherein the target bias weight parameter comprises: At least one of a general bias weight parameter corresponding to a plurality of task types and a task bias weight parameter corresponding to the task type of the task to be processed.
9. The method according to claim 4, wherein the target quantization weight parameter comprises: The initial quantization weight parameter; Alternatively, the initial inverse quantization weight parameter is updated according to the sample data, and the updated inverse quantization weight parameter obtained is quantized to obtain a quantization weight parameter.
10. An electronic device comprising a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program to implement: Get pending tasks; At least determining a target quantization weight parameter and a target bias weight parameter for processing the task to be processed, wherein the expressions of the target quantization weight parameter and the target bias weight parameter meet the target condition, and the precision of the target bias weight parameter is higher than the precision of the target quantization weight parameter; The task to be processed is processed based on the target quantization weight parameter and the target bias weight parameter to obtain a processing result.