Determination Method, Device, Electronic Device and Storage Medium of Data Processing Model

Abstract

The present disclosure relates to a method, apparatus, electronic device, and storage medium for determining a data processing model. The method includes: obtaining a sample data set; converting each sample data in the sample data set into sample vector data, grouping the sample data in the sample data set according to the distribution interval corresponding to each sample vector data, inputting the sample data in each sample data group into the current quantization layer of a preset model, determining the initial quantization coefficient corresponding to each sample data group as the current quantization coefficient of the current quantization layer, and outputting current quantization data based on the current quantization layer; obtaining the target quantization coefficient corresponding to each sample data group based on the sample data, the current quantization coefficient, and the current quantization data in each sample data group; and performing quantization processing on the preset model to obtain a data processing model. The present disclosure can improve the quantization accuracy of the model and reduce the storage space of the model without sacrificing the inference efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular, to a method, apparatus, electronic device, and storage medium for determining a data processing model. Background Art

[0002] In the related art, the LLM model has fully demonstrated its potential and superiority in the AIGC field. However, due to the huge number of parameters and computational volume of the LLM model itself, the cost of its inference is much greater than that of ordinary models. Common LLMs generally have a parameter order of 7B, 13B, and 70B. High computing power and high storage requirements severely limit the widespread application of the LLM model. Summary of the Invention

[0003] This disclosure provides a method, apparatus, electronic device, and storage medium for determining a data processing model to at least solve the problems of large storage space occupied by the data processing model and slow inference speed in the related art. The technical solution of this disclosure is as follows:

[0004] According to the first aspect of the embodiments of this disclosure, a method for determining a data processing model is provided, including:

[0005] Obtain the sample data corresponding to each sample object in the sample object set to obtain a sample data set; the sample data is the characterization data of the sample text or sample image;

[0006] Convert each sample data in the sample data set into sample vector data, group the sample data in the sample data set according to the distribution interval corresponding to each sample vector data, and determine the initial quantization coefficient corresponding to each sample data group;

[0007] Input the sample data in each sample data group into the current quantization layer of the preset model, determine the initial quantization coefficient corresponding to each sample data group as the current quantization coefficient of the current quantization layer, and output the current quantization data based on the current quantization layer;

[0008] Adjust the current quantization coefficient based on the sample data, the current quantization coefficient, and the current quantization data in each sample data group to obtain the target quantization coefficient corresponding to each sample data group;

[0009] Quantize the preset model based on the target quantization coefficient corresponding to each sample data group to obtain a data processing model.

[0010] In an exemplary embodiment, the adjusting the current quantization coefficient based on the sample data, the current quantization coefficient, and the current quantization data in each sample data group to obtain the target quantization coefficient corresponding to each sample data group includes:

[0011] Obtain the initial matrix parameters of the current quantization layer, and use the initial matrix parameters as the current matrix parameters;

[0012] Based on the sample data in each sample data group, the current quantization coefficient, the current quantization data, and the current matrix parameters, determine the current loss data;

[0013] Based on the current loss data, adjust the current quantization coefficient and the current matrix parameters to obtain the target quantization coefficient and the target matrix parameters corresponding to each sample data group;

[0014] The quantization processing of the preset model based on the target quantization coefficient corresponding to each sample data group to obtain a data processing model includes:

[0015] Based on the target quantization coefficient corresponding to each sample data group and the target matrix parameters corresponding to the current quantization layer, perform quantization processing on the preset model to obtain a data processing model.

[0016] In an exemplary embodiment, the adjusting the current quantization coefficient and the current matrix parameters based on the current loss data to obtain the target quantization coefficient and the target matrix parameters corresponding to each sample data group includes:

[0017] Fix the current matrix parameters, and adjust the current quantization coefficient based on the current loss data to obtain the target quantization coefficient corresponding to each sample data group;

[0018] Based on the target quantization coefficient and the current loss data, adjust the current matrix parameters to obtain the target matrix parameters corresponding to the current quantization layer.

[0019] In an exemplary embodiment, the adjusting the current quantization coefficient and the current matrix parameters based on the current loss data to obtain the target quantization coefficient and the target matrix parameters corresponding to each sample data group includes:

[0020] Fix the current matrix parameters, and adjust the current quantization coefficient based on the current loss data to obtain the updated quantization coefficient corresponding to each sample data group;

[0021] Reuse the updated quantization coefficient as the current quantization coefficient, fix the current quantization coefficient, and adjust the current matrix parameters based on the current loss data to obtain updated matrix parameters;

[0022] Regard the updated matrix parameters as the current matrix parameters again, and repeat the steps of fixing the current matrix parameters, adjusting the current quantization coefficients based on the current loss data to obtain the updated quantization coefficients corresponding to each sample data group; and regarding the updated quantization coefficients as the current quantization coefficients again, fixing the current quantization coefficients, and adjusting the current matrix parameters based on the current loss data to obtain the updated matrix parameters until the current loss data meets the preset conditions;

[0023] Determine the current quantization coefficients at the end of the adjustment as the target quantization coefficients, and determine the current matrix parameters at the end of the adjustment as the target matrix parameters.

[0024] In an exemplary embodiment, the method further includes:

[0025] For the target sample data in each sample data group, adjust the current quantization coefficients based on the current loss data to obtain the real-time quantization coefficients corresponding to the target sample data; the target sample data is any data in each sample data group;

[0026] Based on the real-time quantization coefficients corresponding to all the sample data in each sample data group respectively, determine the target quantization coefficients corresponding to each sample data group.

[0027] In an exemplary embodiment, the step of grouping the sample data in the sample data set according to the distribution interval corresponding to each sample vector data and determining the initial quantization coefficients corresponding to each sample data group includes:

[0028] Determine the distribution interval corresponding to each sample vector data;

[0029] Group the sample data in the sample data set according to the distribution interval corresponding to each sample vector data to obtain multiple sample data groups;

[0030] Based on the distribution interval corresponding to each sample data group, determine the initial quantization coefficients corresponding to each sample data group.

[0031] In an exemplary embodiment, after grouping the sample data in the sample data set according to the distribution interval corresponding to each sample vector data to obtain multiple sample data groups, the method further includes:

[0032] According to the sample vector data corresponding to each sample data in the sample data set, determine the maximum value and the minimum value corresponding to each sample data group;

[0033] Based on the ratio between the minimum value corresponding to each sample data group and the maximum value corresponding to each sample data group, generate the grouping identifier corresponding to each sample data group;

[0034] Based on the grouping identifier corresponding to each sample data group and the target quantization coefficient corresponding to each sample data group, establish the corresponding relationship between the grouping identifier and the target quantization coefficient, and obtain the identifier coefficient association relationship;

[0035] The quantization processing of the preset model based on the target quantization coefficient corresponding to each sample data group to obtain a data processing model includes:

[0036] Based on the identifier coefficient association relationship, perform quantization processing on the preset model to obtain a data processing model.

[0037] According to the second aspect of the embodiments of the present disclosure, there is provided a data processing method for a data processing model, the method including:

[0038] Obtain the target data of the target object;

[0039] Input the target data into the data processing model, determine the grouping identifier that matches the target data, and obtain a filtered grouping identifier; the data processing model is obtained by the above determination method;

[0040] Based on the identifier coefficient association relationship, search for the filtered quantization coefficient corresponding to the filtered grouping identifier;

[0041] Based on the quantization layer corresponding to the filtered quantization coefficient in the data processing model, process the target data to obtain target processed data.

[0042] In an exemplary implementation manner, the processing of the target data based on the quantization layer corresponding to the filtered quantization coefficient in the data processing model to obtain target processed data includes:

[0043] Obtain the matrix parameter of the quantization layer corresponding to the target data to obtain a filtered matrix parameter;

[0044] Based on the filtered matrix parameter and the filtered quantization coefficient, determine the target quantization layer in the data processing model;

[0045] Based on the target quantization layer, process the target data to obtain the target processed data.

[0046] According to the third aspect of the embodiments of the present disclosure, there is provided a determination device for a data processing model, including:

[0047] A sample data acquisition module configured to execute acquiring the sample data corresponding to each sample object in the sample object set to obtain a sample data set; the sample data is the characterization data of the sample text or the sample image;

[0048] The grouping module is configured to convert each sample data in the sample dataset into sample vector data, group the sample data in the sample dataset according to the distribution interval corresponding to each sample vector data, and determine the initial quantization coefficient corresponding to each sample data group;

[0049] The current data determination module is configured to input the sample data in each sample data group into the current quantization layer of a preset model, determine the initial quantization coefficient corresponding to each sample data group as the current quantization coefficient of the current quantization layer, and output the current quantization data based on the current quantization layer;

[0050] The adjustment module is configured to adjust the current quantization coefficient based on the sample data in each sample data group, the current quantization coefficient, and the current quantization data to obtain the target quantization coefficient corresponding to each sample data group;

[0051] The model determination module is configured to perform quantization processing on the preset model based on the target quantization coefficient corresponding to each sample data group to obtain a data processing model.

[0052] In an exemplary embodiment, the current data determination module includes:

[0053] The current matrix parameter determination unit is configured to obtain the initial matrix parameter of the current quantization layer and use the initial matrix parameter as the current matrix parameter;

[0054] The current loss determination unit is configured to determine the current loss data based on the sample data in each sample data group, the current quantization coefficient, the current quantization data, and the current matrix parameter;

[0055] The current data determination unit is configured to adjust the current quantization coefficient and the current matrix parameter based on the current loss data to obtain the target quantization coefficient and the target matrix parameter corresponding to each sample data group;

[0056] The model determination module is further configured to perform quantization processing on the preset model based on the target quantization coefficient corresponding to each sample data group and the target matrix parameter corresponding to the current quantization layer to obtain a data processing model.

[0057] In an exemplary embodiment, the adjustment module includes:

[0058] The target quantization coefficient determination unit is further configured to fix the current matrix parameter and adjust the current quantization coefficient based on the current loss data to obtain the target quantization coefficient corresponding to each sample data group;

[0059] The target matrix parameter determination unit is further configured to perform an adjustment on the current matrix parameter based on the target quantization coefficient and the current loss data to obtain the target matrix parameter corresponding to the current quantization layer.

[0060] In an exemplary embodiment, the adjustment module includes:

[0061] The first processing unit is configured to perform fixing the current matrix parameter, adjusting the current quantization coefficient based on the current loss data to obtain an updated quantization coefficient corresponding to each sample data group;

[0062] The second processing unit is configured to perform using the updated quantization coefficient as the current quantization coefficient again, fixing the current quantization coefficient, and adjusting the current matrix parameter based on the current loss data to obtain an updated matrix parameter;

[0063] The third processing unit is configured to perform using the updated matrix parameter as the current matrix parameter again, and repeating the steps of fixing the current matrix parameter, adjusting the current quantization coefficient based on the current loss data to obtain an updated quantization coefficient corresponding to each sample data group; and using the updated quantization coefficient as the current quantization coefficient again, fixing the current quantization coefficient, and adjusting the current matrix parameter based on the current loss data to obtain an updated matrix parameter until the current loss data meets a preset condition;

[0064] The fourth processing unit is configured to perform determining the current quantization coefficient at the end of the adjustment as the target quantization coefficient, and determining the current matrix parameter at the end of the adjustment as the target matrix parameter.

[0065] In an exemplary embodiment, the device further includes:

[0066] The first determination module is configured to perform, for the target sample data in each sample data group, adjusting the current quantization coefficient based on the current loss data to obtain a real-time quantization coefficient corresponding to the target sample data; the target sample data is any data in each sample data group;

[0067] The second determination module is configured to perform determining the target quantization coefficient corresponding to each sample data group based on the real-time quantization coefficients respectively corresponding to all the sample data in each sample data group.

[0068] In an exemplary embodiment, the grouping module includes:

[0069] The distribution interval determination unit is configured to perform determining the distribution interval corresponding to each sample vector data;

[0070] A grouping unit, configured to perform grouping of the sample data in the sample data set according to the distribution interval corresponding to each sample vector data, so as to obtain a plurality of sample data groups;

[0071] An initial coefficient determination unit, configured to perform determination of an initial quantization coefficient corresponding to each sample data group based on the distribution interval corresponding to each sample data group.

[0072] In an exemplary embodiment, the apparatus further includes:

[0073] A numerical value determination module, configured to perform determination of a maximum value and a minimum value corresponding to each sample data group according to the sample vector data corresponding to each sample data in the sample data set;

[0074] A grouping identifier determination module, configured to perform generation of a grouping identifier corresponding to each sample data group based on the ratio between the minimum value corresponding to each sample data group and the maximum value corresponding to each sample data group;

[0075] A relationship construction module, configured to perform construction of a corresponding relationship between the grouping identifier and the target quantization coefficient based on the grouping identifier corresponding to each sample data group and the target quantization coefficient, so as to obtain an identifier - coefficient association relationship.

[0076] In an exemplary embodiment, the model determination module is further configured to perform quantization processing on the preset model based on the identifier - coefficient association relationship, so as to obtain a data processing model.

[0077] According to a fourth aspect of the embodiments of the present disclosure, there is provided a data processing apparatus for a data processing model, including:

[0078] A target data acquisition module, configured to perform acquisition of target data of a target object;

[0079] An identifier grouping module, configured to perform input of the target data into the data processing model, determine a grouping identifier matching the target data, so as to obtain a filtered grouping identifier; the data processing model is obtained by the above - mentioned determination method;

[0080] A coefficient filtering module, configured to perform search for a filtered quantization coefficient corresponding to the filtered grouping identifier based on the identifier - coefficient association relationship;

[0081] A target processing module, configured to perform processing of the target data based on the quantization layer corresponding to the filtered quantization coefficient in the data processing model, so as to obtain target processed data.

[0082] In an exemplary embodiment, the target processing module includes:

[0083] A parameter screening unit, configured to execute obtaining matrix parameters of a quantization layer corresponding to the target data to obtain screened matrix parameters;

[0084] A layer determination unit, configured to execute determining a target quantization layer in the data processing model based on the screened matrix parameters and the screened quantization coefficients;

[0085] A target processing unit, configured to execute processing the target data based on the target quantization layer to obtain the target processed data.

[0086] According to a fifth aspect of the embodiments of the present disclosure, there is provided an electronic device, including:

[0087] A processor;

[0088] A memory for storing instructions executable by the processor;

[0089] Wherein, the processor is configured to execute the instructions to implement the method for determining the data processing model as described above.

[0090] According to a sixth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, enabling the electronic device to execute the method for determining the data processing model as described above.

[0091] According to a seventh aspect of the embodiments of the present disclosure, there is provided a computer program product, including a computer program, and when the computer program is executed by a processor, implementing the method for determining the data processing model as described above.

[0092] The technical solutions provided by the embodiments of the present disclosure at least bring the following beneficial effects:

[0093] The present disclosure obtains sample data corresponding to each sample object in a sample object set to obtain a sample data set; the sample data is characterization data of a sample text or a sample image; converts each sample data in the sample data set into sample vector data, groups the sample data in the sample data set according to the distribution interval corresponding to each sample vector data, and determines an initial quantization coefficient corresponding to each sample data group; inputs the sample data in each sample data group into the current quantization layer of a preset model, determines the initial quantization coefficient corresponding to each sample data group as the current quantization coefficient of the current quantization layer, and outputs current quantization data based on the current quantization layer; adjusts the current quantization coefficient based on the sample data, the current quantization coefficient, and the current quantization data in each sample data group to obtain a target quantization coefficient corresponding to each sample data group; performs quantization processing on the preset model based on the target quantization coefficient corresponding to each sample data group to obtain a data processing model. The present disclosure obtains target quantization coefficients for different input distributions for each layer to be quantized, and the data processing model obtained thereby can improve the quantization accuracy of the model and reduce the storage space of the model without sacrificing the inference efficiency.

[0094] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. Brief Description of the Drawings

[0095] The accompanying drawings here are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure and do not constitute an improper limitation of the present disclosure.

[0096] Figure 1 is an application environment diagram of a method for determining a data processing model shown according to an exemplary embodiment.

[0097] Figure 2 is a flowchart of a method for determining a data processing model shown according to an exemplary embodiment.

[0098] Figure 3 is a flowchart of a method for grouping the sample data in the sample data set according to the distribution interval corresponding to each sample vector data and determining the initial quantization coefficient corresponding to each sample data group shown according to an exemplary embodiment.

[0099] Figure 4 is a flowchart of a method for adjusting the current quantization coefficient based on the sample data, the current quantization coefficient, and the current quantization data in each sample data group to obtain the target quantization coefficient corresponding to each sample data group shown according to an exemplary embodiment.

[0100] Figure 5 It is a flowchart of a method for adjusting the current quantization coefficients and the current matrix parameters based on the current loss data to obtain the target quantization coefficients and the target matrix parameters corresponding to each sample data group according to an exemplary embodiment.

[0101] Figure 6 It is a flowchart of a method for constructing an association relationship of identification coefficients according to an exemplary embodiment.

[0102] Figure 7 It is a flowchart of a data processing method of a data processing model according to an exemplary embodiment.

[0103] Figure 8 It is a block diagram of a determining device of a data processing model according to an exemplary embodiment.

[0104] Figure 9 It is a block diagram of a server according to an exemplary embodiment. Detailed implementation manners

[0105] In order to enable those of ordinary skill in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0106] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that such used data may be interchanged under appropriate circumstances so that the embodiments of the present disclosure described herein can be implemented in an order different from those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0107] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in the present disclosure are all information and data authorized by the user or fully authorized by all parties.

[0108] First, some nouns or terms that appear in the process of describing the embodiments of this specification are explained as follows:

[0109] LLM: A large language model (abbreviated as LLM in English, Large Language Model) refers to a computer model that can process and generate natural language; it represents a major advancement in the field of artificial intelligence and is expected to transform this field with the knowledge it has acquired. An LLM can predict the next word or sentence by learning the statistical patterns and semantic information of language data. As the input dataset and parameter space continue to expand, the capabilities of the LLM will increase accordingly. It is used in various application fields such as robotics, machine learning, machine translation, speech recognition, image processing, etc., so it is called a multi-modal large language model (MLLM).

[0110] Int8 refers to an integer that occupies 8 bits of memory, and fp32 refers to a floating-point number that occupies 32 bits of memory.

[0111] Quantization has been proven to be a very effective model acceleration method in the field of neural networks. Commonly used such as INT8 quantization, which can not only save the storage space occupied by the model (reduced to 1 / 4 of the original), but also, due to the hardware-friendly characteristics of INT8 multiply-accumulate, can accelerate the model inference speed to 1.5 - 2 times the original. There are already very mature quantization schemes in CNNs. However, in LLM models, due to the extremely large fluctuations in their input and output with the change of input tokens, conventional quantization methods cannot meet the quantization requirements of LLM models. The specific quantization method can be expressed as:

[0112]

[0113] Among them, [] represents the rounding operation, and clamp represents the truncation operation. During the truncation process, numbers less than qmin will be set to qmin, and numbers greater than qmax will be set to qmax. s is the quantization mapping coefficient used (abbreviation: quantization coefficient). For INT8, qmin = -128 and qmax = 127;

[0114] For CNNs, quantization has been able to reach the INT4 level. These quantization methods often use static quantization coefficients, but these methods often cannot be migrated to LLM. This is because LLM itself has an extremely large number of parameters, and these static quantization methods often rely on optimization methods based on gradient descent to select the optimal quantization coefficients, which results in huge time and storage costs when these methods are migrated to LLM.

[0115] Please refer to Figure 1 , which shows an application environment diagram of a method for determining a data processing model according to an exemplary embodiment. This application environment may include server 01 and client 02.

[0116] Specifically, in the embodiments of this specification, the server 01 may include an independently operating server, a distributed server, or a server cluster composed of multiple servers. It may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The server 01 may include a network communication unit, a processor, a memory, and so on. Specifically, the server 01 may be used to determine the target quantization coefficient corresponding to each sample data group; based on the target quantization coefficient corresponding to each sample data group, perform quantization processing on the preset model to obtain a data processing model.

[0117] Specifically, in the embodiments of this specification, the client 02 may include physical devices such as smartphones, desktop computers, tablet computers, laptop computers, digital assistants, smart wearable devices, and vehicle terminals. It may also include software running on the physical devices, such as web pages provided by some service providers to users, or applications provided by these service providers to users. Specifically, the client 02 may be used to query the target processed data corresponding to the target data based on the data processing model.

[0118] Figure 2 is a flowchart of a method for determining a data processing model shown according to an exemplary embodiment. As Figure 2 shown, this method may be applied to Figure 1 the server 01 shown, and includes the following steps.

[0119] In step S201, obtain the sample data corresponding to each sample object in the sample object set to obtain a sample data set; the sample data is the characterization data of the sample text or the sample image.

[0120] The sample data may be the characterization data of the sample text or the sample image, and this data may be floating-point data; in a resource recommendation scenario, the sample data may include the object attribute information of the sample object, the historical multimedia resources browsed, etc.; in a question-and-answer scenario, the sample data may include the sample questions of the sample object, the context interaction information of the sample object, etc.

[0121] In step S203, convert each sample data in the sample data set into sample vector data, group the sample data in the sample data set according to the distribution interval corresponding to each sample vector data, and determine the initial quantization coefficient corresponding to each sample data group.

[0122] In the embodiments of the present disclosure, model quantization is to convert a floating-point model into a fixed-point model by a certain method. For example, the weights in the original model are all float32, and through model quantization, the model becomes a fixed-point model with weights all being int8. With the development of deep learning, it has achieved great success in fields such as computer vision (CV) and natural language processing (NLP). Through deep learning, we can obtain high-performance models for handling various tasks. Most of these models are very complex and generally only suitable for inference on GPUs, not suitable for inference on the board side. However, in actual applications, in many scenarios, the model needs to be deployed to the board side. To solve the problem that it is difficult to deploy the model to the board side, we need to reduce the complexity of the model through model quantization. This process will inevitably result in accuracy loss. Model quantization is to establish a mapping relationship between floating-point data and fixed-point data, so as to obtain greater benefits at the cost of less accuracy loss.

[0123] In the embodiments of the present disclosure, the sample data can be fp32. After grouping multiple sample data according to the distribution interval corresponding to each sample data, the initial quantization coefficient corresponding to each sample data group can be determined.

[0124] In the embodiments of the present disclosure, as Figure 3 shown, the steps of grouping the sample data in the sample data set according to the distribution interval corresponding to each sample vector data and determining the initial quantization coefficient corresponding to each sample data group include:

[0125] S20101: Determine the distribution interval corresponding to each sample vector data;

[0126] S20103: Group the sample data in the sample data set according to the distribution interval corresponding to each sample vector data to obtain multiple sample data groups;

[0127] S20105: Based on the distribution interval corresponding to each sample data group, determine the initial quantization coefficient corresponding to each sample data group.

[0128] In the embodiments of the present disclosure, considering the characteristics of the LLM model, for sample data (such as tokens) with similar distributions, the output features of the LLM model are very similar. However, if different input tokens are given, the output range of each layer of the LLM will fluctuate greatly. The sample data can be grouped according to the sampling distribution interval corresponding to each sample data, and multiple sample data groups are obtained; for example, those with a distribution of (-1, 1) are in one group, those with a distribution of (-10, 2) are in one group, and those with a distribution of (-2, 10) are in one group; the data in the same sample data group correspond to the same quantization coefficient, different sample data groups correspond to different quantization coefficients, different quantization coefficients are used for data between different groups, and the same quantization coefficient is used for data within the same group. That is, the initial quantization coefficients corresponding to different sample data groups are different. The initial quantization coefficient corresponding to each sample data group can be set as the initial quantization coefficient of each sample data in each sample data group.

[0129] In the embodiments of the present disclosure, the sample data can be grouped according to the distribution interval corresponding to the sample data, and the quantization coefficient of the sample data in the same group is determined to be the same value, so as to ensure that the output features corresponding to data with similar distributions are similar and avoid large fluctuations in the output data range of each layer of the model.

[0130] In step S205, the sample data in each sample data group is input into the current quantization layer of the preset model, the initial quantization coefficient corresponding to each sample data group is determined as the current quantization coefficient of the current quantization layer, and the current quantization data is output based on the current quantization layer.

[0131] In the embodiments of the present disclosure, the inputting the sample data in each sample data group into the quantization layer of the preset model includes:

[0132] Determine the preset quantization layer corresponding to the sample data in each sample data group in the preset model;

[0133] Input the sample data in each sample data group into the preset quantization layer of the preset model for quantization processing.

[0134] In the embodiments of the present disclosure, the preset model may include, but is not limited to, an LLM, a multimedia resource recommendation model determined based on the LLM, or a dialogue prediction model. The quantized preset model can be applied to an instant messaging application. The preset model may include one or more quantization layers; quantization operations can be performed on some or all of the layers according to requirements; first, determine the preset quantization layer corresponding to the sample data in each sample data group in the preset model; then input the sample data in each sample data group into the preset quantization layer of the preset model for quantization processing.

[0135] In the embodiment of the present disclosure, some or all quantization layers in the preset model may be selected for quantization processing according to actual conditions, thereby improving the flexibility of selecting quantization layers.

[0136] In step S207, based on the sample data in each sample data group, the current quantization coefficient and the current quantization data, the current quantization coefficient is adjusted to obtain a target quantization coefficient corresponding to each sample data group.

[0137] In the embodiment of the present disclosure, Figure 4 As shown, the current quantization coefficient is adjusted based on the sample data in each sample data group, the current quantization coefficient, and the current quantization data to obtain the target quantization coefficient corresponding to each sample data group, including:

[0138] S2071: Obtain initial matrix parameters of the current quantization layer, and use the initial matrix parameters as current matrix parameters;

[0139] S2073: Determine current loss data based on the sample data in each sample data group, the current quantization coefficient, the current quantization data, and the current matrix parameter;

[0140] S2075: Adjust the current quantization coefficient and the current matrix parameter based on the current loss data to obtain the target quantization coefficient and target matrix parameter corresponding to each sample data group.

[0141] In the embodiment of the present disclosure, a loss function may be pre-constructed based on the sample data, the current quantization coefficient, and the current quantization data, and the current loss data may be determined based on the loss function. Exemplarily, the formula corresponding to the loss function is as follows:

[0142] loss=||Q(x)*sx|| p

[0143] Among them, ||.|| p Represents the matrix norm, p is the matrix parameter, s is the quantization coefficient, x is the sample data, and Q(x) is the output quantization data; for each set of calibration data, select s that minimizes the current loss data, and finally take the mean of s corresponding to all sample data in the sample data group as the target quantization coefficient for all sample data in the sample data group. In addition, p can be adjusted according to actual conditions. Selecting a smaller p means that we pay more attention to the global error of Q(x) and x, while selecting a larger p means that we will pay more attention to the quantization error caused by larger values in x. In the application process, the value range of p can be determined according to actual needs. The matrix parameters corresponding to the same quantization layer are the same, the quantization coefficients corresponding to the data in the same sample data group input in the same quantization layer are the same, and the quantization coefficients corresponding to different sample data groups are different.

[0144] In the embodiments of the present disclosure, the initial matrix parameters of the quantization layer can be obtained; according to the sample data in each sample data group, the current quantization coefficient, the current quantization data, and the current matrix parameters, the current loss data can be determined; thus, according to the current loss data and the current matrix parameters, the target quantization coefficient corresponding to each sample data can be accurately determined.

[0145] In the embodiments of the present disclosure, as Figure 5 shown, the adjustment of the current quantization coefficient and the current matrix parameters based on the current loss data to obtain the target quantization coefficient and the target matrix parameters corresponding to each sample data group includes:

[0146] S20751: Fix the current matrix parameters, and adjust the current quantization coefficient based on the current loss data to obtain the updated quantization coefficient corresponding to each sample data group;

[0147] S20753: Reuse the updated quantization coefficient as the current quantization coefficient, fix the current quantization coefficient, and adjust the current matrix parameters based on the current loss data to obtain the updated matrix parameters;

[0148] S20755: Reuse the updated matrix parameters as the current matrix parameters, and repeat the steps of fixing the current matrix parameters, adjusting the current quantization coefficient based on the current loss data to obtain the updated quantization coefficient corresponding to each sample data group; and reusing the updated quantization coefficient as the current quantization coefficient, fixing the current quantization coefficient, and adjusting the current matrix parameters based on the current loss data to obtain the updated matrix parameters until the current loss data meets the preset conditions;

[0149] S20757: Determine the current quantization coefficient at the end of the adjustment as the target quantization coefficient, and determine the current matrix parameters at the end of the adjustment as the target matrix parameters.

[0150] In the embodiments of the present disclosure, the current matrix parameters and the current quantization coefficient can be alternately fixed in sequence, and the current matrix parameters and the current quantization coefficient can be continuously adjusted respectively until the current loss data meets the preset conditions; the preset conditions can include that the current loss data is less than the preset threshold or the number of alternating iterations reaches the preset number. When the adjustment ends, the current quantization coefficient at the end of the adjustment can be determined as the target quantization coefficient, and the current matrix parameters at the end of the adjustment can be determined as the target matrix parameters.

[0151] In the embodiments of the present disclosure, the current matrix parameters and the current quantization coefficient can be alternately fixed and adjusted, so as to further improve the accuracy of the target quantization coefficient and the target matrix parameters, and improve the accuracy of the quantization model.

[0152] Exemplarily, adjusting the current quantization coefficients and the current matrix parameters based on the current loss data to obtain the target quantization coefficients and the target matrix parameters corresponding to each sample data group includes:

[0153] Fixing the current matrix parameters and adjusting the current quantization coefficients based on the current loss data to obtain the target quantization coefficients corresponding to each sample data group;

[0154] Adjusting the current matrix parameters based on the target quantization coefficients and the current loss data to obtain the target matrix parameters corresponding to the current quantization layer.

[0155] In the embodiments of the present disclosure, the current matrix parameters can be fixed first, and the current quantization coefficients can be adjusted according to the current loss data to obtain the target quantization coefficients corresponding to each sample data group; exemplarily, the adjustment of the current quantization coefficients can be terminated until the current loss data is less than a preset threshold, and the current quantization coefficients when the current loss data is less than the preset threshold can be determined as the target quantization coefficients. Further, the current matrix parameters can be adjusted conversely according to the target quantization coefficients to obtain the target matrix parameters.

[0156] In the embodiments of the present disclosure, the current matrix parameters can be fixed first, and the current quantization coefficients can be adjusted according to the current loss data to obtain the target quantization coefficients corresponding to each sample data group; then the target quantization coefficients can be fixed, and the current matrix parameters can be adjusted according to the current loss data to obtain the target matrix parameters corresponding to the current quantization layer; thus, the target quantization coefficients and the target matrix parameters for determining the quantization model can be obtained quickly and accurately.

[0157] Exemplarily, the method further includes:

[0158] Obtaining the initial matrix parameters of the current quantization layer and using the initial matrix parameters as the current matrix parameters;

[0159] Determining the current loss data based on the sample data in each sample data group, the current quantization coefficients, the current quantization data, and the current matrix parameters; the current loss data is used to determine the target matrix parameters corresponding to the current quantization layer.

[0160] In the embodiments of the present disclosure, the method further includes:

[0161] For the target sample data in each sample data group, adjust the current quantization coefficient based on the current loss data to obtain the real-time quantization coefficient corresponding to the target sample data; the target sample data is any data in each sample data group; based on the real-time quantization coefficients corresponding to all the sample data in each sample data group, determine the target quantization coefficient corresponding to each sample data group.

[0162] In the embodiments of the present disclosure, for each group of calibration data, select s that minimizes the current loss data, and finally take the mean of s corresponding to all the sample data in the sample data group as the target quantization coefficient of all the sample data in the sample data group. In addition, p can be adjusted according to the actual situation. Selecting a smaller p means that we pay more attention to the global error between Q(x) and x, while selecting a larger p means that we pay more attention to the quantization error brought by the larger values in x. For the target sample data in each sample data group, the current quantization coefficient can be adjusted based on the current loss data to obtain the real-time quantization coefficient corresponding to the target sample data; then calculate the sum of the real-time quantization coefficients corresponding to all the sample data in each sample data group to determine the target quantization coefficient corresponding to each sample data group; or calculate the weighted sum of the real-time quantization coefficients corresponding to all the sample data in each sample data group to obtain the target quantization coefficient corresponding to each sample data group.

[0163] In the embodiments of the present disclosure, the current quantization coefficient can be adjusted according to the current loss data to obtain the real-time quantization coefficient corresponding to the target sample data, and the real-time quantization coefficients corresponding to all the sample data in each sample data group, so as to determine the target quantization coefficient corresponding to each sample data group, thereby improving the accuracy of the target quantization coefficient.

[0164] In the embodiments of the present disclosure, after grouping the sample data in the sample data set according to the distribution interval corresponding to each sample vector data to obtain multiple sample data groups, as Figure 6 shown, the method further includes:

[0165] S601: Determine the maximum value and the minimum value corresponding to each sample data group according to the sample vector data corresponding to each sample data in the sample data set;

[0166] S603: Generate a grouping identifier corresponding to each sample data group based on the ratio between the minimum value corresponding to each sample data group and the maximum value corresponding to each sample data group;

[0167] S605: Based on the grouping identifier corresponding to each sample data group and the target quantization coefficient corresponding to each sample data group, construct the corresponding relationship between the grouping identifier and the target quantization coefficient to obtain the identifier coefficient association relationship.

[0168] In the disclosed embodiment, the ratio between the minimum value corresponding to each sample data group and the maximum value corresponding to each sample data group can be calculated to generate a group identifier corresponding to each sample data group; or the absolute value of the ratio can be determined as the group identifier; then, based on the group identifier corresponding to each sample data group and the target quantization coefficient, a corresponding relationship between the group identifier and the target quantization coefficient can be constructed to obtain an identifier coefficient association relationship; the group identifier and the target quantization coefficient can be a one-to-one relationship. Since grouping based on the actual sampling distribution will bring about a huge computational overhead, this dynamic quantization cannot be actually applied in the reasoning scenario. The ratio of the current minimum absolute value and the maximum absolute value can be used to estimate the current dynamic distribution group. For example, if the minimum value is -1 and the maximum value is 10, its distribution can be represented by 1 / 10, while if the minimum value is -10 and the maximum value is 1, its distribution can be represented by 10. Based on the group identifiers such as 1 / 10 or 10, only the maximum and minimum values can be used to quickly judge the group identifier to which the current input data belongs, thereby selecting the appropriate quantization coefficient for it.

[0169] In the disclosed embodiment, a group identifier can be generated based on the maximum and minimum values corresponding to each sample data group, and an identification coefficient association relationship between the group identifier and the target quantization coefficient can be constructed, thereby facilitating rapid query of the target quantization coefficient corresponding to the data based on the identification coefficient association relationship.

[0170] In step S209, the preset model is quantized based on the target quantization coefficient corresponding to each sample data group to obtain a data processing model.

[0171] The method of performing quantization processing on the preset model based on the target quantization coefficient corresponding to each sample data group to obtain a data processing model includes:

[0172] Based on the target quantization coefficient corresponding to each sample data group and the target matrix parameters corresponding to the current quantization layer, the preset model is quantized to obtain a data processing model.

[0173] In some embodiments, the quantization processing of the preset model based on the target quantization coefficient corresponding to each sample data group to obtain the data processing model includes:

[0174] Based on the correlation relationship of the identification coefficients, the preset model is quantized to obtain a data processing model.

[0175] The data processing model is used to predict the multimedia resources to be recommended for the target object or to predict the target answer corresponding to the target question raised by the target object.

[0176] Exemplarily, the calibration process for quantizing the entire LLM is summarized as follows:

[0177] a. Starting from the first LLM decoder, step b is sequentially executed for each LLM decoder to be quantized:

[0178] b. For an LLM decoder to be quantized, steps c - h are sequentially executed for each layer to be quantized therein:

[0179] c. For the current layer to be quantized, the initial value of p is given as 2;

[0180] d. Feed all calibration data into this layer and count the possible input distribution of this layer; [[ID=!4]]

[0181] e. Divide it into k groups according to the counted input distribution. Generally, k = 4 is taken. Here, the basis for grouping is the distribution density of data on the positive and negative semi - axes;

[0182] f. For each group of inputs, the s that minimizes the current quantization error can be determined according to loss = ||Q(x)*s - x|| p and finally, the quantization coefficient of this group is determined by the average value of s;

[0183] g. Fix the quantization coefficient s of this quantization layer, gradually adjust p from 2 to 3.2, and select the p value that gives the best final output performance of the model;

[0184] h. Repeat steps f and g until the p value stabilizes.

[0185] In the above process, when each LLM decoder (decoder) is quantized, in the subsequent quantization process, it will maintain the quantized state, and both s and p will remain unchanged. Thus, for each layer to be quantized, s for different input distributions and the optimal p to be used by this layer can be obtained; in the process of selecting p, the distribution of input data is not considered, but the mean value of all inputs is used as the evaluation criterion. Then in the actual inference process, each quantized layer will dynamically select the current quantization coefficient under different input distribution conditions, which not only exceeds the performance of static quantization but also hardly introduces any additional computational overhead.

[0186] In an exemplary embodiment, large language models (LLMs) such as ChatGLM have attracted wide attention in the field of artificial intelligence. These models have demonstrated amazing performance in natural language processing tasks. To better evaluate the performance of these models, researchers have developed a series of evaluation methods, including Perplexity (PPL); ChatGLM is a pre-trained language model based on the Transformer architecture. The lower the PPL, the higher the accuracy of the model in predicting the next word, and the better the performance of the language model. The PPL performance and inference latency of the INT8 quantization models obtained by the prior art solutions and this embodiment on the wikitext-2 dataset are shown in Table 1-2 below. Table 1 shows the PPL performance data of the INT8 quantization models obtained by the prior art solutions and this embodiment on the wikitext-2 dataset; Table 2 shows the inference latency normalization data of the INT8 quantization models obtained by the prior art solutions and this embodiment on the wikitext-2 dataset. Among them, the prior art is the PyTorch dynamic quantization method; PyTorch currently supports quantization in the following three ways: PostTraining Dynamic Quantization, dynamic quantization after the model training is completed; PostTraining StaticQuantization, static quantization after the model training is completed; QAT (Quantization AwareTraining), quantization is enabled during the model training. WikiText English corpus data (The WikiText Long TermDependency LanguageModeling Dataset) is an English corpus data containing 100 million words. These words are extracted from high-quality and benchmark articles on Wikipedia, including two versions, WikText-2 and WiKiText-103. Each word also retains the original article that generated the word, which is especially suitable for scenarios where long-term dependency natural language modeling is required. The Wikitext-2 dataset is extracted from Wikipedia and contains the article text data content in Wikipedia; the Wikitext2 dataset contains the article content of Wikipedia, including information on various topics and fields. These articles have been preprocessed and cleaned to provide clean and trainable text data. Data scale: The scale of the Wikitext-2 dataset is relatively small. It contains text with more than 2,088,628 word tokens, of which 1,915,997 word tokens are used for training, 172,430 word tokens are used for validation, and 186,716 word tokens are used for testing.Data Format: The Wikitext2 dataset is stored in plain text format, and each text file contains the content of a Wikipedia article. The text is segmented into paragraphs and sentences. Usage: The Wikitext-2 dataset is commonly used in language modeling tasks, where the goal of the model is to predict the next word or the next sentence based on the previous context. Additionally, this dataset can also be used in other text generation tasks, such as machine translation, abstract generation, etc. In this embodiment, the inference latency of the quantized model accounts for 5-10% of the total time during the entire inference calculation process.

[0187] Table 1

[0188] LLaMA2-7B LLaMA-13B LLaMA-70B Prior art 6.12 5.22 3.97 This solution 5.87 5.17 3.78

[0189] Table 2

[0190] Prior art 1 This solution 1.05

[0191] In this embodiment, dynamic INT8 quantization is used to accelerate the inference process of the LLM model, and the problem of large variations in the model output for different input tokens is addressed, ensuring the accuracy of the model. Depending on the type of LLM model used, an acceleration effect of 1.5 - 2x is achieved, and at the same time, the storage requirement of the LLM is significantly reduced (1 / 4). Experiments have proven that the solution of this embodiment can achieve performance far exceeding that of dynamic quantization in the prior art while ensuring the INT8 acceleration effect, and there is almost no loss in inference speed compared to the dynamic quantization scheme of the prior art.

[0192] This disclosure obtains the sample data corresponding to each sample object in the sample object set to obtain a sample data set; the sample data is the characterization data of the sample text or the sample image; each sample data in the sample data set is converted into sample vector data, and the sample data in the sample data set is grouped according to the distribution interval corresponding to each sample vector data, and the initial quantization coefficient corresponding to each sample data group is determined; the sample data in each sample data group is input into the current quantization layer of the preset model, and the initial quantization coefficient corresponding to each sample data group is determined as the current quantization coefficient of the current quantization layer, and the current quantization data is output based on the current quantization layer; based on the sample data, the current quantization coefficient, and the current quantization data in each sample data group, the current quantization coefficient is adjusted to obtain the target quantization coefficient corresponding to each sample data group; based on the target quantization coefficient corresponding to each sample data group, the preset model is quantized to obtain a data processing model. This disclosure obtains the target quantization coefficient for different input distributions for each layer that needs to be quantized. The data processing model obtained thereby can improve the quantization accuracy of the model and reduce the storage space of the model without sacrificing the inference efficiency.

[0193] Such asFigure 7 As shown in the figure, this embodiment also discloses a data processing method for a data processing model. The method includes:

[0194] S701: Obtain the target data of the target object;

[0195] S703: Input the target data into the data processing model, determine the grouping identifier that matches the target data, and obtain the screening grouping identifier; the data processing model is obtained by the above determination method;

[0196] S705: Based on the identifier coefficient association relationship, search for the screening quantization coefficient corresponding to the screening grouping identifier;

[0197] S707: Based on the quantization layer corresponding to the screening quantization coefficient in the data processing model, process the target data to obtain the target processed data.

[0198] In the embodiments of the present disclosure, in a resource recommendation scenario, the target data may include the object attribute information of the target object, the historical multimedia resources browsed, etc.; in a question and answer scenario, the target data may include the target question of the target object, the context interaction information of the target object, etc. Each grouping identifier can be parsed to obtain the distribution difference corresponding to each grouping identifier, so as to determine the screening distribution interval corresponding to the target data, and further determine the screening grouping identifier corresponding to the screening distribution interval; then, according to the identifier coefficient association relationship, search for the screening quantization coefficient corresponding to the screening grouping identifier; determine the screening quantization coefficient as the quantization coefficient of the current quantization layer, so as to obtain the quantization data corresponding to the target data, that is, the target processed data.

[0199] In the embodiments of the present disclosure, after constructing the identifier coefficient association relationship, the screening quantization coefficient corresponding to the screening grouping identifier can be quickly and accurately determined according to the screening grouping identifier corresponding to the target data, so that the target processed data corresponding to the target data can be quickly determined.

[0200] Exemplarily, the processing the target data based on the quantization layer corresponding to the screening quantization coefficient in the data processing model to obtain the target processed data includes:

[0201] Obtain the matrix parameter of the quantization layer corresponding to the target data to obtain the screening matrix parameter; [[ID=?]]

[0202] Based on the screening matrix parameter and the screening quantization coefficient, determine the target quantization layer in the data processing model;

[0203] Process the target data based on the target quantization layer to obtain the target processed data.

[0204] In an embodiment of the present disclosure, after determining the screening matrix parameters, the preset model can be quantized according to the target quantization coefficients corresponding to each sample data group and the screening matrix parameters corresponding to the current quantization layer, and a data processing model can be quickly obtained. Among them, in the data processing model, multiple target quantization coefficients and one screening matrix parameter can correspond to the same quantization layer.

[0205] Figure 8 It is a block diagram of a device for determining a data processing model shown according to an exemplary embodiment. Refer to Figure 8 , the device includes:

[0206] A sample data acquisition module 810, configured to execute acquiring sample data corresponding to each sample object in a sample object set to obtain a sample data set; the sample data is the characterization data of a sample text or a sample image;

[0207] A grouping module 820, configured to execute converting each sample data in the sample data set into sample vector data, grouping the sample data in the sample data set according to the distribution interval corresponding to each sample vector data, and determining the initial quantization coefficient corresponding to each sample data group;

[0208] A current data determination module 830, configured to execute inputting the sample data in each sample data group into the current quantization layer of a preset model, determining the initial quantization coefficient corresponding to each sample data group as the current quantization coefficient of the current quantization layer, and outputting current quantization data based on the current quantization layer;

[0209] An adjustment module 840, configured to execute adjusting the current quantization coefficient based on the sample data, the current quantization coefficient, and the current quantization data in each sample data group to obtain the target quantization coefficient corresponding to each sample data group;

[0210] A model determination module 850, configured to execute quantizing the preset model based on the target quantization coefficients corresponding to each sample data group to obtain a data processing model.

[0211] In an exemplary implementation manner, the current data determination module includes:

[0212] A current matrix parameter determination unit, configured to execute acquiring the initial matrix parameter of the current quantization layer and using the initial matrix parameter as the current matrix parameter;

[0213] A current loss determination unit, configured to execute determining current loss data based on the sample data, the current quantization coefficient, the current quantization data, and the current matrix parameter in each sample data group;

[0214] A current data determination unit, configured to perform an adjustment on the current quantization coefficients and the current matrix parameters based on the current loss data, so as to obtain target quantization coefficients and target matrix parameters corresponding to each sample data group;

[0215] The model determination module is further configured to perform a quantization process on the preset model based on the target quantization coefficients corresponding to each sample data group and the target matrix parameters corresponding to the current quantization layer, so as to obtain a data processing model.

[0216] In an exemplary embodiment, the adjustment module includes:

[0217] A target quantization coefficient determination unit, further configured to perform fixing the current matrix parameters, and adjusting the current quantization coefficients based on the current loss data, so as to obtain target quantization coefficients corresponding to each sample data group;

[0218] A target matrix parameter determination unit, further configured to perform an adjustment on the current matrix parameters based on the target quantization coefficients and the current loss data, so as to obtain target matrix parameters corresponding to the current quantization layer.

[0219] In an exemplary embodiment, the adjustment module includes:

[0220] A first processing unit, configured to perform fixing the current matrix parameters, and adjusting the current quantization coefficients based on the current loss data, so as to obtain updated quantization coefficients corresponding to each sample data group;

[0221] A second processing unit, configured to perform re-taking the updated quantization coefficients as the current quantization coefficients, fixing the current quantization coefficients, and adjusting the current matrix parameters based on the current loss data, so as to obtain updated matrix parameters;

[0222] A third processing unit, configured to perform re-taking the updated matrix parameters as the current matrix parameters, and repeating the steps of fixing the current matrix parameters, adjusting the current quantization coefficients based on the current loss data to obtain updated quantization coefficients corresponding to each sample data group; and re-taking the updated quantization coefficients as the current quantization coefficients, fixing the current quantization coefficients, and adjusting the current matrix parameters based on the current loss data to obtain updated matrix parameters until the current loss data meets a preset condition;

[0223] A fourth processing unit, configured to perform determining the current quantization coefficients at the end of the adjustment as the target quantization coefficients, and determining the current matrix parameters at the end of the adjustment as the target matrix parameters.

[0224] In an exemplary embodiment, the apparatus further includes:

[0225] A first determination module, configured to perform, for the target sample data in each sample data group, adjusting the current quantization coefficient based on the current loss data to obtain the real-time quantization coefficient corresponding to the target sample data; the target sample data is any data in each sample data group.

[0226] A second determination module, configured to perform determining the target quantization coefficient corresponding to each sample data group based on the real-time quantization coefficients respectively corresponding to all the sample data in each sample data group.

[0227] In an exemplary embodiment, the grouping module includes:

[0228] A distribution interval determination unit, configured to perform determining the distribution interval corresponding to each sample vector data.

[0229] A grouping unit, configured to perform grouping the sample data in the sample data set according to the distribution interval corresponding to each sample vector data to obtain a plurality of sample data groups.

[0230] An initial coefficient determination unit, configured to perform determining the initial quantization coefficient corresponding to each sample data group based on the distribution interval corresponding to each sample data group.

[0231] In an exemplary embodiment, the apparatus further includes:

[0232] A value determination module, configured to perform determining the maximum value and the minimum value corresponding to each sample data group according to the sample vector data corresponding to each sample data in the sample data set.

[0233] A grouping identifier determination module, configured to perform generating the grouping identifier corresponding to each sample data group based on the ratio between the minimum value corresponding to each sample data group and the maximum value corresponding to each sample data group.

[0234] A relationship construction module, configured to perform constructing the corresponding relationship between the grouping identifier and the target quantization coefficient based on the grouping identifier corresponding to each sample data group and the target quantization coefficient to obtain the identifier-coefficient association relationship.

[0235] In an exemplary embodiment, the model determination module is further configured to perform quantizing the preset model based on the identifier-coefficient association relationship to obtain a data processing model.

[0236] This embodiment further provides a data processing apparatus for a data processing model, including:

[0237] A target data acquisition module, configured to perform acquiring the target data of a target object.

[0238] An identification grouping module, configured to execute inputting the target data into a data processing model, determining a grouping identifier matching the target data, and obtaining a filtered grouping identifier; the data processing model is obtained by the above-mentioned determination method.

[0239] A coefficient filtering module, configured to execute finding a filtered quantization coefficient corresponding to the filtered grouping identifier based on the identification coefficient association relationship.

[0240] A target processing module, configured to execute processing the target data based on a quantization layer corresponding to the filtered quantization coefficient in the data processing model to obtain target processed data.

[0241] In an exemplary embodiment, the target processing module includes:

[0242] A parameter filtering unit, configured to execute obtaining matrix parameters of a quantization layer corresponding to the target data to obtain filtered matrix parameters.

[0243] A layer determination unit, configured to execute determining a target quantization layer in the data processing model based on the filtered matrix parameters and the filtered quantization coefficient.

[0244] A target processing unit, configured to execute processing the target data based on the target quantization layer to obtain the target processed data.

[0245] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0246] In an exemplary embodiment, an electronic device is further provided, including a processor; a memory for storing processor-executable instructions; wherein, when the processor is configured to execute the instructions stored on the memory, the method for determining the data processing model provided in any of the above embodiments is implemented.

[0247] The electronic device may be a terminal, a server or a similar computing device. Taking the electronic device as a server as an example, Figure 9 is a block diagram of an electronic device shown according to an exemplary embodiment, as Figure 9As shown, the server 900 can vary significantly due to differences in configuration or performance, and may include one or more central processing units (CPUs) 910 (the processor 910 may include, but is not limited to, processing devices such as a microprocessor MCU or a field programmable gate array FPGA), a memory 930 for storing data, and one or more storage media 920 for storing application programs 923 or data 922 (such as one or more mass storage devices). Among them, the memory 930 and the storage media 920 can be transient storage or persistent storage. The programs stored in the storage media 920 may include one or more modules, and each module may include a series of instruction operations on the server. Further, the central processing unit 910 can be configured to communicate with the storage media 920 and execute a series of instruction operations in the storage media 920 on the server 900. The server 900 may also include one or more power supplies 960, one or more wired or wireless network interfaces 950, one or more input / output interfaces 940, and / or one or more operating systems 921, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSD TM, and so on.

[0248] The input / output interface 940 can be used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by the communication provider of the server 900. In one example, the input / output interface 940 includes a network interface controller (NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one example, the input / output interface 940 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0249] Those of ordinary skill in the art can understand that Figure 9 the structure shown is only illustrative and does not limit the structure of the above electronic device. For example, the server 900 may also include more or fewer components than those Figure 9 shown, or have a different configuration from that Figure 9 shown.

[0250] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as the memory 930 including instructions, and the above instructions can be executed by the processor 910 of the device 900 to complete the above method. Optionally, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and so on.

[0251] In an exemplary embodiment, a computer program product is further provided, including a computer program, which implements the method for determining the data processing model provided in any of the above embodiments when executed by a processor.

[0252] The present disclosure obtains sample data corresponding to each sample object in a sample object set to obtain a sample data set; the sample data is the characterization data of a sample text or a sample image; each sample data in the sample data set is converted into sample vector data, the sample data in the sample data set is grouped according to the distribution interval corresponding to each sample vector data, and an initial quantization coefficient corresponding to each sample data group is determined; the sample data in each sample data group is input into the current quantization layer of a preset model, the initial quantization coefficient corresponding to each sample data group is determined as the current quantization coefficient of the current quantization layer, and current quantization data is output based on the current quantization layer; the current quantization coefficient is adjusted based on the sample data, the current quantization coefficient, and the current quantization data in each sample data group to obtain a target quantization coefficient corresponding to each sample data group; the preset model is quantized based on the target quantization coefficient corresponding to each sample data group to obtain a data processing model. The present disclosure obtains the target quantization coefficient for different input distributions for each layer to be quantized, and the data processing model obtained thereby can improve the quantization accuracy of the model and reduce the storage space of the model without sacrificing the inference efficiency.

[0253] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. This computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0254] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present disclosure. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and embodiments are only to be considered exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0255] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A method for determining a data processing model, characterized in that: include: Obtaining sample data corresponding to each sample object in the sample object set to obtain a sample data set; the sample data is representation data of the sample text or sample image; Converting each sample data in the sample data set into sample vector data, grouping the sample data in the sample data set according to the distribution interval corresponding to each sample vector data, and determining an initial quantization coefficient corresponding to each sample data group; Inputting the sample data in each sample data group into a current quantization layer of a preset model, determining the initial quantization coefficient corresponding to each sample data group as the current quantization coefficient of the current quantization layer, and outputting the current quantization data based on the current quantization layer; Obtaining initial matrix parameters of the current quantization layer, and using the initial matrix parameters as current matrix parameters; determining current loss data based on the sample data in each sample data group, the current quantization coefficient, the current quantization data, and the current matrix parameter; Adjusting the current quantization coefficient and the current matrix parameter based on the current loss data to obtain a target quantization coefficient and a target matrix parameter corresponding to each sample data group; Based on the target quantization coefficient corresponding to each sample data group and the target matrix parameters corresponding to the current quantization layer, the preset model is quantized to obtain a data processing model.

2. The method according to claim 1, characterized in that The adjusting the current quantization coefficient and the current matrix parameter based on the current loss data to obtain the target quantization coefficient and the target matrix parameter corresponding to each sample data group includes: Fixing the current matrix parameters, and adjusting the current quantization coefficients based on the current loss data to obtain target quantization coefficients corresponding to each sample data group; The current matrix parameters are adjusted based on the target quantization coefficient and the current loss data to obtain target matrix parameters corresponding to the current quantization layer.

3. The method according to claim 1, characterized in that The adjusting the current quantization coefficient and the current matrix parameter based on the current loss data to obtain the target quantization coefficient and the target matrix parameter corresponding to each sample data group includes: Fixing the current matrix parameters, and adjusting the current quantization coefficients based on the current loss data to obtain updated quantization coefficients corresponding to each sample data group; Using the updated quantization coefficient as the current quantization coefficient again, fixing the current quantization coefficient, and adjusting the current matrix parameter based on the current loss data to obtain an updated matrix parameter; The updated matrix parameters are used again as the current matrix parameters, and the steps of fixing the current matrix parameters and adjusting the current quantization coefficients based on the current loss data to obtain updated quantization coefficients corresponding to each sample data group are repeated; and the steps of using the updated quantization coefficients as the current quantization coefficients, fixing the current quantization coefficients, and adjusting the current matrix parameters based on the current loss data to obtain updated matrix parameters are repeated until the current loss data satisfies a preset condition. The current quantization coefficient at the end of the adjustment is determined as the target quantization coefficient, and the current matrix parameter at the end of the adjustment is determined as the target matrix parameter.

4. The method according to claim 1, wherein The method further comprises: For target sample data in each sample data group, adjusting the current quantization coefficient based on the current loss data to obtain a real-time quantization coefficient corresponding to the target sample data; the target sample data is any data in each sample data group; Based on the real-time quantization coefficients corresponding to all the sample data in each sample data group, the target quantization coefficient corresponding to each sample data group is determined.

5. The method according to claim 1, wherein The step of grouping the sample data in the sample data set according to the distribution interval corresponding to each sample vector data, and determining the initial quantization coefficient corresponding to each sample data group, includes: Determine the distribution interval corresponding to each sample vector data; Grouping the sample data in the sample data set according to the distribution interval corresponding to each sample vector data to obtain multiple sample data groups; Based on the distribution interval corresponding to each sample data group, an initial quantization coefficient corresponding to each sample data group is determined.

6. The method according to claim 5, characterized in that After grouping the sample data in the sample data set according to the distribution interval corresponding to each sample vector data to obtain multiple sample data groups, the method further includes: Determine the maximum value and the minimum value corresponding to each sample data group according to the sample vector data corresponding to each sample data in the sample data set; Generate a group identifier corresponding to each sample data group based on a ratio between a minimum value corresponding to each sample data group and a maximum value corresponding to each sample data group; Based on the group identifier corresponding to each sample data group and the target quantization coefficient corresponding to each sample data group, a corresponding relationship between the group identifier and the target quantization coefficient is constructed to obtain an identifier-coefficient correlation relationship; The method of performing quantization processing on the preset model based on the target quantization coefficient corresponding to each sample data group to obtain a data processing model includes: Based on the correlation relationship of the identification coefficients, the preset model is quantized to obtain a data processing model.

7. A data processing method for a data processing model, characterized in that: The method comprises: Get the target data of the target object; Inputting the target data into a data processing model, determining a group identifier that matches the target data, and obtaining a screening group identifier; the data processing model is obtained by the determination method according to any one of claims 1 to 6; Based on the identification coefficient association relationship, searching for the screening quantization coefficient corresponding to the screening group identification; The target data is processed based on the quantization layer corresponding to the screened quantization coefficient in the data processing model to obtain target processed data.

8. The method according to claim 7, characterized in that The step of processing the target data based on the quantization layer corresponding to the screened quantization coefficient in the data processing model to obtain target processed data includes: Obtaining matrix parameters of the quantization layer corresponding to the target data to obtain screening matrix parameters; determining a target quantization level in the data processing model based on the screening matrix parameters and the screening quantization coefficients; The target data is processed based on the target quantization layer to obtain the target processed data.

9. A device for determining a data processing model, characterized in that: include: A sample data acquisition module is configured to acquire sample data corresponding to each sample object in a sample object set to obtain a sample data set; the sample data is representation data of a sample text or a sample image; a grouping module configured to convert each sample data in the sample data set into sample vector data, group the sample data in the sample data set according to a distribution interval corresponding to each sample vector data, and determine an initial quantization coefficient corresponding to each sample data group; a current data determination module configured to input the sample data in each sample data group into a current quantization layer of a preset model, determine the initial quantization coefficient corresponding to each sample data group as the current quantization coefficient of the current quantization layer, and output current quantization data based on the current quantization layer; The current data determination module includes: a current matrix parameter determination unit configured to obtain initial matrix parameters of the current quantization layer and use the initial matrix parameters as current matrix parameters; a current loss determination unit configured to determine current loss data based on sample data in each sample data group, the current quantization coefficient, the current quantization data, and the current matrix parameters; the current data determination unit configured to adjust the current quantization coefficient and the current matrix parameters based on the current loss data to obtain target quantization coefficients and target matrix parameters corresponding to each sample data group; an adjustment module configured to adjust the current quantization coefficient based on the sample data in each sample data group, the current quantization coefficient, and the current quantization data to obtain target quantization coefficients corresponding to each sample data group; The model determination module is configured to perform quantization processing on the preset model based on the target quantization coefficient corresponding to each sample data group and the target matrix parameters corresponding to the current quantization layer to obtain a data processing model.

10. A data processing device for a data processing model, characterized in that: The device comprises: A target data acquisition module is configured to acquire target data of a target object; an identification grouping module configured to input the target data into a data processing model, determine a group identification that matches the target data, and obtain a screening group identification; the data processing model is obtained by the determination method according to any one of claims 1 to 6; A coefficient screening module is configured to search for a screening quantization coefficient corresponding to the screening group identifier based on an identification coefficient association relationship; The target processing module is configured to execute a quantization layer corresponding to the screened quantization coefficient in the data processing model to process the target data to obtain target processed data.

11. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method for determining a data processing model according to any one of claims 1 to 6 or the data processing method for a data processing model according to claim 7 or 8.

12. A computer-readable storage medium, characterized in that When the instructions in the computer-readable storage medium are executed by an electronic device processor, the electronic device is enabled to execute the method for determining a data processing model as described in any one of claims 1 to 6 or the data processing method for a data processing model as described in claim 7 or 8.

13. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the method for determining a data processing model according to any one of claims 1 to 6 or the data processing method of the data processing model according to claim 7 or 8 is implemented.

Images