A data processing method and device, and a computer-readable storage medium

Abstract

The embodiment of the present application discloses a data processing method and device, and a computer-readable storage medium, which can improve the convergence speed when training a quantization model, and improve the accuracy of multimedia data processing by the trained quantization model. The method includes: obtaining multimedia data to be processed; using a target quantization model to process the multimedia data to be processed to obtain a target processing result; the target quantization model is obtained by training the initial quantization model using a preset real target value and an original model demonstration value; the original model demonstration value includes at least one of the original model processing result and the original model weight distribution; wherein the initial quantization model is obtained by quantizing and compressing the original model, and the original model is a network model of the first precision type for processing the multimedia data to be processed; the initial quantization model is a network model of the second precision type.

Description

Technical Field

[0001] The present application relates to the field of software engineering, and in particular to a data processing method and device, and a computer-readable storage medium. Background Art

[0002] At present, in order to improve the processing speed of the neural network model and reduce resource usage, the high-precision model is usually quantized and compressed to obtain a smaller low-bit quantized model. However, it is difficult to guarantee the processing accuracy of the quantized model obtained by direct compression. In order to improve the accuracy of the quantized model, the existing technology usually uses the true value as the target value to retrain the quantized model. However, the current training method still makes it difficult to restore the quantized model to its original accuracy, and the training process is long, the convergence speed is slow, and the accuracy of multimedia data processing using the trained quantized model is reduced. Summary of the invention

[0003] The embodiments of the present application hope to provide a data processing method and device, and a computer-readable storage medium, which can improve the convergence speed when training a quantization model and improve the accuracy of multimedia data processing by the trained quantization model.

[0004] The technical solution of this application is implemented as follows:

[0005] The present application provides a data processing method, including:

[0006] Acquire multimedia data to be processed;

[0007] The multimedia data to be processed is processed using a target quantization model to obtain a target processing result; the target quantization model is obtained by training the initial quantization model using a preset real target value and an original model demonstration value; the original model demonstration value includes at least one of the original model processing result and the original model weight distribution;

[0008] Among them, the initial quantization model is obtained by quantizing and compressing the original model; the original model is a network model of the first precision type for processing the multimedia data to be processed; the initial quantization model is a network model of the second precision type, and the precision of the first precision type is higher than that of the second precision type; the original model processing result is obtained by processing a preset training sample set by the original model; the original model weight distribution represents the weight distribution of the network layer of the original model.

[0009] The present application provides a data processing device, including:

[0010] An acquisition unit, used for acquiring multimedia data to be processed;

[0011] A model processing unit, configured to process the multimedia data to be processed using a target quantization model to obtain a target processing result; the target quantization model is obtained by training an initial quantization model using a preset real target value and an original model demonstration value; the original model demonstration value includes at least one of an original model processing result and an original model weight distribution;

[0012] Among them, the initial quantization model is obtained by quantizing and compressing the original model; the original model is a network model of the first precision type for processing the multimedia data to be processed; the initial quantization model is a network model of the second precision type, and the precision of the first precision type is higher than that of the second precision type; the original model processing result is obtained by processing a preset training sample set by the original model; the original model weight distribution represents the weight distribution of the network layer of the original model.

[0013] The present application provides a data processing device, including:

[0014] A memory for storing executable data instructions;

[0015] The processor is used to implement any of the data processing methods described above when executing the executable instructions stored in the memory.

[0016] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a processor to execute instructions to implement any of the data processing methods described above.

[0017] The data processing method and device provided by the embodiment of the present application, and the computer-readable storage medium, include obtaining multimedia data to be processed; using a target quantization model to process the multimedia data to be processed to obtain a target processing result; the target quantization model is obtained by training the initial quantization model using a preset real target value and an original model demonstration value; the original model demonstration value includes at least one of the original model processing result and the original model weight distribution; wherein the original model is a network model of the first precision type for processing the multimedia data to be processed; the initial quantization model is a network model of the second precision type, and the precision of the first precision type is higher than that of the second precision type; the initial quantization model is obtained by quantizing and compressing the original model. With the above technical implementation scheme, since the target quantization model is obtained by training according to the preset real target value and the original model demonstration value, in the process of training the initial quantization model, the processing result of the original model and at least one of the original model weight distribution can be used as the training target of the initial quantization model on the basis of the preset real target value, so as to achieve more efficient adjustment of the network parameters of each round of training of the initial quantization model, thereby improving the convergence speed of training the initial quantization model and improving the precision of the target quantization model finally trained to process multimedia data. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A method for quantizing, compressing and retraining current high-precision neural network models;

[0019] Figure 2 An optional flow chart of a data processing method provided in an embodiment of the present application;

[0020] Figure 3 An optional flow chart of a data processing method provided in an embodiment of the present application;

[0021] Figure 4 A schematic diagram of a quantization compression method provided in an embodiment of the present application;

[0022] Figure 5 An optional flow chart of a data processing method provided in an embodiment of the present application;

[0023] Figure 6 An optional flow chart of a data processing method provided in an embodiment of the present application;

[0024] Figure 7 A schematic diagram of a method for calculating the second loss and the third loss provided in an embodiment of the present application;

[0025] Figure 8 An optional flow chart of a data processing method provided in an embodiment of the present application;

[0026] Fig. 9 An optional flow chart of a data processing method provided in an embodiment of the present application;

[0027] Fig.10 An optional structural diagram of a data processing device provided in an embodiment of the present application;

[0028] Fig.11 An optional structural diagram of a data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0030] At present, neural network models are widely used in speech recognition, image classification, object detection and other fields. In the neural network model, the data type of the neuron weight of each layer of the network characterizes the data range that the deep neural network model can express. For example, the neuron weight can be a 32-bit floating point type (float), in which 1 bit (bit) represents the symbol, 8 bits represent the integer part, and 23 bits represent the fractional part. A larger data range can make the neural network model perform operations with higher complexity and precision, but it also increases the scale of the neural network model and the resource consumption during operation. In order to reduce the scale of the neural network model without changing the operation effect of the neural network model as much as possible, so that the neural network model has a faster running speed and can be applied to low-power and low-computing resource platforms, such as mobile phones, tablets, etc., the neural network model is usually compressed by low-bit quantization. For example, the 32-bit float type neural network model can be quantized to an integer (int) 8 type. Compression to int8 means memory saving, saving 3 / 4, and also improves computing efficiency, because on hardware such as a graphics processing unit (GPU), the calculation speed of low-bit values ​​is much higher than the calculation speed of high-bit values. However, using a lower bit data type means a smaller data representation range and sparser values, so when quantizing a neural network model, it will cause a loss of numerical precision. For example, after quantizing a float value to an int type, the decimal part will be lost first, and the value in the float value that exceeds the range that the int type can express will also be compressed to the larger or minimum value that the int can express.

[0031] At present, in order to improve the accuracy of the quantization model, it is usually Figure 1As shown, after the high-precision neural network model is quantized and compressed to obtain the quantized model, the quantized model will be retrained; in each round of training, the loss value of the training process is calculated based on the difference between the output value of the quantized model in each round of training and the true value. When the loss value cannot meet the accuracy requirements of the training, the network parameters of the quantized model are adjusted according to the loss value, and the next training is performed based on the adjusted network parameters. The above method is used for iterative training until the loss value meets the accuracy requirements, and the training is terminated to obtain the trained quantized model. The trained quantized model can be used for actual multimedia data processing later. However, the current training method has limited improvement on the accuracy of the quantized model, and it is difficult to restore the original accuracy of the high-precision neural network model.

[0032] The present application embodiment provides a data processing method, such as Figure 2 As shown, the method may include S101-S102, as follows:

[0033] S101: Acquire multimedia data to be processed.

[0034] The method provided in the embodiment of the present application is applicable to scenarios where a compressed neural network model is used to process multimedia data such as speech, images, and videos. By way of example, the compressed neural network model is used for application scenarios such as image classification, image recognition, object detection, speech recognition, and image segmentation. The specific method is selected based on actual conditions and is not limited in the embodiment of the present application.

[0035] In an embodiment of the present application, a data processing device obtains multimedia data to be processed, for example, an image to be identified as a target image, audio to be speech recognized, or video to be intelligently edited.

[0036] S102. Use the target quantization model to process the multimedia data to be processed to obtain a target processing result; the target quantization model is obtained by training the initial quantization model using a preset real target value and an original model demonstration value; the original model demonstration value includes at least one of the original model processing result and the original model weight distribution; wherein the initial quantization model is obtained by quantizing and compressing the original model; the original model is a network model of a first precision type for processing the multimedia data to be processed; the initial quantization model is a network model of a second precision type, and the precision of the first precision type is higher than that of the second precision type; the original model processing result is obtained by processing a preset training sample set with the original model; the original model weight distribution represents the weight distribution of the network layer of the original model.

[0037] In an embodiment of the present application, a data processing device uses a target quantization model to process multimedia data to be processed. By way of example, when the target quantization model is an image classification model and the multimedia data to be processed is a picture, the data processing device performs feature extraction and category probability inference on the input picture through the target quantization model, and then outputs the classification result of the picture as the target processing result.

[0038] In an embodiment of the present application, the original model is a network model of the first precision type for processing the multimedia data to be processed; the initial quantized model is a network model of the second precision type, and the precision of the first precision type is higher than that of the second precision type. The initial quantized model is obtained by quantizing and compressing the original model. In some embodiments, the original model can be a float type or a double precision (double) floating point type, and the initial quantized model can be a low-bit quantized model of a binarized type, 4bit, 8bit or 16bit.

[0039] In the embodiment of the present application, the target quantization model is obtained by training the initial quantization model using the preset real target value and the original model demonstration value. The initial quantization model is obtained by quantizing and compressing the original model. In order to reduce the network scale of the original model and reduce the resource occupation during application, the data processing device can quantize and compress the original model to obtain an initial quantization model suitable for running on a low power consumption and low computing resource platform.

[0040] In the embodiment of the present application, the original model is a network model that has been trained. The data processing accuracy of the original model can meet the needs of practical applications, so it can play a guiding role in the training process of the initial quantization model.

[0041] In an embodiment of the present application, in order to ensure the accuracy of data processing in practical applications, the data processing device can use the preset true target value and the original model demonstration value as target values ​​to train the initial quantization model, and iteratively update the network parameters of the initial quantization model according to the preset true target value and the original model demonstration value during the training process until the preset training target is reached, thereby obtaining the target quantization model.

[0042] In the embodiment of the present application, the preset real target value represents the expected result of data processing on the multimedia data to be processed; the original model demonstration value includes at least one of the original model processing result and the original model weight distribution. Among them, the original model processing result represents the actual result of the original model performing data processing on the multimedia data to be processed, which can be the final output processing result or the result output by the intermediate layer of the original model; the original model weight distribution represents the value distribution of the network parameters of each layer of the original model. Since the network architecture of the model to be processed is the same as that of the original model, the value of its own network parameters can be adjusted by learning the weight distribution of the original model.

[0043] It can be understood that in the embodiment of the present application, since the target quantization model is obtained by training based on the preset true target value and the original model demonstration value, in the process of training the initial quantization model, the processing result of the original model and at least one item in the weight distribution of the original model can be used as the training target of the initial quantization model on the basis of the preset true target value, so as to achieve more efficient adjustment of the network parameters of each round of training of the initial quantization model, thereby improving the convergence speed of training the initial quantization model and improving the accuracy of multimedia data processing by the target quantization model finally trained.

[0044] In the embodiment of the present application, before S102, it is also possible to Figure 3 As shown, including S201-S202, as follows:

[0045] S201, quantize and compress the original model to obtain an initial quantized model.

[0046] In the embodiment of the present application, in order to reduce the scale and resource occupation of the original model, the data processing device can perform linear quantization compression on the original model by back-end quantization to obtain an initial quantized model. Specifically, the data processing device can quantize the original network parameters of the original model according to a preset ratio to obtain an initial quantized model of the target parameter type; the bit number of the target parameter type is lower than the bit number of the original network parameters.

[0047] In the embodiment of the present application, the data processing device uses the network model corresponding to the network parameter of the target parameter type as the initial quantization model.

[0048] In some embodiments, when the original model is a floating-point model, the original network parameters of the float type of the original model can be linearly quantized by formula (1) and mapped to the fixed-point (Quant) data type space to obtain the network parameters of the Quant type, and then obtain the initial quantized model of the Quant type, thereby achieving quantization compression of the original network model of the float type, such as Figure 4 shown.

[0049] quant data =scale*float data (1)

[0050] In formula (1), float data is the original network parameter, scale is the mapping ratio from floating point type to fixed point type, quant data is float dataThe data processing device can use formula (1) to quantize and compress the original floating-point model into an initial fixed-point quantized model.

[0051] S202. Based on a preset loss function, iteratively train the initial quantization model to obtain a target quantization model; the preset loss function is a function that performs a weighted summation of at least two losses in the training process and corresponding at least two weight factors; wherein the at least two losses are obtained based on differential evaluation of each training result of the initial quantization model with a preset true target value and an original model demonstration value, and the at least two weight factors characterize the proportion of the at least two losses in the preset loss function.

[0052] In an embodiment of the present application, since the data processing device causes accuracy loss in the process of compressing the original model into an initial quantization model, the data processing device needs to iteratively train the initial quantization model based on a preset loss function to obtain a target quantization model to ensure that the accuracy of the target quantization model can meet the requirements of practical applications.

[0053] In the embodiment of the present application, the preset loss function is used to calculate the loss of the initial quantization model in each training process based on the training targets of different dimensions. In the embodiment of the present application, the training targets of different dimensions can be preset true target values ​​and original model demonstration values. Therefore, accordingly, the preset loss function of the embodiment of the present application includes at least two losses, wherein at least two losses are errors obtained by performing a difference evaluation on each training result of the initial quantization model, the preset true target value and the original model demonstration value.

[0054] In an embodiment of the present application, in a preset loss function, the preset true target value and the original model processing result can be used as training targets, or the preset true target value and the original model weight distribution can be used as training targets, or the preset true target value, the original model processing result and the original model weight distribution can be used as training targets, and other related items of the original model can be introduced as training targets in the preset loss function according to actual training needs. The specific selection is made according to actual conditions, and the embodiments of the present application are not limited thereto.

[0055] In an embodiment of the present application, corresponding to the above-mentioned training objectives, at least two losses in the preset loss function may be the error of each training result of the initial quantization model in at least one dimension of the original model processing result and the original model weight distribution dimension, and the error in the preset true target value dimension. Specifically, each training result of the initial quantization model may include the current training result output by each training of the initial quantization model and the weight distribution of the initial quantization model in each training; the at least two losses may include at least one of the second loss and the third loss, and the first loss. Among them, the first loss represents the error of each training result in the preset true target value dimension; the second loss represents the error of each training result in the dimension of the original model processing result; the third loss term represents the error of each training result in the dimension of the original model weight distribution.

[0056] In the embodiment of the present application, accordingly, in order to comprehensively reflect the proportion of each loss in the calculation result of the preset loss function, in the embodiment of the present application, the data processing device sets corresponding weights for the first loss, and at least one of the second loss and the third loss, thereby obtaining at least two weight factors. Among them, the at least two weight factors include at least one of the second weight factor and the third weight factor, and the first weight factor; the first weight factor is the weight corresponding to the first loss in the preset loss function, the second weight factor is the weight corresponding to the second loss in the preset loss function, and the third weight factor is the weight corresponding to the third loss in the preset loss function.

[0057] In the embodiment of the present application, the preset loss function may be a function that performs a weighted summation of at least two losses and corresponding at least two weight factors. The data processing device may multiply the first weight factor by the first loss through the preset loss function to obtain a first product; when there is a second loss, the data processing device multiplies the second weight factor by the second loss to obtain a second product; when there is a third loss, the data processing device multiplies the third weight factor by the third loss to obtain a third product. The data processing device uses the sum of all products as the calculation result of the preset loss function.

[0058] In some embodiments, when the original model demonstration value includes the original model processing result and the original model weight distribution, the preset loss function can be as shown in formula (2). The data processing device can use the first weight factor, and at least one of the second weight factor and the third weight factor, and multiply the first loss, and at least one of the second loss and the third loss by formula (2) to obtain the calculation result of the preset loss function, as follows:

[0059] Loss=w0*Loss0+w1*Loss1+w2*Loss2 (2)

[0060] In formula (2), Loss is the calculation result of the preset loss function, Loss0 is the first loss, w0 is the first weight factor, Loss1 is the second loss, w1 is the second weight factor, Loss2 is the third loss, w2 is the third weight factor, and the data processing device calculates the product of Loss0 and w0, the product of Loss1 and w1, and the product of Loss2 and w2, and adds the three products to obtain the calculation result Loss of the preset loss function.

[0061] It should be noted that in the embodiments of the present application, the calculation form of the preset loss function can be defined according to the actual training needs. It can be a weighted summation calculation method, or it can be a summation calculation method after exponential processing of the original model processing result or at least one item in the original model weight distribution and the preset true target value. The specific selection is made according to the actual situation and is not limited in the embodiments of the present application.

[0062] In an embodiment of the present application, the data processing device can be used in each round of training to obtain the corresponding calculation results of the current training based on a preset loss function, and then the initial quantization model can be iteratively trained according to the corresponding calculation results of the current training to finally obtain the target quantization model.

[0063] It can be understood that in the embodiment of the present application, the data processing device can evaluate the loss value of each training result of the initial quantization model from two dimensions, namely the preset true target value and the original model demonstration value, according to the preset loss function, so that the initial quantization model can not only learn the preset true target value during the training process, but also learn at least one of the original model processing result and the original model weight distribution, thereby improving the accuracy of the target quantization model finally trained.

[0064] In the embodiment of the present application, the specific implementation process of S202 can be as follows: Figure 5 As shown, including S2021-S2025 as follows:

[0065] S2021. Based on the initial quantization model, a preset training sample set is processed to obtain a current training result output by the initial quantization model.

[0066] In an embodiment of the present application, for a training process, the data processing device may use an initial quantization model to process a preset training sample set to obtain a current training result output by the initial quantization model.

[0067] In some embodiments of the present application, in order to enable the initial quantization model to learn the data processing results of the original model, the data processing device can also use the original model and the initial quantization model to jointly process a preset training sample set to obtain the current training results output by the initial quantization model and the original model processing results output by the original model to start training the initial quantization model.

[0068] S2022. In the current training, determine at least two losses based on the current training result.

[0069] In an embodiment of the present application, the data processing device may, in the current training, perform a difference evaluation based on the current training result and between a preset true target value and an original model demonstration value, thereby determining at least two losses.

[0070] In the present application embodiment, based on Figure 5 The implementation process of S2022 can be specifically as follows Figure 6 As shown, it is implemented through S301-S304 as follows:

[0071] S301: The difference between the current training result and the preset true target value is taken as the first loss.

[0072] In the embodiment of the present application, the data processing device uses the difference between the training processing result and the preset true target value as the first loss.

[0073] S302. When the original model demonstration value includes the original model processing result, the difference between the original model processing result and the current training result is used as the second loss.

[0074] In an embodiment of the present application, in S202, the initial quantization model is iteratively trained based on a preset loss function. Before obtaining the target quantization model, the data processing device may use the original model to process a preset training sample set, and obtain a first final processing result output by the output layer of the original model as the original model processing result; or, use the original model to process a preset training sample set, and obtain a first intermediate processing result output by the intermediate layer of the original model as the original model processing result.

[0075] In an embodiment of the present application, corresponding to the situation where the data processing device obtains the first final processing result as the original model processing result, that is, when the original model processing result is the first final processing result, the method in S2021 processes the preset training sample set based on the initial quantization model to obtain the current training result output by the initial quantization model is: use the initial quantization model to process the preset training sample set, and obtain the second final processing result output by the output layer of the initial quantization model as the current training result.

[0076] In an embodiment of the present application, corresponding to the situation where the data processing device obtains the first intermediate processing result as the original model processing result, that is, when the original model processing result is the first intermediate processing result, the method in S2021 processes the preset training sample set based on the initial quantization model to obtain the current training result output by the initial quantization model is: using the initial quantization model to process the preset training sample set, and obtaining the second intermediate processing result output by the intermediate layer of the initial quantization model as the current training result.

[0077] That is to say, in an embodiment of the present application, when the original model demonstration value includes the original model processing result, the data processing device may use the difference between the first final result and the second final result as the second loss, or may use the difference between the first intermediate result and the second intermediate result as the second loss.

[0078] S303: When the original model demonstration value includes the original model weight distribution, the difference between the weight distribution of the original model and the weight distribution of the initial quantization model is used as the third loss.

[0079] In the embodiment of the present application, since the initial quantization model can be obtained by quantizing and compressing the network parameters of the original model, the network architecture of the initial quantization model is the same as that of the original model. Among them, the original model includes at least one layer of original network layer, the initial quantization model includes at least one layer of quantized network layer, and the network architecture of at least one layer of original network layer corresponds to the network architecture of at least one layer of quantized network layer. Based on the same network layer structure, the initial quantization model can learn the weight distribution of the original model during training, so as to complete the training process faster and improve the accuracy of the final training result.

[0080] In some embodiments of the present application, when the original model demonstration value includes the original model weight distribution, the data processing device uses the difference between the weight distribution of the original model and the weight distribution of the initial quantization model as the third loss, which can be implemented by S3031-S3034 as follows:

[0081] S3031. When the original model demonstration value includes the original model weight distribution, for each original network layer in at least one original network layer, obtain first weight distribution data of each original network layer.

[0082] In an embodiment of the present application, when the original model demonstration value includes the original model weight distribution, the data processing device obtains the weight distribution data of each original network layer in at least one original network layer of the original model, exemplarily, the weight matrix of each original network layer, as the first weight distribution data.

[0083] S3032. In at least one quantized network layer, for each quantized network layer corresponding to each original network layer, obtain second weight distribution data of each quantized network layer.

[0084] The present application is implemented in two ways. Since at least one original network layer corresponds to at least one quantized network layer one-to-one, the data processing device obtains weight distribution data of each quantized network layer for each quantized network layer corresponding to each original network layer in at least one quantized network layer of the initial quantized model. Exemplarily, the weight matrix of each quantized network layer is used as the second weight distribution data.

[0085] S3033. Calculate the difference between each first weight distribution data and its corresponding second weight distribution data, thereby obtaining at least one weight difference.

[0086] In the embodiment of the present application, the data processing device calculates the difference between the first weight distribution data and the second weight distribution data one by one, thereby obtaining at least one weight difference.

[0087] S3034. Taking the sum of at least one weight distribution difference as the third loss.

[0088] In an embodiment of the present application, the data processing device uses the sum of at least one weight distribution difference as the third loss; the third loss characterizes the difference in network parameter configuration between the initial quantization model and the original model.

[0089] In some embodiments, the original model is a floating-point type network model, and the original model includes at least one original network layer 1, 2...n; the initial quantized model is a fixed-point type network model, and the initial quantized model includes at least one quantized network layer 1, 2...n; Figure 6 The data processing device may calculate the difference 1 between the weight distribution of the original network layer 1 and the weight distribution of the quantized network layer 1, and calculate the difference 2 between the weight distribution of the original network layer 2 and the weight distribution of the quantized network layer 2, and calculate one by one until the difference n between the weight distribution of the original network layer n and the weight distribution of the quantized network layer n is calculated. The data processing device may take the sum of the difference 1 and the difference 2 to the difference n as the third loss.

[0090] In some embodiments, Figure 7 It is also shown that the data processing device processes the original model result according to the original model output and the current training result output by the initial quantization model, and uses the difference between the original model processing result and the current training result as the second loss.

[0091] S304: taking at least one of the second loss and the third loss and the first loss as at least two losses.

[0092] In an embodiment of the present application, the data processing device can set the loss item according to the actual training needs, and according to the above method, calculate at least one of the second loss and the third loss, as well as the first loss, thereby obtaining at least two losses.

[0093] S2023. Determine, based on the at least two losses, at least two weight factors corresponding to the at least two losses in the current training.

[0094] In an embodiment of the present application, the data processing device can dynamically adjust the weight factors corresponding to the first loss, and at least one of the second loss and the third loss, to implement a two-stage training method of first teaching and then comparing with a preset true target value, so as to accelerate the convergence speed of the quantization model and improve the accuracy of the quantization model.

[0095] In an embodiment of the present application, the data processing device can determine the first weight factor in the current training, and at least one of the second weight factor and the third weight factor based on the value of the second loss and / or the third loss calculated in the current training, so as to dynamically adjust the weights corresponding to different losses during the training process, so that the initial quantization model focuses on learning the training targets corresponding to high-weight losses.

[0096] In some embodiments of the present application, the data processing device may adjust the value of the first weight factor in the previous training, and at least one of the second weight factor and the third weight factor based on the convergence speed of the second loss and / or the third loss in the current training compared to the previous training.

[0097] In some embodiments, during the training process, when the data processing device finds that the convergence speed of the second loss and / or the third loss changes from fast to slow, it indicates that the initial quantization model has approached the training target corresponding to the second loss and / or the third loss. The data processing device can gradually reduce the weight of the second weight factor and / or the third weight factor in each subsequent round of training, and gradually increase the weight factors corresponding to other unconverged losses, so that the learning focus of the initial quantization model can be automatically adjusted in real time according to the training progress, thereby determining the first weight factor in this round of training, and at least one of the second weight factor and the third weight factor.

[0098] In some embodiments of the present application, based on Figure 6 , S2023 can be Figure 8 As shown, it is implemented through S401-S402 as follows:

[0099] S401: When the current training is the first training, determine the first weight factor to be zero.

[0100] In an embodiment of the present application, the first weight factor is the weight corresponding to the first loss. During the first training, the data processing device first sets the first weight factor to zero. That is, at the beginning of training, the training processing result of the initial quantization model is not considered to be compared with the preset true target value, but the original model demonstration value is learned first.

[0101] S402: Determine a second weight factor and / or a third weight factor according to a preset total weight value, so that the sum of the second weight factor and / or the third weight factor is the preset total weight value.

[0102] In the embodiment of the present application, the second weight factor is the weight corresponding to the second loss in the preset loss function; the third weight factor is the weight corresponding to the third loss in the preset loss function. During the first training, the data processing device initializes the second weight factor and / or the third weight factor so that the sum of the second weight factor and / or the third weight factor is the preset total weight value.

[0103] In an embodiment of the present application, the data processing device calculates the comprehensive loss of the first training based on the initialized first weight factor and at least one of the second weight factor and the third weight factor, adjusts the network parameters of the initial quantization model based on the comprehensive loss of the first training, and then enters the next round of training.

[0104] It is understandable that the comprehensive loss of the first training is mainly calculated through the second loss and / or the third loss, thereby achieving rapid learning of the original model in the early stage of training the initial quantization model and improving the convergence speed of the training process.

[0105] In some embodiments, when the preset total weight value is 1, the second weight factor may be initialized to 0.6, and the third weight factor may be initialized to 1.

[0106] In some embodiments of the present application, Figure 8 As shown, S2023 can also be implemented through S403-S405 as follows:

[0107] S403: When the current training is not the first training, determine whether the second loss among the at least two losses is less than or equal to a preset second loss threshold; or determine whether the third loss among the at least two losses is less than or equal to a preset third loss threshold.

[0108] In an embodiment of the present application, when the data processing device obtains the second loss and / or third loss of each training, it can determine whether the second loss is less than or equal to a preset second loss threshold; or whether the third loss is less than or equal to a preset third loss threshold.

[0109] In an embodiment of the present application, the preset second loss threshold may be the same value as the preset third loss threshold, or may be different thresholds set separately. The specific selection is made based on actual conditions and is not limited in the embodiment of the present application.

[0110] S404: When the second loss is less than or equal to a preset second loss threshold, determine the first weight factor according to the second weight factor in the previous training, and determine the second weight factor to be zero.

[0111] In the embodiment of the present application, when the second loss is less than or equal to the preset second loss threshold, it means that the initial quantization model has learned the training target corresponding to the second loss during the training process, such as the original model processing result, so the initial quantization model can start learning the preset real target value by reducing the second weight factor and increasing the first weight factor. The data processing device can use the second weight factor in the previous training to assign the first weight factor in this round of training, that is, use the value of the second weight factor in the previous training as the value of the first weight factor in the current training, and set the second weight factor of the current training to zero, thereby starting to learn the preset real target value.

[0112] In an embodiment of the present application, the data processing device may adjust the weight factor of the previous training in each round of training, or may not adjust the weight factor when the second loss is greater than a preset second loss threshold, and only assign the second weight factor of the previous training to the first weight factor of the current training when the second loss is less than or equal to the preset second loss threshold, and set the second weight factor of the current training to zero.

[0113] It should be noted that, in the case where the weight factor is adjusted only when the second loss is less than or equal to the preset second loss threshold, the value of each weight factor of the last training is the same as the value of each weight factor at the time of initialization.

[0114] S405: When the third loss is less than or equal to a preset third loss threshold, determine the first weight factor according to the third weight factor in the previous training, and determine the third weight factor to be zero.

[0115] In the embodiment of the present application, when the third loss is less than or equal to the preset third loss threshold, it means that the initial quantization model has learned the training target corresponding to the third loss during the training process, such as the original model weight distribution, so the initial quantization model can start learning the preset real target value by reducing the third weight factor and increasing the first weight factor. The data processing device can use the third weight factor in the previous training to assign the first weight factor in the current training, that is, use the value of the third weight factor in the previous training as the value of the first weight factor in the current training, and set the third weight factor of the current training to zero, thereby starting to learn the preset real target value.

[0116] In an embodiment of the present application, the data processing device may adjust the weight factor of the previous training in each round of training, or may not adjust the weight factor when the third loss is greater than the preset second loss threshold, and only assign the third weight factor of the previous training to the first weight factor of the current training when the third loss is less than or equal to the preset second loss threshold, and set the third weight factor of the current training to zero.

[0117] It should be noted that, in the case where the weight factor is adjusted only when the third loss is less than or equal to the preset third loss threshold, the value of each weight factor of the last training is the same as the value of each weight factor at the time of initialization.

[0118] In the embodiment of the present application, when the data processing device adjusts at least one of the first weight factor, the second weight factor, and the third weight factor, it is necessary to ensure that the total value of at least one of the first weight factor, the second weight factor, and the third weight factor is equal to the preset total weight value. Exemplarily, when the preset total weight value is 1 and the original model demonstration value includes the original model processing result and the original model weight distribution, no matter how the data processing device adjusts the first weight factor, the second weight factor, and the third weight factor during the training process, the total value of the three needs to be 1.

[0119] It can be understood that in the embodiment of the present application, since the data processing device can dynamically adjust at least one of the first weight factor, the second weight factor and the third weight factor during the training process, a two-stage training method of first teaching and then comparing with the preset true target value is implemented. Therefore, the initial quantization model in the early stage of training can quickly learn the original model demonstration value, thereby improving the training convergence speed, and the initial quantization model in the later stage can learn the preset true target value with a large proportion, thereby further improving the accuracy of the target quantization model.

[0120] S2024. Based on at least two losses and at least two weight factors, a preset loss function is used to calculate the current comprehensive loss of the current training; the current comprehensive loss represents the comprehensive difference between the current training result and the preset true target value and the original model demonstration value.

[0121] In an embodiment of the present application, after determining at least two losses and at least two weight factors in the current training, the data processing device can calculate the current comprehensive loss of the current training through a preset loss function, exemplarily such as formula (2).

[0122] In an embodiment of the present application, the current comprehensive loss is a comprehensive representation of at least one of the difference between the training processing result of the initial quantization model and the processing result of the original model in the current training, the difference between the network weight distribution of the initial quantization model and the network weight distribution of the original model, and the difference between the training processing result of the initial quantization model and the preset true target value.

[0123] S2025. When the current comprehensive loss does not meet the preset comprehensive loss threshold, the network parameters of the initial quantization model are adjusted, and the next training is continued until the Nth comprehensive loss meets the preset comprehensive loss threshold, and the target quantization model is obtained. The Nth time is the last training after the current time.

[0124] In the embodiment of the present application, after the data processing device obtains the comprehensive loss of the current training, it determines whether the current comprehensive loss meets the preset comprehensive loss threshold. When the current comprehensive loss does not meet the preset comprehensive loss threshold, the data processing device adjusts the network parameters of the initial quantization model of the current training with minimizing the current comprehensive loss as the optimization goal, and performs the next training based on the adjusted network parameters; the data processing device iteratively updates the network parameters of the initial quantization model in the above manner until the Nth comprehensive loss meets the preset comprehensive loss threshold, the training ends, and the target quantization model is obtained. Among them, the Nth time is the last training after the current time.

[0125] It should be noted that in the embodiment of the present application, the preset number of training times can also be used as a preset condition for ending the training. The specific setting is based on actual conditions and is not limited in the embodiment of the present application.

[0126] It can be understood that in the embodiment of the present application, the data processing device introduces at least one of the processing results of the original model and the weight distribution of the original model into the training target of the initial quantization model, and can perform a comprehensive evaluation of the training loss of each round of the initial quantization model by setting the weights corresponding to the preset true target value and the original model demonstration value, thereby being able to achieve more efficient adjustment of the network parameters of each round of training of the initial quantization model based on the comprehensive loss obtained by the comprehensive evaluation, thereby improving the convergence speed of training the initial quantization model and improving the accuracy of multimedia data processing by the target quantization model finally obtained through training.

[0127] The following will describe an exemplary application of the present invention in an actual application scenario. When the original model demonstration value includes the original model processing result and the original model weight distribution, the present invention provides a method for model quantization and training. Fig. 9 As shown below:

[0128] S501 , linearly quantize the original model to obtain an initial quantized model.

[0129] In the embodiment of the present application, the data processing device performs linear quantization on the original model to obtain an initial quantization model, which is consistent with the process description of quantization compression in S201 and will not be repeated here.

[0130] S502: Retrain the initial quantization model.

[0131] In the embodiment of the present application, the initial quantization model of the data processing device is retrained to improve the accuracy of the initial quantization model. Specifically, the training process of S502 can be implemented by S5021-S5027 as follows:

[0132] S5021. Input a preset training sample set into the original model and the initial quantization model.

[0133] S5022: In a training session, use the initial quantization model to perform inference calculations to obtain a current training result.

[0134] S5023. Use the original model to perform inference calculations to obtain the original model processing results.

[0135] In the embodiment of the present application, the implementation process of S5021-S5023 is consistent with the description of S2021 and will not be repeated here.

[0136] S5024. Based on the current training processing result, according to the preset true target value, the original model processing result and the original model weight distribution, the first loss, the second loss and the third loss are calculated.

[0137] In the embodiment of the present application, the implementation process of S5024 is consistent with the description of S2022 and will not be repeated here.

[0138] S5025. Determine the first weight factor, the second weight factor, and the third weight factor of the current training according to the second loss and the third loss.

[0139] In the embodiment of the present application, the implementation process of S5025 is consistent with the description of S2023 and will not be repeated here.

[0140] S5026. Calculate the current comprehensive loss through a preset loss function using the first loss, the second loss and the third loss, as well as the first weight factor, the second weight factor and the third weight factor.

[0141] In the embodiment of the present application, the implementation process of S5026 is consistent with the description of S2024 and will not be repeated here.

[0142] S5027. Determine whether the current comprehensive loss meets the preset comprehensive loss threshold. When the current comprehensive loss of the initial quantitative model meets the preset comprehensive loss threshold, execute S503; otherwise, execute S5028.

[0143] S5028. After adjusting the network parameters of the initial quantization model, continue to jump to S5021 for the next training.

[0144] S503: End the training and obtain the target quantization model.

[0145] In the embodiment of the present application, the implementation process of S5027-S503 is consistent with the description of S2025 and will not be repeated here.

[0146] It can be understood that in the embodiment of the present application, the data processing device introduces at least one of the processing results of the original model and the weight distribution of the original model into the training target of the initial quantization model, and can perform a comprehensive evaluation of the training loss of each round of the initial quantization model by setting the weights corresponding to the preset true target value and the original model demonstration value, thereby being able to achieve more efficient adjustment of the network parameters of each round of training of the initial quantization model based on the comprehensive loss obtained by the comprehensive evaluation, thereby improving the convergence speed of training the initial quantization model and improving the accuracy of multimedia data processing by the target quantization model finally obtained through training.

[0147] The present application provides a data processing device. Fig.10 is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application, such as Fig.10 As shown, the data processing device 1 includes: an acquisition unit 11, used to acquire multimedia data to be processed; a model processing unit 12, used to process the multimedia data to be processed using a target quantization model to obtain a target processing result; the target quantization model is obtained by training the initial quantization model using a preset real target value and an original model demonstration value; the original model demonstration value includes at least one of the original model processing result and the original model weight distribution;

[0148] Among them, the initial quantization model is obtained by quantizing and compressing the original model; the original model is a network model of the first precision type for processing the multimedia data to be processed; the initial quantization model is a network model of the second precision type, and the precision of the first precision type is higher than that of the second precision type; the original model processing result is obtained by processing a preset training sample set by the original model; the original model weight distribution represents the weight distribution of the network layer of the original model.

[0149] In some embodiments, the data processing device 1 also includes a compression unit and a model training unit, wherein the compression unit is used to perform quantization compression on the original model to obtain the initial quantization model; the model training unit is used to iteratively train the initial quantization model based on a preset loss function to obtain the target quantization model; the preset loss function is a function that performs a weighted summation of at least two losses in the training process and corresponding at least two weight factors; wherein the at least two losses are obtained based on a difference evaluation of each training result of the initial quantization model with the preset true target value and the original model demonstration value, and the at least two weight factors represent the proportion of the at least two losses in the preset loss function.

[0150] In some embodiments, the model training unit is also used to process the preset training sample set based on the initial quantization model to obtain the current training result output by the initial quantization model; in the current training, based on the current training result, determine the at least two losses; according to the at least two losses, determine the at least two weight factors corresponding to the at least two losses in the current training; according to the at least two losses and the at least two weight factors, use the preset loss function to calculate the current comprehensive loss of the current training; the current comprehensive loss represents the comprehensive difference between the current training result and the preset true target value and the original model demonstration value; when the current comprehensive loss does not meet the preset comprehensive loss threshold, the network parameters of the initial quantization model are adjusted, and the next training is continued until the Nth comprehensive loss meets the preset comprehensive loss threshold, and the target quantization model is obtained, and the Nth time is the last training after the current time.

[0151] In some embodiments, the model training unit is further used to use the difference between the current training result and the preset true target value as the first loss; when the original model demonstration value includes the original model processing result, the difference between the original model processing result and the current training result is used as the second loss; when the original model demonstration value includes the original model weight distribution, the difference between the weight distribution of the original model and the weight distribution of the initial quantization model is used as the third loss; at least one of the second loss and the third loss, and the first loss are used as the at least two losses.

[0152] In some embodiments, the original model includes at least one original network layer, the initial quantized model includes at least one quantized network layer, and the network architecture of the at least one original network layer corresponds one-to-one to the network architecture of the at least one quantized network layer; the model training unit is also used to, when the original model demonstration value includes the original model weight distribution, obtain the first weight distribution data of each original network layer for each original network layer in the at least one original network layer; in the at least one quantized network layer, for each quantized network layer corresponding to each original network layer, obtain the second weight distribution data of each quantized network layer; calculate the difference between each first weight distribution data and its corresponding second weight distribution data, thereby obtaining at least one weight difference; and use the sum of the at least one weight distribution difference as the third loss.

[0153] In some embodiments, the model training unit is also used to iteratively train the initial quantization model based on a preset loss function, and before obtaining the target quantization model, use the original model to process the preset training sample set, and obtain the first final processing result output by the output layer of the original model as the original model processing result; or, use the original model to process the preset training sample set, and obtain the first intermediate processing result output by the intermediate layer of the original model as the original model processing result.

[0154] In some embodiments, when the original model processing result is the first final processing result, the model training unit is also used to use the initial quantization model to process the preset training sample set, and obtain the second final processing result output by the output layer of the initial quantization model as the current training result; when the original model processing result is the first intermediate processing result, the model training unit is also used to use the initial quantization model to process the preset training sample set, and obtain the second intermediate processing result output by the intermediate layer of the initial quantization model as the current training result.

[0155] In some embodiments, the at least two weight factors include at least one of a second weight factor and a third weight factor, and a first weight factor, the first weight factor being the weight corresponding to the first loss in the preset loss function, the second weight factor being the weight corresponding to the second loss in the preset loss function, and the third weight factor being the weight corresponding to the third loss in the preset loss function; the model training unit is also used to determine the first weight factor to be zero when the current training is the first training; determine the second weight factor and / or the third weight factor according to a preset total weight value so that the sum of the second weight factor and / or the third weight factor is the preset total weight value.

[0156] In some embodiments, the model training unit is also used to determine whether the second loss among the at least two losses is less than or equal to a preset second loss threshold when the current training is not the first training; or, to determine whether the third loss among the at least two losses is less than or equal to a preset third loss threshold; when the second loss is less than or equal to the preset second loss threshold, determine the first weight factor according to the second weight factor in the previous training, and determine the second weight factor to be zero; when the third loss is less than or equal to the preset third loss threshold, determine the first weight factor according to the third weight factor in the previous training, and determine the third weight factor to be zero.

[0157] In some embodiments, the compression unit is also used to quantize the original network parameters of the original model according to a preset ratio to obtain network parameters of a target parameter type; the number of bits of the target parameter type is lower than the number of bits of the original network parameters; and the network model corresponding to the network parameters of the target parameter type is used as the initial quantization model.

[0158] It should be noted that the description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0159] The present application also provides a data processing device, Fig.11 is a schematic diagram of the structure of the data processing device 2 provided in the embodiment of the present application, such as Fig.11 As shown, the data processing device 2 includes: a memory 23 and a processor 24, wherein the memory 23 and the processor 24 are connected via a communication bus 25; the memory 23 is used to store executable instructions or executable computer programs; the processor 24 is used to implement the method provided in the embodiment of the present application when executing the executable instructions or executable computer programs stored in the memory 23, for example, the data processing method provided in the embodiment of the present application.

[0160] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing the processor 24 to execute and implement the method provided by the embodiment of the present application, for example, the data processing method provided by the embodiment of the present application.

[0161] In some embodiments of the present application, the storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0162] In some embodiments of the present application, executable instructions may be in the form of programs, software, software modules, scripts or codes, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines or other units suitable for use in a computing environment.

[0163] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0164] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0165] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0166] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0167] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0168] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0169] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. A data processing method, characterized in that: include: Acquire multimedia data to be processed; Using a target quantization model to process the multimedia data to be processed, to obtain a target processing result; The target quantization model is obtained by training the initial quantization model using a preset real target value and an original model demonstration value; the original model demonstration value includes at least one of an original model processing result and an original model weight distribution; The initial quantization model is obtained by quantizing and compressing the original model; The original model is a network model of the first precision type for processing the multimedia data to be processed; the initial quantization model is a network model of the second precision type, and the accuracy of the first precision type is higher than that of the second precision type; the original model processing result is obtained by processing a preset training sample set by the original model; the original model weight distribution represents the weight distribution of the network layer of the original model.

2. The method according to claim 1, characterized in that Before the target quantization model is used to process the multimedia data to be processed to obtain a target processing result, the method further includes: Performing quantization compression on the original model to obtain the initial quantization model; Based on a preset loss function, the initial quantization model is iteratively trained to obtain the target quantization model; the preset loss function is a function that performs a weighted summation of at least two losses in the training process and corresponding at least two weight factors; wherein the at least two losses are obtained based on a difference evaluation of each training result of the initial quantization model with the preset true target value and the original model demonstration value, and the at least two weight factors characterize the proportion of the at least two losses in the preset loss function.

3. The method according to claim 2, characterized in that The iterative training of the initial quantization model based on a preset loss function to obtain the target quantization model includes: Based on the initial quantization model, the preset training sample set is processed to obtain a current training result output by the initial quantization model; In the current training, based on the current training result, determining the at least two losses; Determining, according to the at least two losses, the at least two weight factors corresponding to the at least two losses in the current training; According to the at least two losses and the at least two weight factors, the current comprehensive loss of the current training is calculated using the preset loss function; the current comprehensive loss represents the comprehensive difference between the current training result and the preset true target value and the original model demonstration value; When the current comprehensive loss does not meet the preset comprehensive loss threshold, the network parameters of the initial quantization model are adjusted, and the next training is continued until the Nth comprehensive loss meets the preset comprehensive loss threshold, and the target quantization model is obtained, and the Nth time is the last training after the current time.

4. The method according to claim 3, characterized in that The determining, in the current training, based on the current training result, of the at least two losses includes: The difference between the current training result and the preset true target value is used as the first loss; When the original model demonstration value includes the original model processing result, the difference between the original model processing result and the current training result is used as the second loss; When the original model demonstration value includes the original model weight distribution, taking the difference between the weight distribution of the original model and the weight distribution of the initial quantization model as the third loss; At least one of the second loss and the third loss, and the first loss are used as the at least two losses.

5. The method according to claim 4, characterized in that The original model includes at least one original network layer, the initial quantized model includes at least one quantized network layer, and the network architecture of the at least one original network layer corresponds to the network architecture of the at least one quantized network layer in one-to-one correspondence; when the original model demonstration value includes the original model weight distribution, taking the difference between the weight distribution of the original model and the weight distribution of the initial quantized model as the third loss, including: When the original model demonstration value includes the original model weight distribution, for each original network layer in the at least one original network layer, obtaining first weight distribution data of each original network layer; In the at least one quantized network layer, for each quantized network layer corresponding to each original network layer, obtaining second weight distribution data of each quantized network layer; Calculating the difference between each first weight distribution data and its corresponding second weight distribution data, thereby obtaining at least one weight difference; The sum of the at least one weight distribution difference is taken as the third loss.

6. The method according to any one of claims 2 to 4, characterized in that: Based on a preset loss function, the initial quantization model is iteratively trained to obtain the target quantization model, and the method further includes: Using the original model to process the preset training sample set, and obtaining a first final processing result output by the output layer of the original model as the original model processing result; Alternatively, the preset training sample set is processed using the original model to obtain a first intermediate processing result output by an intermediate layer of the original model as the original model processing result.

7. The method according to claim 6, characterized in that When the original model processing result is the first final processing result, the preset training sample set is processed based on the initial quantization model to obtain a current training result output by the initial quantization model, including: Processing the preset training sample set using the initial quantization model, and obtaining a second final processing result output by an output layer of the initial quantization model as the current training result; When the original model processing result is the first intermediate processing result, the preset training sample set is processed based on the initial quantization model to obtain the current training result output by the initial quantization model, including: The preset training sample set is processed using the initial quantization model, and a second intermediate processing result output by an intermediate layer of the initial quantization model is obtained as the current training result.

8. The method according to any one of claims 2 to 7, characterized in that: The at least two weight factors include at least one of a second weight factor and a third weight factor, and a first weight factor, the first weight factor being a weight corresponding to a first loss in the preset loss function, the second weight factor being a weight corresponding to a second loss in the preset loss function, and the third weight factor being a weight corresponding to a third loss in the preset loss function; Determining, according to the at least two losses, the at least two weight factors corresponding to the at least two losses in the current training, comprising: When the current training is the first training, determining the first weight factor to be zero; The second weighting factor and / or the third weighting factor is determined according to a preset total weight value, so that the sum of the second weighting factor and / or the third weighting factor is the preset total weight value.

9. The method according to claim 8, characterized in that The determining, according to the at least two losses, the at least two weight factors corresponding to the at least two losses in the current training includes: When the current training is not the first training, determining whether the second loss among the at least two losses is less than or equal to a preset second loss threshold; or determining whether the third loss among the at least two losses is less than or equal to a preset third loss threshold; When the second loss is less than or equal to the preset second loss threshold, determining the first weight factor according to the second weight factor in the previous training, and determining the second weight factor to be zero; When the third loss is less than or equal to the preset third loss threshold, the first weight factor is determined according to the third weight factor in the previous training, and the third weight factor is determined to be zero.

10. The method according to claim 2, wherein the quantizing and compressing the original model to obtain the initial quantized model comprises: Quantizing the original network parameters of the original model according to a preset ratio to obtain network parameters of a target parameter type; The number of bits of the target parameter type is lower than the number of bits of the original network parameter; The network model corresponding to the network parameter of the target parameter type is used as the initial quantization model.

11. A data processing device, characterized in that: include: An acquisition unit, used for acquiring multimedia data to be processed; A model processing unit, used to process the to-be-processed multimedia data using a target quantization model to obtain a target processing result; The target quantization model is obtained by training the initial quantization model using a preset real target value and an original model demonstration value; the original model demonstration value includes at least one of an original model processing result and an original model weight distribution; The initial quantization model is obtained by quantizing and compressing the original model; The original model is a network model of the first precision type for processing the multimedia data to be processed; the initial quantization model is a network model of the second precision type, and the accuracy of the first precision type is higher than that of the second precision type; the original model processing result is obtained by processing a preset training sample set by the original model; the original model weight distribution represents the weight distribution of the network layer of the original model.

12. A data processing device, characterized in that: include: A memory for storing executable data instructions; A processor, configured to implement the method according to any one of claims 1 to 10 when executing the executable instructions stored in the memory.

13. A computer-readable storage medium, characterized in that: Executable instructions are stored, which are used to cause a processor to execute and implement the method described in any one of claims 1 to 10.

Images