Data processing methods and apparatus, electronic devices, computer-readable storage media

Abstract

This disclosure provides a data processing method and training method, a data processing device, a training device, an electronic device, and a computer-readable medium. The data processing method includes: acquiring input features of input data corresponding to a target network layer of a target neural network; inputting the input features corresponding to the target network layer into a prediction model to predict the category attribute corresponding to the target network layer, wherein the category attribute characterizes whether the target network layer is suitable for quantization processing of the input data. According to the technical solution of this disclosure, the quantization scheme of the network layer of the target neural network can be determined efficiently and accurately, which is beneficial for the quantized target neural network to ensure that it simultaneously possesses fast inference speed and high inference accuracy during inference operations.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a data processing method and a training method for a prediction model, a data processing device, a training device, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Quantization is a commonly used technique in the field of deep learning. By quantizing the input data of the network layers of a neural network, such as quantizing the precision of the input data from float32 to int8, the inference speed of the neural network model can be effectively improved. However, quantization may also cause a loss of prediction accuracy of the neural network model.

[0003] Therefore, in the process of quantizing neural network models, how to ensure that the neural network has a fast inference speed and high inference accuracy has become an urgent technical problem to be solved. Summary of the Invention

[0004] This disclosure provides a data processing method and a training method for a prediction model, a data processing device, a training device, an electronic device, and a computer-readable storage medium.

[0005] In a first aspect, this disclosure provides a data processing method, the data processing method comprising:

[0006] Obtain the input features of the input data corresponding to the target network layer of the target neural network;

[0007] The input features corresponding to the target network layer are input into the prediction model to predict the category attribute corresponding to the target network layer. The category attribute indicates whether the target network layer is suitable for quantizing the input data.

[0008] Secondly, this disclosure provides a method for training a prediction model, the training method comprising:

[0009] The input features of input data corresponding to multiple network layers of at least one neural network are obtained, wherein each network layer has a corresponding category attribute, and the category attribute indicates whether the corresponding network layer is suitable for quantization processing of the corresponding input data.

[0010] The input features of the input data of each network layer are used as training samples, and the category attributes are used as the labels of the corresponding training samples to obtain the training sample set;

[0011] The initial prediction model is trained based on the training sample set to obtain the prediction model;

[0012] The prediction model is used to predict the category attribute corresponding to the network layer based on the input features of the input data corresponding to the input network layer.

[0013] Thirdly, this disclosure provides a data processing apparatus, the data processing apparatus comprising:

[0014] The first acquisition unit is used to acquire the input features of the input data corresponding to the target network layer of the target neural network;

[0015] The prediction unit is used to input the input features corresponding to the target network layer into the prediction model and predict the category attribute corresponding to the target network layer. The category attribute represents whether the target network layer is suitable for quantizing the input data.

[0016] Fourthly, this disclosure provides a training apparatus for a prediction model, the training apparatus comprising:

[0017] The second acquisition unit is used to acquire the input features of the input data corresponding to the multiple network layers of at least one neural network, wherein each network layer has a corresponding category attribute, and the category attribute indicates whether the corresponding network layer is suitable for quantizing the corresponding input data.

[0018] The sample set unit is used to obtain the training sample set by taking the input features of the input data of each network layer as training samples and the category attribute as the label of the corresponding training sample.

[0019] A training unit is used to train the initial prediction model based on the training sample set to obtain the prediction model.

[0020] The prediction model is used to predict the category attribute corresponding to the network layer based on the input features of the input data corresponding to the input network layer.

[0021] Fifthly, this disclosure provides an electronic device comprising:

[0022] At least one processor;

[0023] and a memory communicatively connected to the at least one processor;

[0024] The memory stores one or more computer programs that can be executed by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the data processing method or training method described above.

[0025] In a sixth aspect, this disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described data processing method or training method.

[0026] According to the technical solutions of the data processing method and prediction model training method, data processing device, training device, electronic device, and computer-readable storage medium provided in the embodiments of this disclosure, the data processing method can be applied to any type of target neural network. For any target network layer of the target neural network, the prediction model is used to predict the category attribute of the target network layer based on the input features of the target network layer. The category attribute can characterize whether the target network layer is suitable for quantization of the input data. Therefore, it is possible to efficiently and accurately determine which network layers in the target neural network are suitable for quantization and which network layers are not suitable for quantization based on the predicted category attribute of each network layer in the target neural network. This allows for efficient and accurate determination of the network layer quantization scheme of the target neural network, and quantization of the network layers of the target neural network based on the quantization scheme. This is beneficial for the quantized target neural network to have both fast inference speed and high inference accuracy when performing inference operations.

[0027] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0028] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0029] Figure 1 A flowchart illustrating a data processing method provided in an embodiment of this disclosure;

[0030] Figure 2 for Figure 1 A flowchart illustrating a specific implementation of step S11;

[0031] Figure 3 for Figure 1 A flowchart illustrating another specific implementation of step S11;

[0032] Figure 4 for Figure 1 A flowchart illustrating another specific implementation of step S11;

[0033] Figure 5 Here is a histogram corresponding to an example of input data;

[0034] Figure 6 A flowchart illustrating a training method for a prediction model provided in an embodiment of this disclosure;

[0035] Figure 7 A flowchart illustrating another method for training a prediction model provided in an embodiment of this disclosure;

[0036] Figure 8 A block diagram of a data processing apparatus provided in an embodiment of this disclosure;

[0037] Figure 9 This is a block diagram of a training device provided in an embodiment of the present disclosure;

[0038] Figure 10 This is a block diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0039] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0040] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0041] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0042] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0043] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0044] In related technologies, a trained neural network of any type can include multiple network layers. To improve inference speed, the input data of all or some of the network layers can be quantized. While quantizing the input data of all network layers can maximize the inference speed of the neural network model, it may significantly reduce the model's prediction accuracy. Therefore, quantization can be performed on the input data of only some of the neural network layers.

[0045] Because different neural networks have different network structures, even for the same neural network, the impact of whether or not to quantize different network layers on the inference accuracy of the quantized neural network is different. Therefore, when quantizing the input data of some network layers of a neural network, a lot of time and manpower are often required to manually debug and determine which network layers' input data should be quantized and which network layers' input data should not be quantized, in order to ensure that the neural network has both fast inference speed and high inference accuracy.

[0046] Therefore, in related technologies, there is a lack of efficient and accurate quantization schemes that can simultaneously enable a neural network to achieve both fast inference speed and high inference accuracy. This is especially true when the neural network has many layers, making it difficult to efficiently and accurately determine which layers require quantization and which do not.

[0047] Therefore, this disclosure provides a data processing method, a prediction model training method, a data processing apparatus, a training apparatus, an electronic device, and a computer-readable storage medium, which are intended to effectively solve at least one of the technical problems existing in the above-mentioned related technologies.

[0048] The data processing method and training method of this disclosure can be executed by a data processing device and a training device, respectively. Both the training device and the data processing device can be integrated into electronic devices such as terminal devices or servers via software and / or hardware. For example, the terminal device can be an in-vehicle device, user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some embodiments, the data processing method of this disclosure can be implemented by a processor calling computer-readable program instructions stored in memory, or the data processing method or training method of this disclosure can be executed by a server.

[0049] Figure 1 This is a flowchart illustrating a data processing method provided in an embodiment of the present disclosure.

[0050] This disclosure provides a data processing method applicable to any type of target neural network, which includes multiple network layers. The data processing method is used to predict the network layers in the target neural network that require quantization and those that do not require quantization.

[0051] In this embodiment of the disclosure, the target neural network can be any type of pre-trained deep learning network. The target neural network can be used to perform any one of image processing tasks, speech processing tasks, text processing tasks, and video processing tasks. The original input data of the target neural network can be any one of image data, speech data, text data, and video data.

[0052] Reference Figure 1 The data processing method may include steps S11 to S12.

[0053] Step S11: Obtain the input features of the input data corresponding to the target network layer of the target neural network.

[0054] In this embodiment of the disclosure, the target network layer is any network layer in the target neural network that is currently being predicted as to whether quantization is required.

[0055] In this embodiment of the disclosure, when the target network layer is the first network layer in the target neural network, the input data corresponding to the target network layer can be the original input data of the target neural network. When the target network layer is another network layer in the target neural network, the input data corresponding to the target network layer can be the output data of the previous network layer connected to the target network layer.

[0056] In this embodiment of the disclosure, the input features of the input data are information that characterizes the data distribution features of the input data. The data distribution feature information of the input data may be, for example, the KL distance of the input data, the dynamic range of the input data, the peak distribution information of the input data, etc.

[0057] Step S12: Input the input features corresponding to the target network layer into the prediction model to predict the category attribute corresponding to the target network layer. The category attribute represents whether the target network layer is suitable for quantizing the input data.

[0058] In this embodiment of the disclosure, the prediction model is a pre-trained network model used to predict whether the network layer needs to quantize the corresponding input data based on the input features of the network layer. In step S12, the input features of the target network layer to be predicted are input into the prediction model. The prediction model predicts the category attribute of the target network layer based on the input features of the target network layer. The category attribute of the target network layer characterizes whether the target network layer is suitable for quantizing the input data.

[0059] In this embodiment of the disclosure, there can be multiple target network layers to be predicted in the target neural network. In step S11, the input features corresponding to each target network layer can be obtained. In step S12, the input features corresponding to each target network layer to be predicted are sequentially input into the prediction model to predict the category attribute corresponding to each target network layer in sequence.

[0060] In this embodiment, the data processing method can be applied to any type of target neural network. For any target network layer of the target neural network, the prediction model is used to predict the category attribute of the target network layer based on the input features of the target network layer. The category attribute can characterize whether the target network layer is suitable for quantization of the input data. Therefore, it is possible to efficiently and accurately determine which network layers in the target neural network are suitable for quantization and which are not suitable for quantization based on the predicted category attribute of each network layer in the target neural network. This allows for efficient and accurate determination of the network layer quantization scheme of the target neural network, and quantization of the network layers of the target neural network is performed based on the quantization scheme. This is beneficial for the quantized target neural network to have both fast inference speed and high inference accuracy when performing inference operations.

[0061] In practice, the Kullback-Leibler distance (KL distance) of the input data characterizes the probability distribution error between the original input data and the quantized input data. A sudden change in the KL distance indicates a large probability distribution error between the quantized and original input data, significantly impacting data accuracy and consequently affecting the inference results of the target neural network. Therefore, the corresponding target network layer is unsuitable for quantizing the input data. The larger the dynamic range of the input data, the greater the impact of quantizing it on the data distribution, leading to a greater impact on the inference results of the target neural network and resulting in larger errors. Therefore, the corresponding target network layer is less suitable for quantizing the input data. The peak distribution information of the input data reflects its peak distribution. The presence of multiple peaks, similar peak influence, and large peak spacing indicates a large error caused by quantizing the input data, significantly impacting data accuracy and consequently affecting the inference results of the target neural network. Therefore, the corresponding target network layer is unsuitable for quantizing the input data. The KL distance, dynamic range, and peak distribution information will be described in detail below and will not be repeated here.

[0062] It is evident that using at least one of the KL distance, dynamic range, and peak distribution information of the input data can be beneficial in analyzing whether the corresponding target network layer is suitable for quantizing the input data. Therefore, in some embodiments, the input characteristics of the input data corresponding to the target network layer may include at least one of the KL distance, dynamic range, and peak distribution information of the input data. The KL distance is also known as KL divergence or relative entropy.

[0063] Figure 2 for Figure 1 A flowchart illustrating a specific implementation of step S11 is shown. In some embodiments, the input features of the input data corresponding to the target network layer include the KL distance of the input data, as referred to... Figure 2 In step S11, obtaining the input features of the input data corresponding to the target network layer of the target neural network may further include steps S111a to S113a.

[0064] Step S111a: Quantize the input data corresponding to the target network layer to obtain the first data.

[0065] In this embodiment of the disclosure, in the neural network, "quantization" refers to converting data from high precision to low precision, such as converting the precision of data from float32 to int8, and the numerical range of the data is readjusted.

[0066] Step S112a: Perform inverse quantization on the first data to obtain the second data.

[0067] In this embodiment of the disclosure, in the neural network, "dequantization" refers to converting the precision of data from low precision to high precision, for example, converting the precision of data from int8 to float32.

[0068] For example, the input data corresponding to the target network layer is represented as [3,4,5], the first data obtained after quantization is represented as [4], and the second data obtained after dequantization is represented as [4,4,4].

[0069] Step S113a: Determine the KL distance of the input data based on the probability distribution information of the input data and the probability distribution information of the second data.

[0070] In some embodiments, the probability distribution information P(x) of the input data and the probability distribution information Q(x) of the second data are statistically analyzed. Based on the probability distribution information P(x) of the input data and the probability distribution information Q(x) of the second data, the KL distance of the input data is calculated using the KL distance calculation formula. Here, D(P||Q) represents the KL distance, and the KL distance calculation formula is:

[0071]

[0072] In some embodiments, each input feature has a corresponding feature attribute, and the feature attribute of KL distance can be used to characterize whether the KL distance of the input data has undergone a sudden change. After obtaining the KL distance of the input data, the KL distances of the input data are sorted in ascending order. When the absolute value of the difference between two adjacent KL distances exceeds a first preset value, it can be determined that the KL distance of the input data has undergone a sudden change. The first preset value can be set according to actual needs, and this embodiment does not impose any special limitations on it.

[0073] Figure 3 for Figure 1 A flowchart illustrating another specific implementation of step S11 is shown. In some embodiments, the input characteristics of the input data corresponding to the target network layer may include the dynamic range of the input data, as described above. Figure 3 In step S11, obtaining the input features of the input data corresponding to the target network layer of the target neural network may further include step S111b.

[0074] Step S111b: Determine the dynamic range of the input data based on the minimum and maximum values ​​in the input data.

[0075] The dynamic range of input data refers to the range of data size in the input data, which can be represented by the maximum and minimum values ​​in the data size. The dynamic range of input data is the range from the minimum value to the maximum value in the input data.

[0076] In some embodiments, each input feature has a corresponding feature attribute, and the dynamic range feature attribute can be used to characterize the size of the dynamic range of the input data. Specifically, the dynamic range of the input data can be determined based on the difference between the maximum and minimum values ​​of the input data. For example, the dynamic range of the input data can be the difference between the maximum and minimum values. When the dynamic range of the input data is greater than a second preset value, it indicates that the dynamic range of the input data is large; otherwise, it indicates that the dynamic range of the input data is small. The second preset value can be set according to actual needs, and this embodiment does not impose any special limitations on it.

[0077] Figure 4 for Figure 1 A flowchart illustrating another specific implementation of step S11 is provided. The input features of the input data corresponding to the target network layer may include the peak distribution information of the input data, as shown in the figure. Figure 4 In step S11, obtaining the input features of the input data corresponding to the target network layer of the target neural network may further include steps S111c to S112c.

[0078] Step S111c: Perform histogram statistics on the input data corresponding to the target network layer.

[0079] In the histogram, the x-axis represents the size of the data in the input data, and the y-axis represents the number of data corresponding to each data size range.

[0080] Specifically, the dynamic range of the input data is divided into multiple data size ranges, and the number of data points corresponding to each data size range is counted. Based on each data size range and the corresponding number of data points, a histogram is drawn.

[0081] Step S112c: Based on the histogram, statistically analyze the peak distribution information of the input data.

[0082] The peak distribution information can include the number of peaks, the peak influence of each peak, and the ratio of the distance between adjacent peaks to the distance threshold. The peak influence of each peak refers to the area corresponding to that peak in the histogram. The area corresponding to that peak in the histogram is the product of the size of the data size range corresponding to that peak and the number of data points. The size of the data size range is the difference between the largest and smallest data points within that range. The distance threshold is the difference between the maximum and minimum values ​​in the input data. The distance between adjacent peaks is the absolute value of the difference between the data sizes corresponding to adjacent peaks. For example, the distance between adjacent peaks can be the absolute value of the difference between the largest and smallest data points within the data size range corresponding to the previous peak. Alternatively, the distance between adjacent peaks can be the absolute value of the difference between the median data size within the data size range corresponding to the previous peak and the median data size within the data size range corresponding to the next peak. The median data size within the data size range is the average of the largest and smallest data sizes within that range.

[0083] Figure 5 Here is an example histogram corresponding to input data, such as... Figure 5 As shown, the dynamic range of the input data is represented as [1, 50]. The maximum value of the input data is 50, and the minimum value is 1. The dynamic range of the input data is divided into multiple data size ranges, for example, into 5 data size ranges: [1, 10], [11, 20], [21, 30], [31, 40], and [41, 50]. Based on the number of data points corresponding to each data size range, a histogram is plotted on the input data to obtain the following results: Figure 5 The histogram shown has two peaks, A and B. Peak A corresponds to a data range of [11, 20] and contains 30 data points. Therefore, the peak influence of peak A is the result of multiplying the data range of peak A by the number of data points, i.e., (20-11)*30 = 270. Peak B corresponds to a data range of [41, 50] and contains 40 data points. Therefore, the peak influence of peak B is the result of multiplying the data range of peak B by the number of data points, i.e., (50-41)*40 = 360. The distance between peaks A and B is 41-20 = 21, and the distance threshold is 50-1 = 49.

[0084] In some embodiments, the input feature may include the number of peaks. Each input feature has a corresponding feature attribute. The feature attribute of the number of peaks can be used to characterize whether the number of peaks in the input data is greater than or equal to 2. When the number of peaks in the input data is greater than or equal to 2, it indicates that there are multiple peaks in the input data.

[0085] In some embodiments, the input features may include peak influence. Each input feature has a corresponding feature attribute. The peak influence attribute can be used to characterize whether the absolute value of the difference between the peak influence of two adjacent peaks is less than a third preset value. For example, if there are peaks 1 and 2 in the histogram, the peak influence of peak 1 and peak 2 are calculated respectively. When the absolute value of the difference between the peak influence of peak 1 and peak 2 is less than the third preset value, it indicates that there are multiple peaks, and the peak influence of peak 1 and peak 2 are close. The third preset value can be set according to actual needs, and this embodiment does not impose any special limitations on it.

[0086] In some embodiments, the input feature may include the ratio of the spacing between adjacent peaks to a spacing threshold. Each input feature has a corresponding feature attribute. This feature attribute of the ratio can be used to characterize whether the input data has an input where the ratio of the spacing between adjacent peaks to the spacing threshold is greater than a fourth preset value. When the ratio of the spacing between adjacent peaks to the spacing threshold is greater than the fourth preset value, it indicates that there are multiple peaks and the interval between adjacent peaks is relatively large. The fourth preset value can be set according to actual needs, and this embodiment does not impose any special limitations on it.

[0087] In some embodiments, the prediction model can predict the category attribute of the target network layer based on the feature attributes of the input features corresponding to the target network layer.

[0088] In some embodiments, the prediction model is a decision tree model trained based on the decision tree algorithm, which is a classification algorithm that makes decisions based on a tree structure.

[0089] Figure 6 This is a flowchart illustrating a training method for a prediction model provided in an embodiment of this disclosure. The embodiment of this disclosure provides a training method for a prediction model, referring to... Figure 6 The training method may include steps S21 to S23.

[0090] Step S21: Obtain the input features of the input data corresponding to the multiple network layers of at least one neural network.

[0091] Each network layer has a corresponding category attribute, which indicates whether the corresponding network layer is suitable for quantizing the corresponding input data.

[0092] In step S21, the neural network can be any type of pre-trained deep learning network. Each neural network can have multiple network layers and can be used to perform any of the following tasks: image processing, speech processing, text processing, and video processing.

[0093] For details on the specific implementation of obtaining the input features of the input data in step S21, please refer to the above description of step S11, which will not be repeated here.

[0094] In some embodiments, the input features of the input data include at least one of the following: KL distance of the input data, dynamic range of the input data, and peak distribution information of the input data. The peak distribution information includes the number of peaks, the peak influence degree corresponding to each peak, and the ratio of the distance between adjacent peaks to a distance threshold. The peak influence degree refers to the area corresponding to a peak in the histogram, the distance threshold refers to the difference between the maximum and minimum values ​​in the input data, and the distance between adjacent peaks refers to the absolute value of the difference between the data sizes corresponding to adjacent peaks.

[0095] In some embodiments, each input feature has a corresponding feature attribute. For example, the feature attribute of KL distance indicates whether the KL distance of the input data undergoes a sudden change; the feature attribute of dynamic range indicates the size of the dynamic range of the input data; the feature attribute of peak distribution information indicates whether the number of peaks in the input data is greater than or equal to 2, or, the feature attribute of peak distribution information indicates whether the absolute value of the difference between the peak influence of two adjacent peaks in the input data is less than a third preset value, or, the feature attribute of peak distribution information indicates whether the ratio of the distance between adjacent peaks to the distance threshold in the input data is greater than a fourth preset value.

[0096] In some embodiments, when the KL distance of the input data changes abruptly, it indicates that the input data is not suitable for quantization; when the dynamic range of the input data is large, it indicates that the input data is not suitable for quantization; when the input data has multiple peaks, it indicates that the input data is not suitable for quantization; when the absolute value of the difference between the peak influence of two adjacent peaks in the input data is less than a third preset value, it indicates that the input data is not suitable for quantization; when the ratio of the distance between adjacent peaks in the input data to the distance threshold is greater than a fourth preset value, it indicates that the input data is not suitable for quantization.

[0097] Step S22: Use the input features of the input data of each network layer as training samples, and use the category attribute as the label of the corresponding training sample to obtain the training sample set.

[0098] For example, a training sample corresponds to three input features: KL distance, dynamic range, and number of peaks. The feature attributes of each input feature are represented by 0 or 1, and the label is also represented by 0 or 1. For example, a training sample is (1, 0, 1, 1), which represents a sudden change in KL distance, a small dynamic range, and multiple peaks. Quantization affects the network accuracy and the network prediction accuracy.

[0099] Step S23: Train the initial prediction model based on the training sample set to obtain the prediction model.

[0100] The prediction model is used to predict the category attribute of the network layer based on the input features of the input data corresponding to the input network layer.

[0101] During model training, training samples are input into the initial prediction model to obtain the corresponding predicted category attributes. Based on the error between the predicted category attributes output by the initial prediction model and the labels of the corresponding input training samples, the network parameters of the initial prediction model are adjusted to iteratively optimize the initial prediction model until the model loss converges, thereby training the prediction model.

[0102] Figure 7 A flowchart illustrating another method for training a prediction model provided in this disclosure, in some embodiments, refers to... Figure 7 Before obtaining the input features of the input data corresponding to the multiple network layers of at least one neural network, the training method further includes steps S201 to S203.

[0103] Step S201: Quantize the input data corresponding to each network layer.

[0104] In some embodiments, in step S201, the input data corresponding to each network layer can be quantized separately. In some embodiments, in step S201, the input data corresponding to multiple network layers can be quantized simultaneously.

[0105] Step S202: Obtain the quantization impact of the input data corresponding to each network layer. The quantization impact represents the degree of influence of quantizing the input data of the network layer on the network prediction accuracy.

[0106] In some embodiments, the quantization impact value can be divided into a first value and a second value based on whether the network prediction accuracy (network prediction precision or recognition rate) changes before and after quantization. When the network prediction accuracy decreases or increases after quantization compared to before quantization, the quantization impact value of the input data corresponding to the network layer is set to the first value, indicating that quantization of the input data of the network layer has an impact on the network prediction accuracy. When the network prediction accuracy remains unchanged after quantization compared to before quantization, the quantization impact value of the input data corresponding to the network layer is set to the second value, indicating that quantization of the input data of the network layer has no impact on the network prediction accuracy. For example, the first value can be set to 1, and the second value can be set to 0. Here, network prediction accuracy refers to the prediction accuracy of the corresponding neural network.

[0107] In some embodiments, the value of quantization influence can be further refined based on the change in network prediction accuracy (network prediction precision or recognition rate) before and after quantization. This disclosure does not impose any special limitations on this aspect.

[0108] Step S203: Determine the category attribute corresponding to the network layer based on the quantization influence of the input data corresponding to the network layer.

[0109] In some embodiments, network layers are of two types: those that require quantization and those that do not.

[0110] In step S203, for example, when the quantization influence of the input data corresponding to the network layer is a first value, it indicates that quantizing the network layer has an impact on the network prediction accuracy. Therefore, the category of the network layer is determined to be a category that does not need to be quantized. Accordingly, the category attribute of the network layer indicates that the corresponding network layer is not suitable for quantizing the corresponding input data. When the quantization influence of the input data corresponding to the network layer is a second value, it indicates that quantizing the network layer has no impact on the network prediction accuracy. Therefore, the category of the network layer is determined to be a category that needs to be quantized. Accordingly, the category attribute of the network layer indicates that the corresponding network layer is suitable for quantizing the corresponding input data.

[0111] In some embodiments, the value of the quantization influence degree is further refined based on the change in network prediction accuracy (network prediction precision or recognition rate) before and after quantization. In step S203, network layers with a quantization influence degree greater than a preset influence degree value can be determined as categories that do not need to be quantized, while network layers with a quantization influence degree less than a preset influence degree value can be determined as categories that need to be quantized.

[0112] In some embodiments, the initial prediction model is a decision tree model, and model training of the initial prediction model based on the training sample set includes: training the initial prediction model based on the decision tree algorithm and the training sample set.

[0113] According to the technical solution of the training method provided in this disclosure, training samples are composed of input features and category attributes of multiple network layers. The initial prediction model is trained to obtain a prediction model for predicting the category attributes of the target network layer based on the input features of the target network layer. The category attributes can characterize whether the network layer to be predicted is suitable for quantization of the input data. Therefore, it is possible to efficiently and accurately determine which network layers in the neural network are suitable for quantization and which are not suitable for quantization based on the category attributes corresponding to each network layer in the predicted neural network. This allows for efficient and accurate determination of the network layer quantization scheme of the neural network. Based on the quantization scheme, the network layers of the neural network are quantized, which is beneficial for the quantized neural network to have both fast inference speed and high inference accuracy when performing inference operations.

[0114] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0115] In addition, this disclosure also provides a data processing apparatus, a training apparatus, an electronic device, and a computer-readable storage medium. The data processing apparatus is used to implement the data processing method provided by this disclosure, the training apparatus is used to implement the training method provided by this disclosure, and the electronic device and the computer-readable storage medium can all be used to implement the data processing method or training method provided by this disclosure. The corresponding technical solutions and descriptions are described in the corresponding descriptions in the method section, and will not be repeated here.

[0116] Figure 8 This is a block diagram of a data processing apparatus provided in an embodiment of the present disclosure, with reference to... Figure 8 This disclosure provides a data processing device 30, which includes a first acquisition unit 301 and a prediction unit 302.

[0117] The first acquisition unit 301 is used to acquire the input features of the input data corresponding to the target network layer of the target neural network.

[0118] The prediction unit 302 is used to input the input features corresponding to the target network layer into the prediction model and predict the category attribute corresponding to the target network layer. The category attribute represents whether the target network layer is suitable for quantizing the input data.

[0119] The data processing apparatus provided in this disclosure is used to implement the data processing method provided in the above embodiments. For a detailed description, please refer to the relevant description in the data processing method of the above embodiments, which will not be repeated here.

[0120] Figure 9 This is a block diagram of a training device provided in an embodiment of the present disclosure, with reference to... Figure 9 This disclosure provides a training device 40 for training a prediction model. The training device 40 includes a second acquisition unit 401, a sample set unit 402, and a training unit 403.

[0121] The second acquisition unit 401 is used to acquire the input features of the input data corresponding to the multiple network layers of at least one neural network, wherein each network layer has a corresponding category attribute, and the category attribute indicates whether the corresponding network layer is suitable for quantization processing of the corresponding input data.

[0122] The sample set unit 402 is used to obtain a training sample set by using the input features of the input data of each network layer as training samples and the category attribute as the label of the corresponding training sample.

[0123] Training unit 403 is used to train the initial prediction model based on the training sample set to obtain the prediction model.

[0124] The prediction model is used to predict the category attribute of the network layer based on the input features of the input data corresponding to the input network layer.

[0125] In some embodiments, the training device further includes a quantization unit, a third acquisition unit, and a labeling unit. The quantization unit performs quantization processing on the input data corresponding to each network layer; the third acquisition unit acquires the quantization impact degree of the input data corresponding to each network layer, whereby the quantization impact degree characterizes the degree of influence of quantization processing on the network prediction accuracy; and the labeling unit determines the category attribute corresponding to the network layer based on the quantization impact degree of the input data corresponding to the network layer.

[0126] The training apparatus provided in this disclosure is used to implement the training method provided in the above embodiments. For a detailed description, please refer to the relevant description in the training method of the above embodiments, which will not be repeated here.

[0127] Figure 10 This is a block diagram of an electronic device provided in an embodiment of the present disclosure, with reference to... Figure 10 This disclosure provides an electronic device, which includes: at least one processor 51; at least one memory 52; and one or more I / O interfaces 53 connected between the processor 51 and the memory 52; wherein the memory 52 stores one or more computer programs that can be executed by the at least one processor 51, and the one or more computer programs are executed by the at least one processor 51 to enable the at least one processor 51 to perform the above-described data processing method or training method.

[0128] This disclosure also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the aforementioned data processing method or training method. The computer-readable storage medium may be volatile or non-volatile.

[0129] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described data processing method or training method.

[0130] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0131] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0132] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0133] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0134] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0135] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0136] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0137] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0138] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0139] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A method for processing input data of a neural network model, characterized in that, The method includes: The input features of the input data corresponding to the target network layer of the target neural network are obtained. The target network layer includes any network layer in the target neural network that is currently being predicted and whether quantization is required. If the target network layer is the first network layer in the target neural network, then the input data corresponding to the target network layer is the original input data of the target neural network. If the target network layer is another network layer in the target neural network, then the input data corresponding to the target network layer is the output data of the previous network layer connected to the target network layer. The original input data of the target neural network includes image data, speech data, text data, and / or video data. The input features corresponding to the target network layer are input into the prediction model to predict the category attribute corresponding to the target network layer. The category attribute represents whether the target network layer is suitable for quantizing the input data. Based on the category attribute corresponding to the target network layer, determine whether the target neural network quantizes the input data corresponding to the target network layer during inference operations.

2. The data processing method according to claim 1, characterized in that, The input features of the input data include at least one of the following: the KL distance of the input data, the dynamic range of the input data, and the peak distribution information of the input data.

3. The data processing method according to claim 2, characterized in that, The input features include the KL distance of the input data, and the process of obtaining the input features of the input data corresponding to the target network layer of the target neural network includes: The input data corresponding to the target network layer is quantized to obtain the first data; The first data is dequantized to obtain the second data; The KL distance of the input data is determined based on the probability distribution information of the input data and the probability distribution information of the second data.

4. The data processing method according to claim 2, characterized in that, The input features include the dynamic range of the input data, and the acquisition of input features corresponding to the input data of the target network layer of the target neural network includes: The dynamic range of the input data is determined based on the minimum and maximum values ​​in the input data.

5. The data processing method according to claim 2, characterized in that, The input features include the peak distribution information of the input data, and the step of obtaining the input features of the input data corresponding to the target network layer of the target neural network includes: Histogram statistics are performed on the input data corresponding to the target network layer; Based on the histogram, the peak distribution information of the input data is statistically analyzed.

6. The data processing method according to claim 5, characterized in that, The peak distribution information includes the number of peaks, the peak influence degree corresponding to each peak, and the ratio of the distance between adjacent peaks to the distance threshold. Wherein, the peak influence degree corresponding to the peak refers to the area size corresponding to the peak in the histogram, and the spacing threshold refers to the difference between the maximum and minimum values ​​in the input data.

7. The data processing method according to any one of claims 1-6, characterized in that, The prediction model is a decision tree model.

8. A method for training a prediction model, characterized in that, include: The input features of input data corresponding to multiple network layers of at least one neural network are obtained, wherein each network layer has a corresponding category attribute, and the category attribute indicates whether the corresponding network layer is suitable for quantization processing of the corresponding input data. The input features of the input data of each network layer are used as training samples, and the category attributes are used as the labels of the corresponding training samples to obtain the training sample set; The initial prediction model is trained based on the training sample set to obtain the prediction model; The prediction model is used to predict the category attribute corresponding to the network layer based on the input features of the input data corresponding to the input network layer; the prediction model is used to implement the processing method according to any one of claims 1-7.

9. The training method according to claim 8, characterized in that, Before obtaining the input features of the input data corresponding to the multiple network layers of at least one neural network, the training method further includes: The input data corresponding to each network layer is quantized. Obtain the quantized impact of the input data corresponding to each network layer. The quantized impact represents the degree of influence of quantizing the input data of the network layer on the network prediction accuracy. The category attribute corresponding to the network layer is determined based on the quantized influence of the input data corresponding to the network layer.

10. A device for processing input data of a neural network model, characterized in that, include: The first acquisition unit is used to acquire the input features of the input data corresponding to the target network layer of the target neural network; the target network layer includes any network layer in the target neural network that is currently being predicted and whether quantization is required. If the target network layer is the first network layer in the target neural network, then the input data corresponding to the target network layer is the original input data of the target neural network; if the target network layer is another network layer in the target neural network, then the input data corresponding to the target network layer is the output data of the previous network layer connected to the target network layer; wherein, the original input data of the target neural network includes image data, speech data, text data, and / or video data; The prediction unit is used to input the input features corresponding to the target network layer into the prediction model and predict the category attribute corresponding to the target network layer. The category attribute characterizes whether the target network layer is suitable for quantizing the input data. Based on the category attribute corresponding to the target network layer, it is determined whether the target neural network should quantize the input data corresponding to the target network layer during inference operations.

11. A training device for a prediction model, characterized in that, include: The second acquisition unit is used to acquire the input features of the input data corresponding to the multiple network layers of at least one neural network, wherein each network layer has a corresponding category attribute, and the category attribute indicates whether the corresponding network layer is suitable for quantizing the corresponding input data. The sample set unit is used to obtain the training sample set by taking the input features of the input data of each network layer as training samples and the category attribute as the label of the corresponding training sample. A training unit is used to train the initial prediction model based on the training sample set to obtain the prediction model. The prediction model is used to predict the category attribute corresponding to the network layer based on the input features of the input data corresponding to the input network layer; the prediction model is used to implement the processing method according to any one of claims 1-7.

12. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the data processing method as described in any one of claims 1-7, or the training method as described in any one of claims 8-9.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the data processing method as described in any one of claims 1-7, or the training method as described in any one of claims 8-9.

Images