Data processing methods, electronic devices and media thereof

Abstract

This application relates to the field of artificial intelligence, specifically a data processing method, electronic device, and medium thereof. The method includes: acquiring a first input dataset for quantizing the model parameters of a first neural network model; selecting data from the first input dataset that meet a first condition to form a second input dataset; adjusting the data in the second input dataset based on an adjustment coefficient to obtain a third input dataset; and using the third input dataset to quantize the model parameters of the first neural network model to obtain a second neural network model. The determined third input dataset has a smaller data volume than the first input dataset, but the types of data in the third input dataset can cover as many types as possible from those in the first input dataset, thus reducing the time consumed in quantizing the neural network model while maintaining the accuracy of the quantized neural network model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a data processing method and its electronic device and medium. Background Technology

[0002] In the field of artificial intelligence, trained neural network models need to be quantized before they can be deployed and run. For example... Figure 1 As shown, quantization here refers to determining the model parameters of each data processing layer of the neural network model 100. That is, adjusting the numerical range of model parameters such as weights and activation values ​​of the trained neural network model to save storage space of the neural network model. For example, converting the model parameters of the neural network model from high precision (original numerical range) to low precision (quantized numerical range).

[0003] Larger input data (validation datasets) are typically used to quantize trained neural network models to ensure their accuracy. However, larger validation datasets also increase the time required for quantization. For example, in large datasets, many sample data features are correlated, increasing the complexity of quantization. Therefore, a method is needed to reduce quantization time while maintaining quantization accuracy as much as possible. Summary of the Invention

[0004] In view of this, embodiments of this application provide a data processing method, a medium, and an electronic device.

[0005] The first aspect of this application provides a data processing method, including:

[0006] Obtain the first input dataset for quantizing the model parameters of the first neural network model;

[0007] The second input dataset is formed by filtering data that meet the first condition from the first input dataset. The first condition includes: the data in the second input dataset has a data correlation with multiple data in the first input dataset that is less than a first correlation threshold.

[0008] The data in the second input dataset are adjusted based on the adjustment coefficient to obtain the third input dataset, where the adjustment coefficient is the data in the first input dataset, and the correlation between the adjustment coefficient and the data in the second input dataset satisfies the second condition.

[0009] The model parameters of the first neural network model are quantized using the third input dataset to obtain the second neural network model.

[0010] In this application, the first neural network model can be a trained but not yet quantized neural network model. The first input dataset can be a validation dataset, that is, the input data used by the server to quantize the trained neural network model. The second input dataset can be a validation dataset, which can be validation data used to quantize the model parameters of the trained neural network model. The total amount of calibration data in the calibration dataset is less than the total amount of validation data in the validation dataset, and the correlation between each calibration data and all validation data is less than a first correlation threshold. A correlation less than the first correlation threshold indicates that the smaller the correlation between the selected calibration data and the validation data, the more types of things (e.g., objects in an image) the calibration dataset can represent, and the more general the calibration dataset is, achieving classification from validation data to calibration data. The third input dataset can be a calibration dataset with features superimposed on the moderating coefficients in the validation dataset. The moderating coefficients can be validation data whose correlation with the calibration data in the calibration dataset meets the second correlation threshold, indicating that the moderating coefficients and calibration data belong to the same category. The second neural network model can be a quantized neural network model.

[0011] It can be seen that although the amount of data in the calibration dataset is smaller than that in the validation dataset, the types of calibration data in the calibration dataset can cover as many types of validation data as possible. This reduces the time required to quantize the neural network model using the calibration dataset. At the same time, by overlaying the features corresponding to the relevant validation data in the validation dataset onto the calibration data in the calibration dataset, the calibration data can include more features of the same type of validation data. That is, it can include more features of validation data that were not included in the calibration dataset. This makes the features included in the calibration dataset closer to those in the validation dataset, thus maintaining the accuracy of the neural network model quantized using the calibration dataset.

[0012] In one possible implementation of the first aspect described above, the data in the second input dataset are not the same.

[0013] In this application, the data in the second input dataset is not duplicated.

[0014] In one possible implementation of the first aspect above, the second condition includes:

[0015] The maximum value of the data correlation between the adjustment coefficient and each data point in the second input dataset is greater than the second correlation threshold, wherein the adjustment coefficient is not the same as each data point in the second input dataset.

[0016] In this application, the adjustment coefficient can be a verification data point where the correlation between the calibration data in the verification dataset and the calibration data in the calibration dataset meets a second correlation threshold, indicating that the adjustment coefficient and the calibration data belong to the same category.

[0017] In one possible implementation of the first aspect described above, the data in the first input dataset and the second input dataset include at least one of the following types: image feature vectors; speech feature vectors; and text feature vectors.

[0018] In this application, the input data used to quantize the first neural network model can be image data, speech data, or text data.

[0019] In one possible implementation of the first aspect above, the adjustment coefficients form an adjustment coefficient vector, and the method further includes: obtaining the correlation value between the adjustment coefficients and each data in the second input dataset by calculating the Euclidean distance or cosine similarity between the adjustment coefficient vector and each data in the second input dataset.

[0020] In one possible implementation of the first aspect above, the adjustment coefficients form an adjustment coefficient vector; and based on the adjustment coefficient vector, a third input dataset is determined by at least one of the following feature vector operations: vector addition, vector multiplication, and vector transpose.

[0021] In this application, the correlation between the adjustment coefficient and the calibration data can be determined by the correlation calculation method between vectors.

[0022] In one possible implementation of the first aspect described above, the data in the third input dataset include some or all of the adjustment coefficient vectors.

[0023] In this application, the features corresponding to the adjustment coefficient and the features corresponding to the calibration data of the maximum correlation value can be partially or fully superimposed (the superposition ratio is determined).

[0024] In one possible implementation of the first aspect above, the first neural network model or the second neural network model is used to implement at least one of the following functions: image recognition, text recognition, and speech recognition.

[0025] In this application, the first neural network model and the second neural network model can be neural network models used for image recognition, speech recognition, or text recognition.

[0026] A second aspect of this application provides a readable medium containing instructions that, when executed by a processor of an electronic device, cause the electronic device to implement the data processing method provided in the first aspect.

[0027] A third aspect of this application provides an electronic device, comprising:

[0028] Memory is used to store instructions executed by one or more processors of an electronic device;

[0029] And a processor, which is one of the processors of an electronic device, is used to run instructions to enable the electronic device to implement the data processing method provided in the first aspect above. Attached Figure Description

[0030] Figure 1 The diagram shown is a schematic diagram of quantizing a neural network model according to an embodiment of this application;

[0031] Figure 2 The diagram shown is an application scenario diagram of a user using a quantized neural network model according to an embodiment of this application;

[0032] Figure 3 This is an architecture diagram of a server 3 for quantifying a trained neural network model, as shown in an embodiment of this application.

[0033] Figure 4 This is an architectural diagram of a terminal device 2 using a quantized neural network model, as shown in an embodiment of this application.

[0034] Figure 5a and Figure 5b The diagram shown is a flowchart illustrating a data processing method provided in an embodiment of this application.

[0035] Figure 6 The diagram shown is a schematic representation of a data processing method provided in an embodiment of this application.

[0036] Figure 7 The diagram shown is a flowchart illustrating a data processing method provided in an embodiment of this application. Detailed Implementation

[0037] The embodiments of this application include, but are not limited to, an electronic device and a method for processing data thereon, as well as a medium. To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0038] Figure 2 This is an application scenario diagram illustrating a user's use of an application on a terminal device, as shown in an embodiment of this application. Figure 2As shown, server 3 can quantize the trained neural network model and deploy the quantized neural network model on terminal device 2. For example, a camera application deployed on terminal device 2 allows user 1 to take a picture of a person (i.e., the input data is an image) using the camera application on terminal device 2, and then segment the person from the picture using an image application based on the neural network model.

[0039] To address the issue mentioned in the background art of reducing the time consumption of quantizing neural network models while maintaining quantization accuracy, this application provides a method for processing neural network models. The method may include: acquiring a validation dataset (first input dataset), and selecting calibration data from the validation dataset to form a calibration dataset (second input dataset). The validation dataset can be a collection of validation data (e.g., image feature data) used to quantize the model parameters of a trained neural network model. The total amount of calibration data in the calibration dataset is less than the total amount of validation data in the validation dataset, and the correlation between each calibration data point and all validation data is less than a first correlation threshold. Correlation here represents the association or relationship between data points. A correlation less than the first correlation threshold indicates a smaller correlation between the selected calibration data and the validation data, resulting in a wider range of representations (e.g., objects in an image) and greater generality of the calibration dataset, thus achieving classification from validation data to calibration data.

[0040] The validation dataset is iterated through, and a validation data point whose correlation with the selected calibration data meets the second correlation threshold is selected as the adjustment coefficient. In other words, the validation data with the highest correlation (most relevant) to the calibration data is selected, meaning it belongs to the same category as the calibration data. For example, the maximum correlation between the validation data and all calibration data is obtained. If the maximum value is greater than the second correlation threshold, this validation data is determined as the adjustment coefficient. The feature corresponding to the adjustment coefficient is partially or fully superimposed on the feature corresponding to the calibration data with the maximum correlation (the superposition ratio is determined) until every calibration data point in the calibration dataset has one or more features corresponding to the same category of adjustment coefficients from the validation dataset. This enriches the features of data of the same category in the calibration dataset, resulting in the processed calibration dataset (the third input dataset). The calibration dataset is then used as input data to quantize the neural network model and determine its parameters.

[0041] It can be seen that the total amount of data in the calibration dataset determined by the above neural network model processing method is less than the total amount of data in the validation dataset used for quantizing and training the neural network model. However, the types of calibration data in the calibration dataset can cover as many types of validation data as possible, so that the time consumed in quantizing the neural network model using the calibration dataset is less. At the same time, after superimposing the features corresponding to the relevant validation data in the validation dataset to the calibration data in the calibration dataset, the calibration data can include more features of the same type of validation data. That is, it can include more features of validation data that were not included in the calibration dataset, so that the features included in the calibration dataset are closer to those in the validation dataset, thus maintaining the accuracy of the neural network model quantized using the calibration dataset.

[0042] Figure 3 This diagram illustrates the architecture of a server 3 for quantifying a trained neural network model, as shown in an embodiment of this application. Server 3 may include a processor 300, internal memory 310, power management module 320, and communication module 330. Server 3 may include application servers, cloud servers, etc.

[0043] The processor 300 may include one or more processing units, such as a central processing unit (CPU). The processor 300 is used to train the neural network model stored in the internal memory 310, adjusting the neural network model before deploying it on a terminal device. For example, a validation dataset can be set up to quantize the trained neural network model, a calibration dataset can be determined from the validation dataset, and the neural network model can be quantized using the calibration dataset to obtain the target neural network model.

[0044] Internal memory 310 can be used to store computer executable program code, including instructions. Internal memory 310 may include a program storage area and a data storage area. The program storage area may store neural network models, as well as validation and calibration datasets.

[0045] The power management module 320 is used to supply power to the processor 300 and the internal memory 310.

[0046] The communication module 330 is used to communicate with at least one terminal device and send the adjusted neural network model to the terminal device.

[0047] It is understood that the neural network model quantized using the processing method provided in this application embodiment can be applied to any terminal device capable of running a neural network model, including but not limited to mobile phones, wearable devices (such as smartwatches), tablets, desktop computers, laptops, handheld computers, and notebook computers, etc., and this application embodiment does not limit the application. To facilitate understanding of the technical solutions of this application embodiment, the structure of the applicable electronic device for the data processing method provided in this application embodiment is described using terminal device 1 as an example.

[0048] further, Figure 4 According to some embodiments of this application, a schematic diagram of the structure of a terminal device 1 is shown. For example... Figure 4 As shown, terminal device 1 includes one or more processors 401, system memory 402, non-volatile memory (NVM) 403, communication interface 404, input / output (I / O) device 405, and system control logic 406 for coupling the processor 401, system memory 402, non-volatile memory 403, communication interface 404, and input / output (I / O) device 405. Wherein:

[0049] Processor 401 may include one or more processing units, such as a processing module or processing circuit that may include a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a micro-programmed control unit (MCU), an AI (Artificial Intelligence) processor, or a programmable logic device (FPGA), a neural network processing unit (NPU), etc., and may include one or more single-core or multi-core processors.

[0050] System memory 402 is volatile memory, such as random-access memory (RAM), double data rate synchronous dynamic random access memory (DDR SDRAM), etc.

[0051] The non-volatile memory 403 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, the non-volatile memory 403 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as a hard disk drive (HDD), a compact disc (CD), a digital versatile disc (DVD), a solid-state drive (SSD), etc. In some embodiments, the non-volatile memory 403 may also be a removable storage medium, such as a Secure Digital (SD) memory card, etc.

[0052] Specifically, system memory 402 and non-volatile memory 403 may each include a temporary copy and a permanent copy of instruction 407. Instruction 407 may include, when executed by at least one of processors 401, causing terminal device 1 to implement the data processing method provided in the embodiments of this application.

[0053] The communication interface 404 may include a transceiver for providing a wired or wireless communication interface for the terminal device 1, thereby enabling communication with any other suitable device through one or more networks. In some embodiments, the communication interface 404 may be integrated into other components of the terminal device 1, for example, the communication interface 404 may be integrated into the processor 401. In some embodiments, the terminal device 1 may communicate with other devices through the communication interface 404. For example, the terminal device 1 may obtain a neural network model and a first lookup table and a second lookup table corresponding to the neural network model from other electronic devices through the communication interface 404.

[0054] Input / output (I / O) device 405 may include input devices such as keyboard and mouse, and output devices such as monitor. Users can interact with terminal device 1 through input / output (I / O) device 405.

[0055] System control logic 406 may include any suitable interface controller to provide any suitable interface to other modules of terminal device 1. For example, in some embodiments, system control logic 406 may include one or more memory controllers to provide an interface to system memory 402 and non-volatile memory 403.

[0056] In some embodiments, at least one of the processors 401 may be packaged together with the logic of one or more controllers for system control logic 406 to form a system in package (SiP). In other embodiments, at least one of the processors 401 may also be integrated on the same chip with the logic of one or more controllers for system control logic 406 to form a system-on-chip (SoC).

[0057] Understandable. Figure 4 The structure of the terminal device 1 shown is merely an example. In other embodiments, the terminal device 1 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0058] The neural network model processing method provided in this application embodiment can be executed by the processor 300 in the server 2 described above. Figure 5 is a flowchart illustrating a neural network model processing method according to an application embodiment. The method includes:

[0059] S501: Obtain validation data for the neural network model.

[0060] For example, the validation data here can be the input data used by server 2 to quantize the trained neural network model, such as: the validation data can be like... Figure 1 The image feature data stored on server 2 shown is called the verification dataset.

[0061] In some embodiments, the validation data may come from the training data, or data that is different from the training data.

[0062] S502: Adjust the numerical range of the verification data.

[0063] For example, adjusting the numerical range of the validation data here can refer to the process of quantizing and mapping the numerical range x of the validation data to the numerical range y, where the domain of the numerical range x is larger than the domain of the numerical range y. The numerical range x of the validation data used when training the neural network model on server 2 can be of full 32-bit precision (FP32), for example: the domain of the numerical range x can be [±1.18*10^2 / 24]. -38 ±3.4*10 38After the neural network model is trained, the numerical range of the validation data can be reduced, for example, to the precision of FP16 or INT8, and then the model parameters of the neural network model can be determined. This results in the quantized neural network model occupying less storage space and running faster.

[0064] In some embodiments, X can be used to represent the verification data before quantization mapping, and Q can be used to represent the verification data after quantization mapping to b bits. For example, when b is a value of 8, X can be quantized and mapped to 8 bits of signed or unsigned data. As shown in formula (1) below, when b is a value of 8, the numerical range of the verification data after quantization mapping can be [-128, 127] or [0, 255].

[0065]

[0066] In some embodiments, the formula (2) can be used: Qant(q_X|b)=clip(X*scale-zerop,Q min Q max The numerical range of the validation data is quantized and mapped. Wherein, Q... min Q represents the minimum value of the validation data after quantization mapping. max This represents the maximum value of the verification data after quantization mapping, and clip represents quantizing and mapping X*scale-zerop to the numerical range of Q. Here, scale can be expressed by formula (3): It is determined that zerop can be obtained using formula (4): zerop = scale * X min -Q min Confirmed. `scale` can represent the precision of the quantization mapping, that is, the range of change in the numerical range of the verification data before and after quantization. `zerop` can represent the zero point of the quantization mapping, that is, the integer value of floating-point zero after quantization. Here, X... max X min It can represent the maximum and minimum values ​​of the validation data before quantization mapping.

[0067] In some embodiments, X may be determined without counting all validation data. max X min Alternatively, you can simply draw a suitable sample from the validation set (validation dataset), which consists of all the validation data, for statistical analysis. For example, It is the i-th calibration set sample. Using formula (5): Update X min X is obtained using the same method. max Here, α can be a filtering coefficient used to improve the statistical value X. minSimilarly, the accuracy of X can also be determined using formula (5). max .

[0068] Figure 6 This is a schematic diagram illustrating verification data for a quantized mapping neural network model, as shown in the embodiment of the application. Figure 6 As shown, for example, the numerical range of the verification data after quantization mapping can be a number between -128 and 127. The numerical range of the verification data before quantization mapping, from -max(|X|) (the minimum value of the verification data) to max(|X|) (the maximum value of the verification data), will be quantized and mapped to a value between -128 and 127.

[0069] S503: Determine calibration data based on verification data.

[0070] For example, the verification data here can be represented by V, where V = {D0, D1, D2, ... D}. n The calibration data can be represented by C, where C = {C0, C1, C2, ..., C}. m In this context, the first quantity n and the second quantity m can both be natural numbers, with m being less than n. After determining the calibration data, the features of the verification data are superimposed onto the calibration data to obtain updated calibration data. The process of determining the calibration data based on the verification data may include the following steps.

[0071] S503a: Select a second number of validation data from all validation data whose correlation is less than a preset first correlation threshold and determine them as calibration data.

[0072] For example, a method for determining calibration data may include: first determining a calibration set (calibration dataset) C with a quantity of 0, i.e., empty calibration data; and then randomly selecting a verification data D from the verification set V of verification data. n As calibration data C m Add it to calibration set C. Additionally, randomly select a validation data point D from validation set V. n Compare with each calibration data C in calibration set C respectively m Perform a correlation comparison in D n With all calibration data in calibration set C m When the correlation between them is less than a preset first correlation threshold, D will be... n The calibration data is added to the calibration set C until the number of calibration data in the calibration set C reaches a second number m. The preset first correlation threshold here can be a value of 0.065. The value is exemplary and is not limited in this embodiment.

[0073] It is understandable that this can be achieved through formula (6): Determine the verification data D n With calibration data Cm The correlation between them, where x represents the validation data D n y represents the calibration data C m δ represents the standard deviation, and E represents the variance.

[0074] In some embodiments, the verification data D here n With calibration data C m The correlation between them can also be determined by correlation calculation methods such as cosine similarity and Euclidean distance. This application does not impose specific limitations on the embodiments.

[0075] S503b: Select a test verification data from all verification data and compare its correlation with each calibration data in the established calibration data.

[0076] For example, a test validation data D is randomly selected from the validation set V of the validation data. j D j Each calibration data C in the calibration set C determined by step S503a is compared with the calibration data C in step S503a. m A correlation comparison is performed. It can be understood that the test verification data D can be determined using the above formula (6). j With calibration data C m The correlation between them. It can be understood that the range of j here can be greater than or equal to 0 and less than or equal to n.

[0077] S503c: Record the calibration data corresponding to the maximum correlation between the test verification data and the established calibration data.

[0078] For example, when testing and verifying data D j With each calibration data C in the already determined calibration data m After performing a correlation comparison, the test validation data D was obtained. j With each calibration data C m The framework determines the correlation, sorting all correlation values ​​in ascending or descending order to identify the highest correlation. Simultaneously, it records and verifies the test data D. j The calibration data C with the highest correlation exist. i This serves as the target calibration data. It can be understood that the range of i here can be greater than or equal to 0 and less than or equal to m.

[0079] S503d: Determine whether the maximum correlation is greater than the preset second correlation threshold and whether the number of times the target calibration data is superimposed on the test verification data has reached the preset superposition threshold.

[0080] For example, in the test verification data D j With target calibration data C iThe maximum correlation between them is greater than the preset second correlation threshold, and the target calibration data C i Superimposed test verification data D j When the number of occurrences is less than the preset stacking threshold, the test verification data D here is... j This can be referred to as the adjustment coefficient. Step S503e is executed to perform feature decomposition on the adjustment coefficient and the target calibration data; otherwise, step S503b is executed to continue selecting the next test verification data.

[0081] In some embodiments, the condition for determining whether to overlay test verification data may also be that the minimum correlation between the test verification data and the already determined calibration data is less than a second correlation threshold. In some embodiments, the preset second correlation threshold may be 0.25, and the preset overlay quantity threshold may be represented by L, for example, the value of L may be 10. This application does not impose limitations on the embodiments.

[0082] S503e: Perform feature decomposition on test verification data and target calibration data, and select features with a preset proportion.

[0083] For example, the feature decomposition here can be PCA (Principal Component Analysis) decomposition, used to transform high-dimensional data into low-dimensional data while retaining the main features of the high-dimensional data. This reduces computational burden, improves model efficiency, and avoids problems such as overfitting. Taking image data as an example, PCA decomposition integrates multiple image datasets into a single image dataset, allowing one image dataset to contain information from multiple images. This achieves the effect of using a validation set with a smaller calibration set.

[0084] In some embodiments, the preset ratio feature may include the test verification data D after feature decomposition. j (Adjustment coefficient) and target calibration data C i The first 20% or 40% of the principal components, where the principal components can be feature data of the same dimension as the test verification data or target calibration data, are used to indicate which feature data in the test verification data or target calibration data has the greatest impact on the test verification data or target calibration data. This 20% or 40% is exemplary and not limited in the embodiments of this application.

[0085] The following example illustrates how to perform feature decomposition on test verification data or target calibration data to obtain principal components.

[0086] Assuming test verification data

[0087] First calculate the test verification data D jThe covariance matrix M is used to describe the test and validation data D. j The relationship between each pair of feature data.

[0088] covariance matrix Here, T represents matrix transpose.

[0089] Calculate the eigenvectors and eigenvalues ​​of the covariance matrix M. Here, the eigenvectors describe the main directions in the feature dataset, while the eigenvalues ​​describe the variance in each direction.

[0090] Obtain eigenvalues

[0091] Sort the eigenvectors by their eigenvalues ​​from largest to smallest, and select the eigenvector with the largest eigenvalue. This indicates that the feature data is more accurate. For example:

[0092] Test verification data D j Projecting the data onto the previously selected feature vectors to obtain the test and validation data D after feature decomposition. j ,

[0093] Select test verification data D of features with a preset proportion j That is, test and verification data D j The principal component, with a preset proportion of 40%, was tested and validated using data D. j The principal component can be

[0094] S503f: Use the selected preset ratio of features to recover test verification data and target calibration data, and overlay the test verification data onto the target calibration data.

[0095] For example, recovering test verification data and target calibration data using features of a selected preset ratio can be achieved by performing feature decomposition on the selected preset ratio of features in reverse to obtain test verification data and target calibration data.

[0096] For example: for test verification data D j It can be done Reverse execution feature decomposition based on Obtain the restored test verification data D j .

[0097] In some embodiments, it can be achieved through formula (7): C m =μ*C i +(1-μ)*D j The restored test verification data D j and target calibration data Ci Feature overlay, that is, combining the test and verification data D j Superimposed onto target calibration data C i Here, μ can be a filter coefficient used to improve the calibration data C after superimposing the test and verification data. m The accuracy.

[0098] In some embodiments, formulas (5) and (7) can be combined to obtain: f(α*C) i +(1-α)*D j )=α*f(C i )+(1-α)*f(D j In other words, considering formula (5) as a transfer function is... The first-order low-pass filtering, combined with formulas (5) and (7), transforms this process into The second-order low-pass filter makes the calibration data C m It is closer to the true value. Here, Y(s) and X(s) can represent the restored target calibration data C. i and test verification data D j .

[0099] S504: Quantize the neural network model using calibration data to obtain the target neural network model.

[0100] For example, server 3 can use calibration data superimposed with validation data to quantize the trained neural network model, configure the model parameters of the neural network model, obtain the target neural network model, and send the target neural network model to terminal device 2 for deployment. Since the target neural network model has fewer model parameters, the efficiency of the target neural network model can be improved. At the same time, the calibration data superimposed with the features of validation data makes the calibration data have more comprehensive feature data, so that the quantized target neural network model has higher accuracy.

[0101] In some embodiments, the process of quantizing the model parameters of a neural network model may include preprocessing, model parameter fine-tuning, and other steps. Preprocessing includes removing batch normalization (BN) layers, where BN layers are used to normalize the model parameters during training, that is, to determine the numerical range of the model parameters (e.g., between 0 and 1) so that the model parameter values ​​are not too large or too small. Since the model parameters of the quantized neural network model are not frequently modified, the BN layers in the neural network model can be removed during the quantization process. Model parameter quantization. Model parameter fine-tuning includes fine-tuning the model parameters of the quantized neural network model to improve accuracy. The process of model parameter fine-tuning is similar to the process of training a neural network model, but the total amount of data in the dataset for model parameter fine-tuning is smaller. For example, the calibration data determined in steps S501 to S504 can be used to fine-tune the model parameters of the neural network model.

[0102] In some embodiments, the process of quantizing the model parameters of a neural network model may further include converting the numerical range of the model parameters of the neural network model from high precision to low precision, for example, from 32-bit floating-point numbers to 8-bit floating-point numbers.

[0103] Figure 7 This is a flowchart illustrating another method for processing a neural network model as shown in the application embodiment. The method includes:

[0104] S701: Initialize calibration dataset.

[0105] For example, the calibration dataset C here can be a set of input data selected from the validation dataset V for quantizing the neural network model. For example, the total number of data in the validation dataset can be 10,000, and the total number of data in the calibration dataset can be 1,000. The above values ​​are exemplary and are not limited in this application embodiment. The initial number of calibration datasets is 0, that is, empty.

[0106] S702: Randomly select a validation data point from the validation dataset.

[0107] For example, one validation data point is randomly selected from the validation dataset.

[0108] S703: Perform a correlation comparison between the validation data and each calibration data in the calibration dataset.

[0109] For example, when the number of calibration datasets is 0, a validation data point can be randomly selected from the validation dataset and added to the calibration dataset as calibration data. When the number of calibration datasets is not 0, a validation data point is randomly selected from the validation dataset and its correlation is compared with each calibration data point in the calibration dataset.

[0110] S704: Determine whether the correlation between the verification data and all calibration data is less than the first correlation threshold.

[0111] For example, if the correlation between the verification data and all calibration data in the calibration dataset is less than a preset first correlation threshold, step S705 is executed to add the verification data to the calibration dataset until the number of calibration data in the calibration dataset reaches the threshold. Otherwise, the process returns to step S702 to select the next set of verification data.

[0112] S705: Update the verification data to the calibration dataset.

[0113] For example, verification data with a correlation less than a preset first correlation threshold are added to the calibration dataset as calibration data.

[0114] S706: Determine whether the total amount of data in the calibration dataset has reached the threshold.

[0115] For example, when the total amount of data in the calibration dataset reaches the threshold, step S707 is executed; otherwise, step S702 is returned to continue selecting the next verification data.

[0116] S707: Randomly select a validation data point from the validation dataset.

[0117] For example, one validation data point is randomly selected from the validation dataset.

[0118] S708: Compare the verification data with each calibration data in the calibration dataset for correlation.

[0119] For example, a validation data point is randomly selected from the validation dataset and its correlation is compared with each calibration data point in the calibration dataset.

[0120] S709: Record the maximum correlation between the verification data and the calibration data, as well as the calibration data corresponding to the maximum correlation.

[0121] For example, the maximum correlation between the verification data and the calibration data is recorded, as well as the location of the calibration data corresponding to the maximum correlation in the calibration dataset.

[0122] S710: Determine whether the maximum correlation is greater than the preset second correlation threshold and whether the number of times the calibration data is superimposed on the verification data reaches the preset superposition threshold.

[0123] For example, if the maximum correlation between the verification data and the calibration data is greater than a preset second correlation threshold, and the number of times the calibration data corresponding to the maximum correlation has been superimposed with the verification data has not reached a preset superposition threshold, step S711 is executed to perform feature decomposition on the verification data and calibration data, select features of a preset proportion, use the features of the preset proportion to recover the verification data and calibration data, and superimpose the verification data onto the calibration data. Otherwise, return to step S707 and continue to select the next verification data.

[0124] S711: Perform feature decomposition on the verification data and calibration data, and select features with a preset proportion.

[0125] S712: Restore verification data and calibration data, and overlay the verification data onto the calibration data.

[0126] For example, the feature decomposition here can be principal component analysis decomposition, which is used to transform high-dimensional data into low-dimensional data while retaining the main features of the high-dimensional data. Recovering the verification data and calibration data using the selected preset proportion of features can be achieved by performing feature decomposition in reverse on the selected preset proportion of features to obtain the verification data and calibration data.

[0127] S713: Determine whether the number of times all calibration data in the calibration dataset is superimposed with verification data has reached the preset superposition threshold.

[0128] For example, the execution ends when the number of times the calibration data corresponding to the maximum correlation has been superimposed with the verification data reaches a preset superposition threshold.

[0129] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0130] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0131] It should be noted that, in the examples and description of this patent, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0132] Although this application has been illustrated and described with reference to certain preferred embodiments thereof, those skilled in the art should understand that various changes in form and detail may be made thereto without departing from the spirit and scope of this application.

Claims

1. A data processing method, characterized in that, include: Obtain the first input dataset for quantizing the model parameters of the first neural network model; The data that meets the first condition are selected from the first input dataset to form the second input dataset. The first condition includes: the data in the second input dataset has a data correlation with multiple data in the first input dataset that is less than a first correlation threshold. The data in the first input dataset and the second input dataset include at least one of the following types: image feature vector, speech feature vector, and text feature vector. A third input dataset is obtained by adjusting the data in the second input dataset based on an adjustment coefficient, wherein the adjustment coefficient is derived from the data in the first input dataset, and the correlation between the adjustment coefficient and the data in the second input dataset satisfies a second condition. The adjustment coefficients form an adjustment coefficient vector. The correlation values ​​between the adjustment coefficients and each data point in the second input dataset are obtained by calculating the Euclidean distance or cosine similarity between the adjustment coefficient vector and each data point in the second input dataset. The third input dataset is determined based on the adjustment coefficient vector using at least one of the following feature vector operations: vector addition, vector multiplication, and vector transpose. The second condition includes: the maximum value of the data correlation between the adjustment coefficient and each data in the second input dataset is greater than a second correlation threshold, wherein the adjustment coefficient is not the same as each data in the second input dataset; The model parameters of the first neural network model are quantized using the third input dataset to obtain the second neural network model.

2. The data processing method according to claim 1, characterized in that, The data in the second input dataset are different.

3. The data processing method according to claim 1, characterized in that, Each data point in the third input dataset includes some or all of the adjustment coefficient vectors.

4. The method according to claim 1, characterized in that, The first neural network model or the second neural network model is used to achieve at least one of the following functions: image recognition, text recognition, and speech recognition.

5. A readable medium, characterized in that, The readable medium contains instructions that, when executed by the processor of the electronic device, cause the electronic device to implement the data processing method of any one of claims 1 to 4.

6. An electronic device, characterized in that, include: Memory is used to store instructions executed by one or more processors of an electronic device; And a processor, one of the processors of the electronic device, for running the instructions to enable the electronic device to implement the data processing method of any one of claims 1 to 4.

Images