A quantization method, data processing method, and related apparatus for a neural network model.

Abstract

This specification provides a method for quantizing a neural network model, a data processing method, and related apparatus. By introducing a regularization term into the original loss function, minimizing the loss function as the objective function, and optimizing the expectation as the constraint, a portion of the multiple first weights are sparsified into second weights during multiple iterations. This achieves the goal of sparsifying some of the first weights into second weights while ensuring that the sum of the loss function and the regularization term of the target neural network model is minimized. As a result, the target neural network model can balance operating efficiency and computational accuracy, that is, while expanding the application scenarios of the target neural network model, it ensures that the target neural network model has good accuracy.

Description

Technical Field

[0001] The embodiments described in this specification relate to the field of computer application technology, specifically to machine learning technology in the field of computer application technology, and more specifically, to a method for quantizing a neural network model, a data processing method, and related apparatus. Background Technology

[0002] With the continuous development of machine learning technology, neural network models are becoming increasingly smaller, enabling them to be applied to low-performance computing devices or embedded devices, thus expanding the applicable scenarios of neural network models.

[0003] Quantization of neural network models is a means to compress model size, improve running efficiency, and reduce power consumption, but current quantization methods struggle to balance the relationship between neural network model performance and running efficiency. Summary of the Invention

[0004] This specification provides a method for quantizing a neural network model, a data processing method, and related apparatus through various embodiments, in order to achieve a balance between the performance and operating efficiency of the neural network model.

[0005] Firstly, one embodiment of this specification provides a method for quantizing a neural network model, comprising:

[0006] Obtain a neural network model trained on target data, wherein the target data includes image data and / or speech data;

[0007] The weights of the neural network model are quantized and trained to obtain multiple first weights; the first weights are represented in integer format.

[0008] With minimizing the loss function as the objective function and optimizing the expectation as the constraint, during multiple iterations, some of the multiple first weights are sparsed into second weights to obtain a target neural network model. The accuracy of the first weights is greater than the accuracy of the second weights. The target neural network model is used to obtain processing results based on the input speech and / or image.

[0009] The loss function includes a regularization term, which is used to characterize the performance of the neural network model. The regularization term includes a penalty rule when the first weight is sparsed into the second weight. The optimization expectation includes the ratio of the number of the second weight to the total number of weights in the neural network model.

[0010] Secondly, one embodiment of this specification provides a data processing method, including:

[0011] Acquire data to be processed, the data to be processed including: voice and / or images;

[0012] In response to the processing instruction, the data to be processed is input into the target neural network model to obtain the processing result;

[0013] The target neural network model is obtained by quantizing a neural network model trained on target data, wherein the target data includes: image data and / or speech data;

[0014] The quantization process includes:

[0015] The weights of the neural network model are quantized and trained to obtain multiple first weights; the first weights are represented in integer format.

[0016] With minimizing the loss function as the objective function and optimizing the expectation as the constraint, during multiple iterations, some of the multiple first weights are sparsed into second weights to obtain a target neural network model. The accuracy of the first weights is greater than the accuracy of the second weights. The target neural network model is used to obtain processing results based on the input speech and / or image.

[0017] The loss function includes a regularization term, which is used to characterize the performance of the neural network model. The regularization term includes a penalty rule when the first weight is sparsed into the second weight. The optimization expectation includes the ratio of the number of the second weight to the total number of weights in the neural network model.

[0018] Thirdly, one embodiment of this specification provides a quantization apparatus for a neural network model, comprising:

[0019] The model acquisition module is used to acquire a neural network model trained on target data, wherein the target data includes image data and / or voice data;

[0020] The quantization training module is used to quantize and train the weights of the neural network model to obtain multiple first weights; the first weights are represented in integer format.

[0021] The weighting coefficient module is used to minimize the loss function as the objective function and optimize the expectation as the constraint. During multiple iterations, a portion of the multiple first weights are sparsed into second weights to obtain a target neural network model. The accuracy of the first weights is greater than that of the second weights. The target neural network model is used to obtain processing results based on the input speech and / or image.

[0022] The loss function includes a regularization term, which is used to characterize the performance of the neural network model. The regularization term includes a penalty rule when the first weight is sparsed into the second weight. The optimization expectation includes the ratio of the number of the second weight to the total number of weights in the neural network model.

[0023] Fourthly, one embodiment of this specification provides a data processing apparatus, comprising:

[0024] The data acquisition module is used to acquire data to be processed, which includes: voice and / or images;

[0025] The data processing module is used to input the data to be processed into the target neural network model in response to the processing instruction to obtain the processing result;

[0026] The target neural network model is obtained by quantizing a neural network model trained on target data, wherein the target data includes: image data and / or speech data;

[0027] The quantization process includes:

[0028] The weights of the neural network model are quantized and trained to obtain multiple first weights; the first weights are represented in integer format.

[0029] With minimizing the loss function as the objective function and optimizing the expectation as the constraint, during multiple iterations, some of the multiple first weights are sparsed into second weights to obtain a target neural network model. The accuracy of the first weights is greater than the accuracy of the second weights. The target neural network model is used to obtain processing results based on the input speech and / or image.

[0030] The loss function includes a regularization term, which is used to characterize the performance of the neural network model. The regularization term includes a penalty rule when the first weight is sparsed into the second weight. The optimization expectation includes the ratio of the number of the second weight to the total number of weights in the neural network model.

[0031] Fifthly, one embodiment of this specification provides a computing device, including: a processor and a memory;

[0032] The memory is connected to the processor and is used to store computer programs;

[0033] The processor is configured to implement the quantization method or data processing method of the neural network model described above by running a computer program stored in the memory.

[0034] Sixthly, one embodiment of this specification provides a storage medium storing a computer program that, when executed by a processor, implements the quantization method or data processing method for the neural network model described above.

[0035] In a seventh aspect, one embodiment of this specification provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium; a processor of the computer device reads the computer instructions from the computer-readable storage medium, and when the processor executes the computer instructions, it implements the steps of the above-described quantization method or data processing method for a neural network model.

[0036] The various implementation methods provided in this specification introduce a regularization term into the original loss function of the neural network model trained on the target data. Using minimizing the loss function as the objective function and optimizing the expectation as the constraint, a portion of the multiple first weights is sparsified to second weights during multiple iterations. This achieves the goal of sparsifying some of the first weights to second weights while minimizing the sum of the loss function and the regularization term of the target neural network model. This allows the target neural network model to balance operational efficiency and computational accuracy when processing speech and / or images, thus ensuring good accuracy while expanding the application scenarios of the target neural network model. Furthermore, by using minimizing the loss function as the objective function and optimizing the expectation as the constraint, and sparsifying a portion of the multiple first weights to second weights during multiple iterations, the mixed-precision quantization strategy problem is transformed into a discrete-space optimization problem, reducing the difficulty of mixed-precision quantization and improving the execution efficiency of the method. Attached Figure Description

[0037] Figure 1 This specification provides a feasible application scenario for a quantization method for a neural network model, as one embodiment of the present invention.

[0038] Figure 2 A flowchart illustrating a method for quantizing a neural network model, provided as one embodiment of this specification;

[0039] Figure 3 A schematic diagram illustrating a process of sparsifying a first weight, provided as one embodiment of this specification;

[0040] Figure 4 A schematic diagram illustrating another process for sparsifying the first weight, provided as one embodiment of this specification;

[0041] Figure 5A flowchart illustrating another method for quantizing a neural network model, provided as one embodiment of this specification;

[0042] Figure 6 A flowchart illustrating a data processing method provided for one embodiment of this specification;

[0043] Figure 7 A schematic diagram of the structure of a quantization device for a neural network model provided in one embodiment of this specification;

[0044] Figure 8 A schematic diagram of the structure of a data processing apparatus provided for one embodiment of this specification;

[0045] Figure 9 This is a schematic diagram of the structure of a computing device provided for one embodiment of this specification. Detailed Implementation

[0046] Unless otherwise defined, the technical or scientific terms used in the embodiments of this specification shall have the ordinary meaning understood by one of ordinary skill in the art to which this specification pertains. The terms "first," "second," and similar terms used in the embodiments of this specification do not indicate any order, quantity, or importance, but are merely used to avoid confusion of constituent elements.

[0047] Unless the context otherwise requires, throughout this specification, "a plurality of" means "at least two," and "including" is interpreted as open-ended or encompassing, that is, "including, but not limited to." In the description of this specification, terms such as "one embodiment," "some embodiments," "exemplary embodiment," "example," "specific example," or "some examples" are intended to indicate that a particular feature, structure, material, or characteristic associated with that embodiment or example is included in at least one embodiment or example of this specification. The illustrative representations of the above terms do not necessarily refer to the same embodiment or example.

[0048] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0049] Overview

[0050] Quantization techniques for neural network models can compress the model, improve operating efficiency, and reduce power consumption, making the quantized neural network model suitable for low-performance devices (such as embedded devices). This can reduce the power consumption of neural network models and expand their application scenarios.

[0051] The principle of quantization in neural network models can be summarized as converting floating-point model parameters and computational variables into integer or low-bit-width formats to compress the model, improve operational efficiency, and reduce power consumption. By compressing floating-point parameters into integers or smaller floating-point numbers, it can significantly reduce the model size, thereby achieving better performance in model storage and transmission. Simultaneously, quantization to integers and low-bit-width computation can be executed faster on hardware accelerators, thus improving the model's running speed.

[0052] Current quantization methods for neural network models typically quantize the entire model into an 8-bit integer format. With this method, the quantized neural network model generally achieves accuracy close to that of the original floating-point representation. However, this quantization method does not significantly improve the compression effect or operational efficiency of the neural network model, meaning that some low-performance devices still cannot process the quantized neural network model.

[0053] To address this issue, the inventors discovered that the model could be quantized into a lower-precision neural network model, such as a model represented in 4-bit or 6-bit integer format. This significantly accelerates the model's inference process, achieving good results in model compression, improved operating efficiency, and reduced power consumption. However, converting the entire neural network model to a lower-precision model results in a significant loss of precision. In this context, the inventors, through research, found that mixed-precision quantization is a feasible solution. Mixed-precision quantization refers to representing some parameters in 4-bit or 6-bit integer format and others in 8-bit integer format during the quantization process of the neural network model. For example, some time-consuming operators can be run in a lower-precision manner, significantly reducing computational cost and energy consumption while maintaining relatively high computational accuracy. However, in mixed-precision quantization methods, choosing which parameters to represent with lower precision (e.g., 4-bit or 6-bit integer format) remains a challenge.

[0054] Further research by the inventors revealed that by introducing a regularization term into the original loss function of the neural network model trained on the target data, minimizing the loss function as the objective function, and optimizing the expectation as the constraint, a portion of the multiple first weights is sparsified to second weights during multiple iterations. This achieves the goal of converting some first weights to second weights while minimizing the sum of the loss function and the regularization term of the target neural network model. This allows the target neural network model to balance operational efficiency and computational accuracy when processing speech and / or images, thus expanding the application scenarios of the target neural network model while ensuring good accuracy. Furthermore, by using minimizing the loss function as the objective function and optimizing the expectation as the constraint, and sparsely converting some of the multiple first weights to second weights during multiple iterations, the mixed-precision quantization strategy problem is transformed into a discrete-space optimization problem, reducing the difficulty of mixed-precision quantization and improving the execution efficiency of the method.

[0055] Based on the above-described inventive concept, this specification provides a method for quantizing a neural network model. The method for quantizing the neural network model will be described exemplarily below with reference to the accompanying drawings.

[0056] Scenario Example

[0057] refer to Figure 1 , Figure 1 This paper illustrates a feasible application scenario for quantization methods of neural network models. In this scenario, the neural network model is pre-trained and possesses certain data processing capabilities, such as face recognition based on input images or speech recognition based on input sounds. At this point, the weights of the neural network model are represented by 32-bit floating-point numbers, resulting in relatively high data processing accuracy (i.e., a high probability of outputting accurate results based on input data). However, this neural network model is large in size, has low operating efficiency, and high power consumption, making it unsuitable for low-performance computing devices. Therefore, it is necessary to quantize the neural network model, representing some weights in 8-bit integer format and others in lower-bit integer format (e.g., 4 or 6 bits). This allows for compression, improved operating efficiency, and reduced power consumption while maintaining relatively high computational accuracy.

[0058] The quantized neural network model can be mounted on computing devices (such as mobile smart terminals) to perform corresponding tasks (such as image feature extraction, speech recognition, etc.). For example, in Figure 1In this context, a neural network model mounted on a smartphone can process the input speech and output the speech recognition result. In some cases, the computing device can be not only a smartphone, but also a desktop computer, tablet computer, laptop computer, digital assistant, smart wearable device, shopping guide terminal, television, smart speaker, microphone, etc. Smart wearable devices include, but are not limited to, smart bracelets, smartwatches, smart glasses, smart helmets, and smart necklaces.

[0059] Example method

[0060] One embodiment of this specification provides a method for quantizing a neural network model, such as... Figure 2 As shown, it includes:

[0061] S201: Obtain a neural network model trained on target data, wherein the target data includes image data and / or voice data.

[0062] The target data can be labeled training samples. In some cases, depending on the purpose of the training samples during training, they can be divided into a training set and a test set. A neural network model can gradually improve its prediction accuracy and generalization ability by repeatedly learning and adjusting the data in the training set. The test set is a dataset used to test the performance and generalization ability of the neural network model. By testing the neural network model using the test set and evaluating its accuracy, it can be verified whether the predictive ability of the neural network model has reached the expected level. A neural network model trained on the target data can possess corresponding functions, such as speech recognition, image segmentation, etc.

[0063] S202: Quantize the weights of the neural network model to obtain multiple first weights; the first weights are represented in integer format.

[0064] Quantization-aware training (QAT) is a training method that optimizes the parameters of a neural network by quantization. Traditional neural network training typically uses floating-point numbers to represent parameters, but this consumes significant computational resources and memory, and is also detrimental to the deployment of neural networks on mobile devices. Quantization training can compress the size of the neural network model, improving its computational efficiency and thus meeting the needs of deploying neural network models on various computing devices.

[0065] In step S202, the weights in the neural network model can be quantized into first weights through quantization training. These first weights can be, for example, weights represented in 8-bit integer format. Through the quantization training in step S202, the weights in the neural network model are initially quantized into first weights with higher precision.

[0066] S203: Using minimizing the loss function as the objective function and optimizing the expectation as the constraint, during multiple iterations, some of the multiple first weights are sparsed into second weights to obtain a target neural network model, wherein the accuracy of the first weights is greater than the accuracy of the second weights; the target neural network model is used to obtain processing results based on the input speech and / or image;

[0067] The loss function includes a regularization term, which is used to characterize the performance of the neural network model. The regularization term includes a penalty rule when the first weight is sparsed into the second weight. The optimization expectation includes the ratio of the number of the second weight to the total number of weights in the neural network model.

[0068] For example, suppose the optimization expectation is that the number of the second weights accounts for 50% of the total number of weights in the neural network model. Then, when the number of the second weights in the neural network model / (the number of the first weights + the number of the second weights) = 50%, the optimization expectation is considered to be satisfied.

[0069] Sparsity is a method to lightweight neural network models. Regularization is a technique used to prevent overfitting during the iteration process of neural network models. It avoids excessive complexity in the neural network model parameters by introducing additional regularization terms during model training. A regularization term is a penalty rule for the weights in a neural network model. For example, in this embodiment, the regularization term may specifically include a penalty rule when the first weight is sparsified to a second weight. Assume the penalty rule includes: if the effective data carried by the 4 bits of the sparsified first weight is not the minimum effective data amount, while ensuring that the second weight is a consecutive 4 bits of the first weight, then a penalty is imposed. (Reference) Figure 3 and Figure 4 , Figure 3 and Figure 4 This illustrates two ways to assign the same first weight coefficient as the second weight. Figure 3 In the first sparse weight, the two bits before and after it result in a second weight of "1101". Two of the sparsified bits are non-zero. Therefore... Figure 3 In the sparse method, the 4 bits of the first weight that are sparsified carry 2 bits of effective data. And... Figure 4In the first four bits of the sparse first weight, the second weight is "0101". Three of the sparsified bits are not zero. Figure 4 In the sparse method, the 4 bits of the first weight that are sparsified carry 3 bits of effective data, which satisfies the above penalty rule and can be used to target... Figure 4 The sparse manner imposes penalties.

[0070] In some implementations, the process of making the first weight sparse into the second weight includes: discarding N bits of data in the first weight and using the remaining consecutive bits of data as the second weight.

[0071] The regularization term may include: during the process of the first weight being sparse into the second weight, if the logic for discarding N bits of data in the first weight does not meet the preset logic, then a penalty value is assigned to the second weight.

[0072] The preset logic may include, for example, discarding the first two and last two digits of the first weight, or discarding the first four digits of the first weight, or discarding the four digits carrying the least amount of effective data while ensuring that the second weight is a series of consecutive digits of the first weight. This specification does not limit this.

[0073] In this embodiment, by retaining consecutive multiple data points as the second weight, the relationship between multiple data points can be preserved, reducing the accuracy loss caused by sparsening the first weight into the second weight.

[0074] In summary, the quantization method for the neural network model introduces a regularization term into the original loss function of the neural network model trained on the target data. With minimizing the loss function as the objective function and optimizing the expectation as the constraint, the method iteratively converts a portion of the multiple first weights into second weights. This achieves the goal of converting some first weights into second weights while minimizing the sum of the loss function and the regularization term of the target neural network model. This allows the target neural network model to balance operational efficiency and computational accuracy when processing speech and / or images, thus expanding the application scenarios of the target neural network model while ensuring good accuracy. Furthermore, by using minimizing the loss function as the objective function and optimizing the expectation as the constraint, and converting a portion of the multiple first weights into second weights during multiple iterations, the mixed-precision quantization strategy problem is transformed into a discrete-space optimization problem, reducing the difficulty of mixed-precision quantization and improving the execution efficiency of the method.

[0075] During quantization training, the gradient of the weights in the neural network model does not match the first weights obtained after quantization. If the gradient of the weights is directly used to update the first weights during the learning process, it will make learning very difficult and may cause the learning process to fail to converge. To solve this problem, in some implementations, step S202 may include:

[0076] S2021: Using the quantization-perception training method, the neural network model is subjected to first-precision integer quantization to obtain the quantization parameters of the neural network model mapped to the first-precision neural network model;

[0077] The quantization parameters may include bit width, scaling factor, zero point, etc. The bit width specifies the precision of the quantized value, the scaling factor maps floating-point numbers to integers, and the zero point represents the zero value of an integer.

[0078] S2022: Using the quantization parameters, convert the weights of the neural network model from floating-point format to the first weights with first precision;

[0079] S2023: Perform gradient backpropagation on the neural network model using approximate scaling of the weight gradient to update the first weight of the neural network model; the approximate scaling of the weight gradient includes the product of the weight gradient and the quantization parameter.

[0080] The weight gradient refers to the gradient of the weights in the neural network model before quantization learning. Since the weights in the neural network model before quantization learning are represented as 32-bit floating-point numbers, their gradients are also represented as 32-bit floating-point numbers. Because the first weight obtained in step S2022 is represented in first-precision integer format, the first weight and its gradient do not match. If the weight gradient is directly used to update the first weight during quantization learning, the learning process will be difficult to converge. Therefore, in step S2023, an approximate scaling of the weight gradient is used for gradient backpropagation of the neural network model, so that the approximate scaling of the weight gradient matches the first weight, thereby solving the above problem. According to the derivative formula, the approximate scaling of the weight gradient can be approximately represented by the product of the weight gradient and the quantization parameter.

[0081] Optionally, the first precision can be 8-bit precision, and the precision of the second weight can be a second precision, which can be 4-bit or 6-bit precision. This specification does not limit this.

[0082] As mentioned above, step S203 transforms the mixed-precision quantization strategy problem into a discrete-space optimization problem, providing a feasible solution to the mixed-precision quantization strategy problem. However, discrete-space optimization problems are more complex than ordinary optimization problems. One embodiment of this specification provides a simpler method for solving this problem. Specifically, step S203 may include:

[0083] The Alternating Direction Method of Multipliers (ADMM) is used to minimize the loss function as the objective function, and the first, second and third steps are executed alternately in multiple iterations.

[0084] The first step includes using the Lagrange multiplier method to fix the optimization expectation and iterating the loss function; the second step includes using the Lagrange multiplier method to fix the loss function and iterating the optimization expectation; the third step includes fixing the optimization expectation and the loss function and updating the scaling of the Lagrange multipliers; the scaling of the Lagrange multipliers is used to balance the magnitudes of the objective function and the constraints.

[0085] The optimization problem of mixed precision can be expressed in the form of alternating direction multiplier method as follows:

[0086] Among them, W i b represents the weight of the i-th layer in the neural network model. i Let f() represent the bias of the i-th layer in the neural network model, and g represent the original loss function in the neural network model. i () represents the regularization term added to the i-th layer in the neural network model, Z i This represents the expected optimization of the neural network model, where minimize means to minimize and subject to means constraint.

[0087] Solving the above problem using the alternating direction multiplier method can be summarized as solving two subproblems. The first subproblem is an optimization problem concerning the objective function, and the second subproblem is the dynamic updating of the multipliers in each iteration. The first subproblem can be solved through the first and second steps, and the second subproblem can be solved through the third step. The first, second, and third steps can be expressed by the following formula:

[0088]

[0089] In this context, the superscripts k and k+1 of each parameter represent the number of iterations, for example... This represents the scaling of the Lagrange multipliers in the i-th layer during the k-th iteration of the neural network model. represents the scaling of the Lagrange multipliers in the i-th layer during the (k+1)-th iteration of the neural network model; argmin represents finding the minimum value; Lρ() represents the function of the augmented Lagrange multiplier method.

[0090] The method of alternating direction multipliers can transform the optimization problem in discrete space into an alternating optimization problem of weights and expectations. By alternating the optimization of weights and expectations, the optimization objective can be solved step by step.

[0091] refer to Figure 5 , Figure 5 This illustrates a flowchart of a relatively complete quantization method for a neural network model. Figure 5 The method may include:

[0092] Step 1: Perform quantization training with 8-bit precision integer shaping based on the original 32-bit floating-point representation of the neural network model to obtain quantization parameters mapped to 8-bit precision.

[0093] Step 2: Use quantization parameters to convert the weights of the neural network model from 32-bit floating-point numbers to 8-bit integer format;

[0094] Step 3: Modify the gradient backpropagation path and use the approximate scaling of the weight gradient to perform gradient backpropagation on the neural network model.

[0095] Step 4: Add a regularization term to the original loss function of the neural network model to obtain a new loss function;

[0096] Step 5: Using minimizing the loss function as the objective function and optimizing the expectation as the constraint, ADMM is used to alternately optimize the loss function, optimize the expectation, and update the scaling of the Lagrange multipliers, thereby sparse some of the first weights in the neural network model into second weights.

[0097] Step 6: Obtain the compressed neural network model, which includes multiple first weights and multiple second weights. This greatly improves the computational efficiency of the neural network model while minimizing the loss of accuracy.

[0098] Accordingly, the embodiments of this specification also provide a data processing method, such as... Figure 6 As shown, it includes:

[0099] S601: Obtain data to be processed, the data to be processed including: voice and / or images;

[0100] S602: In response to the processing instruction, the data to be processed is input into the target neural network model to obtain the processing result;

[0101] The target neural network model is obtained by quantizing a neural network model trained on target data, wherein the target data includes: image data and / or speech data;

[0102] The quantization process includes:

[0103] The weights of the neural network model are quantized and trained to obtain multiple first weights; the first weights are represented in integer format.

[0104] With minimizing the loss function as the objective function and optimizing the expectation as the constraint, during multiple iterations, some of the multiple first weights are sparsed into second weights to obtain a target neural network model. The accuracy of the first weights is greater than the accuracy of the second weights. The target neural network model is used to obtain processing results based on the input speech and / or image.

[0105] The loss function includes a regularization term, which is used to characterize the performance of the neural network model. The regularization term includes a penalty rule when the first weight is sparsed into the second weight. The optimization expectation includes the ratio of the number of the second weight to the total number of weights in the neural network model.

[0106] For a detailed description of the quantization process, please refer to the description of the quantization method for neural network models above. This manual will not repeat it here.

[0107] Optionally, in one embodiment, the processing instruction carries a target processing method, and the step of inputting the data to be processed into the target neural network model in response to the processing instruction to obtain the processing result includes:

[0108] In response to the processing instruction, the data to be processed is input into the target neural network model corresponding to the target processing method, so as to obtain the processing result output by the target neural network model;

[0109] The target processing method includes at least one of the following: image segmentation, image feature extraction, image restoration, image reconstruction, speech recognition, speech synthesis, speech feature extraction, speech restoration, and speech reconstruction.

[0110] In this embodiment, the data processing method supports responses to processing instructions carrying different target processing methods. Specifically, multiple neural network models with different processing method response capabilities can be pre-set. After obtaining processing instructions carrying different processing methods, the data to be processed can be input into the target neural network model with the corresponding response capability to realize responses to various target processing methods and improve the adaptability of the data processing method.

[0111] Example device

[0112] Based on the same concept, this specification also provides a quantization device for a neural network model, such as... Figure 7 As shown, it includes:

[0113] The model acquisition module 701 is used to acquire a neural network model trained on target data, wherein the target data includes image data and / or speech data;

[0114] The quantization training module 702 is used to quantize and train the weights of the neural network model to obtain multiple first weights; the first weights are represented in integer format.

[0115] The weighting coefficient module 703 is used to minimize the loss function as the objective function and optimize the expectation as the constraint. During multiple iterations, a portion of the multiple first weights are sparsed into second weights to obtain a target neural network model. The accuracy of the first weights is greater than the accuracy of the second weights. The target neural network model is used to obtain processing results based on the input speech and / or image.

[0116] The loss function includes a regularization term, which is used to characterize the performance of the neural network model. The regularization term includes a penalty rule when the first weight is sparsed into the second weight. The optimization expectation includes the ratio of the number of the second weight to the total number of weights in the neural network model.

[0117] The quantization device for the neural network model provided in this embodiment belongs to the same concept as the quantization method for the neural network model provided in the above embodiments of this specification. It can execute the quantization method for the neural network model provided in any of the above embodiments of this specification and has the corresponding functional modules and beneficial effects for executing the quantization method for the neural network model. Technical details not described in detail in this embodiment can be found in the specific processing content of the quantization method for the neural network model provided in the above embodiments of this specification, and will not be repeated here.

[0118] Based on the same concept, this specification also provides a data processing apparatus, such as... Figure 8 As shown, it includes:

[0119] Data acquisition module 801 is used to acquire data to be processed, the data to be processed including: voice and / or images;

[0120] The data processing module 802 is used to input the data to be processed into the target neural network model in response to the processing instruction to obtain the processing result;

[0121] The target neural network model is obtained by quantizing a neural network model trained on target data, wherein the target data includes: image data and / or speech data;

[0122] The quantification process includes:

[0123] The weights of the neural network model are quantized and trained to obtain multiple first weights; the first weights are represented in integer format.

[0124] With minimizing the loss function as the objective function and optimizing the expectation as the constraint, during multiple iterations, some of the multiple first weights are sparsed into second weights to obtain a target neural network model. The accuracy of the first weights is greater than the accuracy of the second weights. The target neural network model is used to obtain processing results based on the input speech and / or image.

[0125] The loss function includes a regularization term, which is used to characterize the performance of the neural network model. The regularization term includes a penalty rule when the first weight is sparsed into the second weight. The optimization expectation includes the ratio of the number of the second weight to the total number of weights in the neural network model.

[0126] The data processing apparatus provided in this embodiment belongs to the same concept as the data processing method provided in the above embodiments of this specification. It can execute the data processing method provided in any of the above embodiments of this specification and has the corresponding functional modules and beneficial effects for executing the data processing method. Technical details not described in detail in this embodiment can be found in the specific processing content of the data processing method provided in the above embodiments of this specification, and will not be repeated here.

[0127] Example computing devices, storage media, and software

[0128] Another embodiment of this specification also proposes a computing device, see [link to documentation]. Figure 9 As shown, an exemplary embodiment of this specification also provides a computing device, including: a memory and a processor, the memory storing a computer program, the processor executing the computer program to perform steps in the quantization method or data processing method of the neural network model according to various embodiments of this specification described above.

[0129] The internal structure of the computing device can be as follows: Figure 9 As shown, the computing device includes a processor, memory, network interface, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory of the central control device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface of the computing device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it follows the steps of the quantization method or data processing method for the neural network model according to various embodiments of this specification as described in the above embodiments.

[0130] The processor may include the main processor, as well as baseband chips, modems, etc.

[0131] The memory stores a computer program that executes the technical solution of this invention, and may also store an operating system and other key programs. Specifically, the computer program may include program code, which includes computer operation instructions. More specifically, the memory may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0132] The processor can be a general-purpose processor, such as a general-purpose processor (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present invention. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0133] Input devices may include devices that receive data and information input by the user, such as keyboards, mice, cameras, scanners, light pens, voice input devices, touch screens, pedometers, or gravity sensors.

[0134] Output devices may include devices that allow information to be output to a user, such as displays, printers, speakers, etc.

[0135] The communication interface may include any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0136] The processor executes the computer program stored in the memory and calls other devices, which can be used to implement the various steps of any of the quantization methods or data processing methods of the neural network model provided in the above embodiments of this application.

[0137] The computing device may also include a display component and a voice component. The display component may be a liquid crystal display screen or an e-ink display screen. The input device of the computing device may be a touch layer covering the display component, or a button, trackball or touchpad set on the casing of the computing device, or an external keyboard, touchpad or mouse, etc.

[0138] Those skilled in the art will understand that Figure 9 The structures shown are merely block diagrams of some structures related to the solutions in this specification and do not constitute a limitation on the computing devices on which the solutions in this specification are applied. Specific computing devices may include more or fewer components than those shown in the figures, or combine certain components, or have different component arrangements.

[0139] In addition to the methods and devices described above, the quantization method or data processing method for the neural network model provided in the embodiments of this specification can also be a computer program product, which includes a computer program that, when run by a processor, causes the processor to perform the steps in the quantization method or data processing method for the neural network model according to various embodiments of this specification as described in the "Exemplary Methods" section above.

[0140] The computer program product described herein can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments described herein. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0141] Furthermore, embodiments of this specification also provide a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor of the steps in the quantization method or data processing method of the neural network model according to various embodiments of this specification as described in the "Example Methods" section above.

[0142] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0143] It is understood that the specific examples in this document are only intended to help those skilled in the art better understand the implementation methods of this specification, and are not intended to limit the scope of this specification.

[0144] It is understood that in the various embodiments described in this specification, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments described in this specification.

[0145] It is understood that the various implementation methods described in this specification can be implemented individually or in combination, and the implementation methods in this specification are not limited in this respect.

[0146] Unless otherwise stated, all technical and scientific terms used in the embodiments of this specification have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this specification. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items. The singular forms "a," "the," and "the" as used in the embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0147] It is understood that the processor in the embodiments of this specification can be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this specification. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this specification can be directly implemented by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above methods.

[0148] It is understood that the memory in the embodiments of this specification may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM). It should be noted that the memory in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0149] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this specification.

[0150] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the aforementioned method implementations, and will not be repeated here.

[0151] In the several embodiments provided in this specification, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0152] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0153] In addition, the functional units in the various embodiments of this specification can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0154] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of this specification, in essence, or the parts that contribute to the prior art, or parts of the technical solutions, can be embodied in the form of software products. These computer software products are stored in a storage medium and include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this specification. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0155] The above description is merely a specific embodiment of this specification, but the scope of protection of this specification is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this specification should be included within the scope of protection of this specification. Therefore, the scope of protection of this specification should be determined by the scope of the claims.

Claims

1. A quantization method for a neural network model, characterized in that, include: Obtain a neural network model trained on target data, wherein the target data includes image data and / or speech data; The weights of the neural network model are quantized and trained to obtain multiple first weights; the first weights are represented in integer format. With minimizing the loss function as the objective function and optimizing the expectation as the constraint, during multiple iterations, some of the multiple first weights are sparsed into second weights to obtain a target neural network model. The accuracy of the first weights is greater than the accuracy of the second weights. The target neural network model is used to obtain processing results based on the input speech and / or image. The loss function includes a regularization term, which is used to characterize the performance of the neural network model. The regularization term includes a penalty rule when the first weight is sparsed into the second weight. The optimization expectation includes the ratio of the number of the second weight to the total number of weights in the neural network model.

2. The method according to claim 1, characterized in that, The step of quantizing and training the weights of the neural network model to obtain multiple first weights includes: Using the quantization-perception training method, the neural network model is subjected to first-precision integer quantization to obtain the quantization parameters that map the neural network model to the first-precision neural network model. The weights of the neural network model are converted from floating-point format to the first weights with first precision using the quantization parameters. The neural network model is backpropagated using an approximate scaling of the weight gradient to update the first weights of the neural network model; the approximate scaling of the weight gradient includes the product of the weight gradient and the quantization parameter.

3. The method according to claim 1, characterized in that, The process of minimizing the loss function as the objective function and optimizing the expectation as the constraint, and then sparsely converting a portion of the multiple first weights into second weights during multiple iterations to obtain the target neural network model includes: Using the alternating direction multiplier method, with minimizing the loss function as the objective function, the first, second, and third steps are executed alternately during multiple iterations; The first step includes using the Lagrange multiplier method to fix the optimization expectation and iterating the loss function; the second step includes using the Lagrange multiplier method to fix the loss function and iterating the optimization expectation; the third step includes fixing the optimization expectation and the loss function and updating the scaling of the Lagrange multipliers; the scaling of the Lagrange multipliers is used to balance the magnitudes of the objective function and the constraints.

4. The method according to claim 1, characterized in that, The process of converting the first weight into the second weight includes: Discard N bits of data from the first weight and use the remaining consecutive bits as the second weight.

5. The method according to claim 4, characterized in that, The regularization terms include: During the process of converting the first weight into the second weight, if the logic for discarding N bits of data in the first weight does not meet the preset logic, then a penalty value is assigned to the second weight.

6. A data processing method, characterized in that, include: Acquire data to be processed, the data to be processed including: voice and / or images; In response to the processing instruction, the data to be processed is input into the target neural network model to obtain the processing result; The target neural network model is obtained by quantizing a neural network model trained on target data, wherein the target data includes: image data and / or speech data; The quantification process includes: The weights of the neural network model are quantized and trained to obtain multiple first weights; the first weights are represented in integer format. With minimizing the loss function as the objective function and optimizing the expectation as the constraint, during multiple iterations, some of the multiple first weights are sparsed into second weights to obtain a target neural network model. The accuracy of the first weights is greater than the accuracy of the second weights. The target neural network model is used to obtain processing results based on the input speech and / or image. The loss function includes a regularization term, which is used to characterize the performance of the neural network model. The regularization term includes a penalty rule when the first weight is sparsed into the second weight. The optimization expectation includes the ratio of the number of the second weight to the total number of weights in the neural network model.

7. The method according to claim 6, characterized in that, The processing instruction carries a target processing method, and the step of inputting the data to be processed into the target neural network model in response to the processing instruction to obtain the processing result includes: In response to the processing instruction, the data to be processed is input into the target neural network model corresponding to the target processing method, so as to obtain the processing result output by the target neural network model; The target processing method includes at least one of the following: image segmentation, image feature extraction, image restoration, image reconstruction, speech recognition, speech synthesis, speech feature extraction, speech restoration, and speech reconstruction.

8. A quantization device for a neural network model, characterized in that, include: The model acquisition module is used to acquire a neural network model trained on target data, wherein the target data includes image data and / or voice data; The quantization training module is used to quantize and train the weights of the neural network model to obtain multiple first weights; the first weights are represented in integer format. The weighting coefficient module is used to minimize the loss function as the objective function and optimize the expectation as the constraint. During multiple iterations, a portion of the multiple first weights are sparsed into second weights to obtain a target neural network model. The accuracy of the first weights is greater than that of the second weights. The target neural network model is used to obtain processing results based on the input speech and / or image. The loss function includes a regularization term, which is used to characterize the performance of the neural network model. The regularization term includes a penalty rule when the first weight is sparsed into the second weight. The optimization expectation includes the ratio of the number of the second weight to the total number of weights in the neural network model.

9. A data processing apparatus, characterized in that, include: The data acquisition module is used to acquire data to be processed, which includes: voice and / or images; The data processing module is used to input the data to be processed into the target neural network model in response to the processing instruction to obtain the processing result; The target neural network model is obtained by quantizing a neural network model trained on target data, wherein the target data includes: image data and / or speech data; The quantization process includes: The weights of the neural network model are quantized and trained to obtain multiple first weights; the first weights are represented in integer format. With minimizing the loss function as the objective function and optimizing the expectation as the constraint, during multiple iterations, some of the multiple first weights are sparsed into second weights to obtain a target neural network model. The accuracy of the first weights is greater than the accuracy of the second weights. The target neural network model is used to obtain processing results based on the input speech and / or image. The loss function includes a regularization term, which is used to characterize the performance of the neural network model. The regularization term includes a penalty rule when the first weight is sparsed into the second weight. The optimization expectation includes the ratio of the number of the second weight to the total number of weights in the neural network model.

10. A computing device, characterized in that, include: Processor and memory; The memory is connected to the processor and is used to store computer programs; The processor is configured to implement the quantization method of the neural network model as described in any one of claims 1 to 5 or the data processing method as described in any one of claims 6 to 7 by running a computer program stored in the memory.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the quantization method of the neural network model as described in any one of claims 1 to 5 or the data processing method as described in any one of claims 6 to 7.

Images