Method for quantizing neural network model, data processing method, and related apparatuses

By introducing regularization terms and loss functions in the quantization process of neural network models and sparse part of the weight, the problem of difficult to balance model performance and efficiency in the existing technology is solved, and efficient application and accuracy guarantee on low-performance devices are achieved.

WO2024244594A1PCT designated stage expired Publication Date: 2024-12-05HANGZHOU ALICLOUD FEITIAN INFORMATION TECH CO LTD

Patent Information

Application Number
PCT/CN2024/080769
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-31
Filing Date
2024-03-08
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

The existing neural network model quantization method is difficult to balance the relationship between model performance and operational efficiency, making it difficult to effectively apply on low-performance devices.

Method used

By introducing regularization terms in the quantization process of neural network models, using minimizing loss functions and optimization expectations as constraints, part of the weights are sparse to lower-precision weights during multiple iterations, thereby achieving mixed precision quantization. Taking into account both operating efficiency and calculation accuracy.

Benefits of technology

It realizes the feasibility of carrying a neural network model on low-performance devices, significantly reducing computing costs and energy consumption, while ensuring high computing accuracy, and expanding the application scenarios of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024080769_05122024_PF_FP_ABST
    Figure CN2024080769_05122024_PF_FP_ABST
Patent Text Reader

Abstract

Provided in the embodiments of the present specification are a method for quantizing a neural network model, a data processing method, and related apparatuses. A regularization item is introduced into an original loss function, a minimized loss function is taken as an objective function, an optimization expectation is taken as a constraint condition and, during a plurality of iteration processes, some first weights among a plurality of first weights are sparsified into second weights, so as to achieve the aim of sparsifying some first weights into the second weights while ensuring that the sum of the loss function of a target neural network model and the regularization item is as small as possible, such that the target neural network model can take both operation efficiency and calculation precision into account; that is, on the basis of expanding the scenarios to which the target neural network model can be applied, good precision of the target neural network model is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

A quantization method for a neural network model, a data processing method and related devices

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on May 31, 2023, with application number 202310640797.0 and invention name “A quantization method, data processing method and related device for a neural network model”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments in this specification relate to the field of computer application technology, specifically, to machine learning technology under the field of computer application technology, and more specifically, to a quantization method, data processing method and related devices of a neural network model. Background Art

[0003] With the continuous development of machine learning technology, neural network models are constantly developing towards miniaturization, allowing neural network models to be applied to low-performance computing devices or embedded devices, expanding the applicable scenarios of neural network models.

[0004] Quantization of neural network models is a means to compress model size, improve operating efficiency and reduce power consumption, but current quantization methods make it difficult to balance the relationship between the performance and operating efficiency of neural network models.

[0005] Summary of the Invention

[0006] Multiple embodiments in this specification provide a quantization method, data processing method and related devices for a neural network model to achieve the purpose of balancing the relationship between the performance and operating efficiency of the neural network model.

[0007] In a first aspect, an embodiment of the present specification provides a quantization method for a neural network model, comprising: obtaining a neural network model trained with target data, wherein the target data includes image data and / or speech data; performing quantization training on the weights of the neural network model to obtain a plurality of first weights; the first weights are represented in an integer format; using minimization of the loss function as the objective function and optimization expectation as the constraint condition, in a plurality of iterations, thinning out some of the plurality of first weights into second weights to obtain a target neural network model, wherein the accuracy of the first weights is greater than the accuracy of the second weights; the target neural network model is used to obtain a processing result based on the input speech and / or image; the loss function includes a regularization term, the loss function is used to characterize the performance of the neural network model, the regularization term includes a penalty rule when thinning out the first weights into the second weights, and the optimization expectation includes the ratio of the number of the second weights to the total number of weights in the neural network model.

[0008] In a second aspect, an embodiment of the present specification provides a data processing method, comprising: obtaining data to be processed, the data to be processed comprising: speech and / or image; in response to a processing instruction, inputting the data to be processed into a target neural network model to obtain a processing result; the target neural network model comprises quantizing a neural network model trained on target data, the target data comprising: image data and / or speech data; the quantization process comprises: quantizing and training the weights of the neural network model to obtain a plurality of first weights; the first weights are represented in an integer format; minimizing a loss function as an objective function and optimizing expectations as constraints, and in multiple iterations, thinning out some of the plurality of first weights into second weights to obtain a target neural network model, the accuracy of the first weights being greater than the accuracy of the second weights; the target neural network model is used to obtain a processing result based on the input speech and / or image; the loss function comprises a regularization term, the loss function is used to characterize the performance of the neural network model, the regularization term comprises a penalty rule when thinning out the first weights into the second weights, and the optimization expectation comprises the ratio of the number of the second weights to the total number of weights in the neural network model.

[0009] In a third aspect, an embodiment of the present specification provides a quantization device for a neural network model, comprising: a model acquisition module for acquiring a neural network model trained with target data, wherein the target data includes image data and / or speech data; a quantization training module for quantizing the weights of the neural network model to obtain a plurality of first weights; the first weights are represented in an integer format; a weight coefficient module for thinning out some of the plurality of first weights into second weights in a plurality of iterations with minimizing the loss function as the objective function and optimizing the expectation as the constraint condition to obtain a target neural network model, wherein the accuracy of the first weights is greater than the accuracy of the second weights; the target neural network model is used to obtain a processing result based on the input speech and / or image; the loss function includes a regularization term, the loss function is used to characterize the performance of the neural network model, the regularization term includes a penalty rule when thinning out the first weights into the second weights, and the optimization expectation includes the ratio of the number of the second weights to the total number of weights in the neural network model.

[0010] In a fourth aspect, an embodiment of the present specification provides a data processing device, comprising: a data acquisition module for acquiring data to be processed, wherein the data to be processed includes: speech and / or image; a data processing module for inputting the data to be processed into a target neural network model in response to a processing instruction to obtain a processing result; the target neural network model includes quantizing a neural network model trained with target data, wherein the target data includes: image data and / or speech data; the quantization process includes: quantizing the weights of the neural network model to obtain a plurality of first weights; the first weights are represented in an integer format; minimizing the loss function as the objective function and optimizing the expectation as the constraint condition, in multiple iterations, some of the plurality of first weights are thinned into second weights to obtain a target neural network model, wherein the accuracy of the first weights is greater than the accuracy of the second weights; the target neural network model is used to obtain a processing result based on the input speech and / or image; the loss function includes a regularization term, wherein the loss function is used to characterize the performance of the neural network model, the regularization term includes a penalty rule when thinning the first weights into the second weights, and the optimization expectation includes the ratio of the number of the second weights to the total number of weights in the neural network model.

[0011] In a fifth aspect, an embodiment of the present specification provides a computing device comprising: a processor and a memory; wherein the memory is connected to the processor, and the memory is used to store a computer program; the processor is used to implement the quantization method or data processing method of the neural network model as described above by running the computer program stored in the memory.

[0012] In a sixth aspect, an embodiment of the present specification provides a storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the quantization method or data processing method of the neural network model as described above is implemented.

[0013] In a seventh aspect, an embodiment of the present specification provides a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; the processor of the computer device reads the computer instructions from the computer-readable storage medium, and when the processor executes the computer instructions, it implements the steps of the above-mentioned neural network model quantization method or data processing method. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar components or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed in this application and should not be construed as limiting the scope of this application.

[0015] FIG1 is a feasible application scenario of a quantization method for a neural network model provided by one embodiment of this specification.

[0016] FIG2 is a flow chart of a quantization method for a neural network model provided in one embodiment of this specification.

[0017] FIG3 is a schematic diagram of a process for thinning the first weight provided by one embodiment of this specification.

[0018] FIG4 is a schematic diagram of another process of thinning the first weight provided by one embodiment of this specification.

[0019] FIG5 is a flow chart of another method for quantizing a neural network model provided in accordance with an embodiment of the present disclosure.

[0020] FIG6 is a flow chart of a data processing method provided in one embodiment of this specification.

[0021] FIG7 is a schematic structural diagram of a quantization device for a neural network model provided in one embodiment of this specification.

[0022] FIG8 is a schematic structural diagram of a data processing device provided in one embodiment of this specification.

[0023] FIG9 is a schematic diagram of the structure of a computing device provided in accordance with one embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] Unless otherwise defined, technical or scientific terms used in the embodiments of this specification should have the same ordinary meaning as those understood by persons of ordinary skill in the art to which this specification pertains. The terms "first," "second," and similar terms used in the embodiments of this specification do not denote any order, quantity, or importance, but are provided solely to avoid confusion between constituent elements.

[0025] Unless the context requires otherwise, throughout this specification, the term "plurality" means "at least two," and "including" is to be interpreted as open and inclusive, meaning "including, but not limited to." Throughout this specification, the terms "one embodiment," "some embodiments," "exemplary embodiments," "example," "specific example," or "some examples" are intended to indicate that a particular feature, structure, material, or characteristic associated with the embodiment or example is included in at least one embodiment or example of this specification. The schematic representations of these terms do not necessarily refer to the same embodiment or example.

[0026] The following will be combined with the drawings in the embodiments of this specification to clearly and completely describe the technical solutions in the embodiments of this specification. Obviously, the embodiments described are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this specification.

[0027] Overview

[0028] The quantization technology of neural network models can achieve the purpose of compressing models, improving operating efficiency and reducing power consumption, so that the quantized neural network models can be applied to low-performance devices (such as embedded devices), which can achieve the purpose of reducing the power consumption of neural network models and expanding the usage scenarios.

[0029] The principle of quantization in neural network models can be summarized as converting model parameters and computational variables represented by floating-point numbers into integer or low-bit-width formats to achieve model compression, improve operational efficiency, and reduce power consumption. By compressing floating-point parameters into integers or smaller floating-point numbers, this significantly reduces model size, resulting in better performance in model storage and transmission. Simultaneously, quantization to integers and low-bit-width calculations allows for faster execution in hardware accelerators, thereby increasing model execution speed.

[0030] Current quantization methods for neural network models typically convert the entire model to an 8-bit integer format. With this quantization method, the resulting neural network model generally achieves accuracy close to that of the original floating-point representation. However, this quantization method does not significantly improve the compression and operational efficiency of the neural network model, making it difficult for some low-performance devices to process data using quantized neural network models.

[0031] In order to solve this problem, the embodiments of the present application have found that the model can be quantized into a neural network model with lower precision, for example, the neural network model can be quantized into a model represented by a 4-bit or 6-bit integer format, which can greatly accelerate the reasoning process of the model and achieve good model compression, improved operating efficiency and reduced power consumption. However, the loss of precision caused by converting the entire neural network model into a model with lower precision is difficult to ignore. In this case, the embodiments of the present application have found through investigation and research that mixed precision quantization is a feasible solution. Mixed precision quantization refers to the process of expressing part of the parameters in a 4-bit or 6-bit integer format and the other part of the parameters in an 8-bit integer format during the quantization process of the neural network model. For example, some operators that take a long time to run in a low-precision manner can significantly reduce the computing cost and energy consumption while maintaining relatively high computing accuracy. However, in the mixed precision quantization method, how to choose which parameters to represent with lower precision (for example, 4-bit or 6-bit integer format) is a difficult problem.

[0032] Through further research, the implementation method of the present application has found that by introducing a regularization term into the original loss function of the neural network model trained on the target data, minimizing the loss function as the objective function and optimizing the expectation as the constraint condition, in the process of multiple iterations, some of the first weights in the multiple first weights are sparsely converted to second weights. While ensuring that the sum of the loss function and the regularization term of the target neural network model is minimized, the purpose of converting some of the first weights to second weights is achieved, so that the target neural network model can take into account both operating efficiency and computational accuracy when processing speech and / or images, that is, on the basis of expanding the scenarios in which the target neural network model can be applied, it is ensured that the target neural network model has good accuracy. In addition, by minimizing the loss function as the objective function and optimizing the expectation as the constraint condition, in the process of multiple iterations, some of the first weights in the multiple first weights are sparsely converted to second weights, the mixed precision quantization strategy problem is converted into an optimization problem in a discrete space, which reduces the difficulty of mixed precision quantization and improves the execution efficiency of the method.

[0033] Based on the above-mentioned inventive concept, an embodiment of this specification provides a quantization method for a neural network model. The quantization method for the neural network model will be exemplarily described below with reference to the accompanying drawings.

[0034] Scenario Example

[0035] Referring to Figure 1, Figure 1 shows a feasible application scenario of the quantization method of the neural network model. In this application scenario, the neural network model has been pre-trained and has a certain data processing capability, such as the ability to perform face recognition based on the input image or voice recognition based on the input sound. At this time, the weights of the neural network model are all represented by 32-bit floating point numbers, which has a relatively high data processing accuracy (that is, the probability of outputting accurate results based on the input data is high). However, the size of the neural network model at this time is large, the operating efficiency is low, and the power consumption is high, which is not suitable for lower performance computing devices. Therefore, it is necessary to quantize the neural network model, represent some of the weights of the neural network model in 8-bit integer format, and represent the other part of the weights in a lower bit number (such as 4-bit or 6-bit integer format), so that on the basis of achieving compression of the neural network model, improving operating efficiency and reducing power consumption, the neural network model is guaranteed to have relatively high computing accuracy.

[0036] The quantized neural network model can be carried on a computing device (for example, a mobile smart terminal) to perform corresponding tasks (such as image feature extraction, speech recognition, etc.). For example, in Figure 1, the neural network model carried on a smart phone can process the input voice and obtain the speech recognition result for output. In some cases, in addition to being a smart phone, the computing device can also be a desktop computer, tablet computer, laptop computer, digital assistant, smart wearable device, shopping guide terminal, television, smart speaker, microphone, etc. Among them, smart wearable devices include but are not limited to smart bracelets, smart watches, smart glasses, smart helmets, smart necklaces, etc.

[0037] Example Method

[0038] One embodiment of this specification provides a quantization method for a neural network model, as shown in FIG2 , including:

[0039] S201: Acquire a neural network model trained with target data, where the target data includes image data and / or speech data.

[0040] The target data can be labeled training samples. In some cases, the training samples can be divided into training sets and test sets based on their different uses during the training process. A neural network model can gradually improve its predictive accuracy and generalization capabilities by repeatedly learning and adjusting the data in the training set. The test set is a dataset used to test the performance and generalization capabilities of the neural network model. By testing the neural network model with the test set and evaluating its accuracy, it can be verified whether the neural network model's predictive capabilities have reached the expected level. A neural network model trained with target data can possess corresponding functions, such as speech recognition, image segmentation, and so on.

[0041] S202: Perform quantization training on the weights of the neural network model to obtain a plurality of first weights; the first weights are represented in an integer format.

[0042] Quantization training, also known as Quantization Aware Training (QAT), is a training method that optimizes the parameters of neural networks using quantitative methods. Traditional neural network training typically uses floating-point numbers to represent neural network parameters, but this approach consumes significant computing resources and memory space, hindering the deployment of neural networks on mobile devices. Quantization training can compress the size of neural network models and improve their operational efficiency, thereby meeting the needs of deploying neural network models on various computing devices.

[0043] In step S202, the weights in the neural network model can be quantized to first weights through quantization training. The first weights can be, for example, weights represented in an 8-bit integer format. Through the quantization training in step S202, preliminary quantization of the weights in the neural network model is achieved, and the weights in the neural network model are quantized to first weights with higher accuracy.

[0044] S203: Using minimization of the loss function as an objective function and optimization expectation as a constraint condition, in a plurality of iterations, thinning out some of the plurality of first weights into second weights to obtain a target neural network model, wherein the accuracy of the first weights is greater than the accuracy of the second weights; the target neural network model is used to obtain a processing result based on the input speech and / or image;

[0045] The loss function includes a regularization term, which is used to characterize the performance of the neural network model. The regularization term includes a penalty rule when the first weight is thinned to the second weight. The optimization expectation includes the ratio of the number of the second weights to the total number of weights in the neural network model.

[0046] For example, assuming that the optimization expectation is that in the neural network model, the number of second weights accounts for 50% of the total number of weights, then when the number of second weights in the neural network model / (the number of first weights + the number of second weights) = 50%, it is considered to meet the optimization expectation.

[0047] Sparsification is a method of making neural network models lightweight. Regularization is a technique used to prevent overfitting problems in neural network models during the iterative process. By introducing additional regularization terms during the model training process, the neural network model parameters are prevented from becoming overly complex. The regularization term is a penalty rule for the weights in the neural network model. For example, in this embodiment, the regularization term may specifically include a penalty rule for thinning the first weight to the second weight. Assume that the penalty rule includes: while ensuring that the second weight is a continuous 4-bit portion of the first weight, if the amount of valid data carried by the 4 bits in the thinned-out first weight is not the minimum amount of valid data, a penalty is imposed. Referring to Figures 3 and 4, Figures 3 and 4 show two ways of using the same first weight coefficient as the second weight. In Figure 3, the first and second bits of the first weight are thinned, and the obtained second weight is "1101". Two of the sparsed bits are not zero. Therefore, in the sparse method of Figure 3, the amount of valid data carried by the 4 bits of the thinned-out first weight is 2 bits. In Figure 4, the first four bits of the sparse first weight obtain a second weight of "0101", and 3 of the sparse bits are not zero. Therefore, in the sparse method of Figure 4, the amount of valid data carried by the sparse 4 bits of the first weight is 3 bits, which meets the above-mentioned penalty rules, and a penalty can be imposed on the sparse method of Figure 4.

[0048] That is, in some embodiments, the process of thinning the first weight to the second weight includes: discarding N bits of data in the first weight and using the retained continuous multi-bit data as the second weight.

[0049] The regularization term may include: in the process of thinning the first weight to the second weight, if the discarding logic of N-bit data in the first weight does not meet the preset logic, assigning a penalty value to the second weight.

[0050] The preset logic may, for example, include: discarding the first two bits and the last two bits of data of the first weight, or discarding the first four bits of data of the first weight, or discarding the four bits of data carrying the least amount of valid data on the basis of ensuring that the second weight is a continuous multi-bit data in the first weight, etc. This specification does not limit this.

[0051] In this embodiment, by retaining continuous multi-bit data as the second weight, the relationship between the multi-bit data can be retained, reducing the accuracy loss caused by thinning the first weight into the second weight.

[0052] In general, the quantization method of the neural network model introduces a regularization term into the original loss function of the neural network model trained on the target data, takes minimizing the loss function as the objective function, takes optimizing the expectation as the constraint condition, and in a plurality of iterations, thins out some of the plurality of first weights into second weights. This achieves the purpose of thinning out some of the first weights into second weights while ensuring that the sum of the loss function and the regularization term of the target neural network model is minimized, so that the target neural network model can take into account both operational efficiency and computational accuracy when processing speech and / or images, that is, on the basis of expanding the scenarios in which the target neural network model can be applied, it ensures that the target neural network model has good accuracy. In addition, by taking minimizing the loss function as the objective function and optimizing the expectation as the constraint condition, and in a plurality of iterations, thinning out some of the plurality of first weights into second weights, the mixed precision quantization strategy problem is transformed into an optimization problem in a discrete space, thereby reducing the difficulty of mixed precision quantization and improving the execution efficiency of the method.

[0053] During the quantization training process, the gradient of the weight of the neural network model does not match the first weight obtained after quantization. If the gradient of the weight is directly used to update the first weight in the learning process, it will make learning very difficult and may cause the learning process to be difficult to converge. To solve this problem, in some embodiments, step S202 may include:

[0054] S2021: Performing shaping quantization of the neural network model with a first precision using a quantization-aware training method to obtain quantization parameters of the neural network model mapped to a neural network model with the first precision;

[0055] The quantization parameters may include bit width, scaling factor, zero point, etc. The bit width specifies the numerical precision after quantization, the scaling factor is used to map floating point numbers to integers, and the zero point is used to represent the zero value of the integer.

[0056] S2022: Convert the weight of the neural network model from a floating point format to the first weight of a first precision using the quantization parameter;

[0057] S2023: Perform gradient backpropagation on the neural network model using approximate scaling of the weight gradient to update the first weight of the neural network model; the approximate scaling of the weight gradient includes the product of the weight gradient and the quantization parameter.

[0058] The weight gradient refers to the gradient of the weight of the neural network model before quantization learning. Since the weights of the neural network model before quantization learning are all represented by 32-bit floating point numbers, correspondingly, the gradient of the weight is also represented by 32-bit floating point numbers. Since the first weight obtained in step S2022 is represented in the integer format of the first precision, this makes the first weight and the gradient of the weight not match. If the weight gradient is directly used to update the first weight during the quantization learning process, it will make the learning process difficult to converge. Therefore, in step S2023, the neural network model is gradient-backed using the approximate scaling of the weight gradient, so that the approximate scaling of the weight gradient matches that of the first weight, thereby solving the above problem. According to the derivation formula, the approximate scaling of the weight gradient can be approximately represented by the product of the gradient of the weight and the quantization parameter.

[0059] Optionally, the first precision may be 8-bit precision, the second weight precision may be second precision, and the second precision may be 4-bit or 6-bit precision, which is not limited in this specification.

[0060] As described above, step S203 converts the mixed-precision quantization strategy problem into a discrete-space optimization problem, providing a feasible solution to the mixed-precision quantization strategy problem. However, the discrete-space optimization problem is more complex than the ordinary optimization problem. One embodiment of this specification provides a relatively simple method for solving this problem. Specifically, step S203 may include:

[0061] Using the Alternating Direction Method of Multipliers (ADMM), minimizing the loss function is used as the objective function, and the first step, the second step, and the third step are alternately performed in multiple iterations;

[0062] The first step includes using the Lagrange multiplier method to fix the optimization expectation and iterate the loss function. The second step includes using the Lagrange multiplier method to fix the loss function and iterate the optimization expectation. The third step includes fixing the optimization expectation and the loss function and updating the scaling of the Lagrange multiplier; the scaling of the Lagrange multiplier is used to balance the magnitude of the objective function and the constraint conditions.

[0063] The mixed precision optimization problem can be expressed as follows in the form of alternating direction multipliers:

[0064] Among them, W i represents the weight of the i-th layer in the neural network model, b irepresents the bias of the i-th layer in the neural network model, f() represents the original loss function in the neural network model, g i () represents the regularization term added to the i-th layer in the neural network model, Z i It represents the optimization expectation of the neural network model, minimize represents minimization, and subject to represents constraints.

[0065] Solving the above problem using the alternating direction multiplier method can be summarized as solving two subproblems. The first subproblem is to optimize the objective function, and the second subproblem is to dynamically update the multiplier during each iteration. The first subproblem can be solved by the first and second steps, and the second subproblem can be solved by the third step. The first, second, and third steps can be expressed as the following process:

[0066] The superscripts k and k+1 of each parameter represent the number of iterations, for example represents the scaling of the Lagrange multiplier of the i-th layer in the k-th iteration of the neural network model, It represents the scaling of the Lagrange multiplier of the i-th layer in the k+1-th iteration process in the neural network model; argmin represents the minimum value; Lρ() represents the function of the augmented Lagrange multiplier method.

[0067] Based on the alternating direction multiplier method, the optimization problem in discrete space can be transformed into an alternating optimization problem of weights and expectations. By optimizing the weights and expectations alternately, the optimization goal can be gradually solved.

[0068] Referring to FIG5 , FIG5 shows a flow chart of a relatively complete method for quantizing a neural network model. In FIG5 , the method may include:

[0069] Step 1: Perform quantization training with 8-bit precision shaping based on the original 32-bit floating-point representation of the neural network model to obtain the quantization parameters mapped to 8-bit precision.

[0070] Step 2: Use the quantization parameters to convert the weights of the neural network model from 32-bit floating point numbers to 8-bit integer format.

[0071] Step 3: Modify the gradient return path and use the approximate scaling of the weight gradient to return the gradient of the neural network model.

[0072] Step 4: Add a regularization term to the original loss function of the neural network model to obtain a new loss function.

[0073] Step 5: Taking minimizing the loss function as the objective function and optimizing the expectation as the constraint, use ADMM to alternately optimize the loss function, optimize the expectation, and update the scaling of the Lagrange multiplier to achieve the sparse conversion of part of the first weights in the neural network model to the second weights.

[0074] Step 6: Obtain a compressed neural network model, which includes multiple first weights and multiple second weights, and greatly improves the computational efficiency of the neural network model while minimizing the loss of accuracy.

[0075] Accordingly, the embodiment of this specification further provides a data processing method, as shown in FIG6 , including:

[0076] S601: Acquire data to be processed, where the data to be processed includes: voice and / or image.

[0077] S602: In response to the processing instruction, the data to be processed is input into the target neural network model to obtain a processing result.

[0078] The target neural network model is obtained by quantizing a neural network model trained with target data, and the target data includes: image data and / or speech data.

[0079] The quantization process includes:

[0080] The weights of the neural network model are quantized and trained to obtain a plurality of first weights; the first weights are represented in an integer format.

[0081] Taking minimizing the loss function as the objective function and optimizing the expectation as the constraint condition, during multiple iterations, some of the multiple first weights are thinned into second weights to obtain a target neural network model, and the accuracy of the first weights is greater than the accuracy of the second weights; the target neural network model is used to obtain processing results based on the input speech and / or image.

[0082] The loss function includes a regularization term, which is used to characterize the performance of the neural network model. The regularization term includes a penalty rule when the first weight is thinned to the second weight. The optimization expectation includes the ratio of the number of the second weights to the total number of weights in the neural network model.

[0083] For a detailed description of the quantization process, please refer to the above description of the quantization method for the neural network model, which will not be elaborated in this manual.

[0084] Optionally, in one embodiment, the processing instruction carries a target processing mode, and in response to the processing instruction, inputting the data to be processed into a target neural network model to obtain a processing result includes:

[0085] In response to a processing instruction, inputting the data to be processed into a target neural network model corresponding to the target processing mode to obtain a processing result output by the target neural network model;

[0086] The target processing method includes: at least one of: image segmentation processing, image feature extraction processing, image repair processing, image reconstruction processing, speech recognition processing, speech synthesis processing, speech feature extraction processing, speech repair processing and speech reconstruction processing.

[0087] In this embodiment, the data processing method supports responding to processing instructions carrying different target processing methods. Specifically, multiple neural network models with different processing method response capabilities can be pre-set. After obtaining processing instructions carrying different processing methods, the data to be processed can be input into the target neural network model with corresponding response capabilities to achieve response to various target processing methods, thereby improving the adaptability of the data processing method.

[0088] Example device

[0089] Based on the same concept, the embodiment of this specification also provides a quantization device for a neural network model, as shown in FIG7 , including:

[0090] A model acquisition module 701 is configured to acquire a neural network model trained with target data, wherein the target data includes image data and / or speech data;

[0091] A quantization training module 702 is configured to perform quantization training on the weights of the neural network model to obtain a plurality of first weights, wherein the first weights are represented in an integer format;

[0092] A weight coefficient module 703 is configured to thin out some of the plurality of first weights into second weights during multiple iterations, using minimization of the loss function as an objective function and optimization expectation as a constraint condition, to obtain a target neural network model, wherein the accuracy of the first weights is greater than the accuracy of the second weights; the target neural network model is used to obtain a processing result based on the input speech and / or image;

[0093] The loss function includes a regularization term, which is used to characterize the performance of the neural network model. The regularization term includes a penalty rule when the first weight is thinned to the second weight. The optimization expectation includes the ratio of the number of the second weights to the total number of weights in the neural network model.

[0094] The quantization device for the neural network model provided in this embodiment is based on the same application concept as the quantization method for the neural network model provided in the above embodiments of this specification. It can execute the quantization method for the neural network model provided in any of the above embodiments of this specification and has the corresponding functional modules and beneficial effects for executing the quantization method for the neural network model. For technical details not fully described in this embodiment, please refer to the specific processing content of the quantization method for the neural network model provided in the above embodiments of this specification, and will not be repeated here.

[0095] Based on the same concept, the embodiment of this specification further provides a data processing device, as shown in FIG8 , including:

[0096] The data acquisition module 801 is used to acquire data to be processed, wherein the data to be processed includes: voice and / or image;

[0097] The data processing module 802 is configured to input the data to be processed into a target neural network model in response to a processing instruction to obtain a processing result;

[0098] The target neural network model is obtained by quantizing a neural network model trained with target data, wherein the target data includes: image data and / or speech data;

[0099] The quantization process includes:

[0100] Quantizing and training the weights of the neural network model to obtain a plurality of first weights, wherein the first weights are represented in an integer format;

[0101] Taking minimizing the loss function as the objective function and optimizing the expectation as the constraint condition, in multiple iterations, thinning some of the multiple first weights into second weights to obtain a target neural network model, where the accuracy of the first weights is greater than the accuracy of the second weights; the target neural network model is used to obtain a processing result based on the input speech and / or image;

[0102] The loss function includes a regularization term, which is used to characterize the performance of the neural network model. The regularization term includes a penalty rule when the first weight is thinned to the second weight. The optimization expectation includes the ratio of the number of the second weights to the total number of weights in the neural network model.

[0103] The data processing device provided in this embodiment is based on the same application concept as the data processing method provided in the above embodiments of this specification. It can execute the data processing method provided in any of the above embodiments of this specification and has the corresponding functional modules and beneficial effects of executing the data processing method. For technical details not fully described in this embodiment, please refer to the specific processing content of the data processing method provided in the above embodiments of this specification and will not be repeated here.

[0104] Example computing devices, storage media, and software

[0105] Another embodiment of this specification also proposes a computing device. As shown in Figure 9, an exemplary embodiment of this specification also provides a computing device, including: a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it executes the steps of the quantization method or data processing method of the neural network model according to various embodiments of this specification described in the above embodiments of this specification.

[0106] The internal structure of the computing device can be shown in Figure 9. The computing device includes a processor, a memory, a network interface and an input device connected through a system bus. Among them, the processor of the computing device is used to provide computing and control capabilities. The memory of the central control device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computing device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it follows the steps of the quantization method or data processing method of the neural network model according to various embodiments of the present specification described in the above embodiments of the present specification.

[0107] The processor may include a main processor, and may also include a baseband chip, a modem, etc.

[0108] The memory stores a computer program that executes the technical solution of the present invention, and may also store an operating system and other key programs. Specifically, the computer program may include program code, which includes computer operating instructions. More specifically, the memory may include read-only memory (ROM), other types of static storage devices that can store static information and instructions, random access memory (RAM), other types of dynamic storage devices that can store information and instructions, disk storage, flash memory, etc.

[0109] The processor may be a general-purpose processor, such as a CPU or microprocessor, or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present invention. Alternatively, the processor may be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware components.

[0110] Input devices may include devices that receive data and information input by a user, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0111] Output devices may include means that allow information to be output to a user, such as display screens, printers, speakers, and the like.

[0112] The communication interface may include any device such as a transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0113] The processor executes the computer program stored in the memory and calls other devices, which can be used to implement the various steps of the quantization method or data processing method of any neural network model provided in the above embodiments of the present application.

[0114] The computing device may also include a display component and a voice component. The display component may be a liquid crystal display or an electronic ink display. The input device of the computing device may be a touch layer covering the display component, or a button, trackball or touchpad provided on the housing of the computing device, or an external keyboard, touchpad or mouse.

[0115] Those skilled in the art will understand that the structure shown in FIG9 is merely a block diagram of a portion of the structure related to the solution of this specification, and does not constitute a limitation on the computing device to which the solution of this specification is applied. The specific computing device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0116] In addition to the above-mentioned methods and devices, the quantization method or data processing method of the neural network model provided in the embodiments of this specification may also be a computer program product, which includes a computer program, which, when executed by a processor, enables the processor to execute the steps of the quantization method or data processing method of the neural network model according to various embodiments of this specification described in the above "Exemplary Method" section of this specification.

[0117] The computer program product may be written in any combination of one or more programming languages ​​to implement the operations of the embodiments of this specification, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0118] In addition, an embodiment of this specification also provides a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to perform the steps of the quantization method or data processing method of the neural network model according to various embodiments of this specification described in the above "Example Method" section of this specification.

[0119] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0120] It should be understood that the specific examples in this document are only intended to help those skilled in the art better understand the implementation methods of this specification, and are not intended to limit the scope of this specification.

[0121] It can be understood that in the various implementations of this specification, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the implementation methods of this specification.

[0122] It can be understood that the various embodiments described in this specification can be implemented individually or in combination, and the embodiments in this specification are not limited to this.

[0123] Unless otherwise indicated, all technical and scientific terms used in the embodiments of this specification have the same meaning as those commonly understood by those skilled in the art in the technical field of this specification. The terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the scope of this specification. The term "and / or" used in this specification includes any and all combinations of one or more related listed items. The singular forms "a", "above", and "the" used in the embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise.

[0124] It is understood that the processor in the embodiments of this specification can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiment can be completed by hardware integrated logic circuits in the processor or software instructions. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this specification can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this specification can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0125] It will be understood that the memory in the embodiments of this specification may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (Programmable ROM, PROM), an erasable programmable read-only memory (Erasable PROM, EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory may be a random access memory (RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0126] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this specification.

[0127] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, devices and units can refer to the corresponding processes in the aforementioned method implementation methods and will not be repeated here.

[0128] In the several embodiments provided in this specification, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0129] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of this embodiment.

[0130] In addition, each functional unit in each embodiment of this specification may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0131] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this specification, or the part that contributes to the prior art, or the part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of this specification. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0132] The above description is merely a specific embodiment of this specification, but the scope of protection of this specification is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this specification should be included in the scope of protection of this specification. Therefore, the scope of protection of this specification should be based on the scope of protection of the claims.

Claims

1. A quantization method for a neural network model, characterized in that: include: Acquire a neural network model trained with target data, wherein the target data includes image data and / or speech data; Performing quantization training on the weights of the neural network model to obtain a plurality of first weights; the first weights are represented in an integer format; Taking minimizing the loss function as the objective function and optimizing the expectation as the constraint condition, in multiple iterations, thinning out some of the multiple first weights into second weights to obtain a target neural network model, wherein the accuracy of the first weights is greater than the accuracy of the second weights; the target neural network model is used to obtain a processing result based on the input speech and / or image; The loss function includes a regularization term, which is used to characterize the performance of the neural network model. The regularization term includes a penalty rule when the first weight is thinned to the second weight. The optimization expectation includes the ratio of the number of the second weights to the total number of weights in the neural network model.

2. The method according to claim 1, characterized in that The step of quantizing and training the weights of the neural network model to obtain a plurality of first weights comprises: Using a quantization-aware training method, the neural network model is quantized with a first precision to obtain a quantization parameter of a neural network model mapped to the first precision; Using the quantization parameter, converting the weight of the neural network model from a floating point format to the first weight of a first precision; The neural network model is gradient-backed using approximate scaling of the weight gradient to update the first weight of the neural network model; the approximate scaling of the weight gradient includes the product of the weight gradient and the quantization parameter.

3. The method according to claim 1, characterized in that The method of taking minimizing the loss function as the objective function and optimizing the expectation as the constraint condition, and thinning out some of the multiple first weights into second weights in multiple iterations to obtain the target neural network model includes: Using an alternating direction multiplier method, minimizing the loss function is used as the objective function, and alternately performing the first step, the second step, and the third step in multiple iterations; The first step includes using the Lagrange multiplier method to fix the optimization expectation and iterate the loss function. The second step includes using the Lagrange multiplier method to fix the loss function and iterate the optimization expectation. The third step includes fixing the optimization expectation and the loss function and updating the scaling of the Lagrange multiplier. The scaling of the Lagrange multiplier is used to balance the magnitude of the objective function and the constraint conditions.

4. The method according to claim 1, characterized in that: The process of thinning the first weight to the second weight includes: The N bits of data in the first weight are discarded, and the retained continuous multiple bits of data are used as the second weight.

5. The method according to claim 4, characterized in that The regularization term includes: In the process of thinning the first weight to the second weight, if the discarding logic of N-bit data in the first weight does not meet the preset logic, a penalty value is assigned to the second weight.

6. A data processing method, characterized in that: include: Acquiring data to be processed, wherein the data to be processed includes: voice and / or image; In response to the processing instruction, the data to be processed is input into the target neural network model to obtain a processing result; The target neural network model includes quantizing a neural network model trained with target data, wherein the target data includes: image data and / or speech data; The quantification process includes: Performing quantization training on the weights of the neural network model to obtain a plurality of first weights; the first weights are represented in an integer format; Taking minimizing the loss function as the objective function and optimizing the expectation as the constraint condition, in multiple iterations, thinning out some of the multiple first weights into second weights to obtain a target neural network model, wherein the accuracy of the first weights is greater than the accuracy of the second weights; the target neural network model is used to obtain a processing result based on the input speech and / or image; The loss function includes a regularization term, which is used to characterize the performance of the neural network model. The regularization term includes a penalty rule when the first weight is thinned to the second weight. The optimization expectation includes the ratio of the number of the second weights to the total number of weights in the neural network model.

7. The method according to claim 6, characterized in that The processing instruction carries a target processing mode, and in response to the processing instruction, inputting the data to be processed into a target neural network model to obtain a processing result includes: In response to the processing instruction, the data to be processed is input into a target neural network model corresponding to the target processing mode to obtain a processing result output by the target neural network model; The target processing method includes: at least one of image segmentation processing, image feature extraction processing, image repair processing, image reconstruction processing, speech recognition processing, speech synthesis processing, speech feature extraction processing, speech repair processing and speech reconstruction processing.

8. A quantization device for a neural network model, characterized in that: include: A model acquisition module, used to acquire a neural network model trained with target data, wherein the target data includes image data and / or speech data; A quantization training module, used for performing quantization training on the weights of the neural network model to obtain a plurality of first weights; the first weights are represented in an integer format; A weight coefficient module, which is used to take minimizing the loss function as the objective function and optimizing the expectation as the constraint condition, and in multiple iterations, thin out some of the multiple first weights into second weights to obtain a target neural network model, wherein the accuracy of the first weights is greater than the accuracy of the second weights; the target neural network model is used to obtain a processing result based on the input speech and / or image; The loss function includes a regularization term, which is used to characterize the performance of the neural network model. The regularization term includes a penalty rule when the first weight is thinned to the second weight. The optimization expectation includes the ratio of the number of the second weights to the total number of weights in the neural network model.

9. A data processing device, characterized in that: include: A data acquisition module, used to acquire data to be processed, wherein the data to be processed includes: voice and / or image; A data processing module, used for inputting the data to be processed into a target neural network model in response to a processing instruction to obtain a processing result; The target neural network model includes quantizing a neural network model trained with target data, wherein the target data includes: image data and / or speech data; The quantification process includes: Performing quantization training on the weights of the neural network model to obtain a plurality of first weights; the first weights are represented in an integer format; Taking minimizing the loss function as the objective function and optimizing the expectation as the constraint condition, in multiple iterations, thinning out some of the multiple first weights into second weights to obtain a target neural network model, wherein the accuracy of the first weights is greater than the accuracy of the second weights; the target neural network model is used to obtain a processing result based on the input speech and / or image; The loss function includes a regularization term, which is used to characterize the performance of the neural network model. The regularization term includes a penalty rule when the first weight is thinned to the second weight. The optimization expectation includes the ratio of the number of the second weights to the total number of weights in the neural network model.

10. A computing device, characterized in that include: Processor and memory; Wherein, the memory is connected to the processor, and the memory is used to store a computer program; The processor is used to implement the quantization method of the neural network model as described in any one of claims 1 to 5 or the data processing method as described in any one of claims 6 to 7 by running the computer program stored in the memory.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the computer program implements the quantization method of the neural network model according to any one of claims 1 to 5 or the data processing method according to any one of claims 6 to 7.

Citation Information

Patent Citations

  • Neural network model compression method and device based on compressed sensing and storage medium

    CN112352249A

  • Neural network model compression method, device and system

    CN115496207A

  • Quantitative training method and device of model, electronic equipment and readable storage medium

    CN116070689A

  • Neural network model quantification method, data processing method and related device

    CN116562345A

  • Method and Apparatus for Training Neural Network and Computer Server

    US20190385059A1

Cited By

  • Compression method, system and equipment of speech synthesis model

    CN121075309A

  • Suspension control-oriented lightweight neural network model deployment and real-time calculation method

    CN121189393A