Neural network model quantization method and apparatus, computer device, and storage medium

Abstract

The application relates to a neural network model quantization method and device, computer equipment and a storage medium. The method comprises the following steps: obtaining second-order partial derivative information corresponding to a target layer of an initial neural network model according to a backward calculation graph of the initial neural network model and training samples; wherein the target layer is a network layer participating in training in the initial neural network model; and obtaining target bit positions for quantizing each network layer of the initial neural network model according to the second-order partial derivative information of the target layer and a hardware constraint condition corresponding to the initial neural network model. The method can improve the quantization precision of the neural network model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of neural network, and particularly relates to a neural network model quantization method and device, computer equipment and a storage medium. BACKGROUND

[0002] With the wide application of deep learning algorithms in the fields of computer vision, speech processing, automatic driving and the like, neural network models are designed to be increasingly large to meet complex applications, high precision, multi-task and the like application scenarios. However, when deploying a neural network model on an edge device or a mobile platform which has higher limitations on power, delay, memory and the like resources, it is often necessary to perform quantization processing on the neural network model.

[0003] In the prior art, when quantizing a neural network model, a higher precision bit width is allocated to a more sensitive layer in the neural network model, and a lower precision bit width is allocated to a less sensitive layer in the neural network model, so as to obtain a quantized neural network model.

[0004] However, the traditional method has the problem of low quantization precision. SUMMARY

[0005] The embodiments of the present application provide a neural network model quantization method, device, computer equipment and storage medium, which can improve the quantization precision of a neural network model.

[0006] In a first aspect, the embodiments of the present application provide a neural network model quantization method, comprising:

[0007] According to the backpropagation graph of the initial neural network model and the training sample, second-order derivative information corresponding to a target layer of the initial neural network model is obtained; wherein the target layer is a network layer participating in training in the initial neural network model;

[0008] According to the second-order derivative information of the target layer and a hardware constraint condition corresponding to the initial neural network model, target bit widths of each network layer of the initial neural network model are obtained.

[0009] In a second aspect, the embodiments of the present application provide a neural network model quantization device, comprising:

[0010] A first obtaining module is configured to obtain, according to a backpropagation graph of an initial neural network model and a training sample, second-order derivative information corresponding to a target layer of the initial neural network model; wherein the target layer is a network layer participating in training in the initial neural network model;

[0011] The second obtaining module is configured to obtain target bit numbers of each network layer of the initial neural network model according to the second-order partial derivative information of the target layer and a hardware constraint condition corresponding to the initial neural network model.

[0012] In a third aspect, an embodiment of the present application provides a computer device, including a memory and a processor, the memory storing a computer program, and the computer program, when executed by the processor, causes the processor to perform the steps of the neural network model quantization method in the first aspect.

[0013] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program, when executed by a processor, implements the steps of the method in the first aspect.

[0014] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, and the computer program, when executed by a processor, implements the steps of the method in the first aspect.

[0015] The neural network model quantization method, device, computer device and storage medium can obtain the second-order partial derivative information of the target layer participating in training in the initial neural network model according to the backward computation graph of the initial neural network model and the training sample, and can obtain the target bit numbers of each network layer of the initial neural network model according to the second-order partial derivative information of the target layer in the initial neural network model and the hardware constraint condition corresponding to the initial neural network model. Since the hardware constraint condition corresponding to the initial neural network model and the second-order partial derivative information of the target layer participating in training in the initial neural network model are considered in the process of obtaining the target bit numbers of each network layer of the initial neural network model, more fine-grained configuration can be performed under the hardware constraint condition corresponding to the initial neural network model, the precision of the target bit numbers of each network layer of the initial neural network model obtained is high, and thus the precision of the quantization of the initial neural network model by using the target bit numbers of each network layer obtained is improved. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0017] Figure 1 A flowchart of the neural network model quantization method in an embodiment;

[0018] Figure 2A flow chart of a neural network model quantization method in another embodiment;

[0019] Figure 3 A flow chart of a neural network model quantization method in another embodiment;

[0020] Figure 4 A structural block diagram of a neural network model quantization device in one embodiment;

[0021] Figure 5 A structural block diagram of a neural network model quantization device in another embodiment;

[0022] Figure 6 A structural block diagram of a neural network model quantization device in another embodiment;

[0023] Figure 7 A structural block diagram of a neural network model quantization device in another embodiment;

[0024] Figure 8 A structural block diagram of a neural network model quantization device in another embodiment;

[0025] Figure 9 A structural block diagram of a neural network model quantization device in another embodiment;

[0026] Figure 10 A structural block diagram of a neural network model quantization device in another embodiment;

[0027] Figure 11 A structural schematic diagram of a computer device in one embodiment. DETAILED DESCRIPTION

[0028] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0029] In one embodiment, as shown in Figure 1 a neural network model quantization method is provided, and the present embodiment takes the method applied to a computer device as an example for illustration. It should be understood that the method can also be applied to a server. In the present embodiment, the method includes the following steps:

[0030] S201, obtaining second-order derivative information corresponding to a target layer of an initial neural network model according to a backward computation graph of the initial neural network model and a training sample; wherein the target layer is a network layer participating in training in the initial neural network model.

[0031] Firstly, it needs to be explained that the initial neural network model in the embodiment is a trained neural network model, that is, the network structure of the initial neural network model has been determined. Optionally, the initial neural network model in the embodiment can be a neural network model used in the fields of image processing, speech processing, natural language processing, control processing, etc. Optionally, the initial neural network model can be a floating-point type (neural network model with floating-point parameters), or a fixed-point model, an integer model, etc. with any bit width (for example, 16-bit, 12-bit, 10-bit, 8-bit, etc.). In addition, the backpropagation graph of the initial neural network model in the embodiment is obtained by performing backpropagation differential processing on the weight parameters of each layer of the initial neural network model and the input parameters of each layer.

[0032] It can be understood that some of all the network layers constituting the neural network model do not participate in the training process of the neural network model. Optionally, in the embodiment, the network layers participating in the training of the initial neural network model can be determined as target layers according to the labels of each network layer of the initial neural network model. Further, as an optional implementation, in the embodiment, the training samples can be input into the initial neural network model, and the second-order derivative information corresponding to the target layers of the initial neural network model can be obtained according to the backpropagation graph of the initial neural network model and the outputs of each network layer of the initial neural network model. Optionally, the training samples in the embodiment can be the samples used for training the initial neural network model, or a part of the samples used for training the initial neural network model, or a part of the samples randomly selected from the samples used for training the initial neural network model, or new training samples different from the samples used for training the initial neural network model.

[0033] S202, according to the second-order derivative information of the target layer and the hardware constraint condition corresponding to the initial neural network model, obtaining the target bit of each network layer of the quantized initial neural network model.

[0034] It can be understood that in the embodiment, the initial neural network model is quantized by using the target bit of each network layer, which can compress the initial neural network model, so that the quantized neural network model is more lightweight and suitable for deployment on devices with higher limitations on power, delay, memory, etc.

[0035] Optionally, in the embodiment, the optimization function can be constructed according to the second-order derivative information of the target layer and the quantization bit of each network layer of the initial neural network model under the constraint of the hardware constraint condition corresponding to the initial neural network model, and the optimization function is solved to obtain the target bit of each network layer of the quantized initial neural network model. Optionally, the target bit of each network layer of the initial neural network model obtained can be the bit corresponding to each network layer of the initial neural network model under the above hardware constraint condition and the minimum second-order derivative information of the target layer. Optionally, the target bits of each network layer of the initial neural network model can be the same or different. For example, the target bits of each network layer of the initial neural network model can be 2, 4, 6, 8, etc. integer multiples of 2 bits.

[0036] In the above neural network model quantization method, the second-order derivative information corresponding to the target layer participating in the training in the initial neural network model can be obtained according to the backpropagation graph of the initial neural network model and the training sample, so that the bits of each network layer of the quantized initial neural network model can be obtained according to the second-order derivative information of the target layer in the initial neural network model and the hardware constraint condition corresponding to the initial neural network model. Since the hardware constraint condition corresponding to the initial neural network model and the second-order derivative information of the target layer participating in the training in the initial neural network model are considered in the process of obtaining the target bits of each network layer of the quantized initial neural network model, the initial neural network model can be configured in a more fine-grained manner under the hardware constraint condition corresponding to the initial neural network model, and the target bits of each network layer of the quantized initial neural network model with high precision can be obtained, thereby improving the quantization precision of the initial neural network model with the obtained target bits of each network layer.

[0037] The embodiment will describe in detail the process of obtaining the target bits of each network layer of the quantized initial neural network model according to the second-order derivative information of the target layer in the initial neural network model and the hardware constraint condition corresponding to the initial neural network model. In one embodiment, S202 includes minimizing the second-order derivative information of the target layer under the hardware resource constraint condition corresponding to the initial neural network model, and obtaining the target bits of each network layer of the quantized initial neural network model.

[0038] Optionally, in this embodiment, the hardware resource constraint condition corresponding to the initial neural network model can be used as a constraint to establish an optimization function including the second-order derivative information of the target layer of the initial neural network model and the bit of each network layer of the initial neural network model, and the bit of each network layer of the initial neural network model is obtained by minimizing the second-order derivative information of the target layer of the initial neural network model and solving the optimization function. The bit of each network layer of the initial neural network model corresponding to the minimum second-order derivative information of the target layer of the initial neural network model is determined as the target bit of each network layer of the initial neural network model.

[0039] As an optional implementation, in this embodiment, the amount of hardware resources consumed by each network layer of the initial neural network model under different bits, including but not limited to power consumption, memory, etc., can be obtained in advance, and then the target performance index required to be achieved by the initial neural network model and the amount of hardware resources consumed by each network layer of the initial neural network model under different bits are used to determine the hardware resource constraint condition corresponding to the initial neural network model when the target performance index is achieved. For example, the minimum amount of hardware resources consumed by the initial neural network model under different bits when achieving the target performance can be determined as the hardware resource constraint condition corresponding to the initial neural network model.

[0040] In this embodiment, by minimizing the second-order derivative information of the target layer participating in training in the initial neural network model under the hardware resource constraint condition corresponding to the initial neural network model, each network layer of the initial neural network model can be configured more granularly, and the target bit of each network layer of the quantized initial neural network model with high precision can be obtained, thereby improving the precision of the obtained target bit of each network layer of the quantized initial neural network model.

[0041] In some scenarios, after obtaining the target bit of each network layer of the quantized initial neural network model, the initial neural network model can be quantized using the target bit, and when the precision of the quantized neural network model does not meet the preset precision condition, the target bit of each network layer of the quantized initial neural network model is adjusted. In one embodiment, as shown in Figure 2 The above method further includes:

[0042] S301, quantizing each network layer of the initial neural network model using each target bit to obtain a quantized neural network model.

[0043] In this embodiment, after obtaining the target bit of each network layer of the initial neural network model, the target bit can be used to quantize each network layer of the initial neural network model, that is, the bit of each network layer of the initial neural network model is modified to the corresponding target bit, and a quantized neural network model is obtained. Further, the accuracy of the quantized neural network model can be tested by using a test data set to determine whether the accuracy of the quantized neural network model meets the preset accuracy condition. For example, if the test data set is input into the quantized neural network model, the error between the output obtained and the gold standard data corresponding to the test data set is small, it can be determined that the accuracy of the quantized neural network model meets the preset accuracy condition, otherwise, if the error between the output obtained and the gold standard data corresponding to the test data set is large, it can be determined that the accuracy of the quantized neural network model does not meet the preset accuracy condition.

[0044] In S302, if the accuracy of the quantized neural network model does not meet the preset accuracy condition, a new training sample is used as a training sample, and the step of obtaining the second-order derivative information corresponding to the target layer of the initial neural network model according to the reverse calculation graph and the training sample is performed again.

[0045] Optionally, in this embodiment, the new training sample can be a part of the sample used for training the initial neural network model, or it can also be a part of the sample randomly selected from the sample used for training the initial neural network model, or the new training sample can be a sample different from the sample used for training the initial neural network model, which is not limited in this embodiment.

[0046] In this embodiment, if it is determined that the accuracy of the quantized neural network model does not meet the preset accuracy condition, the above-mentioned new training sample can be used as the training sample in step S201, and step S201 is performed again to obtain new second-order derivative information corresponding to the target layer of the initial neural network model, and based on the newly obtained second-order derivative information corresponding to the target layer and the hardware constraint condition corresponding to the initial neural network model, new target bits of each network layer of the quantized initial neural network model are obtained, and the initial neural network model is quantized using the new target bits of each network layer, until the accuracy of the quantized neural network model meets the above-mentioned preset accuracy condition.

[0047] In this embodiment, the target bit of each network layer of the initial neural network model is used to quantize each network layer of the initial neural network model, and a quantized neural network model can be obtained. Therefore, when the accuracy of the quantized neural network model does not meet the preset accuracy condition, the new second-order derivative information corresponding to the target layer of the initial neural network model can be reacquired according to the corresponding back propagation graph and the new training sample of the initial neural network model, and the new target bit of each network layer of the initial neural network model can be reacquired. Through the above steps, the quantized neural network model obtained finally is the neural network model that meets the preset accuracy condition, and the accuracy of the quantized neural network model is ensured.

[0048] In some scenarios, after obtaining the quantized neural network model, the quantized neural network model can be further compressed. In one embodiment, the above method further includes: compressing the quantized neural network model; and the compression includes at least one of pruning and parameter sharing.

[0049] In the pruning of the neural network model, based on the premise that the trained neural network model has certain redundancy, according to the compression ratio requirement of the neural network model compression task, the compression ratio is allocated to all weight layers (such as convolution layer Conv, fully connected layer Dense, etc.) that need to be pruned in the neural network model according to a certain allocation algorithm, and the importance of the weights in different dimensions of each weight layer is measured according to a certain metrics, the weights with low importance are deleted, and the weights with high importance are retained, so that the original neural network model becomes a new neural network model with less parameters, and then the weights of the new neural network model are retrained and fine-tuned to make the performance close to or even better than that of the original neural network model. In the parameter sharing of the neural network model, based on the premise that the trained neural network model has certain parameter redundancy, the network layers with redundant parameters in the neural network model share the parameters to reduce the memory and calculation amount required for running the neural network model.

[0050] In this embodiment, after obtaining the quantized neural network model, the quantized neural network model can be further pruned, or the quantized neural network model can be further subjected to parameter sharing, or the quantized neural network model can be further subjected to pruning and parameter sharing at the same time. Further, as an optional implementation, after obtaining the quantized neural network model, the quantized neural network model can be preprocessed for actual deployment according to actual deployment.

[0051] In this embodiment, by performing compression processing on the quantized neural network model, the memory and computation required for running the neural network model can be further reduced, so that the processed neural network model is more suitable for deployment on devices with high constraints on power, delay, memory and other resources.

[0052] In the above scenario of obtaining the second-order partial derivative information corresponding to the target layer of the initial neural network model according to the initial neural network model reverse computation graph and the training sample, the initial neural network model reverse computation graph needs to be obtained first. The detailed process of obtaining the initial neural network model reverse computation graph will be described in this embodiment. In one embodiment, the above method further comprises: differentiating the weight parameters of each layer of the initial neural network model and the input parameters of each layer according to the value of the loss function of the initial neural network model and the network structure of the initial neural network model, and constructing the reverse computation graph.

[0053] In this embodiment, according to the network structure of the initial neural network model, the output of the initial neural network model and the value of the loss function can be used as the input parameters of the last network layer of the initial neural network model. The weight parameters and input parameters of the last network layer of the initial neural network model are differentiated in reverse, the input of the last layer of the initial neural network model is used as the input parameters of the previous network layer of the last network layer of the initial neural network model, the weight parameters and input parameters of the previous network layer of the last network layer of the initial neural network model are differentiated in reverse, and the above operation is performed on each layer of the initial neural network model. The weight parameters of each layer of the initial neural network model and the input parameters of each layer are differentiated in reverse to obtain the differentiation results of each layer of the initial neural network model. According to the differentiation results of each layer of the initial neural network model, the reverse computation graph of the initial neural network model is constructed.

[0054] In this embodiment, the process of differentiating the weight parameters of each layer of the initial neural network model and the input parameters of each layer according to the value of the loss function of the initial neural network model and the network structure of the initial neural network model is relatively simple, which can quickly construct the reverse computation graph of the initial neural network model, and improve the efficiency of constructing the reverse computation graph of the initial neural network model.

[0055] The process of obtaining the second-order partial derivative information corresponding to the target layer of the initial neural network model according to the initial neural network model reverse computation graph and the training sample will be described in detail in this embodiment. In one embodiment, S201 comprises: differentiating the weight parameters of the target layer according to the reverse computation graph and the training sample to obtain the second-order partial derivative information corresponding to the target layer.

[0056] In the embodiment, the training sample can be input into the initial neural network model, and the weight parameters of the target layer participating in the training in the initial neural network model are differentiated according to the reverse calculation graph of the initial neural network model obtained above, to obtain the second-order partial derivative information of the loss function of the target layer in the initial neural network model with respect to the weight parameters.

[0057] In the embodiment, the weight parameters of the target layer of the initial neural network model can be differentiated quickly according to the reverse calculation graph of the initial neural network model and the training sample, thereby improving the efficiency of obtaining the second-order partial derivative information corresponding to the target layer of the initial neural network model.

[0058] For the convenience of those skilled in the art, as shown in the accompanying drawings, Figure 3 The neural network model quantization method provided by the present disclosure is described in detail below, which can include:

[0059] S1, differentiating the weight parameters of each layer of the initial neural network model and the input parameters of each layer according to the value of the loss function of the initial neural network model and the network structure of the initial neural network model, and constructing a reverse calculation graph of the initial neural network model.

[0060] S2, differentiating the weight parameters of the target layer according to the reverse calculation graph and the training sample, to obtain the second-order partial derivative information corresponding to the target layer of the initial neural network model; wherein the target layer is a network layer participating in the training in the initial neural network model.

[0061] S3, determining the hardware resource constraint condition corresponding to the initial neural network model according to the amount of hardware resources consumed by each network layer of the initial neural network model under different bit positions and the target performance indicator of the initial neural network model.

[0062] S4, minimizing the second-order partial derivative information of the target layer under the hardware resource constraint condition corresponding to the initial neural network model, to obtain the target bit position of each network layer of the quantized initial neural network model.

[0063] S5, quantizing each network layer of the initial neural network model using each target bit position to obtain a quantized neural network model.

[0064] S6, if the accuracy of the quantized neural network model does not satisfy the preset accuracy condition, a new training sample is used as the training sample, and the step of differentiating the weight parameters of the target layer according to the reverse calculation graph and the training sample to obtain the second-order partial derivative information corresponding to the target layer of the initial neural network model is executed again.

[0065] S7, performing compression processing on the quantized neural network model; the compression processing includes at least one of pruning processing and parameter sharing processing.

[0066] It should be noted that the description in the above steps can refer to the description of the related description in the above embodiments, and the effect is similar, and the present embodiment will not be repeated here.

[0067] It should be understood that although each step in the flowchart involved in the above embodiments is displayed in sequence according to the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise stated herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in the above embodiments can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately executed with other steps or steps or stages in other steps.

[0068] Based on the same inventive concept, the embodiments of the present application also provide a neural network model quantization device for implementing the above-mentioned neural network model quantization method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more neural network model quantization device embodiments provided below can refer to the limitations of the neural network model quantization method in the above, and will not be repeated here.

[0069] In one embodiment, as shown in Figure 4 A neural network model quantization device is provided, comprising: a first acquisition module 10 and a second acquisition module 11, wherein:

[0070] The first acquisition module 10 is configured to acquire second-order derivative information corresponding to a target layer of an initial neural network model according to a backpropagation graph of the initial neural network model and a training sample; wherein the target layer is a network layer participating in training in the initial neural network model.

[0071] The second acquisition module 11 is configured to acquire target bit positions of each network layer of a quantized initial neural network model according to the second-order derivative information of the target layer and a hardware constraint condition corresponding to the initial neural network model.

[0072] The neural network model quantization device provided in the present embodiment can execute the above-mentioned method embodiments, and the implementation principles and technical effects are similar, which will not be repeated here.

[0073] On the basis of the above embodiments, as shown in Figure 5 Optionally, the second acquisition module 11 comprises a first acquisition unit 111, wherein:

[0074] The first obtaining unit 111 is configured to obtain quantized target bit positions of each network layer of the initial neural network model by minimizing the second-order partial derivative information of the target layer under the hardware resource constraint condition corresponding to the initial neural network model.

[0075] The neural network model quantization apparatus provided in the embodiment can execute the method embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0076] Based on the above-mentioned embodiments, as shown in Figure 6 Optionally, the apparatus further includes a determination module 12, wherein:

[0077] The determination module 12 is configured to determine the hardware resource constraint condition corresponding to the initial neural network model according to the amount of hardware resources consumed by each network layer of the initial neural network model under different bit positions and the target performance indicator of the initial neural network model.

[0078] The neural network model quantization apparatus provided in the embodiment can execute the method embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0079] Based on the above-mentioned embodiments, as shown in Figure 7 Optionally, the apparatus further includes a quantization module 13 and an operation module 14, wherein:

[0080] The quantization module 13 is configured to quantize each network layer of the initial neural network model by using each target bit position to obtain a quantized neural network model.

[0081] The operation module 14 is configured to, if the accuracy of the quantized neural network model does not satisfy the preset accuracy condition, take a new training sample as the training sample, and return to execute the step of obtaining the second-order partial derivative information corresponding to the target layer of the initial neural network model according to the backward calculation graph and the training sample.

[0082] The neural network model quantization apparatus provided in the embodiment can execute the method embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0083] Based on the above-mentioned embodiments, as shown in Figure 8 Optionally, the apparatus further includes a processing module 15, wherein:

[0084] The processing module 15 is configured to perform compression processing on the quantized neural network model, and the compression processing includes at least one of pruning processing and parameter sharing processing.

[0085] The neural network model quantization apparatus provided in the embodiment can execute the method embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0086] On the basis of the above-mentioned embodiments, as shown in Figure 9 Optionally, the apparatus further includes a construction module 16, wherein:

[0087] The construction module 16 is configured to perform differential processing on the weight parameters of each layer and the input parameters of each layer of the initial neural network model according to the value of the loss function of the initial neural network model and the network structure of the initial neural network model, and construct a reverse calculation graph.

[0088] The neural network model quantization apparatus provided in the embodiment can execute the method embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0089] On the basis of the above-mentioned embodiments, as shown in Figure 10 Optionally, the first acquisition module 10 includes a second acquisition unit 101, wherein:

[0090] The second acquisition unit 101 is configured to perform differential processing on the weight parameters of the target layer according to the reverse calculation graph and the training sample, and obtain the second-order partial derivative information corresponding to the target layer.

[0091] The neural network model quantization apparatus provided in the embodiment can execute the method embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0092] The modules in the neural network model quantization apparatus can be all or partially implemented by software, hardware or a combination thereof. The modules can be embedded in or independent of a processor in a computer device in a hardware form, or stored in a memory in a computer device in a software form, so as to be called and executed by a processor to perform the operations corresponding to the modules.

[0093] In one embodiment, a computer device is provided, which can be a server, and an internal structure diagram of the computer device can be as shown in Figure 11As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through the system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the target bit of each network layer of the initial neural network model. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with the terminal outside through the network connection. The computer program is executed by the processor to implement a neural network model quantization method.

[0094] Those skilled in the art can understand that, Figure 11 The skilled in the art can understand that,

[0095] The embodiment of the present application also provides a computer readable storage medium. One or more non-volatile computer readable storage media containing computer executable instructions, when the computer executable instructions are executed by one or more processors, make the processor execute the steps of the neural network model quantization method.

[0096] The embodiment of the present application also provides a computer program product containing instructions, when it runs on the computer, makes the computer execute the neural network model quantization method.

[0097] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0098] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0099] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A neural network model quantization method, characterized in that, The method comprises the following steps: According to the network structure of the initial neural network model, the output of the initial neural network model and the value of the loss function are taken as the input parameters of the last network layer of the initial neural network model, the weight parameters and the input parameters of the last network layer of the initial neural network model are processed by reverse differentiation, the input of the last layer of the initial neural network model is taken as the input parameters of the previous network layer of the last network layer of the initial neural network model, the weight parameters and the input parameters of the previous network layer of the last network layer of the initial neural network model are processed by reverse differentiation, and the process is repeated for each layer of the initial neural network model, so that the weight parameters of each layer of the initial neural network model and the input parameters of each layer are processed by reverse differentiation to obtain the differential results of each layer of the initial neural network model, and a reverse calculation graph of the initial neural network model is constructed according to the differential results of each layer of the initial neural network model; According to the reverse calculation graph and the training sample, the second-order partial derivative information corresponding to the target layer of the initial neural network model is obtained; wherein the target layer is a network layer participating in the training of the initial neural network model; The sum of the second-order partial derivative information of the target layer is minimized under the constraint condition of the hardware resources corresponding to the initial neural network model, and the target bit of each network layer of the initial neural network model is obtained.

2. The method of claim 1, wherein, The sum of the second-order partial derivative information of the target layer is minimized under the constraint condition of the hardware resources corresponding to the initial neural network model, and the target bit of each network layer of the initial neural network model is obtained. The constraint condition of the hardware resources corresponding to the initial neural network model is taken as a constraint, an optimization function including the sum of the second-order partial derivative information of the target layer of the initial neural network model and the bit of each network layer of the initial neural network model is established under the constraint, the optimization function is solved by minimizing the sum of the second-order partial derivative information of the target layer of the initial neural network model, the bit of each network layer of the initial neural network model corresponding to the minimum sum of the second-order partial derivative information of the target layer of the initial neural network model is determined as the target bit of each network layer of the initial neural network model.

3. The method of claim 2, wherein, The method further comprises: According to the amount of hardware resources consumed by each network layer of the initial neural network model under different bits and the target performance index of the initial neural network model, the constraint condition of the hardware resources corresponding to the initial neural network model is determined.

4. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: Each network layer of the initial neural network model is quantized by using each target bit to obtain a quantized neural network model; If the accuracy of the quantized neural network model does not meet the preset accuracy condition, a new training sample is taken as the training sample, and the step of obtaining the second-order partial derivative information corresponding to the target layer of the initial neural network model according to the reverse calculation graph and the training sample is performed again.

5. The method of claim 4, wherein, The method further comprises: The compressed processing of the quantized neural network model includes at least one of pruning processing and parameter sharing processing.

6. The method according to any one of claims 1 to 3, characterized in that, The second-order partial derivative information corresponding to the target layer of the initial neural network model is obtained according to the back propagation graph of the initial neural network model and the training sample. The second-order partial derivative information corresponding to the target layer is obtained by differentiating the weight parameters of the target layer according to the back propagation graph and the training sample.

7. A neural network model quantization apparatus, comprising: The second-order partial derivative information corresponding to the target layer is obtained by differentiating the weight parameters of the target layer according to the back propagation graph and the training sample. The second-order partial derivative information corresponding to the target layer is obtained by differentiating the weight parameters of the target layer according to the back propagation graph and the training sample. The second-order partial derivative information corresponding to the target layer is obtained by differentiating the weight parameters of the target layer according to the back propagation graph and the training sample. The second-order partial derivative information corresponding to the target layer is obtained by differentiating the weight parameters of the target layer according to the back propagation graph and the training sample. 8.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is characterized in that, The second-order partial derivative information corresponding to the target layer is obtained by differentiating the weight parameters of the target layer according to the back propagation graph and the training sample.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to perform the steps of the neural network model quantization method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to perform the steps of the neural network model quantization method according to any one of claims 1 to 6. The computer program is executed by the processor to perform the steps of the neural network model quantization method according to any one of claims 1 to 6.

Images