A data quantization method, device and computer-readable storage medium based on neural network
By using adaptive iterative update quantization parameters and statistical distribution information optimization in neural networks, the problem of inapplicable quantization parameters in the existing technology is solved, the quantization performance and efficiency are improved, and it is suitable for edge computing.
Patent Information
- Application Number
- CN201911217982.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-12-03
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2039-12-03
AI Technical Summary
The quantization parameters used in neural networks in the prior art are not suitable for actual work scenarios and affect the quantization performance.
By using adaptive iteratively updated quantization parameters in the quantization operation of neural networks, the quantization parameters are updated iteratively according to the output activation in the sequence of frames to be measured and the quantization bit width iteratively updated. The quantization parameters are optimized using sliding average or alpha average processing, combined with statistical distribution information and peak-cutting processing determines the initial quantization boundaries and parameters.
It reduces quantization errors and improves the quantization effect. It is suitable for edge computing scenarios, saves storage space and improves data reading and writing efficiency.
Smart Images

Figure CN111008701B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of neural network computing, and in particular relates to a data quantization method and device based on a neural network, and a computer-readable storage medium. Background Art
[0002] This section is intended to provide a background or context to embodiments of the invention that are recited in the claims. No description herein is admitted to be prior art by inclusion in this section.
[0003] In recent years, with the rapid development of neural networks, the computing performance of neural networks has been continuously improved. However, due to the disadvantage of large amount of computation, neural networks are limited in their deployment and use on edge devices or low-power devices. Data quantization is one of the methods to solve the above problem. However, in the prior art, the quantization parameters used to quantize the data of neural networks may not be suitable for actual working scenarios, affecting the quantization performance. Summary of the invention
[0004] In view of the problem that the quantization parameters used in the quantization process of the neural network in the above-mentioned prior art may not be suitable for actual working scenarios, the embodiments of the present invention propose a data quantization method, device and computer-readable storage medium for a neural network, which can solve the above-mentioned problem.
[0005] The following solutions are provided in the embodiments of the present invention.
[0006] On the one hand, a data quantization method based on a neural network is provided, including: determining a first output activation of a target layer of a neural network according to any preset frame in a frame sequence to be tested, and performing a quantization operation on the first output activation based on a current quantization parameter; iteratively updating the current quantization parameter according to the first output activation of the target layer and a target quantization bit width; determining a second output activation of the target layer according to a next frame of any preset frame in the frame sequence to be tested, and performing a quantization operation on the second output activation based on the updated current quantization parameter.
[0007] In some possible implementations, if any preset frame is not the first frame in the frame sequence to be tested, the current quantization parameter is iteratively updated according to the first output activation of the target layer and the target quantization bit width, and it also includes: determining the third output activation of the target layer, the third output activation is associated with the previous frame or frames of any preset frame in the frame sequence to be tested; performing iterative update according to the first output activation, the third output activation and the target quantization bit width of the target layer.
[0008] In some possible implementations, the method further includes: using a preset sliding window to determine any preset frame and the previous frame or frames of any preset frame, and performing sliding average processing or alpha average processing on the current quantization parameter to iteratively update according to the preset sliding window.
[0009] In some possible implementations, the method further includes: if any preset frame is the first frame in the frame sequence to be tested, the current quantization parameter before updating is an initial quantization parameter determined in advance by inputting a correction set into the neural network.
[0010] In some possible implementations, the method further includes: inputting each sample in the correction set into the neural network in advance, and obtaining statistical distribution information of the output activation of the target layer; determining an initial quantization boundary of the target layer according to the statistical distribution information of the output activation of the target layer; and determining an initial quantization parameter according to the initial quantization boundary of the target layer and the target quantization bit width.
[0011] In some possible implementations, the initial quantization boundary of the target layer is determined based on the statistical distribution information of the output activation of the target layer, including: by calculating the relative entropy of the output activation of the target layer; and / or by obtaining the maximum and minimum values of the output activation of the target layer; and / or by performing peak clipping on the output activation of the target layer to determine the initial quantization interval of the target layer.
[0012] In some possible implementations, iteratively updating the current quantization parameter according to the first output activation of the target layer and the target quantization bit width also includes: performing iterative update according to the maximum / minimum value, maximum value interval / minimum value interval, or histogram information of the first output activation of the target layer.
[0013] In some possible implementations, the iterative update is a replacement update or a weighted update.
[0014] In a second aspect, a data quantization device based on a neural network is provided, comprising a quantization module and an update module, wherein the quantization module is used to determine a first output activation of a target layer of the neural network according to any preset frame in a frame sequence to be tested, and perform a quantization operation on the first output activation based on a current quantization parameter; the update module is used to iteratively update the current quantization parameter according to the first output activation of the target layer and a target quantization bit width; the quantization module is used to determine a second output activation of the target layer according to a next frame of any preset frame in a frame sequence to be tested, and perform a quantization operation on the second output activation based on the updated current quantization parameter.
[0015] In some possible implementations, the update module is used to: if any preset frame is not the first frame in the frame sequence to be tested, determine the third output activation of the target layer, and the third output activation is associated with the previous frame or frames of the any preset frame in the frame sequence to be tested; perform iterative update according to the first output activation, the third output activation and the target quantization bit width of the target layer.
[0016] In some possible implementations, the update module is used to: use a preset sliding window to determine any preset frame and the previous frame or frames of any preset frame, and perform sliding average processing or alpha average processing on the current quantization parameter according to the preset sliding window to perform iterative update.
[0017] In some possible implementations, the quantization module is used for: if any preset frame is the first frame in the frame sequence to be tested, the current quantization parameter before updating is an initial quantization parameter determined in advance by inputting a correction set into the neural network.
[0018] In some possible implementations, an initial parameter module is also included, which is used to: pre-input each sample in the correction set into the neural network and obtain the statistical distribution information of the output activation of the target layer; determine the initial quantization boundary of the target layer according to the statistical distribution information of the output activation of the target layer; determine the initial quantization parameter according to the initial quantization boundary of the target layer and the target quantization bit width.
[0019] In some possible implementations, the update module is used to: determine the initial quantization interval of the target layer by calculating the relative entropy of the output activation of the target layer; and / or by obtaining the maximum and minimum values of the output activation of the target layer; and / or by performing peak clipping on the output activation of the target layer.
[0020] In some possible implementations, the update module is used to perform iterative updates based on the maximum / minimum value, maximum value interval / minimum value interval, or histogram information of the first output activation of the target layer.
[0021] In some possible implementations, the iterative update is a replacement update or a weighted update.
[0022] In a third aspect, a data quantization device based on a neural network is provided, comprising: one or more multi-core processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more multi-core processors, the one or more multi-core processors are enabled to implement: determining a first output activation of a target layer of the neural network according to any preset frame in a frame sequence to be tested, and performing a quantization operation on the first output activation based on a current quantization parameter; iteratively updating the current quantization parameter according to the first output activation of the target layer and a target quantization bit width; determining a second output activation of the target layer according to a next frame of any preset frame in the frame sequence to be tested, and performing a quantization operation on the second output activation based on the updated current quantization parameter.
[0023] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a program, and when the program is executed by a multi-core processor, the multi-core processor executes the method of the first aspect as described above.
[0024] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: the embodiments of the present invention can reduce quantization errors and have better quantization effects by adopting adaptive iteratively updated quantization parameters in the quantization operation of the neural network.
[0025] It should be understood that the above description is only an overview of the technical solution of the present invention, so that the technical means of the present invention can be more clearly understood and implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described below by way of example. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] The advantages and benefits described herein and other advantages and benefits will be apparent to those of ordinary skill in the art by reading the detailed description of the exemplary embodiments below. The accompanying drawings are only used for the purpose of illustrating exemplary embodiments and are not to be considered as limiting the present invention. Moreover, the same reference numerals are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0027] Figure 1 It is a schematic diagram of the structure of a convolutional neural network;
[0028] Figure 2 A schematic diagram of a flow chart of a data quantization method according to an embodiment of the present invention;
[0029] Figure 3 is a schematic frame sequence to be tested;
[0030] Figure 4 is a flow chart of a data quantization method according to another embodiment of the present invention;
[0031] Figure 5is a structural schematic diagram of a data quantization device according to an embodiment of the present invention;
[0032] Figure 6 FIG. 4 is a schematic diagram of the structure of a data quantization device according to another embodiment of the present invention.
[0033] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION
[0034] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0035] In the present invention, it should be understood that terms such as “including” or “having” are intended to indicate the existence of features, numbers, steps, behaviors, components, parts or a combination thereof disclosed in the specification, and are not intended to exclude the possibility of the existence of one or more other features, numbers, steps, behaviors, components, parts or a combination thereof.
[0036] It should also be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0037] Figure 1 is a schematic diagram of the structure of a convolutional neural network 100. Figure 1 As shown, the convolutional neural network may have a layer structure, and the output of a specific layer may be the input of the next layer. For example, the convolutional neural network 100 may include multiple convolutional layers, the first layer to the nth layer L1, L2, L3, ... and Ln, and the output of each of the first layer to the nth layer L1, L2, L3, ... and Ln may be input to the next layer. In each layer of the convolutional neural network 100, a convolution module 101, a batch normalization module 102, and an activation module 103 are sequentially included, wherein the output of each activation module 103 is also the output of each convolutional layer, wherein, since there are a large number of multiplication and accumulation calculations in the calculation process performed in the convolution module 101, the batch normalization module 102 and the activation module 103, in order to ensure the calculation accuracy, the intermediate result needs to retain a result with a wider bit width, such as floating point 32 bits (hereinafter referred to as FP32) data, which results in occupying a large storage space and data read and write bandwidth.
[0038] Furthermore, in order to save storage space and data read and write bandwidth, this embodiment can quantize the output activation data of each layer of the convolutional neural network. The above quantization process can specifically be to quantize floating-point numbers (such as 32-bit floating-point numbers) to fixed-point numbers with lower bits, so as to compress the output activation data of each layer of the convolutional neural network to fixed-point 8-bit (hereinafter referred to as INT8) data or lower, thereby reducing the number of computing resources used. The embodiment of the present invention is preferably applied to edge computing scenarios. Specifically, in edge computing scenarios, since edge computing devices are usually subject to large computing resource constraints, it is generally necessary to use a quantized fixed-point implementation for the neural network, that is, each time a layer of convolution is calculated, the output of the activation module needs to be stored and read and written. The embodiment of the present invention uses adaptive quantization parameters to perform quantization processing on the output activation of each layer of convolution of the neural network, thereby saving storage space and improving write-in and write-out efficiency.
[0039] Figure 2 A schematic diagram of a method flow of a neural network-based data quantization method 200 according to an embodiment of the present invention is shown.
[0040] like Figure 2 As shown, the method 200 includes:
[0041] Step 201: determining a first output activation of a target layer of a neural network according to any preset frame in a frame sequence to be tested, and performing a quantization operation on the first output activation based on a current quantization parameter;
[0042] Step 202: iteratively update the current quantization parameter according to the first output activation of the target layer and the target quantization bit width;
[0043] Step 203: Determine the second output activation of the target layer according to the next frame of any preset frame in the frame sequence to be tested, and perform a quantization operation on the second output activation based on the updated current quantization parameter.
[0044] Specifically, the method shown in this embodiment can be applied to each convolution layer of a neural network. The target layer of the neural network can be any convolution layer in the neural network. For example, the neural network can be Figure 1 In the convolutional neural network 100 shown in FIG. , the target layer may be Figure 1Each of the first to nth layers L1, L2, L3, ... and Ln in the convolutional neural network 100 is shown. The above-mentioned frame sequence to be tested can be a data frame stream to be tested with time continuity. Among them, any one of the above-mentioned preset frames can be any one of the specified frame or specified multiple frames of data. When the neural network processes any one of the preset frames, an update of the current quantization parameter is performed once, so that when the neural network processes the next frame or subsequent multiple frames of the any one of the preset frames, the updated current quantization parameter is adopted until the next update. The current quantization parameters before or after the above-mentioned update all include quantization parameters for the target layer of the neural network. For example, the distribution range of the floating-point intermediate values output in layer 1 and the distribution range of the floating-point intermediate values used in layer 2 may be different from each other. In this embodiment, the output activation of the target layer of the neural network can be quantized into fixed-point expression values with different decimal lengths based on the above-mentioned current quantization parameters. The above-mentioned target quantization bit width is used to indicate the target quantization degree, for example, it can be 8 bits, 4 bits, etc. Optionally, any one of the above preset frames may also be inferred and determined according to a preset update rule, for example, the current quantization parameter may be updated once every five frames, etc., which is not specifically limited in the embodiment of the present invention.
[0045] For example, Figure 3 A frame sequence 30 to be tested is shown, including frame 1, frame 2, frame 3, ..., frame m. Assuming that frame 1, frame 3, etc. are preset frames, the embodiment of the present invention can be Figure 1 The convolutional neural network 100 shown sequentially inputs the frames 1, 2, 3, ..., m in the above-mentioned frame sequence to be tested to perform the neural network operation of each layer. In the neural network operation process for each frame of the data to be tested, the output activation of the target layer of the convolutional neural network is sequentially obtained, and the output activation of each layer is quantized according to the above-mentioned current quantization parameter, so as to achieve accelerated neural network operation. The above-mentioned current quantization parameter is not fixed in the neural network operation process, but it is iteratively updated after processing any preset frame. For example, after the above-mentioned neural network processes frame 1, the current quantization parameter can be updated once according to the output activation of the target layer of the neural network corresponding to frame 1 in the above-mentioned neural network operation process. Since the frame sequence to be tested has time continuity, the image content of the previous and next frame data itself also has continuity. Therefore, compared with the fixed quantization parameter determined by the sample data set in the online inference stage of the neural network, the embodiment of the present invention adopts the quantization parameter that can be adaptively iteratively updated in the actual operation of the neural network, which can reduce the quantization error and have a better quantization effect.
[0046] based on Figure 2 The neural network-based data quantization method, some embodiments of the present application also provide some specific implementation plans of the data quantization method, and extended solutions, which are described below.
[0047] In a possible implementation, if any one of the preset frames is not the first frame in the frame sequence to be tested, the above step 202 also includes: determining the third output activation of the target layer of the neural network, the third output activation being associated with the previous frame or frames of the any one of the preset frames in the frame sequence to be tested; and performing iterative update according to the first output activation, the third output activation and the target quantization bit width of the target layer of the neural network.
[0048] For example, if Figure 3 As shown, assuming that frame 1, frame 3, etc. are preset frames, after the above neural network processes frame 3, not only can the current quantization parameter be updated according to the output activation of the target layer of the neural network corresponding to frame 3 during the above neural network operation, but the update can also be performed based on one or more frames of data that have been processed by the neural network before frame 3. For example, the current quantization parameter can be updated in combination with the output activation of the target layer of the neural network corresponding to frame 2. For another example, the current quantization parameter can be updated in combination with the output activation of the target layer of the neural network corresponding to frames 1 and 2. Preferably, the quantization boundary can be comprehensively determined based on the maximum / minimum value, maximum value interval / minimum value interval, or histogram information of the first output activation and the third activation output of the target layer of the neural network to perform the above update.
[0049] In a possible implementation, step 202 may further include: using a preset sliding window to determine any preset frame and the previous frame or frames of any preset frame, and performing sliding average processing or alpha average processing on the current quantization parameter to perform iterative update according to the preset sliding window.
[0050] The step size and window length of the preset sliding window determine the arbitrary preset frame and the previous frame or frames of the arbitrary preset frame. For example, assuming that the step size of the preset sliding window is 2 and the window length is 3, the preset sliding window step size can be used to determine the previous frame or frames of the arbitrary preset frame. Figure 3 Frame 2, frame 4, frame 6, ..., etc. can be selected as preset frames, wherein it is assumed that any preset frame in this embodiment is frame 4, then the window data corresponding to frame 4 includes (frame 2, frame 3, frame 4), then the previous frame or frames of any preset frame are determined to be frame 2 and frame 3 according to the window length of the preset sliding window, and further, the first output activation can be obtained according to frame 4, and the third output activation can be obtained according to frame 2 and frame 3. Among them, the sliding average processing or alpha average processing can be used to estimate the local mean of the parameters used in the update process, so that the iterative update of the current quantization parameter is related to the value taken within a period of time, thereby making the update of the current quantization parameter smoother.
[0051] In a possible implementation, in the above step 201, if the any one preset frame is the first frame in the frame sequence to be tested, the current quantization parameter before updating is an initial quantization parameter determined in advance by inputting a correction set into the neural network.
[0052] For example, the calibration set may contain multiple samples, such as 500-1000 images, and then each sample in the calibration set may be input into the trained neural network for forward operation, and then the data distribution of the output activation of the target layer of the neural network is counted, and the initial quantization parameter is confirmed based on the statistical data distribution. Furthermore, the initial quantization parameter is used as the iteration initial value and iterative update is performed to determine the current quantization parameter.
[0053] Optionally, if any preset frame is the first frame in the frame sequence to be tested, the current quantization parameter before updating may also be other preset values, such as preset values determined based on experience.
[0054] In one possible implementation, Figure 4 A schematic diagram 400 of a process for determining an initial quantization parameter of a neural network is shown. The process may occur during the inference process of the neural network, such as Figure 4 As shown, including:
[0055] Step 401: input each sample in the calibration set into the neural network, and obtain the statistical distribution information of the output activation of the target layer of the neural network;
[0056] Step 402: determining an initial quantization boundary of the target layer according to statistical distribution information of output activations of the target layer;
[0057] Step 403: Determine an initial quantization parameter according to the initial quantization boundary of the target layer and the target quantization bit width:
[0058] For example, assuming that the output activation of the neural network is FP32 data, and it is hoped to quantize it to INT8 data, the above quantization process is equivalent to constructing a mapping relationship from FP32 data to INT8 data, and converting the initial quantization boundary of the floating-point type data [X min ,X max ] is mapped to the target interval of integer type data (for example, INT8 data is [-128, 127]). In this embodiment, the statistical distribution information can be formed by counting the output activation of each layer when the neural network performs neural network operation on each sample in the calibration set, and the initial quantization boundary [X] corresponding to the target layer of the neural network can be determined by using the statistical distribution information. min ,X max Of course, the initial quantization boundary can also be determined directly by using the maximum / minimum value of the output activation of the target layer of the neural network [Xmin ,X max ].
[0059] In a possible implementation, the above step 402 may further include: determining the initial quantization interval of the target layer by calculating the relative entropy of the statistical distribution information of the output activation of the target layer, and / or by obtaining the maximum and minimum values in the statistical distribution information of the output activation of the target layer, and / or by performing peak clipping on the statistical distribution information of the output activation of the target layer.
[0060] Specifically, the relative entropy is also called KL divergence (Kullback-Leibler divergence). The peak clipping of the statistical distribution information of the output activation of the target layer of the neural network may include discarding the top a% of data and discarding the bottom b% of data in the statistical distribution information, wherein the value range of a and b may be 0.5 to 4, thereby reducing the quantization error.
[0061] In a possible implementation, step 202 may also include: performing iterative updates according to the maximum / minimum value, maximum interval / minimum interval, or histogram information of the first output activation of the target layer. For example, the quantization boundary can be directly determined by the above-mentioned maximum / minimum value, or by calculating the median or mean of the interval in the maximum interval / minimum interval, where the maximum interval / minimum interval can be the first 10% and the last 10% of the first output activation of each layer. The quantization boundary can also be selected through histogram information so that data greater than a preset proportion (such as 90%) is within the quantization boundary.
[0062] In a possible implementation, the update includes: a replacement update or a weighted update. The replacement update refers to completely replacing the original current quantization parameter with the calculated quantization parameter to form an updated current quantization parameter. The weighted update refers to performing weighted accumulation on the calculated quantization parameter and the original current quantization parameter to form an updated current quantization parameter.
[0063] Based on the same technical concept, an embodiment of the present invention further provides a data quantization device based on a neural network, which is used to execute the data quantization method based on a neural network provided in any of the above embodiments. Figure 5 A schematic diagram of the structure of a data quantization device based on a neural network provided in an embodiment of the present invention.
[0064] like Figure 5As shown, the device 500 includes a quantization module 501 and an update module 502, wherein the quantization module 501 is used to determine the first output activation of the target layer of the neural network according to any preset frame in the frame sequence to be tested, and perform a quantization operation on the first output activation based on the current quantization parameter; the update module 502 is used to iteratively update the current quantization parameter according to the first output activation of the target layer and the target quantization bit width; and the quantization module 501 is also used to determine the second output activation of the target layer according to the next frame of any preset frame in the frame sequence to be tested, and perform a quantization operation on the second output activation based on the updated current quantization parameter.
[0065] In some possible implementations, the update module 502 is used to: if any preset frame is not the first frame in the frame sequence to be tested, determine the third output activation of the target layer, and the third output activation is associated with the previous frame or frames of the any preset frame in the frame sequence to be tested; perform iterative update according to the first output activation, the third output activation and the target quantization bit width of the target layer.
[0066] In some possible implementations, the update module 502 is used to: use a preset sliding window to determine any preset frame and the previous frame or frames of any preset frame, and perform sliding average processing or alpha average processing on the current quantization parameter according to the preset sliding window to perform iterative update.
[0067] In some possible implementations, the quantization module 501 is used for: if any preset frame is the first frame in the frame sequence to be tested, the current quantization parameter before updating is an initial quantization parameter determined in advance by inputting a correction set into the neural network.
[0068] In some possible implementations, an initial parameter module is also included, which is used to: pre-input each sample in the correction set into the neural network and obtain the statistical distribution information of the output activation of the target layer; determine the initial quantization boundary of the target layer according to the statistical distribution information of the output activation of the target layer; determine the initial quantization parameter according to the initial quantization boundary of the target layer and the target quantization bit width.
[0069] In some possible implementations, the update module 502 is used to: determine the initial quantization interval of the target layer by calculating the relative entropy of the output activation of the target layer; and / or by obtaining the maximum and minimum values of the output activation of the target layer; and / or by performing peak clipping on the output activation of the target layer.
[0070] In some possible implementations, the update module 502 is used to perform iterative updates according to the maximum / minimum value, maximum value interval / minimum value interval, or histogram information of the first output activation of the target layer.
[0071] In some possible implementations, the iterative update is a replacement update or a weighted update.
[0072] It should be noted that the neural network-based data quantization device in the embodiment of the present application can implement the various processes of the aforementioned neural network-based data quantization method embodiment and achieve the same effects and functions, which will not be repeated here.
[0073] Figure 6 is a schematic diagram of a data quantization device based on a neural network according to an embodiment of the present application, which is used to perform the following steps: Figure 2 The neural network-based data quantization method shown, the device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute: determining the first output activation of the target layer according to a preset frame in the frame sequence to be tested, and performing a quantization operation on the first output activation based on the current quantization parameter; iteratively updating the current quantization parameter according to the first output activation of the target layer and the target quantization bit width; determining the second output activation of the target layer according to the next frame of any one of the preset frames in the frame sequence to be tested, and performing a quantization operation on the second output activation based on the updated current quantization parameter.
[0074] According to some embodiments of the present application, a computer-readable storage medium stores a program, and when the program is executed by a multi-core processor, the multi-core processor executes: determining a first output activation of the target layer according to a preset frame in a frame sequence to be tested, and performing a quantization operation on the first output activation based on a current quantization parameter; iteratively updating the current quantization parameter according to the first output activation of the target layer and a target quantization bit width; determining a second output activation of the target layer according to a next frame of any one of the preset frames in the frame sequence to be tested, and performing a quantization operation on the second output activation based on the updated current quantization parameter.
[0075] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and computer-readable storage medium embodiments, since they are basically similar to the method embodiments, their descriptions are simplified, and the relevant parts can be referred to the partial description of the method embodiments.
[0076] The apparatus and computer-readable storage medium provided in the embodiments of the present application correspond one-to-one to the method. Therefore, the apparatus, equipment and computer-readable storage medium also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the apparatus, equipment and computer-readable storage medium will not be repeated here.
[0077] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0078] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0079] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0080] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0081] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0082] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0083] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. In addition, although the operations of the method of the present invention are described in a particular order in the accompanying drawings, this does not require or imply that these operations must be performed in this particular order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.
[0084] Although the spirit and principle of the present invention have been described with reference to several specific embodiments, it should be understood that the present invention is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and such division is only for the convenience of expression. The present invention is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims.
[0085] Although the spirit and principle of the present invention have been described with reference to several specific embodiments, it should be understood that the present invention is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and such division is only for the convenience of expression. The present invention is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims.
Claims
1. A data quantization method based on a neural network, characterized in that: include: Determine the first output activation of the target layer of the neural network according to any preset frame in the frame sequence to be tested, and perform a quantization operation on the first output activation based on a current quantization parameter; the frame to be tested is an image frame, and the previous and next frames of the frame sequence to be tested have time continuity and image content continuity; Determine a quantization boundary according to distribution information of the first output activation of the target layer, and iteratively update the current quantization parameter according to the quantization boundary and the target quantization bit width; The second output activation of the target layer is determined according to the next frame of any one of the preset frames in the frame sequence to be tested, and a quantization operation is performed on the second output activation based on the updated current quantization parameter.
2. The method according to claim 1, characterized in that If any one of the preset frames is a non-first frame in the frame sequence to be tested, iteratively updating the current quantization parameter according to the first output activation of the target layer and the target quantization bit width, further comprising: Determine a third output activation of the target layer, wherein the third output activation is associated with a frame or frames preceding any one of the preset frames in the sequence of frames to be tested; The iterative update is performed according to the first output activation, the third output activation and the target quantization bit width of the target layer.
3. The method according to claim 2, characterized in that Also includes: A preset sliding window is used to determine the arbitrary preset frame and the previous frame or frames of the arbitrary preset frame, and sliding average processing or alpha average processing is performed according to the preset sliding window to perform iterative update on the current quantization parameter.
4. The method according to claim 1, characterized in that Also includes: If any one of the preset frames is the first frame in the frame sequence to be tested, the current quantization parameter before updating is an initial quantization parameter determined in advance by inputting a correction set into the neural network.
5. The method according to claim 4, characterized in that Also includes: Input each sample in the calibration set into the neural network in advance, and obtain statistical distribution information of output activation of the target layer; Determining an initial quantization boundary of the target layer according to statistical distribution information of output activations of the target layer; The initial quantization parameter is determined according to the initial quantization boundary of the target layer and the target quantization bit width.
6. The method according to claim 5, characterized in that Determining an initial quantization boundary of the target layer according to statistical distribution information of output activations of the target layer includes: The initial quantization interval of the target layer is determined by calculating the relative entropy of the output activation of the target layer; and / or by obtaining the maximum and minimum values of the output activation of the target layer; and / or by performing peak clipping on the output activation of the target layer.
7. The method according to claim 1, characterized in that Iteratively updating the current quantization parameter according to the first output activation of the target layer and the target quantization bit width, further comprising: The iterative update is performed according to the maximum / minimum value, maximum interval / minimum interval, or histogram information of the first output activation of the target layer.
8. The method according to claim 1, characterized in that The iterative update is a replacement update or a weighted update.
9. A data quantization device based on a neural network, characterized in that: It includes a quantization module and an update module, where: The quantization module is used to determine the first output activation of the target layer of the neural network according to any preset frame in the frame sequence to be tested, and perform a quantization operation on the first output activation based on the current quantization parameter; the frame to be tested is an image frame, and the previous and next frames of the frame sequence to be tested have time continuity and image content continuity; The updating module is used to determine a quantization boundary according to the distribution information of the first output activation of the target layer, and iteratively update the current quantization parameter according to the quantization boundary and the target quantization bit width; The quantization module is used to determine the second output activation of the target layer according to the next frame of any preset frame in the frame sequence to be tested, and perform a quantization operation on the second output activation based on the updated current quantization parameter.
10. The device according to claim 9, characterized in that The update module is used to: If any one of the preset frames is a non-first frame in the sequence of frames to be tested, determining a third output activation of the target layer, wherein the third output activation is associated with a previous frame or frames of the any one of the preset frames in the sequence of frames to be tested; The iterative update is performed according to the first output activation, the third output activation and the target quantization bit width of the target layer.
11. The device according to claim 10, characterized in that The update module is used to: A preset sliding window is used to determine the arbitrary preset frame and the previous frame or frames of the arbitrary preset frame, and sliding average processing or alpha average processing is performed according to the preset sliding window to perform iterative update on the current quantization parameter.
12. The device according to claim 9, characterized in that The quantization module is used for: If any one of the preset frames is the first frame in the frame sequence to be tested, the current quantization parameter before updating is an initial quantization parameter determined in advance by inputting a correction set into the neural network.
13. The device according to claim 12, characterized in that Also includes the initial parameters module for: Input each sample in the calibration set into the neural network in advance, and obtain statistical distribution information of output activation of the target layer; Determining an initial quantization boundary of the target layer according to statistical distribution information of output activations of the target layer; The initial quantization parameter is determined according to the initial quantization boundary of the target layer and the target quantization bit width.
14. The device according to claim 9, characterized in that The update module is used to: The initial quantization interval of the target layer is determined by calculating the relative entropy of the output activation of the target layer; and / or by obtaining the maximum and minimum values of the output activation of the target layer; and / or by performing peak clipping on the output activation of the target layer.
15. The device according to claim 9, characterized in that The update module is used to: The iterative update is performed according to the maximum / minimum value, maximum interval / minimum interval, or histogram information of the first output activation of the target layer.
16. The device according to claim 9, characterized in that The iterative update is a replacement update or a weighted update.
17. A data quantization device based on a neural network, characterized in that: include: at least one processor; as well as, a memory communicatively connected to at least one processor; wherein, The memory stores instructions executable by at least one processor, and the instructions are executed by at least one processor to enable the at least one processor to perform: Determine the first output activation of the target layer of the neural network according to any preset frame in the frame sequence to be tested, and perform a quantization operation on the first output activation based on a current quantization parameter; the frame to be tested is an image frame, and the previous and next frames of the frame sequence to be tested have time continuity and image content continuity; Determine a quantization boundary according to distribution information of the first output activation of the target layer, and iteratively update the current quantization parameter according to the quantization boundary and the target quantization bit width; The second output activation of the target layer is determined according to the next frame of any one of the preset frames in the frame sequence to be tested, and a quantization operation is performed on the second output activation based on the updated current quantization parameter.
18. A computer-readable storage medium storing a program, wherein when the program is executed by a multi-core processor, the multi-core processor executes the method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Activation quantity quantification method and device for a deep neural network
CN109583561A
Object recognition method and device
CN110163240A
Video frame encoding method, video frame encoding equipment and device with storage function
CN110198444A
Optimization and application method and device of multi-layer neural network model and storage medium
CN110245741A
Artificial neural network adjustment method and device
CN110413255A
