A neural network-based partial and quantization processing method and device
By partitioning and array quantizing the input data and weights of the convolution kernel, the memory wall problem of traditional accelerators is solved, improving computational efficiency and reducing hardware complexity and cost, thus adapting to the CIM architecture.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIHANG UNIV
- Filing Date
- 2022-10-12
- Publication Date
- 2026-05-29
AI Technical Summary
Traditional accelerators suffer from the memory wall problem, which limits energy efficiency improvements. Quantized neural networks require high-precision ADCs in in-memory computing, increasing hardware overhead.
The input data and weights of the convolution kernel are partitioned, and array quantization and analog-to-digital conversion are performed. A learnable scaling factor is used to reduce quantization error and reduce the accuracy of analog-to-digital conversion to adapt to the CIM architecture.
It improves computational efficiency, reduces hardware complexity while maintaining good data accuracy, adapts to the CIM architecture, and reduces hardware costs.
Smart Images
Figure CN115640837B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and specifically to a partial and quantization processing method and apparatus based on neural networks. Background Technology
[0002] Convolutional neural networks typically require a large number of parameters and complex computations to achieve high performance, leading to a continuous increase in the demand for memory bandwidth and hardware resources.
[0003] Currently, traditional accelerators based on the von Neumann architecture suffer from the memory wall, hindering further improvements in energy efficiency. Computing-In-Memory (CIM) technology addresses this problem by moving computation into memory, thereby avoiding large amounts of data movement and reducing power consumption. Quantization can effectively convert floating-point data into low-bit data, significantly reducing latency and energy consumption; quantized neural networks are widely used in CIM-based accelerators. However, CIM memory arrays typically have limited capacity. Therefore, matrix vector multiplication (MVM) in a single convolution operation needs to be split into multiple small partitions, involving partial sums. Due to the limited precision of analog-to-digital converters (ADCs), additional quantization steps are required for the partial sums. Most often, high-precision ADCs are used for partial sums, incurring additional hardware overhead. Summary of the Invention
[0004] To address the problems in the prior art, embodiments of the present invention provide a partial summation and quantization processing method and apparatus based on neural networks, which can at least partially solve the problems existing in the prior art.
[0005] In a first aspect, the present invention proposes a partial summation and quantization processing method based on a neural network, comprising:
[0006] The input data and weights of each convolutional kernel are partitioned to obtain the data of each partition of each convolutional kernel;
[0007] Array quantization is performed on each partition of the data of each convolution kernel to obtain the quantized data of each partition of each convolution kernel;
[0008] Multiply and accumulate the quantized data of each partition of each convolution kernel to obtain the partial sum of each partition of each convolution kernel;
[0009] Modulus-to-digital conversion is performed on the partial sums of each partition of each convolutional kernel to obtain the integer output data of each partition of each convolutional kernel;
[0010] The integer output data of each partition of each convolution kernel is dequantized to obtain the floating-point output data of each convolution kernel.
[0011] Furthermore, the step of array quantization of each partition data of each convolutional kernel to obtain quantized data of each partition of each convolutional kernel includes:
[0012] According to the formula Obtain the quantization weights in the partitioned quantization data.
[0013] According to the formula Obtain the quantization input data from the partitioned quantization data.
[0014] Among them, W f A represents the weights in the quantized data of the f-th partition of the convolution kernel. f This represents the input data in the quantized data of the f-th partition of the convolution kernel. This represents the weighting factor in the quantized data of the f-th partition. This represents the scaling factor of the input data in the f-th partition quantization data, b indicates that the data is quantized to b bits, and the clip function indicates that the target data is truncated to [Q]. n Q p Within the range, This indicates the rounding operation. For unsigned numbers, [Q] n Q p ] = [0,2 b -1], for signed numbers, [Q] n Q p ] = [-2 b-1 ,2 b-1 -1], where f is a positive integer.
[0015] Furthermore, before performing an analog-to-digital transformation on the partial sums of each partition of each convolutional kernel, the following steps are also included:
[0016] The partial sums of each partition of each convolutional kernel are quantized using a learnable scaling factor.
[0017] Furthermore, the accuracy of the analog-to-digital conversion is lower than the preset accuracy.
[0018] In a second aspect, the present invention provides a partial quantization processing apparatus based on a neural network, comprising:
[0019] The acquisition unit is used to partition the input data and weights of each convolutional kernel separately to obtain the partition data of each convolutional kernel;
[0020] The array quantization unit is used to perform array quantization on the data of each partition of each convolution kernel to obtain the quantized data of each partition of each convolution kernel.
[0021] The multiplication and accumulation unit is used to perform multiplication and accumulation operations on the quantized data of each partition of each convolution kernel to obtain the partial sum of each partition of each convolution kernel;
[0022] The analog-to-digital conversion unit is used to perform analog-to-digital conversion on the partial sums of each partition of each convolutional kernel to obtain the integer output data of each partition of each convolutional kernel;
[0023] The dequantization unit is used to dequantize the integer output data of each partition of each convolutional kernel to obtain the floating-point output data of each convolutional kernel.
[0024] Furthermore, the array quantization unit is specifically used for:
[0025] According to the formula Obtain the quantization weights in the partitioned quantization data.
[0026] According to the formula Obtain the quantization input data from the partitioned quantization data.
[0027] Among them, W f A represents the weights in the quantized data of the f-th partition of the convolution kernel. f This represents the input data in the quantized data of the f-th partition of the convolution kernel. This represents the weighting factor in the quantized data of the f-th partition. This represents the scaling factor of the input data in the f-th partition quantization data, b indicates that the data is quantized to b bits, and the clip function indicates that the target data is truncated to [Q]. n Q p Within the range, This indicates the rounding operation. For unsigned numbers, [Q] n Q p ] = [0,2 b -1], for signed numbers, [Q] n Q p ] = [-2 b-1 ,2 b-1 -1], where V represents the continuous variable added to the weights, and f is a positive integer.
[0028] Furthermore, the neural network-based partial quantization processing method provided in this embodiment of the invention further includes:
[0029] Partial sum quantization units are used to quantize the partial sums of each partition of each convolutional kernel using a learnable scaling factor.
[0030] Furthermore, the accuracy of the analog-to-digital conversion is lower than the preset accuracy.
[0031] Thirdly, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the neural network-based partial and quantization processing method described in any of the above embodiments.
[0032] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the neural network-based partial and quantization processing method described in any of the above embodiments.
[0033] The partial sum quantization processing method and apparatus based on neural networks provided in this invention partitions the input data and weights of each convolutional kernel to obtain the partition data of each convolutional kernel. Array quantization is then performed on the partition data of each convolutional kernel to obtain the quantized data of each partition of each convolutional kernel. Multiplication and accumulation operations are then performed on the quantized data of each partition of each convolutional kernel to obtain the partial sum of each partition of each convolutional kernel. Modulus-to-digital conversion is then performed on the partial sum of each partition of each convolutional kernel to obtain the integer output data of each partition of each convolutional kernel. Finally, inverse quantization is performed on the integer output data of each partition of each convolutional kernel to obtain the floating-point output data of each convolutional kernel. By partitioning and array quantizing the input data and weights of each convolutional kernel, the partitioned quantized data is better adapted to the CIM architecture, improving computational efficiency. Attached Figure Description
[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:
[0035] Figure 1 This is a schematic diagram of a typical architecture of a CIM-based accelerator provided in the first embodiment of the present invention.
[0036] Figure 2 This is a flowchart illustrating the partial and quantization processing method based on a neural network provided in the second embodiment of the present invention.
[0037] Figure 3 This is a schematic diagram of the structure of the neural network-based partial and quantization processing device provided in the third embodiment of the present invention.
[0038] Figure 4This is a schematic diagram of the structure of the neural network-based partial and quantization processing device provided in the fourth embodiment of the present invention.
[0039] Figure 5 This is a schematic diagram of the physical structure of the electronic device provided in the fifth embodiment of the present invention. Detailed Implementation
[0040] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments and descriptions of the present invention are used to explain the present invention, but are not intended to limit the present invention. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other.
[0041] To facilitate understanding of the technical solution provided in this application, the relevant content of the technical solution in this application will be explained below.
[0042] Typically, Ohm's law and Kirchhoff's laws are the foundation for simulating CIM. Figure 1 This is a schematic diagram of a typical architecture of the CIM-based accelerator provided in the first embodiment of the present invention, as shown below. Figure 1 As shown, the CIM array performs simulated MVM in parallel, where the weight matrix is stored in the storage cells at the array intersections, g i,j It is the conductance of the storage cell. The input vector is used as voltage (V). i This is applied to the cross-array rows converted by a digital-to-analog converter (DAC). The accumulated current (IA) is... j The values are obtained from the columns of the cross array and converted by the ADC to obtain the calculated output.
[0043] The massive computations of convolutional neural networks come from convolution operations composed of MVMs. The size of the weight matrix in a convolution kernel is (K h ,K w C in C out This can be converted into a form of size (K). h ×K w ×C in C out The weight matrix is a two-dimensional matrix, which is then mapped to the CIM array. Since the manufacturing size of storage arrays is typically limited, much smaller than the weight matrix, the matrix needs to be partitioned, resulting in partial sums.
[0044] There are two main types of quantization granularity for low bit-width convolutional neural networks: (1) Layer-by-layer quantization, which uses the same quantization parameters for the input data of the same layer and the weights of all channels by statistically analyzing the information of all parameters within the same layer. Although this method is simple to implement in hardware, it has lower accuracy. (2) Channel quantization, which uses fixed values for the weights of each channel, independent of other channels. This ensures better quantization resolution and usually has higher accuracy, but increases hardware complexity.
[0045] As mentioned above, due to the size limitation of the CIM array, CIM can only process a portion of the input channels and a portion of the output channels in parallel for a single convolution operation. Therefore, this application proposes a novel partial quantization processing method based on the characteristics of the CIM storage array. This method performs array quantization on the input data and weights, that is, it groups the input data and weights, which is more suitable for CIM architecture to perform convolutional neural network calculations, reducing hardware complexity while maintaining good data accuracy.
[0046] Figure 2 This is a flowchart illustrating the partial and quantization processing method based on a neural network provided in the second embodiment of the present invention, as shown below. Figure 2 As shown, the partial quantization processing method based on neural networks provided in this embodiment of the invention includes:
[0047] S201. Partition the input data and weights of each convolution kernel to obtain the partition data of each convolution kernel.
[0048] Specifically, for each convolutional kernel of the neural network, the input data and weights of the convolutional kernel are partitioned to obtain the data of each partition of the convolutional kernel. The partition size is fixed for the CIM memory array. The input data and weights are floating-point data. It is understood that the input data and weights can be matrices or vectors, set according to the actual situation, and this embodiment of the invention does not impose any limitations. The execution entity of the neural network-based partial quantization processing method provided in this embodiment of the invention includes, but is not limited to, a server.
[0049] S202. Perform array quantization on the data of each partition of each convolution kernel to obtain the quantized data of each partition of each convolution kernel;
[0050] Specifically, for each convolutional kernel of the neural network, array quantization is performed on each partition of the convolutional kernel, that is, each data in each partition is converted from floating-point data to integers to obtain the quantized data of each partition of the convolutional kernel.
[0051] S203. Perform multiplication and summation operations on the quantized data of each partition of each convolution kernel to obtain the partial sum of each partition of each convolution kernel;
[0052] Specifically, for each convolutional kernel of the neural network, multiplication and accumulation operations are performed on the quantized data of each partition of the convolutional kernel to obtain the partial sum of each partition of each convolutional kernel.
[0053] S204. Perform modulo-digital conversion on the partial sums of each partition of each convolutional kernel to obtain the integer output data of each partition of each convolutional kernel.
[0054] Specifically, for each convolutional kernel of the neural network, the partial sum of each partition of the convolutional kernel is converted from analog to digital using an analog-to-digital converter to obtain the integer output data of each partition. The precision of the analog-to-digital conversion is set according to actual needs, and this embodiment of the invention does not impose any limitations.
[0055] S205. Perform dequantization on the integer output data of each partition of each convolution kernel to obtain the floating-point output data of each convolution kernel.
[0056] Specifically, for each convolutional kernel of the neural network, the integer output data of each partition of the convolutional kernel is dequantized, that is, the integer output data of each partition is multiplied by a scaling factor to obtain the floating-point output data of the convolutional kernel.
[0057] The partial sum quantization processing method based on neural networks provided in this invention partitions the input data and weights of each convolutional kernel to obtain the partition data of each convolutional kernel. Array quantization is then performed on the partition data of each convolutional kernel to obtain the quantized data of each partition. Multiplication and accumulation operations are then performed on the quantized data of each partition to obtain the partial sum of each partition. Modulus-to-digital conversion is then performed on the partial sum of each partition to obtain the integer output data of each partition. Finally, inverse quantization is performed on the integer output data of each partition to obtain the floating-point output data of each convolutional kernel. By partitioning and array quantizing the input data and weights of each convolutional kernel, the partitioned quantized data is better adapted to the CIM architecture, improving computational efficiency. Furthermore, compared to channel quantization, it reduces hardware complexity while maintaining good data accuracy.
[0058] Based on the above embodiments, further, the step of array quantization of each partition data of each convolutional kernel to obtain quantized data of each partition of each convolutional kernel includes:
[0059] According to the formula Obtain the quantization weights in the partitioned quantization data.
[0060] According to the formula Obtain the quantization input data from the partitioned quantization data.
[0061] Among them, W f A represents the weights in the quantized data of the f-th partition of the convolution kernel. f This represents the input data in the quantized data of the f-th partition of the convolution kernel. This represents the weighting factor in the quantized data of the f-th partition. This represents the scaling factor of the input data in the f-th partition quantization data, b indicates that the data is quantized to b bits, and the clip function indicates that the target data is truncated to [Qn, Q]. p Within the range, This indicates the rounding operation. For unsigned numbers, [Q] n Q p ] = [0, 2 b -1], for signed numbers, [Q] n Q p ] = [-2 b-1 ,2 b-1 -1], where V represents the continuous variable added to the weights, and f is a positive integer.
[0062] Specifically, the basic principle of array quantization in this application can be expressed as follows:
[0063]
[0064] Where x represents the floating-point data to be quantized, s is the scaling factor, and b represents quantizing the data to b bits. The `clip` function represents rounding operations. Truncation to [Q] n Q p Within the range of ], for unsigned numbers, [Q n Q p ] = [0, 2 b -1], for signed numbers, [Q] n Q p ] = [-2 b-1 ,2 b-1 -1].
[0065] For traditional quantization, the optimization objective is:
[0066]
[0067] Where W represents the weight, A represents the input data, and s W s A s respectively W The scaling factor, s, represents the weight. A V represents the scaling factor of the input data, V represents the continuous variable added to the weights, and W represents the final weights obtained after quantization and training.b =W+V.
[0068] Suppose the input data and weights of a neural network's convolutional kernel are divided into F partitions. In the f-th partition, the scaling factor of the weights is... The input scaling factor is f is a positive integer and f is less than or equal to F, where F is a positive integer. The quantized weights corresponding to the f-th partition data are... and input data They are respectively:
[0069]
[0070]
[0071] Based on formulas (1) and (2), we can obtain:
[0072]
[0073] Based on formulas (1) and (3), we can obtain:
[0074]
[0075] For the array quantization in this application, the optimization objective becomes:
[0076]
[0077] Here, argmin represents taking the minimum value.
[0078] Based on the above embodiments, further, before performing analog-to-digital conversion on the partial sums of each partition of each convolutional kernel, the method further includes:
[0079] The partial sums of each partition of each convolutional kernel are quantized using a learnable scaling factor.
[0080] Specifically, in CIM, MVM for integer data is performed in the analog domain. The accumulated current or voltage of the partial sums may exceed the input range of the ADC, introducing truncation error. Therefore, a learnable scaling factor is introduced in the analog domain to reduce truncation error and prevent the accumulated current and voltage of the partial sums from exceeding the ADC's input range. The partial sums of each partition of each convolution kernel can be quantized using the learnable scaling factor, and then analog-to-digital conversion is performed. Quantizing the partial sums using a learnable scaling factor in the analog domain allows them to be converted to digital by a low-resolution ADC and reduces quantization error.
[0081] For example, introducing a learnable scaling factor into the simulation domain, the formula... (4) becomes:
[0082]
[0083] in, Indicates partial sum, The learnable scaling factor represents the partial sum, which is used to scale the cumulative current or cumulative voltage of the partial sum. The data for each partition of each convolutional kernel is represented as current or voltage in the CIM.
[0084] During backpropagation, P represents a partial sum. The gradient calculation formula is as follows:
[0085]
[0086] Based on the above embodiments, the accuracy of the analog-to-digital conversion is further lower than the preset accuracy.
[0087] Specifically, the accuracy of the analog-to-digital conversion can be set lower than the preset accuracy to perform low-precision analog-to-digital conversion, thereby reducing the amount of computation and improving operational efficiency. Furthermore, it can also reduce hardware requirements and lower hardware costs.
[0088] Figure 3 This is a schematic diagram of the structure of the neural network-based partial and quantization processing device provided in the third embodiment of the present invention, as shown below. Figure 3 As shown, the neural network-based partial quantization processing device provided in this embodiment of the invention includes an acquisition unit 301, an array quantization unit 302, a multiplication and accumulation unit 303, an analog-to-digital conversion unit 304, and an inverse quantization unit 305, wherein:
[0089] The acquisition unit 301 is used to partition the input data and weights of each convolutional kernel to obtain the partition data of each convolutional kernel; the array quantization unit 302 is used to perform array quantization on the partition data of each convolutional kernel to obtain the quantized data of each partition of each convolutional kernel; the multiplication and accumulation unit 303 is used to perform multiplication and accumulation on the quantized data of each partition of each convolutional kernel to obtain the partial sum of each partition of each convolutional kernel; the analog-to-digital conversion unit 304 is used to perform analog-to-digital conversion on the partial sum of each partition of each convolutional kernel to obtain the integer output data of each partition of each convolutional kernel; and the dequantization unit 305 is used to perform dequantization on the integer output data of each partition of each convolutional kernel to obtain the floating-point output data of each convolutional kernel.
[0090] Specifically, for each convolutional kernel of the neural network, the obtaining unit 301 partitions the input data and weights of the convolutional kernel to obtain the partition data of each convolutional kernel. The partition size is fixed for the CIM memory array. The input data and weights are floating-point data. It is understood that the input data and weights can be matrices or vectors, set according to the actual situation; this embodiment of the invention does not impose any limitations.
[0091] For each convolutional kernel of the neural network, the array quantization unit 302 performs array quantization on each partition of the convolutional kernel, that is, converts each data in each partition from floating-point data to integers to obtain the quantized data of each partition of the convolutional kernel.
[0092] For each convolutional kernel of the neural network, the multiplication and accumulation unit 303 performs multiplication and accumulation operations on the quantized data of each partition of the convolutional kernel to obtain the partial sum of each partition of each convolutional kernel.
[0093] For each convolutional kernel of the neural network, the analog-to-digital conversion unit 304 performs analog-to-digital conversion on the partial sum of each partition of the convolutional kernel using an analog-to-digital converter, thereby obtaining integer output data for each partition. The accuracy of the analog-to-digital conversion is set according to actual needs, and this embodiment of the invention does not impose any limitations.
[0094] For each convolutional kernel of the neural network, the dequantization unit 305 performs a dequantization operation on the integer output data of each partition of the convolutional kernel, that is, multiplies the integer output data of each partition by a scaling factor to obtain the floating-point output data of the convolutional kernel.
[0095] The partial sum quantization processing device based on neural networks provided in this invention partitions the input data and weights of each convolutional kernel to obtain partition data for each convolutional kernel. It then performs array quantization on the partition data of each convolutional kernel to obtain quantized data for each partition. Next, it performs multiplication and accumulation operations on the quantized data of each partition to obtain the partial sum of each partition. Finally, it performs analog-to-digital conversion on the partial sum of each partition to obtain the integer output data of each partition. Finally, it performs dequantization on the integer output data of each partition to obtain the floating-point output data of each convolutional kernel. By partitioning and array quantizing the input data and weights of each convolutional kernel, the partitioned quantized data is better adapted to the CIM architecture, improving computational efficiency. Furthermore, compared to channel quantization, it reduces hardware complexity while maintaining good data accuracy.
[0096] Based on the above embodiments, the array quantization unit 304 is further specifically used for:
[0097] According to the formula Obtain the quantization weights in the partitioned quantization data.
[0098] According to the formula Obtain the quantization input data from the partitioned quantization data.
[0099] Among them, W f A represents the weights in the quantized data of the f-th partition of the convolution kernel. f This represents the input data in the quantized data of the f-th partition of the convolution kernel. This represents the weighting factor in the quantized data of the f-th partition. This represents the scaling factor of the input data in the f-th partition quantization data, b indicates that the data is quantized to b bits, and the clip function indicates that the target data is truncated to [Q]. n Q p Within the range, This indicates the rounding operation. For unsigned numbers, [Q] n Q p ] = [0,2 b -1], for signed numbers, [Q] n Q p ] = [-2 b-1 ,2 b-1 -1].
[0100] Figure 4 This is a schematic diagram of the structure of the neural network-based partial quantization processing device provided in the fourth embodiment of the present invention, as shown below. Figure 4 As shown, based on the above embodiments, the neural network-based partial quantization processing device provided in this embodiment further includes a partial quantization unit 306, wherein:
[0101] The partial sum quantization unit 306 is used to quantize the partial sums of each partition of each convolutional kernel using a learnable scaling factor.
[0102] Based on the above embodiments, the accuracy of the analog-to-digital conversion is further lower than the preset accuracy.
[0103] The embodiments of the device provided in this invention can be used to execute the processing flow of the above-described method embodiments. Its functions will not be repeated here, but can be referred to the detailed description of the above-described method embodiments.
[0104] Figure 5 This is a schematic diagram of the physical structure of the electronic device provided in the fifth embodiment of the present invention, as shown below. Figure 5As shown, the electronic device may include a processor 501, a communication interface 502, a memory 503, and a communication bus 504. The processor 501, communication interface 502, and memory 503 communicate with each other via the communication bus 504. The processor 501 can call logical instructions in the memory 503 to execute the following methods: partitioning the input data and weights of each convolutional kernel to obtain the partition data of each convolutional kernel; performing array quantization on the partition data of each convolutional kernel to obtain the quantized data of each partition of each convolutional kernel; performing multiplication and accumulation operations on the quantized data of each partition of each convolutional kernel to obtain the partial sum of each partition of each convolutional kernel; performing analog-to-digital conversion on the partial sum of each partition of each convolutional kernel to obtain the integer output data of each partition of each convolutional kernel; and performing dequantization on the integer output data of each partition of each convolutional kernel to obtain the floating-point output data of each convolutional kernel.
[0105] Furthermore, the logical instructions in the aforementioned memory 503 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0106] This embodiment discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can perform the methods provided in the above-described method embodiments, such as: partitioning the input data and weights of each convolutional kernel to obtain partition data of each convolutional kernel; performing array quantization on the partition data of each convolutional kernel to obtain quantized data of each partition of each convolutional kernel; performing multiplication and accumulation operations on the quantized data of each partition of each convolutional kernel to obtain the partial sum of each partition of each convolutional kernel; performing analog-to-digital conversion on the partial sum of each partition of each convolutional kernel to obtain integer output data of each partition of each convolutional kernel; and performing dequantization on the integer output data of each partition of each convolutional kernel to obtain floating-point output data of each convolutional kernel.
[0107] This embodiment provides a computer-readable storage medium storing a computer program that causes a computer to execute the methods provided in the above-described method embodiments. For example, the methods include: partitioning the input data and weights of each convolutional kernel to obtain partition data for each convolutional kernel; performing array quantization on the partition data of each convolutional kernel to obtain quantized data for each partition of each convolutional kernel; performing multiplication and accumulation operations on the quantized data of each partition of each convolutional kernel to obtain a partial sum of each partition of each convolutional kernel; performing analog-to-digital conversion on the partial sum of each partition of each convolutional kernel to obtain integer output data for each partition of each convolutional kernel; and performing dequantization on the integer output data of each partition of each convolutional kernel to obtain floating-point output data for each convolutional kernel.
[0108] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0109] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0110] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0111] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0112] In the description of this specification, the references to terms such as "an embodiment," "a specific embodiment," "some embodiments," "for example," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0113] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A partial summation and quantization processing method based on neural networks, characterized in that, include: The input data and weights of each convolutional kernel are partitioned to obtain the data of each partition of each convolutional kernel; Array quantization is performed on each partition of the data of each convolution kernel to obtain quantized data for each partition of each convolution kernel; wherein, array quantization on each partition of the data of the convolution kernel is to convert each data in each partition from floating-point data to integer; Multiply and accumulate the quantized data of each partition of each convolution kernel to obtain the partial sum of each partition of each convolution kernel; The partial sums of each partition of each convolutional kernel are quantized using a learnable scaling factor; Modulus-to-digital conversion is performed on the partial sums of each partition of each convolutional kernel to obtain the integer output data of each partition of each convolutional kernel; The integer output data of each partition of each convolution kernel is dequantized to obtain the floating-point output data of each convolution kernel.
2. The method according to claim 1, characterized in that, The step of array quantization of each partition of data for each convolutional kernel to obtain quantized data for each partition of each convolutional kernel includes: According to the formula Obtain the quantization weights in the partitioned quantization data. ; According to the formula Obtain the quantized input data from the partitioned quantized data. ; in, The convolution kernel is represented by the first... The weights in the partitioned quantized data, where V represents the continuous variable added to the weights. The convolution kernel is represented by the first... Input data in partitioned quantized data, Indicates the first The weighting factor in the quantized data of each partition. Indicates the first The scaling factor of the input data in each partitioned quantized data. This indicates quantizing the data to b bits, and the clip function indicates truncating the target data to [...]. Within the range, This indicates the rounding operation. For unsigned numbers, [ ]=[0, For signed numbers, [ ]= [ ], It is a positive integer.
3. The method according to claim 1 or 2, characterized in that, The accuracy of the analog-to-digital conversion is lower than the preset accuracy.
4. A partial summation and quantization processing device based on a neural network, characterized in that, include: The acquisition unit is used to partition the input data and weights of each convolutional kernel separately to obtain the partition data of each convolutional kernel; The array quantization unit is used to perform array quantization on the data of each partition of each convolution kernel to obtain the quantized data of each partition of each convolution kernel; wherein, array quantization on the data of each partition of the convolution kernel is to convert the data in each partition from floating point data to integer; The multiplication and accumulation unit is used to perform multiplication and accumulation operations on the quantized data of each partition of each convolution kernel to obtain the partial sum of each partition of each convolution kernel; Partial sum quantization units are used to quantize the partial sums of each partition of each convolutional kernel using a learnable scaling factor; The analog-to-digital conversion unit is used to perform analog-to-digital conversion on the partial sums of each partition of each convolutional kernel to obtain the integer output data of each partition of each convolutional kernel; The dequantization unit is used to dequantize the integer output data of each partition of each convolutional kernel to obtain the floating-point output data of each convolutional kernel.
5. The apparatus according to claim 4, characterized in that, The array quantization unit is specifically used for: According to the formula Obtain the quantization weights in the partitioned quantization data. ; According to the formula Obtain the quantized input data from the partitioned quantized data. ; in, The convolution kernel is represented by the first... The weights in the partitioned quantized data, where V represents the continuous variable added to the weights. The convolution kernel is represented by the first... Input data in partitioned quantized data, Indicates the first The weighting factor in the quantized data of each partition. Indicates the first The scaling factor of the input data in each partitioned quantized data. This indicates quantizing the data to b bits, and the clip function indicates truncating the target data to [...]. Within the range, This indicates the rounding operation. For unsigned numbers, [ ]=[0, For signed numbers, [ ]= [ ], This represents a continuous variable added to the weights. It is a positive integer.
6. The apparatus according to claim 4 or 5, characterized in that, The accuracy of the analog-to-digital conversion is lower than the preset accuracy.
7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 3.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 3.