Convolutional Neural Network Accelerator for Data Processing

Abstract

The present invention provides a convolutional neural network accelerator for data processing, comprising: a data processing unit and an adder array unit; the data processing unit includes a first preset number of random processing array modules, each random processing array module is used to process first data to be processed based on the fast finite impulse response (FIR) algorithm and bilinear stochastic computing to obtain a first output result; the adder array unit is used to perform cumulative summation on the first output result to obtain a target convolution result. By processing the data to be processed based on the fast finite impulse response (FIR) algorithm and bilinear stochastic computing, since the fast finite impulse response (FIR) algorithm only includes simple multiplication and addition calculations, and bilinear stochastic computing has the advantages of low complexity and high fault tolerance, the combination of the two effectively reduces the complexity of convolution calculations in the neural network, thereby reducing the hardware consumption caused by convolution calculations and improving the hardware efficiency.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular, to a convolutional neural network accelerator for data processing. Background Art

[0002] With the rapid development of deep learning technology, convolutional neural networks have gradually been widely applied in fields such as object detection, image classification, and video surveillance. However, as the tasks to be processed become more and more complex, the size and depth of convolutional neural networks are also increasing rapidly. Subsequently, the growth of power consumption and area limits their practical applications on some mobile devices and small embedded devices.

[0003] Currently, a large amount of work has attempted to apply stochastic computing (SC) to the hardware implementation of convolutional neural networks. Stochastic computing is an approximate computing method that uses the probability of the occurrence of 1 in a 0-1 random bit stream to represent a numerical value, and has the advantages of low complexity and high fault tolerance. Therefore, it has great potential in the design of dedicated hardware accelerators for convolutional neural networks. Moreover, since convolutional computing accounts for more than 90% of the computing power consumption and the main computing time in convolutional neural networks, some existing studies have attempted to combine fast convolutional algorithms with SC to reduce the complexity of convolutional computing.

[0004] Although the existing SC convolutional neural network accelerators based on fast convolutional algorithms can indeed reduce the computing cost, these algorithms usually require relatively complex preprocessing operations, such as division and remainder operations, which are not conducive to hardware implementation. Summary of the Invention

[0005] The present invention provides a convolutional neural network accelerator for data processing, which is used to solve the problem of huge hardware consumption existing in existing convolutional neural networks, and realizes effectively reducing the hardware consumption brought by convolutional computing and improving the hardware efficiency on the premise of ensuring the basic unchanged recognition performance of the neural network.

[0006] The present invention provides a convolutional neural network accelerator for data processing, including:

[0007] A data processing unit and an adder array unit;

[0008] The data processing unit includes a first preset number of random processing array modules, and each random processing array module is used to process first to-be-processed data based on a fast finite impulse response (FIR) algorithm and bilinear stochastic computing to obtain a first output result;

[0009] The adder array unit is used to perform cumulative summation on the first output result to obtain a target convolutional result.

[0010] Optionally, for a convolutional neural network accelerator for data processing provided by the present invention, the input mode of the random processing array module is a parallel input mode, and the parallel input mode indicates that the random processing array module can receive at least two of the first data to be processed in parallel.

[0011] Optionally, for a convolutional neural network accelerator for data processing provided by the present invention, the random processing array module includes a second preset number of parallel modules;

[0012] Among them, different parallel modules correspond to different delay lines, one parallel module corresponds to at least one delay line, and the at least one delay line corresponding to the one parallel module is used to receive at least one of the first data to be processed in parallel, obtain the at least one of the first data to be processed transmitted through the at least one delay line as the first data, and input the first data into the corresponding parallel module.

[0013] Optionally, for a convolutional neural network accelerator for data processing provided by the present invention, the parallel module includes a first random conversion module and a data preprocessing module;

[0014] The first random conversion module is used to convert the first data into second data, and the form of the second data is in the form of a random bit stream;

[0015] The data preprocessing module is used to perform scaling addition on the second data output by the first random conversion module to obtain a second output result.

[0016] Optionally, for a convolutional neural network accelerator for data processing provided by the present invention, the first random conversion module includes a first sequence generator and a first comparator;

[0017] The first sequence generator is used to generate a first pseudo-random number;

[0018] The first comparator is used to compare the first pseudo-random number with the first data, and output 1 when the first pseudo-random number is greater than the first data, otherwise output 0.

[0019] Optionally, for a convolutional neural network accelerator for data processing provided by the present invention, the random processing array module includes a bilinear random computing inner product module;

[0020] The bilinear random computing inner product module includes a second sequence generator, a second comparator, and a logic AND gate;

[0021] The second sequence generator and the second comparator are used to convert the first convolutional kernel weight into a second convolutional kernel weight, and the form of the second convolutional kernel weight is a random bit stream form;

[0022] The logic AND gate is used to calculate the product of the second output result and the second convolutional kernel weight to obtain a first product result.

[0023] Optionally, for a convolutional neural network accelerator for data processing provided by the present invention, a demultiplexer is configured after the logic AND gate;

[0024] The demultiplexer is used to determine whether the first product result is positive or negative according to the sign bit of the second convolutional kernel weight;

[0025] When the first product result is positive, the first product result is classified into the positive set;

[0026] When the first product result is negative, the first product result is classified into the negative set.

[0027] Optionally, for a convolutional neural network accelerator for data processing provided by the present invention, the bilinear stochastic computing inner product module further includes a first approximate accumulation counter, a second approximate accumulation counter, and a first subtractor;

[0028] The first approximate accumulation counter is used to accumulate the third data in the positive set to obtain a first accumulation result;

[0029] The second approximate accumulation counter is used to accumulate the fourth data in the negative set to obtain a second accumulation result;

[0030] The first subtractor is used to subtract the second accumulation result from the first accumulation result to obtain a second output result.

[0031] Optionally, for a convolutional neural network accelerator for data processing provided by the present invention, it further includes:

[0032] A crossbar switch, the data processing unit is connected to the adder array unit through the crossbar switch, and the crossbar switch is used to adjust the third output result of the data processing unit.

[0033] Optionally, for a convolutional neural network accelerator for data processing provided by the present invention, the first sequence generator is shared by the second preset number of parallel modules.

[0034] A convolutional neural network accelerator for data processing provided by the present invention first processes first data to be processed through a random processing array module of a data processing unit based on a fast finite impulse response (FIR) algorithm and bilinear stochastic computing to obtain a first output result, and then accumulates and sums the first output result through an adder array unit to obtain a target convolution result. Since the fast FIR algorithm only includes simple multiplication and addition calculations, and combines with a bilinear stochastic computing method with low complexity and high fault tolerance, the complexity of convolution calculations in a neural network is effectively reduced, thereby reducing the hardware consumption caused by convolution calculations and improving the hardware efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0036] Figure 1 is one of the structural schematic diagrams of the convolutional neural network accelerator for data processing provided by the present invention;

[0037] Figure 2 is another structural schematic diagram of the convolutional neural network accelerator for data processing provided by the present invention;

[0038] Figure 3 is the structural schematic diagram of the random processing array module provided by the present invention;

[0039] Figure 4 is the structural schematic diagram of the bilinear stochastic computing inner product module provided by the present invention;

[0040] Figure 5 is the change curve graph of the recognition rate of different convolution architectures of the LeNet-5 convolutional neural network based on the MNIST data set with the change of sequence length / quantization bit width provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention in conjunction with the drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments in the present invention belong to the scope of protection of the present invention.

[0042] The terms "first", "second", etc. in the description and claims of the present invention are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type and do not limit the number of objects. For example, the first object can be one or more. In addition, "and / or" in the description and claims means at least one of the connected objects, and the character " / " generally indicates an "or" relationship between the associated objects before and after.

[0043] The following will describe Figures 1-5 the convolutional neural network accelerator for data processing provided by the present invention.

[0044] Figure 1 FIG. is one of the schematic structural diagrams of the convolutional neural network accelerator for data processing provided by the present invention. As Figure 1 shown, the convolutional neural network accelerator includes:

[0045] a data processing unit 110 and an adder array unit 120;

[0046] The data processing unit 110 includes a first preset number of stochastic processing array modules 130. Each stochastic processing array module 130 is used to process first data to be processed based on a fast finite impulse response (FIR) algorithm and bilinear stochastic calculation to obtain a first output result.

[0047] The adder array unit 120 is used to perform cumulative summation on the first output result to obtain a target convolution result.

[0048] Optionally, the convolutional neural network accelerator may include a data processing unit 110 and an adder array unit 120.

[0049] For example, Figure 2 FIG. is another schematic structural diagram of the convolutional neural network accelerator for data processing provided by the present invention. As Figure 2 shown, it mainly includes two units: a parallel processing unit (Parallel Process Unit) and an adder array unit (Adder Array), which are connected by a crossbar.

[0050] Optionally, the data processing unit 110 may include a first preset number of stochastic processing array modules 130 (Stochastic Processing Element, SPE).

[0051] Optionally, the number of random processing array modules 130 can be determined according to the size of the convolution kernel.

[0052] For example, when calculating a k×k convolution, a data processing unit 110 may include k random processing array modules 130.

[0053] Optionally, each random processing array module 130 can be used to process the first data to be processed based on the Fast Finite Impulse Response (FIR) algorithm and bilinear random calculation to obtain a first output result.

[0054] Optionally, for an N-point FIR filter, its output can be expressed in the following form:

[0055]

[0056] Optionally, the input mode of the random processing array module 130 can be a parallel input mode.

[0057] Optionally, for the three-parallel Fast FIR (Fast FIR Algorithm, FFA) algorithm, it can first decompose an N-tap FIR filter into three N / 3-tap sub-filters.

[0058] Optionally, the first data to be processed, that is, the input data X, can be decomposed into X(z) = X0 + z -1 X1 + z -2 X2.

[0059] Among them, X0 corresponds to the time-domain sequence x(3k), k = 0, 1,..., X1 corresponds to the time-domain sequence x(3k + 1), k = 0, 1,..., X2 corresponds to the time-domain sequence x(3k + 2), k = 0, 1,....

[0060] Optionally, the filter parameter H can be decomposed into H(z) = H0 + z -1 H1 + z -2 H2, and the output Y can be decomposed into Y(z) = Y0 + z -1 Y1 + z -2 Y2.

[0061] Optionally, the traditional direct three-parallel FIR algorithm can calculate Y0, Y1, and Y2 through the following formula:

[0062]

[0063] Optionally, by reusing the sub-filters, the formula for calculating Y0, Y1, and Y2 can be transformed into the following form:

[0064]

[0065] It should be noted that, from the foregoing, it can be seen that the traditional direct three - parallel FIR algorithm requires 3N multiplications and 3N - 3 additions, while the fast FIR algorithm only requires 2N multiplications and 2N + 4 additions.

[0066] Optionally, for Two - line Stochastic Computing (TLSC), the value x represented by a two - line stochastic sequence can be expressed by the following formula:

[0067]

[0068] where S(X i ) represents the i - th bit of the symbol sequence, M(X i ) represents the i - th bit of the amplitude sequence, and L represents the length of the bit stream required for the calculation.

[0069] It should be noted that, compared with the traditional single - line stochastic computing, the two - line stochastic computing introduces an additional symbol bit stream, which can more accurately represent signed numbers, thereby shortening the length of the bit stream required for the calculation.

[0070] Optionally, the adder array unit 120 can be used to accumulate and sum the first output result to obtain the target convolution result.

[0071] Optionally, for the three - parallel fast FIR algorithm, and when calculating a k×k convolution, at every L clock cycles, the data processing unit 110 outputs 3k partial sums. Therefore, 2k adders can be used to accumulate and sum them to obtain the target convolution result.

[0072] The convolutional neural network accelerator for data processing provided by the present invention can be used in applicable application scenarios such as image processing, speech recognition, and target detection.

[0073] In order to reduce the hardware consumption brought by the deep convolutional neural network, the present invention combines the fast finite impulse response (FIR) algorithm and the two - line stochastic computing to process the data to be processed. Since the fast finite impulse response (FIR) algorithm only includes simple multiplication and addition calculations, and the two - line stochastic computing has the advantages of low complexity and high fault tolerance, the combination of the two effectively reduces the complexity of the convolution calculation in the neural network, thereby reducing the hardware consumption brought by the convolution calculation and improving the hardware efficiency.

[0074] A convolutional neural network accelerator for data processing provided by the present invention first processes first data to be processed through a random processing array module of a data processing unit based on a fast finite impulse response (FIR) algorithm and bilinear stochastic computing to obtain a first output result, and then accumulates and sums the first output result through an adder array unit to obtain a target convolution result. Since the fast FIR algorithm only includes simple multiplication and addition calculations, and combines with a bilinear stochastic computing method with low complexity and high fault tolerance, it effectively reduces the complexity of convolution calculations in the neural network, thereby reducing the hardware consumption caused by convolution calculations and improving the hardware efficiency.

[0075] Optionally, the input mode of the random processing array module 130 is a parallel input mode, and the parallel input mode indicates that the random processing array module 130 can receive at least two of the first data to be processed in parallel.

[0076] Optionally, the input mode of the random processing array module 130 can be a parallel input mode.

[0077] Optionally, the parallel input mode can indicate that the random processing array module 130 can receive at least two first data to be processed in parallel.

[0078] Optionally, the parallel input mode can indicate that the random processing array module 130 can receive at least two first data to be processed simultaneously in parallel.

[0079] For example, Figure 3 is a schematic structural diagram of the random processing array module provided by the present invention. As Figure 3 shown, the random processing array module 130 adopts a 3-parallel input mode, that is, the feature maps X0, X1, and X2 can be input into the random processing array module 130 for parallel processing simultaneously.

[0080] Optionally, the specific parallel input mode adopted by the random processing array module 130 (for example, 2-parallel input mode, 3-parallel input mode, or 5-parallel input mode) can be set arbitrarily according to requirements, and the present invention does not make specific limitations on this.

[0081] A convolutional neural network accelerator for data processing provided by the present invention speeds up the convolution calculation rate in the neural network by adopting a parallel input mode, thereby reducing the hardware consumption caused by convolution calculations.

[0082] Optionally, the random processing array module 130 includes a second preset number of parallel modules;

[0083] Among them, different ones of the parallel modules correspond to different delay lines, one of the parallel modules corresponds to at least one of the delay lines, and the at least one of the delay lines corresponding to the one parallel module is configured to receive at least one of the first data to be processed in parallel, obtain the at least one of the first data to be processed that has been transmitted through the at least one of the delay lines as first data, and input the first data into the corresponding parallel module.

[0084] Optionally, the random processing array module 130 may include a second preset number of parallel modules.

[0085] Optionally, different parallel modules may correspond to different delay lines.

[0086] Optionally, one parallel module may correspond to at least one delay line.

[0087] Optionally, the at least one delay line corresponding to one parallel module may be configured to receive at least one first data to be processed in parallel, obtain the at least one first data to be processed that has been transmitted through the at least one delay line as first data, and may input the first data into the corresponding parallel module.

[0088] Optionally, the number of parallel modules may be determined based on the size of the convolution kernel and the length of the feature map amplitude sequence after passing through the delay line, and the present invention does not make specific limitations thereto.

[0089] For example, in the case of calculating a k×k convolution and the length of the feature map amplitude sequence after passing through the delay line is m, the number of parallel modules may be determined to be where the symbol represents the ceiling value.

[0090] For example, as Figure 3 shown, the feature maps X0, X1, and X2 first pass through the delay lines corresponding to different parallel modules respectively, and then are input into the corresponding parallel modules (Parallel-module) for operation.

[0091] A convolutional neural network accelerator for data processing provided by the present invention parallelizes the data to be processed by setting parallel modules, thereby accelerating the convolutional calculation rate in the neural network and reducing the hardware consumption caused by convolutional calculation.

[0092] Optionally, the parallel module includes a first random conversion module and a data preprocessing module;

[0093] The first random conversion module is configured to convert the first data into second data, and the form of the second data is in the form of a random bit stream;

[0094] The data preprocessing module is used to perform scaling addition on the second data output by the first random conversion module to obtain a second output result.

[0095] Optionally, the parallel module may include a first random conversion module and a data preprocessing module.

[0096] Optionally, the first random conversion module may be used to convert the first data into second data in the form of a random bit stream.

[0097] Optionally, the data preprocessing module may be used to perform scaling addition on the second data output by the first random conversion module to obtain a second output result.

[0098] For example, in the case where the random processing array module 130 adopts a 3-parallel input mode, the data preprocessing module may be used to perform scaling addition on the 3 data output by the first random conversion module to obtain 3 corresponding output results.

[0099] Optionally, the first random conversion module may be used to convert the first data into second data in the form of a random bit stream based on the Sobol sequence.

[0100] For example, as Figure 3 shown, each parallel module (Parallel-module) includes a binary-to-stochastic (B-to-S) conversion module and a preprocessing unit (Pre-Processing Unit). In a binary-to-stochastic conversion module, a Sobol sequence generator (Sobol Gen) generates a pseudo-random number source S(i), i = 1, 2,..., L, which are respectively compared with the input X0, X1, and X2. If S(i) > X1, then 1 is output; otherwise, 0 is output. The random sequence output by the binary-to-stochastic conversion module is then sent to the preprocessing unit, where three data selectors are used to perform scaling addition respectively: (X0 + X1) / 2, (X1 + X2) / 2, and (X0 + X1 + X2) / 4.

[0101] A convolutional neural network accelerator for data processing provided by the present invention realizes the processing of data to be processed based on a random calculation method through a first random conversion module and a data preprocessing module, effectively reducing the complexity of convolutional calculation in the neural network, thereby reducing the hardware consumption caused by convolutional calculation and improving the hardware efficiency. Optionally, the first random conversion module includes a first sequence generator and a first comparator;

[0102] The first sequence generator is used to generate a first pseudo-random number;

[0103] The first comparator is used to compare the first pseudo-random number with the first data. When the first pseudo-random number is greater than the first data, it outputs 1; otherwise, it outputs 0.

[0104] Optionally, the first random transformation module may include a first sequence generator and a first comparator.

[0105] Optionally, the first sequence generator is used to generate a first pseudo-random number.

[0106] For example, Figure 3 the Sobol sequence generator in [reference] generates a pseudo-random number source S(i), where i = 1, 2,..., L.

[0107] Optionally, the first comparator can be used to compare the first pseudo-random number with the first data.

[0108] Optionally, when the first pseudo-random number is greater than the first data, 1 can be output; otherwise, 0 can be output.

[0109] For example, if S(i) > X1, then 1 is output; otherwise, 0 is output.

[0110] Optionally, the random processing array module 130 includes a bilinear random computing inner product module;

[0111] The bilinear random computing inner product module includes a second sequence generator, a second comparator, and a logical AND gate;

[0112] The second sequence generator and the second comparator are used to convert the first convolutional kernel weight into a second convolutional kernel weight, and the form of the second convolutional kernel weight is a random bit stream form;

[0113] The logical AND gate is used to calculate the product of the second output result and the second convolutional kernel weight to obtain a first product result.

[0114] Optionally, the random processing array module 130 may include a bilinear random computing inner product module.

[0115] Optionally, the bilinear random computing inner product module may include a second sequence generator, a second comparator, and a logical AND gate.

[0116] Optionally, the second sequence generator and the second comparator can be used to convert the first convolutional kernel weight into a second convolutional kernel weight.

[0117] Optionally, the logical AND gate can be used to calculate the product of the second output result and the second convolutional kernel weight to obtain a first product result.

[0118] For example, Figure 4 is a schematic structural diagram of the bilinear random computing inner product module provided by the present invention, as shown inFigure 4 As shown, the outputs of the preprocessing units in different parallel modules are input into the bilinear stochastic computing inner product module (TLSC-based Inner-Product, TLSC-IP). For the TLSC-IP module, the input of the feature map is already in the form of a bitstream, while the weights of the filter are still in binary form. Therefore, another Sobol sequence generator and corresponding comparator are required to convert the weights of the convolution kernel into the form of a random bitstream, and the logical AND gate implements the product of the bitstream forms of the feature map input and the convolution kernel weights.

[0119] A convolutional neural network accelerator for data processing provided by the present invention processes the data to be processed based on the bilinear stochastic computing method through the bilinear stochastic computing inner product module, further effectively reducing the complexity of the convolutional calculation in the neural network, thereby reducing the hardware consumption caused by the convolutional calculation and improving the hardware efficiency.

[0120] Optionally, a demultiplexer is configured after the logical AND gate;

[0121] The demultiplexer is used to determine whether the first product result is positive or negative according to the sign bit of the second convolution kernel weight;

[0122] In the case where the first product result is positive, the first product result is classified into the positive set;

[0123] In the case where the first product result is negative, the first product result is classified into the negative set.

[0124] Optionally, a demultiplexer can be configured after the logical AND gate.

[0125] Optionally, the demultiplexer can be used to determine whether the first product result is positive or negative according to the sign bit of the second convolution kernel weight.

[0126] Optionally, in the case where the first product result is positive, the first product result can be classified into the positive set.

[0127] Optionally, in the case where the first product result is negative, the first product result can be classified into the negative set.

[0128] For example, as Figure 4 shown, a demultiplexer is configured after each logical AND gate to classify the corresponding product result into the positive set and the negative set according to the sign bit of the convolution kernel weight.

[0129] Compared with traditional single-linear stochastic computing, the bilinear stochastic computing proposed by the present invention introduces an additional symbolic bit stream, which can more accurately represent signed numbers, thereby shortening the length of the bit stream required for convolution computing.

[0130] A convolutional neural network accelerator for data processing provided by the present invention processes data to be processed based on the bilinear stochastic computing method through a bilinear stochastic computing inner product module, further effectively reducing the complexity of convolution computing in the neural network, thereby reducing the hardware consumption caused by convolution computing and improving the hardware efficiency.

[0131] Optionally, the bilinear stochastic computing inner product module further includes a first approximate accumulation counter, a second approximate accumulation counter, and a first subtractor;

[0132] The first approximate accumulation counter is used to accumulate the third data in the positive number set to obtain a first accumulation result;

[0133] The second approximate accumulation counter is used to accumulate the fourth data in the negative number set to obtain a second accumulation result;

[0134] The first subtractor is used to subtract the second accumulation result from the first accumulation result to obtain a second output result.

[0135] Optionally, the bilinear stochastic computing inner product module may further include a first approximate accumulation counter, a second approximate accumulation counter, and a first subtractor.

[0136] Optionally, the first approximate accumulation counter may be used to accumulate the third data in the positive number set to obtain a first accumulation result.

[0137] Optionally, the second approximate accumulation counter may be used to accumulate the fourth data in the negative number set to obtain a second accumulation result.

[0138] Optionally, the first subtractor may be used to subtract the second accumulation result from the first accumulation result to obtain a second output result.

[0139] For example, as Figure 4 shown, the data in the positive number set and the negative number set are respectively input into two approximate accumulation counters (Approximate Parallel Counter, APC) for accumulation, and the results output by the two APCs are input into a subtractor (Subtractor) for subtraction, and finally the result is output.

[0140] A convolutional neural network accelerator for data processing provided by the present invention processes data to be processed based on a bilinear stochastic calculation method through a bilinear stochastic calculation inner product module, further effectively reducing the complexity of convolutional calculations in a neural network, thereby reducing the hardware consumption caused by convolutional calculations and improving hardware efficiency.

[0141] Optionally, the convolutional neural network accelerator for data processing further includes:

[0142] A crossbar, the data processing unit is connected to the adder array unit through the crossbar, and the crossbar is used to adjust the third output result of the data processing unit.

[0143] Optionally, the convolutional neural network accelerator may further include a crossbar.

[0144] Optionally, the data processing unit 110 may be connected to the adder array unit 120 through a crossbar.

[0145] Optionally, the crossbar may be used to adjust the third output result of the data processing unit 110.

[0146] For example, as Figure 2 shown, the crossbar (Crossbar) adjusts the third output result of the data processing unit 110 so that the third output result can be correctly input into the corresponding adder.

[0147] Optionally, the adder array unit 120 may include a stochastic-to-binary conversion module (Stochastic-to-Binary, S-to-B).

[0148] Optionally, the stochastic-to-binary conversion module may be used to convert data in the form of a random bit stream into data in binary form.

[0149] For example, as Figure 2 shown, after the result output from the parallel processing unit enters the adder array unit 120, the result is first accumulated by an adder, and then the accumulated result is input into the stochastic-to-binary conversion module, and the output result is converted into binary and output as the final convolutional result.

[0150] Optionally, the first sequence generator is shared by the second preset number of parallel modules.

[0151] Optionally, the first sequence generator may be shared by the second preset number of parallel modules.

[0152] For example, as Figure 3 shown, the Sobol sequence generator module is It is shared by multiple parallel modules, that is to say, all parallel modules only need one sequence generator parallel module.

[0153] The present invention will be introduced below through a specific embodiment.

[0154] For example, during the image processing process, when calculating a 3×3 convolution, the convolution neural network accelerator proposed by the present invention may include 3 data processing units, and each data processing unit can implement a 3-parallel 3-point fast FIR algorithm based on stochastic computing. The specific implementation process is as follows:

[0155] (1) Input the data to be processed, such as feature maps X0, X1, and X2. After passing through the delay lines corresponding to different parallel modules, they are input to the corresponding parallel modules. For example, after feature maps X0, X1, and X2 pass through the delay lines corresponding to different parallel modules, X0′, X1′, and X2′ are obtained and input to the first parallel module;

[0156] (2) In the first parallel module, X0′, X1′, and X2′ in binary form are first converted into X0″, X1″, and X2″ in random bitstream form by the random conversion module. Then, the preprocessing unit uses three data selectors to perform scaling addition on X0″, X1″, and X2″: (X0″ + X1″) / 2, (X1″ + X2″) / 2, and (X0″ + X1″ + X2″) / 4, and inputs the obtained results y0, y1, and y2 to the bilinear stochastic computing inner product module;

[0157] (3) In the bilinear stochastic computing inner product module, the filter weights w0, w1, and w2 in binary form are first converted into filter weights w0′, w1′, and w2′ in random bitstream form. Then, the logical AND gates implement multiplying y0, y1, and y2 with w0′, w1′, and w2′ respectively. The obtained product results are r0, r1, and r2 respectively. The demultiplexer classifies r0, r1, and r2 into the positive number set or the negative number set according to the sign bits of w0′, w1′, and w2′. For example, r0 is classified into the negative number set, r1 and r2 are classified into the positive number set, r0 is input to the second approximate accumulator, r1 and r2 are input to the first approximate accumulator for accumulation, and finally the accumulation results of the two accumulators are input to the subtractor to obtain the output data R;

[0158] (4) Within every L clock cycles, the data processing unit outputs 3×3 partial sums (R0, R2, …, R8). The cross-switch readjusts these partial sums and then sends them to the corresponding adder for accumulation summation to obtain the target convolution result.

[0159] Taking the application of the present invention in image processing as an example, Figure 5It is a graph showing the variation of the recognition rate of different convolutional architectures of the LeNet-5 convolutional neural network based on the MNIST dataset provided by the present invention with the change of sequence length / quantization bit width. As Figure 5 shown, the abscissa represents the sequence length / quantization bit width (Quantization), and the ordinate represents the recognition rate (Accuracy). To achieve an approximate recognition accuracy of 98.8%, the traditional design based on bipolar stochastic computing and approximate parallel accumulator counter (BSC-APC) requires a sequence about 256-bit long, which is about 16 times longer than that of the design based on unipolar stochastic computing and approximate parallel accumulator counter (USC-APC) and the present invention (This work). In contrast, the present invention and USC-APC can achieve performance similar to that of fixed-point binary under the same quantization degree as fixed-point binary.

[0160] Table 1 is a comparison of the hardware integration results of the present invention based on the SMIC-65nm process. It can be seen from Table 1 that due to the simple operation characteristics of stochastic computing, all designs based on stochastic computing (the present invention, USC-APC, and BSC-APC) have achieved high system frequencies, low area, and low power consumption. However, due to the need for a longer bit stream in the traditional BSC-APC, the BSC-APC design is lower than the fixed-point binary implementation in terms of energy efficiency and area efficiency. Similarly, although the traditional USC-APC design can achieve high area efficiency, it is still lower than the fixed-point binary design in terms of the energy efficiency index. Compared with the fixed-point binary design, the design of the present invention can achieve a 1.11-fold increase in energy efficiency and a 1.64-fold increase in area efficiency. Moreover, the energy and area efficiency of the present invention are 19 times and 22 times that of BSC-APC respectively. Compared with USC-APC, the energy and area efficiency of the present invention also reach about 1.4 times and 1.3 times that of it.

[0161] Table 1 Comparison of Hardware Integration Results Based on SMIC-65nm Process

[0162] Comparison category The present invention USC-APC BSC-APC Fixed-point binary Sequence length / Quantization 16 16 256 5 Frequency [MHz] 909 909 909 680 <![CDATA[Area [um 2 > 5,887 2,543 2,681 23,161 Power consumption 3.91 1.78 1.54 10.4 Degree of parallelism 3 1 1 3 Throughput rate 8.52 2.84 0.178 20.4 Energy efficiency 2,179 1,595 115.6 1,962 Area efficiency 1,448 1,117 66.39 880.8

[0163] A convolutional neural network accelerator for data processing provided by the present invention first processes the first data to be processed through the stochastic processing array module of the data processing unit based on the fast finite impulse response FIR algorithm and bilinear stochastic computing to obtain a first output result, and then accumulates and sums the first output result through the adder array unit to obtain the target convolution result. Since the fast finite impulse response FIR algorithm only includes simple multiplication and addition calculations, and combines with the bilinear stochastic computing method with low complexity and high fault tolerance, it effectively reduces the complexity of the convolution calculation in the neural network, thereby reducing the hardware consumption brought by the convolution calculation and improving the hardware efficiency.

[0164] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A convolutional neural network accelerator for data processing, characterized in that, Including: A data processing unit and an adder array unit; The data processing unit includes a first preset number of random processing array modules, and each random processing array module is used to process first data to be processed based on the fast finite impulse response (FIR) algorithm and bilinear stochastic calculation to obtain a first output result; The adder array unit is used to perform cumulative summation on the first output result to obtain a target convolution result; The random processing array module includes a bilinear stochastic calculation inner product module; The bilinear stochastic calculation inner product module includes a second sequence generator, a second comparator, and a logic AND gate; The second sequence generator and the second comparator are used to convert a first convolution kernel weight into a second convolution kernel weight, and the form of the second convolution kernel weight is a random bit stream form; The logic AND gate is used to calculate the product of a second output result and the second convolution kernel weight to obtain a first product result; The second output result is obtained by scaling second data obtained through a first data conversion; The first data belongs to the first data to be processed.

2. The convolutional neural network accelerator for data processing according to claim 1, characterized in that The input mode of the random processing array module is a parallel input mode, and the parallel input mode indicates that the random processing array module can receive at least two pieces of the first data to be processed in parallel.

3. The convolutional neural network accelerator for data processing according to claim 1 or 2, characterized in that, The random processing array module includes a second preset number of parallel modules; Wherein, different parallel modules correspond to different delay lines, one parallel module corresponds to at least one delay line, and the at least one delay line corresponding to the one parallel module is used to receive at least one piece of the first data to be processed in parallel, obtain the at least one piece of the first data to be processed transmitted through the at least one delay line as first data, and input the first data into the corresponding parallel module.

4. The convolutional neural network accelerator for data processing according to claim 3, characterized in that, The parallel module includes a first random conversion module and a data preprocessing module; The first random conversion module is used to convert the first data into second data, and the form of the second data is a random bit stream form; The data preprocessing module is used to perform scaling addition on the second data output by the first random conversion module to obtain a second output result.

5. The convolutional neural network accelerator for data processing according to claim 4, wherein The first random conversion module includes a first sequence generator and a first comparator; The first sequence generator is used to generate a first pseudo-random number; The first comparator is used to compare the first pseudo-random number with the first data, and output 1 when the first pseudo-random number is greater than the first data, otherwise output 0.

6. The convolutional neural network accelerator for data processing according to claim 1, wherein The adder array unit includes a random-to-binary conversion module, and the random-to-binary conversion module is used to convert data in the form of a random bit stream into data in the binary form.

7. The convolutional neural network accelerator for data processing according to claim 1, characterized in that A demultiplexer is configured after the logic AND gate; The demultiplexer is used to judge whether the first product result is positive or negative according to the sign bit of the second convolution kernel weight; When the first product result is positive, classify the first product result into the positive number set; When the first product result is negative, classify the first product result into the negative number set.

8. The convolutional neural network accelerator for data processing according to claim 7, wherein The bilinear stochastic computing inner product module further includes a first approximate accumulation counter, a second approximate accumulation counter, and a first subtractor; The first approximate accumulation counter is used to accumulate the third data in the positive number set to obtain a first accumulation result; The second approximate accumulation counter is used to accumulate the fourth data in the negative number set to obtain a second accumulation result; The first subtractor is used to subtract the second accumulation result from the first accumulation result to obtain a second output result.

9. The convolutional neural network accelerator for data processing according to claim 1, characterized in that It further includes: A crossbar switch. The data processing unit is connected to the adder array unit through the crossbar switch, and the crossbar switch is used to adjust the third output result of the data processing unit.

10. The convolutional neural network accelerator for data processing according to claim 5, wherein The first sequence generator is shared by the second preset number of parallel modules.

Images