Handwritten numeral recognition method and accelerator based on FPGA and Winograd algorithm

Abstract

The invention relates to the technical field of convolutional neural networks, and discloses a handwritten number recognition method and accelerator based on an FPGA and a Winograd algorithm, an image sensor is used for collecting a handwritten number image, the handwritten number image is preprocessed to obtain the preprocessed handwritten number image, an MNIST database is used as a training sample library, and the training sample library is used for training the handwritten number image. The method comprises the following steps: building a handwritten numeral recognition network by taking a LeNet-5 network model as a basic model, inputting a preprocessed handwritten numeral image as a feature map into the handwritten numeral recognition network improved based on a Winograd algorithm, carrying out convolution processing on an FPGA (Field Programmable Gate Array) by using the Winograd algorithm, entering a full connection layer after pooling operation, realizing the full connection layer on the FPGA, and obtaining a handwritten numeral recognition result. Multiplication and accumulation operations of the weight and the input feature map are completed, and an identification result of the handwritten numerals is obtained through an output layer; according to the method, the parallel processing capacity of the FPGA and optimization of the Winograd algorithm on convolution operation are utilized, the single convolution speed is increased, the recognition speed and accuracy are improved, and the method can meet application scenes with high real-time requirements.

Description

Technical Field

[0001] The present invention relates to the technical field of convolutional neural networks, and in particular to a handwritten digit recognition method and an accelerator based on FPGA and Winograd algorithm. Background Art

[0002] Handwritten digit recognition has a wide range of applications in many fields, such as postal code recognition, financial bill processing, handwriting input systems, etc. Traditional software-based recognition methods have low computational efficiency when processing large-scale data and are difficult to meet application scenarios with high real-time requirements. With the development of hardware technology, FPGA has gradually become an ideal platform for implementing efficient image processing and pattern recognition algorithms due to its advantages such as reconfigurability, parallel processing capabilities and low power consumption. At the same time, the Winograd algorithm can significantly reduce the amount of multiplication operations in convolution operations and improve the calculation speed. Therefore, it is of great significance to study a handwritten digit recognition network accelerator based on FPGA and Winograd algorithm. Summary of the invention

[0003] The purpose of the present invention is to solve the above problems and design a handwritten digit recognition method and accelerator based on FPGA and Winograd algorithm.

[0004] A first aspect of the present invention provides a handwritten digit recognition method based on FPGA and Winograd algorithm, the handwritten digit recognition method based on FPGA and Winograd algorithm comprising the following steps:

[0005] Using an image sensor to collect a handwritten digital image, and preprocessing the handwritten digital image to obtain a preprocessed handwritten digital image, wherein the preprocessing includes at least grayscale conversion and denoising;

[0006] The MNIST database is used as a training sample library, and a handwritten digit recognition network is built based on the LeNet-5 network model. The preprocessed handwritten digit image is used as a feature map to input the handwritten digit recognition network improved based on the Winograd algorithm, wherein the handwritten digit recognition network is a seven-layer convolutional neural network, and the convolutional neural network adopts a parallel computing architecture of multiple convolution kernels;

[0007] The Winograd algorithm is used for convolution processing on the FPGA. The Winograd convolution converts the input feature map and the convolution kernel. After the convolution processing, the Relu activation function is applied for output. The output of the activation function is processed by the maximum pooling operation. After the pooling operation, it enters the fully connected layer. The fully connected layer is implemented on the FPGA to complete the multiplication and accumulation operations of the weights and the input feature map, and the recognition result of the handwritten digits is obtained through the output layer.

[0008] Optionally, in a first implementation of the first aspect of the present invention, the step of collecting a handwritten digital image using an image sensor and preprocessing the handwritten digital image to obtain a preprocessed handwritten digital image includes:

[0009] Using an image sensor to collect a handwritten digital image, performing grayscale processing on the handwritten digital image, and converting a color RGB image into a grayscale image;

[0010] The Gaussian function is used to perform convolution operation on the grayscale handwritten digital image, the Gaussian noise is smoothed, and the gradient amplitude and direction of the smoothed handwritten digital image are calculated using the Sobel operator, the non-maximum value of the gradient amplitude is suppressed, the local maximum value point of the gradient amplitude is retained, the non-edge points are suppressed, and the edge is refined;

[0011] By setting high thresholds and low thresholds, the gradient amplitude is divided into strong edges, weak edges and non-edges. By connecting weak edges and strong edges, the preprocessed handwritten digital image is obtained, where a strong edge is a point with a gradient amplitude greater than the high threshold, a weak edge is a point with a gradient amplitude between the high threshold and the low threshold, and a non-edge is a point with a gradient amplitude less than the low threshold.

[0012] Optionally, in a second implementation of the first aspect of the present invention, the convolution operation is performed on the grayscale handwritten digital image using a Gaussian function, the Gaussian noise is smoothed, the gradient amplitude and direction of the smoothed handwritten digital image are calculated using a Sobel operator, the non-maximum values ​​of the gradient amplitude are suppressed, the maximum points of the local gradient amplitude are retained, the non-edge points are suppressed, and the edge is refined, including:

[0013] Traverse each pixel in the grayscale handwritten digital image, construct an N×N structure with the current pixel as the center, select the K pixel values ​​closest to the current pixel value in the N×N structure, calculate the median of the K pixels, and replace the current pixel value, and use the convolution operation to apply the Gaussian filter to the grayscale handwritten digital image for smoothing, where N=5, K=9;

[0014] The pixels of the smoothed handwritten digital image are obtained in turn through a 3×3 sliding window. The pixel values ​​obtained by the 3×3 sliding window are convolved with the 8-directional convolution factor template to obtain the gradient value. The absolute value average of the calculated gradients in each direction is taken as the gradient amplitude of the pixel:

[0015]

[0016] Where G0, G1, G2, G3, G4, G5, G6, and G7 represent the gradient values ​​calculated by the convolution factor templates in 8 directions, and G0, G1, G2, G3, G4, G5, G6, and G7 correspond to 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°, respectively.

[0017] Optionally, in a third implementation of the first aspect of the present invention, the step of dividing the gradient amplitude into strong edges, weak edges, and non-edges by setting a high threshold and a low threshold, and obtaining a preprocessed handwritten digital image by connecting the weak edges and the strong edges includes:

[0018] Calculate the local mean μ(x,y) and local standard deviation σ(x,y) of the gradient magnitude:

[0019]

[0020] M norm =M(x,y) / max(M(x,y))

[0021] Where N represents the size of the local window, M norm represents the normalized gradient magnitude, (x, y) represents the coordinates of the current pixel, and (i, j) represents the coordinates within the local window;

[0022] The local high threshold and low threshold are determined according to the local mean and standard deviation, and the image is processed using the threshold to obtain the initial edge matrix. The pixel ratio value E is obtained according to the echo main-sidelobe ratio. R :

[0023]

[0024] Where P m is the main lobe peak, P s is the sidelobe peak, E max is the maximum intensity of the echo information;

[0025] Find the initial edge matrix that is not 0 and has an amplitude greater than the pixel ratio value E R , calculate the mean of the found pixels, and perform edge connection.

[0026] Optionally, in a fourth implementation of the first aspect of the present invention, the MNIST database is used as a training sample library, and a handwritten digit recognition network is built based on a LeNet-5 network model, including:

[0027] In one-dimensional convolution, let the input signal be d = [d0, d1, d2, d3] T , the convolution kernel is g = [g0, g1, g2] T, converting the convolution process into matrix multiplication:

[0028]

[0029] In the formula, r0=d0×g0+d1×g1+d2×g2, r1=d1×g0+d2×g1+d3×g2 performs six multiplications and four additions;

[0030] Introducing the Winograd algorithm, let r0 = m1 + m2 + m3, r1 = m2 - m3 - m4, reducing the number of multiplications to four and sorting the intermediate variables to obtain:

[0031]

[0032] Calculate the output F after the convolution operation:

[0033] F=A T [(Gg)⊙(B T d)]

[0034]

[0035]

[0036] In the formula, ⊙ represents the multiplication of corresponding positions, A T represents the transpose of the transformation matrix A, B T represents the transpose of the transformation matrix BB, G represents the convolution kernel matrix, and g represents the convolution kernel.

[0037] Optionally, in a fifth implementation of the first aspect of the present invention, the MNIST database is used as a training sample library, and a handwritten digit recognition network is built based on a LeNet-5 network model, further comprising:

[0038] In two-dimensional convolution, the matrix is ​​multiplied and then split, and the resulting matrix multiplication format is:

[0039]

[0040] In the formula, K0, W0, and R0 represent the vectors corresponding to the segmented areas in the above figure.

[0041] Let D0=[k0,k1,k2,k3] T represents the elements in the 0th row of the image window, D1, D2, D3 and so on represent the elements in the 1st, 2nd and 3rd rows of the image window, W0 = [w0,w Q ,w2] T Represents the weight value of the first row in the convolution kernel, W1, W2 and so on represent the weight values ​​of the second and third rows in the convolution kernel:

[0042]

[0043] Winograd convolution converts the input feature map and convolution kernel to obtain the intermediate result, and obtains the output result through the conversion matrix:

[0044] A T [[G[W0,W1,W2]G T ]⊙[B T [D0,D1,D2,D3]B]]A=A T [[GgG T ]⊙[B T dB]]A

[0045] In the formula, A, B, and G all represent transformation matrices. A and B are used to convert the input feature map and convolution kernel into intermediate results, and G is used to convert the intermediate results into the final output results.

[0046] Optionally, in a sixth implementation of the first aspect of the present invention, algorithm acceleration is provided for a convolutional neural network through ARM and FPGA, a convolutional neural network is constructed on the FPGA using the Verilog hardware description language in the FPGA part, and various calculations in the convolutional neural network are processed in parallel, and data is exchanged between ARM and FPGA through the AXI protocol.

[0047] The second aspect of the present invention provides a handwritten digit recognition accelerator based on FPGA and Winograd algorithm, comprising:

[0048] An image acquisition module, used to acquire a handwritten digital image using an image sensor, and preprocess the handwritten digital image to obtain a preprocessed handwritten digital image, wherein the preprocessing includes at least grayscale conversion and denoising;

[0049] A model building module is used to use the MNIST database as a training sample library, build a handwritten digit recognition network based on the LeNet-5 network model, and input the preprocessed handwritten digit image as a feature map into the handwritten digit recognition network improved based on the Winograd algorithm, wherein the handwritten digit recognition network is a seven-layer convolutional neural network, and the convolutional neural network adopts a parallel computing architecture of multiple convolution kernels;

[0050] The model recognition module is used to perform convolution processing using the Winograd algorithm on the FPGA. The Winograd convolution converts the input feature map and the convolution kernel. After the convolution processing, the Relu activation function is applied for output. The output of the activation function is processed by the maximum pooling operation. After the pooling operation, it enters the fully connected layer. The fully connected layer is implemented on the FPGA to complete the multiplication and accumulation operations of the weights and the input feature map, and the recognition result of the handwritten digits is obtained through the output layer.

[0051] The third aspect of the present invention provides an electronic device, comprising: a memory and at least one processor, wherein instructions are stored in the memory; at least one processor calls the instructions in the memory so that the electronic device executes the above-mentioned handwritten digit recognition method based on FPGA and Winograd algorithm.

[0052] A fourth aspect of the present invention provides a computer-readable storage medium, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the above-mentioned handwritten digit recognition method based on FPGA and Winograd algorithm.

[0053] In the technical solution provided by the present invention, an image sensor is used to collect a handwritten digital image, and the handwritten digital image is preprocessed to obtain a preprocessed handwritten digital image, an MNIST database is used as a training sample library, a handwritten digital recognition network is built based on a LeNet-5 network model, the preprocessed handwritten digital image is used as a feature map to input a handwritten digital recognition network improved based on a Winograd algorithm, and a convolution process is performed on an FPGA using the Winograd algorithm. The Winograd convolution converts the input feature map and the convolution kernel, and after the convolution process, a Relu activation function is applied for output, and a maximum pooling operation is used to process the output of the activation function, and after the pooling operation, a fully connected layer is entered, and the fully connected layer is implemented on the FPGA, and multiplication and accumulation operations of weights and input feature maps are completed, and a recognition result of handwritten digits is obtained through the output layer. The present invention utilizes the parallel processing capability of the FPGA and the optimization of the convolution operation by the Winograd algorithm to accelerate the single convolution speed, improve the speed and accuracy of handwritten digital recognition, and can meet application scenarios with high real-time requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the following detailed description of the preferred embodiment.The drawings are only for the purpose of illustrating the preferred embodiments and are not to be construed as limiting the invention.

[0055] Figure 1A flowchart of a handwritten digit recognition method based on FPGA and Winograd algorithm provided in an embodiment of the present invention;

[0056] Figure 2 A one-dimensional Winograd algorithm to two-dimensional convolution process diagram provided in an embodiment of the present invention;

[0057] Figure 3 A structural hierarchical diagram of a convolutional neural network provided by an embodiment of the present invention;

[0058] Figure 4 A flow chart of a convolutional neural network provided by an embodiment of the present invention;

[0059] Figure 5 A schematic diagram of the parallel structure of the convolutional layers of the convolutional neural network provided in an embodiment of the present invention;

[0060] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0062] In view of the problems of high computing density and low energy efficiency when running convolutional neural network forward reasoning on FPGA platform, the embodiment of the present invention starts from the direction of improving hardware computing speed, reducing power consumption and cost, and studies the design and application of convolutional neural network acceleration system based on ZYNQ. A convolutional neural network accelerator is designed by adopting the idea of ​​software-hardware collaborative design, which can perform the task of handwritten digit recognition. The embodiment of the present invention adopts the software-hardware collaborative design method to divide the functions of the accelerator and proposes an overall architecture. The design is based on ZYNQ7010 platform and includes two main parts: ARM (PS) and FPGA (PL). In the PL part, the embodiment of the present invention draws on NVDL A hardware architecture design is based on the FPGA, and a configurable convolutional neural network accelerator IP is implemented on the FPGA using the Verilog hardware description language. The accelerator IP has independently modularized the various computing modules in the convolutional neural network to achieve flexible deployment and optimization of various network models. In the PS part, a control system circuit with an ARM processor as the core is used to manage and control the operation of the entire system. The PS and PL interact with each other through the AXI protocol to achieve efficient data transmission and communication. The PL and PS are integrated to form the entire SoC hardware system, which provides reliable hardware support and basic platform for the acceleration of the convolutional neural network algorithm.

[0063] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 The flowchart of the handwritten digit recognition method based on FPGA and Winograd algorithm provided in an embodiment of the present invention specifically comprises the following steps:

[0064] Step 101: using an image sensor to collect a handwritten digital image, and preprocessing the handwritten digital image to obtain a preprocessed handwritten digital image;

[0065] Step 102: Use the MNIST database as a training sample library, build a handwritten digit recognition network based on the LeNet-5 network model, and input the preprocessed handwritten digit image as a feature map into the handwritten digit recognition network improved based on the Winograd algorithm;

[0066] Step 103: Use the Winograd algorithm to perform convolution processing on the FPGA. The Winograd convolution converts the input feature map and the convolution kernel. After the convolution processing, the Relu activation function is applied for output. The output of the activation function is processed by the maximum pooling operation. After the pooling operation, it enters the fully connected layer. The fully connected layer is implemented on the FPGA to complete the multiplication and accumulation operations of the weight and the input feature map, and the recognition result of the handwritten digits is obtained through the output layer.

[0067] In this embodiment, the preprocessing includes at least graying and denoising; using an image sensor to collect a handwritten digital image, graying the handwritten digital image, and converting a color RGB image into a grayscale image; using a Gaussian function to perform a convolution operation on the grayed handwritten digital image, smoothing the Gaussian noise, using a Sobel operator to calculate the gradient amplitude and direction of the smoothed handwritten digital image, suppressing the non-maximum value of the gradient amplitude, retaining the local maximum value point of the gradient amplitude, suppressing non-edge points, and performing edge refinement; by setting a high threshold and a low threshold, the gradient amplitude is divided into a strong edge, a weak edge, and a non-edge, and by connecting the weak edge and the strong edge, a preprocessed handwritten digital image is obtained, wherein a strong edge is a point whose gradient amplitude is greater than a high threshold, a weak edge is a point whose gradient amplitude is between a high threshold and a low threshold, and a non-edge is a point whose gradient amplitude is less than a low threshold;

[0068] Traverse each pixel in the grayscale handwritten digital image, construct an N×N structure with the current pixel as the center, select the K pixel values ​​closest to the current pixel value in the N×N structure, calculate the median of the K pixels, and replace the current pixel value, and use the convolution operation to apply the Gaussian filter to the grayscale handwritten digital image for smoothing, where N=5, K=9;

[0069] The pixels of the smoothed handwritten digital image are obtained in turn through a 3×3 sliding window. The pixel values ​​obtained by the 3×3 sliding window are convolved with the 8-directional convolution factor template to obtain the gradient value. The absolute value average of the calculated gradients in each direction is taken as the gradient amplitude of the pixel:

[0070]

[0071] Where G0, G1, G2, G3, G4, G5, G6, and G7 represent the gradient values ​​calculated by the convolution factor templates in 8 directions, and G0, G1, G2, G3, G4, G5, G6, and G7 correspond to 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°, respectively;

[0072] Calculate the local mean μ(x,y) and local standard deviation σ(x,y) of the gradient magnitude:

[0073]

[0074] M norm =M(x,y) / max(M(x,y))

[0075] Where N represents the size of the local window, M norm represents the normalized gradient magnitude, (x, y) represents the coordinates of the current pixel, and (i, j) represents the coordinates within the local window;

[0076] The local high threshold and low threshold are determined according to the local mean and standard deviation, and the image is processed using the threshold to obtain the initial edge matrix. The pixel ratio value E is obtained according to the echo main-sidelobe ratio. R :

[0077]

[0078] Where P m is the main lobe peak, P s is the sidelobe peak, E max is the maximum intensity of the echo information;

[0079] Find the initial edge matrix that is not 0 and has an amplitude greater than the pixel ratio value E R , calculate the mean of the found pixels, and perform edge connection.

[0080] In this embodiment, the handwritten digit recognition network is a seven-layer convolutional neural network, and the convolutional neural network adopts a parallel computing architecture of multiple convolution kernels; the handwritten digit recognition network is built, the LeNet-5 network model is used as the basic model, the MNIST database is used as the training sample library, and a seven-layer convolutional neural network (Convolutional Neural Network) is built, referred to as CNN;

[0081] Due to the small size, low power consumption and low cost of edge embedded devices, their computing power is also limited to a certain extent. The embodiment of the present invention strikes a balance between computing power and power consumption. In order to enable the network system to be successfully deployed on the hardware platform, firstly, in terms of calculation, the method of converting floating-point weights and biases into fixed-point calculations is adopted to reduce the amount of calculation and storage requirements. Secondly, the embodiment of the present invention also expands the loop structure in blocks, divides the large loop into shorter loops, and expands them in combination with optimization instructions to make full use of the computing resources of the device. The above strategy can improve the stability and robustness of the CNN model, while accelerating convolution calculations and reducing system latency, achieving a balance between the accelerator's computing power and power consumption.

[0082] The embodiment of the present invention uses the Winograd algorithm, which is an algorithm for optimizing convolution operations. By converting the input, output and convolution kernel into smaller matrices, the number of multiplications is reduced while speeding up the calculation. In one-dimensional convolution, the input signal is d = [d0, d1, d2, d3] T , the convolution kernel is g = [g0, g1, g2] T , converting the convolution process into matrix multiplication:

[0083]

[0084] In the formula, r0=d0×g0+d1×g1+d2×g2, r1=d1×g0+d2×g1+d3×g2 performs six multiplications and four additions;

[0085] Introducing the Winograd algorithm, let r0 = m1 + m2 + m3, r1 = m2 - m3 - m4, reducing the number of multiplications to four and sorting the intermediate variables to obtain:

[0086]

[0087] Calculate the output F after the convolution operation:

[0088] F=A T [(Gg)⊙(B T d)]

[0089]

[0090]

[0091] In the formula, ⊙ represents the multiplication of corresponding positions, A T represents the transpose of the transformation matrix A, B T represents the transpose of the transformation matrix BB, G represents the convolution kernel matrix, and g represents the convolution kernel;

[0092] Then derive the one-dimensional Winograd algorithm to the two-dimensional convolution process as follows Figure 2 As shown in the figure, during two-dimensional convolution, the matrix is ​​multiplied and then divided, and the resulting matrix multiplication format is:

[0093]

[0094] In the formula, K0, W0, and R0 represent the vectors corresponding to the segmented areas in the above figure.

[0095] Let D0=[k0,k1,k2,k3] T Represents the 0th row element in the image window, D1, D2, D3 and so on represent the 1st, 2nd and 3rd row elements in the image window, W0 = [w0, w1, w2] T Represents the weight value of the first row in the convolution kernel, W1, W2 and so on represent the weight values ​​of the second and third rows in the convolution kernel:

[0096]

[0097] The observation structure is similar to the output matrix of the one-dimensional convolution, which can be simply regarded as the convolution of two row vectors [W0, W1, W2] and [D0, D1, D2, D3]. Since the two-dimensional derivation just now is a column vector, the convolution of the row vector only needs to be right-multiplied by the transposed matrix of the transformation matrix, that is:

[0098] A T [[G[W0,W1,W2]G T ]⊙[B T [D0,D1,D2,D3]B]]A=A T [[GgG T ]⊙[B T dB]]A

[0099] In the formula, A, B, and G all represent transformation matrices. A and B are used to convert the input feature map and convolution kernel into intermediate results, and G is used to convert the intermediate results into the final output results.

[0100] The transformation reduces the number of convolution operations and the number of multiplication and addition operations, thereby improving computational efficiency. Specifically, Winograd convolution converts the input feature map and the convolution kernel to obtain the intermediate result, and obtains the output result through the transformation matrix. The calculation of the transformation matrix only needs to be performed once and can be performed offline, so it can greatly improve computational efficiency in practical applications.

[0101] In terms of network structure, simplify the network structure level, such as Figure 3 As shown in the figure, the number of convolutional layers and pooling layers is reduced, and the number of layers of the entire network is reduced to achieve the effect of streamlining the network and thus accelerating the entire recognition process; in terms of hardware acceleration, the data flow is accelerated through pipeline processing, and the data flow is as follows Figure 4 As shown in the figure, Winograd algorithm is used for convolution processing on FPGA. Winograd convolution converts the input feature map and convolution kernel. After convolution processing, Relu activation function is applied for output. The output of activation function is processed by maximum pooling operation. After pooling operation, it enters the fully connected layer. The fully connected layer is implemented on FPGA to complete the multiplication and accumulation operations of weights and input feature maps. The recognition result of handwritten digits is obtained through the output layer. A multi-layer convolution kernel parallel computing architecture is designed, as shown in Figure 5 As shown, in order to save the time of reading the convolution kernel weights and the feature map input feature values, a parallel structure is designed as shown in the figure. This structure combines the input feature map in parallel and the output feature map in parallel, reads ni input feature map values ​​and ni*no weight values ​​in each cycle, and writes no output neuron values.

[0102] In this embodiment, considering the limited on-chip storage, the traditional CNN network structure is adjusted, the number of convolution layers and pooling layers is reduced, and the on-chip storage pressure is reduced. For FPGA, since the multiplication speed is slow, the Winograd algorithm is used to reduce the number of multiplications in the convolution and accelerate the single convolution speed; for the entire convolution layer, a parallel structure is adopted to implement parallel convolution calculations for the same sampling window, so as to achieve the purpose of accelerating the entire convolution layer; for the pooling layer, since average pooling requires division and reduces the operation speed, the present invention adopts maximum pooling, which can accurately realize the recognition of handwritten digits and achieve the acceleration effect.

[0103] The handwritten digit recognition accelerator based on FPGA and Winograd algorithm in the embodiment of the present invention is described below. The handwritten digit recognition accelerator based on FPGA and Winograd algorithm in the embodiment of the present invention includes:

[0104] An image acquisition module is used to acquire a handwritten digital image using an image sensor, and preprocess the handwritten digital image to obtain a preprocessed handwritten digital image, wherein the preprocessing includes at least grayscale conversion and denoising;

[0105] The model building module is used to use the MNIST database as a training sample library, build a handwritten digit recognition network based on the LeNet-5 network model, and input the preprocessed handwritten digit image as a feature map into the handwritten digit recognition network improved based on the Winograd algorithm. The handwritten digit recognition network is a seven-layer convolutional neural network, and the convolutional neural network adopts a parallel computing architecture with multiple convolution kernels;

[0106] The model recognition module is used to perform convolution processing using the Winograd algorithm on the FPGA. The Winograd convolution converts the input feature map and the convolution kernel. After the convolution processing, the Relu activation function is applied for output. The output of the activation function is processed by the maximum pooling operation. After the pooling operation, it enters the fully connected layer. The fully connected layer is implemented on the FPGA to complete the multiplication and accumulation operations of the weights and the input feature map, and the recognition result of the handwritten digits is obtained through the output layer.

[0107] The electronic device in the embodiment of the present invention is described in detail from the perspective of hardware processing. Figure 66 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device 600 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 610 (for example, one or more processors) and a memory 620, and one or more storage media 630 (for example, one or more mass storage devices) storing application programs 633 or data 632. Among them, the memory 620 and the storage medium 630 can be short-term storage or permanent storage. The program stored in the storage medium 630 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the electronic device 600. Furthermore, the processor 610 can be configured to communicate with the storage medium 630, and execute a series of instruction operations in the storage medium 630 on the electronic device 600 to implement the handwritten digit recognition method based on FPGA and Winograd algorithm provided in the above implementation.

[0108] The electronic device 600 may also include one or more power supplies 640, one or more wired or wireless network interfaces 650, one or more input and output interfaces 660, and / or one or more operating devices 631, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 6 The electronic device structure shown does not constitute a limitation on the computer device provided by the present invention, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0109] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes each step of the handwritten digit recognition method based on FPGA and Winograd algorithm.

[0110] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A handwritten digit recognition method based on FPGA and Winograd algorithm, characterized in that: The handwritten digit recognition method based on FPGA and Winograd algorithm comprises the following steps: Using an image sensor to collect a handwritten digital image, and preprocessing the handwritten digital image to obtain a preprocessed handwritten digital image, wherein the preprocessing includes at least grayscale conversion and denoising; The MNIST database is used as a training sample library, and a handwritten digit recognition network is built based on the LeNet-5 network model. The preprocessed handwritten digit image is used as a feature map to input the handwritten digit recognition network improved based on the Winograd algorithm, wherein the handwritten digit recognition network is a seven-layer convolutional neural network, and the convolutional neural network adopts a parallel computing architecture of multiple convolution kernels; The Winograd algorithm is used for convolution processing on the FPGA. The Winograd convolution converts the input feature map and the convolution kernel. After the convolution processing, the Relu activation function is applied for output. The output of the activation function is processed by the maximum pooling operation. After the pooling operation, it enters the fully connected layer. The fully connected layer is implemented on the FPGA to complete the multiplication and accumulation operations of the weights and the input feature map, and the recognition result of the handwritten digits is obtained through the output layer.

2. A handwritten digit recognition method based on FPGA and Winograd algorithm as claimed in claim 1, characterized in that: The method of collecting a handwritten digital image using an image sensor and preprocessing the handwritten digital image to obtain a preprocessed handwritten digital image includes: Using an image sensor to collect a handwritten digital image, performing grayscale processing on the handwritten digital image, and converting a color RGB image into a grayscale image; The Gaussian function is used to perform convolution operation on the grayscale handwritten digital image, the Gaussian noise is smoothed, and the gradient amplitude and direction of the smoothed handwritten digital image are calculated using the Sobel operator, the non-maximum value of the gradient amplitude is suppressed, the local maximum value point of the gradient amplitude is retained, the non-edge points are suppressed, and the edge is refined; By setting high thresholds and low thresholds, the gradient amplitude is divided into strong edges, weak edges and non-edges. By connecting weak edges and strong edges, the preprocessed handwritten digital image is obtained, where a strong edge is a point with a gradient amplitude greater than the high threshold, a weak edge is a point with a gradient amplitude between the high threshold and the low threshold, and a non-edge is a point with a gradient amplitude less than the low threshold.

3. A handwritten digit recognition method based on FPGA and Winograd algorithm as claimed in claim 2, characterized in that: The convolution operation is performed on the grayscale handwritten digital image using a Gaussian function, the Gaussian noise is smoothed, the gradient amplitude and direction of the smoothed handwritten digital image are calculated using a Sobel operator, the non-maximum value of the gradient amplitude is suppressed, the maximum value point of the local gradient amplitude is retained, the non-edge points are suppressed, and the edge is refined, including: Traverse each pixel in the grayscale handwritten digital image, construct an N×N structure with the current pixel as the center, select the K pixel values ​​closest to the current pixel value in the N×N structure, calculate the median of the K pixels, and replace the current pixel value, and use the convolution operation to apply the Gaussian filter to the grayscale handwritten digital image for smoothing, where N=5, K=9; The pixels of the smoothed handwritten digital image are obtained in turn through a 3×3 sliding window. The pixel values ​​obtained by the 3×3 sliding window are convolved with the 8-directional convolution factor template to obtain the gradient value. The absolute value average of the calculated gradients in each direction is taken as the gradient amplitude of the pixel: Where G0, G1, G2, G3, G4, G5, G6, and G7 represent the gradient values ​​calculated by the convolution factor templates in 8 directions, and G0, G1, G2, G3, G4, G5, G6, and G7 correspond to 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°, respectively.

4. A handwritten digit recognition method based on FPGA and Winograd algorithm as claimed in claim 2, characterized in that: The method of dividing the gradient amplitude into strong edge, weak edge and non-edge by setting a high threshold and a low threshold, and obtaining the preprocessed handwritten digital image by connecting the weak edge and the strong edge, comprises: Calculate the local mean μ(x,y) and local standard deviation σ(x,y) of the gradient magnitude: M norm =M(x,y) / max(M(x,y)) Where N represents the size of the local window, M norm represents the normalized gradient magnitude, (x, y) represents the coordinates of the current pixel, and (i, j) represents the coordinates within the local window; The local high threshold and low threshold are determined according to the local mean and standard deviation, and the image is processed using the threshold to obtain the initial edge matrix. The pixel ratio value E is obtained according to the echo main-sidelobe ratio. R : Where P m is the main lobe peak, P s is the sidelobe peak, E max is the maximum intensity of the echo information; Find the initial edge matrix that is not 0 and has an amplitude greater than the pixel ratio value E R , calculate the mean of the found pixels, and perform edge connection.

5. A handwritten digit recognition method based on FPGA and Winograd algorithm as claimed in claim 1, characterized in that: The MNIST database is used as a training sample library, and a handwritten digit recognition network is built based on the LeNet-5 network model, including: In one-dimensional convolution, let the input signal be d = [d0, d1, d2, d3] T , the convolution kernel is g = [g0, g1, g2] T , converting the convolution process into matrix multiplication: In the formula, r0=d0×g0+d1×g1+d2×g2, r1=d1×g0+d2×g1+d3×g2 performs six multiplications and four additions; Introducing the Winograd algorithm, let r0 = m1 + m2 + m3, r1 = m2 - m3 - m4, reducing the number of multiplications to four and sorting the intermediate variables to obtain: Calculate the output F after the convolution operation: F=A T [(Gg)⊙(B T d)] In the formula, ⊙ represents the multiplication of corresponding positions, A T represents the transpose of the transformation matrix A, B T represents the transpose of the transformation matrix BB, G represents the convolution kernel matrix, and g represents the convolution kernel.

6. A handwritten digit recognition method based on FPGA and Winograd algorithm as claimed in claim 1, characterized in that: The method uses the MNIST database as a training sample library and uses the LeNet-5 network model as a basic model to build a handwritten digit recognition network, and also includes: In two-dimensional convolution, the matrix is ​​multiplied and then split, and the resulting matrix multiplication format is: In the formula, K0, W0, and R0 represent the vectors corresponding to the segmented areas in the above figure. Let D0=[k0,k1,k2,k3] T Represents the 0th row of elements in the image window, D1, D2, D3 and so on represent the 1st, 2nd and 3rd row of elements in the image window, W0 = [w0, w1, w2] T Represents the weight value of the first row in the convolution kernel, W1, W2 and so on represent the weight values ​​of the second and third rows in the convolution kernel: Winograd convolution converts the input feature map and convolution kernel to obtain the intermediate result, and obtains the output result through the conversion matrix: A T [[G[W0,W1,W2]G T ]⊙[B T [D0,D1,D2,D3]B]]A=A T [[GgG T ]⊙[B T dB]]A In the formula, A, B, and G all represent transformation matrices. A and B are used to convert the input feature map and convolution kernel into intermediate results, and G is used to convert the intermediate results into the final output results.

7. A handwritten digit recognition method based on FPGA and Winograd algorithm as claimed in claim 1, characterized in that: ARM and FPGA are used to provide algorithm acceleration for convolutional neural networks. In the FPGA part, the Verilog hardware description language is used to build a convolutional neural network on the FPGA. The various calculations in the convolutional neural network are processed in parallel, and data is exchanged between ARM and FPGA through the AXI protocol.

8. An accelerator used in the method according to any one of claims 1 to 7, characterized in that: include: An image acquisition module, used to acquire a handwritten digital image using an image sensor, and preprocess the handwritten digital image to obtain a preprocessed handwritten digital image, wherein the preprocessing includes at least grayscale conversion and denoising; A model building module is used to use the MNIST database as a training sample library, build a handwritten digit recognition network based on the LeNet-5 network model, and input the preprocessed handwritten digit image as a feature map into the handwritten digit recognition network improved based on the Winograd algorithm, wherein the handwritten digit recognition network is a seven-layer convolutional neural network, and the convolutional neural network adopts a parallel computing architecture of multiple convolution kernels; The model recognition module is used to perform convolution processing using the Winograd algorithm on the FPGA. The Winograd convolution converts the input feature map and the convolution kernel. After the convolution processing, the Relu activation function is applied for output. The output of the activation function is processed by the maximum pooling operation. After the pooling operation, it enters the fully connected layer. The fully connected layer is implemented on the FPGA to complete the multiplication and accumulation operations of the weights and the input feature map, and the recognition result of the handwritten digits is obtained through the output layer.

9. An electronic device, characterized in that: include: A memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory so that the electronic device executes the handwritten digit recognition method based on FPGA and Winograd algorithm as described in any one of claims 1 to 7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the handwritten digit recognition method based on FPGA and Winograd algorithm as described in any one of claims 1 to 7 is implemented.

Images