A handwritten numeral image recognition system
By deploying a handwritten digit image recognition system in FPGA hardware and utilizing RAM and inference accelerator modules for pure hardware neural network operations, the problems of high power consumption and low efficiency in existing technologies are solved, achieving low-cost, low-power, and high-efficiency recognition results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- PEKING UNIV
- Filing Date
- 2023-12-12
- Publication Date
- 2026-07-24
AI Technical Summary
Existing handwritten digit image recognition methods consume a lot of power and have low computational efficiency, making them difficult to deploy effectively in embedded systems with low power consumption and high real-time requirements.
The handwritten digit image recognition algorithm is deployed in FPGA hardware. The receiving module receives the image pixel values, the RAM control module stores and caches them, the inference accelerator module performs calculations using the weighted reassembly of a pre-trained fully connected neural network, and finally the sending module outputs the recognition results, realizing the function of a pure hardware neural network.
It reduces processing costs and power consumption, improves computing efficiency and real-time performance, and achieves low-power, high-efficiency recognition results.
Smart Images
Figure CN117953516B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the design of artificial intelligence inference accelerators, and more particularly to a handwritten digit image recognition system. Background Technology
[0002] In recent years, with the rapid development of Artificial Intelligence (AI) technology, the continuous maturation of machine learning algorithms, and the improvement of computing power, AI has gradually entered people's lives, providing convenience. Artificial Intelligence & Internet of Things (AIoT), ultra-low power consumption, and in-sensor computing are among the main directions for the future development of AI. AIoT applications typically require AI processors to have low power consumption and high real-time performance. However, the training and inference processes of neural network algorithms usually involve massive amounts of data computation, making it difficult for ordinary embedded systems to meet the performance and power consumption requirements of AIoT. How to apply neural network models to embedded edge devices has become a major research hotspot. Since the training process of neural network algorithms is currently difficult to simplify to be implemented in low-power, low-computing-power embedded systems, current AIoT research focuses on deploying the inference process of neural networks to embedded systems. How to miniaturize neural networks and deploy them to low-power, high-real-time embedded systems has become a current research focus.
[0003] Image recognition is an important application of artificial intelligence. Handwritten digit recognition, based on AI and machine learning, aims to analyze and recognize handwritten digit images and convert them into corresponding numerical results. Currently, traditional handwritten digit recognition methods mainly use neural networks. Deploying neural networks for handwritten digit recognition requires a combination of software and hardware. Some neural network calculations require the participation of a general-purpose CPU, resulting in unnecessary power consumption and low computational efficiency, thus necessitating improvement. Summary of the Invention
[0004] This invention provides a handwritten digit image recognition system to address the shortcomings of existing technologies that combine hardware and software to achieve handwritten digit image recognition, which suffer from high power consumption and low computational efficiency. The system fully deploys the handwritten digit image recognition algorithm on FPGA hardware for processing, thereby reducing the processing cost and power consumption of handwritten digit image recognition.
[0005] This invention provides a handwritten digit image recognition system, the system being deployed in an FPGA, the system comprising:
[0006] The receiving module is used to receive the pixel values corresponding to the pixels of the handwritten digit image to be recognized;
[0007] The RAM control module, which is connected to the receiving module, is used to store the pixel values corresponding to each pixel in the handwritten digit image to be recognized into RAM, and to load the pixel values in RAM into the cache sequentially with a preset cache capacity.
[0008] An inference accelerator module is connected to the RAM control module. The inference accelerator module includes a ROM, which stores multiple weighted sets with different digital labels. Each weighted set includes a preset weight value corresponding to each pixel. The inference accelerator module is used to perform calculations on the pixel values loaded into the cache and the preset weight values in the ROM each time, determine the target weighted set based on the calculation results, and output the digital label corresponding to the target weighted set as the recognition result. The preset weight values in the multiple weighted sets are determined according to the model parameters of a pre-trained fully connected neural network.
[0009] A sending module, which is connected to the inference accelerator module, is used to send the recognition result output by the inference accelerator module.
[0010] In some possible implementations, the preset cache capacity is equal to the data size corresponding to four pixels.
[0011] In some possible implementations, the inference accelerator module further includes: an address generation module, a multiplication module corresponding to each pixel value in the cache, an accumulation module corresponding to the weighted set, a comparison module, and an output module;
[0012] The address generation module is used to generate a ROM access address based on the pixel corresponding to the pixel value in each cache, and to extract a preset weight value corresponding to each pixel value in the cache from each weight set of the ROM based on the ROM access address.
[0013] The multiplication module is used to calculate the product of the pixel value in the cache with the preset weight value taken from each weight set and corresponding to each pixel value in the cache.
[0014] The accumulation module is used to accumulate the products belonging to the same weight group to obtain the sum corresponding to each weight group.
[0015] The comparison module is used to compare the sums of each weighted combination after the accumulation is completed, and to take the weighted combination with the largest sum as the target weighted combination.
[0016] The output module is used to match the target weighted reorganization with the preset mapping relationship to obtain the target digital label and output it. The preset mapping relationship represents the correspondence between the weighted reorganization and the digital label.
[0017] In some possible implementations, there are ten weighted reorganizations, with the ten weighted reorganizations corresponding to the numbers 0 to 9.
[0018] In some possible implementations, one ROM of the inference accelerator module is used to store all preset weight values for a weight reassembly.
[0019] In some possible implementations, the pre-trained fully connected neural network is trained according to the following steps:
[0020] Import and load the MNIST dataset;
[0021] A fully connected neural network model is constructed based on a preset mathematical model and the image size in the MNIST dataset. The preset mathematical model is Y = WX, where X represents the input of the fully connected neural network, the dimension of X is equal to the number of pixels corresponding to any image in the MNIST dataset after flattening it from two dimensions to one dimension, Y represents the output of the fully connected neural network, the dimension of Y is equal to the preset total number of expected predicted numbers, and W represents the connection layer weights between the input and output of the fully connected neural network.
[0022] The fully connected neural network model was trained in multiple rounds using the training and test sets in the MNIST dataset.
[0023] The fully connected neural network model trained through multiple rounds is used as the pre-trained fully connected neural network.
[0024] In some possible implementations, the preset weight values in multiple weight reassemblies are calculated according to the following steps:
[0025] Calculate the quantization coefficient according to Formula 1;
[0026]
[0027] Where s is the quantization coefficient, a is the maximum absolute value of the model parameters of the pre-trained fully connected neural network, and b is the logarithm of the maximum range of the quantized data with respect to 2.
[0028] The model parameters of the fully connected neural network are symmetrically quantized according to Formula 2 to obtain the quantized model parameter matrix.
[0029] M q =round(M·s) Formula 2;
[0030] Where M is the model parameter matrix of the fully connected neural network to be quantized, M q M·s is the quantized parameter matrix, s is the quantization coefficient, and the function round() represents rounding all elements in the M·s matrix.
[0031] If the input to a fully connected neural network is a row matrix, then one column of the quantized model parameter matrix corresponds to a preset weight value for a set of weights.
[0032] If the input to a fully connected neural network is a column matrix, then each row of the quantized model parameter matrix corresponds to a preset weight value for a set of weights.
[0033] In some possible implementations, the handwritten digit image recognition system further includes a clock generator;
[0034] The clock generator is used to provide a first clock signal with a frequency of 100MHz to the multiplication module, the accumulation module and the comparison module, so that any multiplication operation performed by the multiplication module, any addition operation performed by the accumulation module and any comparison operation performed by the comparison module are all completed within the period of the first clock signal.
[0035] In some possible implementations, the clock generator is also used to provide a second clock signal with a frequency of 50MHz to the receiving module, the transmitting module and the RAM control module.
[0036] In some possible implementations, the handwritten digit image recognition system further includes a conversion module;
[0037] The conversion module is used to convert the handwritten digit image to be recognized into a grayscale image before the receiving module receives the handwritten digit image to be recognized.
[0038] The handwritten digit image recognition system of the present invention receives the pixel values of the handwritten digit image to be recognized by deploying a receiving module in an FPGA and storing the pixel values by deploying a RAM control module. The inference accelerator module stores the preset weight values of each weight group determined by the model parameters of a pre-trained fully connected neural network in a ROM. The pixel values are then calculated with the preset weight values stored in the ROM, and the corresponding digit tag is obtained based on the calculation results. Finally, the digit tag is sent as the recognition result through a sending module. The system implements the function of a fully connected neural network in pure hardware, and has the advantages of low cost, low power consumption, high computational efficiency, and good real-time performance. Attached Figure Description
[0039] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0040] Figure 1 This is a schematic diagram of the overall structure of the handwritten digit image recognition system provided in an embodiment of the present invention;
[0041] Figure 2 This is a schematic diagram of the internal structure of the inference accelerator module provided in an embodiment of the present invention;
[0042] Figure 3 This is a schematic diagram of the overall structure of another handwritten digit image recognition system provided in an embodiment of the present invention;
[0043] Figure 4 This is a schematic diagram of the structure of a fully connected neural network provided in an embodiment of the present invention;
[0044] Figure 5 This is a flowchart of the neural network training process algorithm provided in an embodiment of the present invention;
[0045] Figure 6 This is a schematic diagram of the data processing flow of the handwritten digit image recognition system provided in an embodiment of the present invention;
[0046] Figure 7 This is a schematic diagram of the image set within the MNIST dataset provided in an embodiment of the present invention;
[0047] Figure 8 This is a schematic diagram of the prediction results of the host computer display handwritten digit image recognition system provided by the present invention.
[0048] [Attached image labels]
[0049] 100: Receiver module;
[0050] 200: RAM controller; 210: RAM; 220: Cache;
[0051] 300: Inference accelerator module; 310: ROM; 320: Address generation module; 330: Multiplication module; 340: Accumulation module; 350: Comparison module;
[0052] 400: Sending module;
[0053] 500: Clock generator;
[0054] 600: Conversion module. Detailed Implementation
[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0056] The terms "first" and "second" in this specification may explicitly or implicitly include one or more of the features. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.
[0057] In related technologies, the main hardware implementation platforms for neural networks are general-purpose CPUs, DSP accelerator boards, and GPUs. The primary method for implementing neural network models using general-purpose CPUs is through serial software implementation, which has low computational efficiency and poor real-time performance. Since the inference process of neural networks typically involves numerous multiplication and addition operations with relatively little flow control, chips with numerous parallel processing units, such as DSPs and GPUs, are more suitable. While DSPs, as digital signal processors, undoubtedly possess strong data parallel processing capabilities, their versatility is poor, their hardware flexibility is insufficient, and their parallel processing capabilities are also limited. Furthermore, the inference process of neural networks implemented using DSPs still requires some software involvement. GPUs are currently the mainstream hardware implementation platform for neural network training and inference. They have a large number of computing cores, enabling extensive parallel floating-point operations, making them very suitable for deploying neural network models. However, GPUs have high power consumption and cost, making them difficult to deploy at edge devices.
[0058] The following is combined Figures 1 to 8 The present invention describes a handwritten digit image recognition system. Figure 1 This is a schematic diagram of the overall structure of a handwritten digit image recognition system provided in an embodiment of the present invention. The system is deployed in an FPGA and includes a receiving module 100, a RAM control module 200, an inference accelerator module 300, and a transmitting module 400. Each component will be described in detail below:
[0059] The receiving module 100 is used to receive the pixel values corresponding to the pixels of the handwritten digit image to be recognized.
[0060] Specifically, the handwritten digit image to be recognized refers to the image that needs to be identified, which contains unknown handwritten digits. In practice, the handwritten digit image to be recognized can be acquired through an electronic device's touchscreen, or it can be extracted from a photograph or video captured by an image acquisition device. The image to be recognized can also be any existing digital image in a database, such as images of bank checks or forms containing handwritten digits. Furthermore, it should be noted that the handwritten digit image to be recognized can be of any size, but a 28*28 grayscale image is preferred. It should be noted that this embodiment does not limit the method of acquiring the handwritten digit image to be recognized; the image acquisition method, image size, and format described above are merely illustrative examples.
[0061] RAM control module 200, connected to receiving module 100, is used to store the pixel values corresponding to each pixel in the handwritten digit image to be recognized into RAM 210. After receiving one image, the pixel values in RAM 210 are sequentially loaded into cache 220.
[0062] In this embodiment, the receiving module 100 can be constructed using existing or future communication interfaces of the FPGA, such as a high-speed serial interface, a UART universal asynchronous receiver / transmitter (UART interface), an SPI interface, or an IIC interface.
[0063] An inference accelerator module 300 is connected to the RAM control module 200. The inference accelerator module 300 includes a ROM 310, which stores multiple weighted sets with different digital tags. Each weighted set includes a preset weight value corresponding to each pixel. The inference accelerator module 300 is used to perform calculations on the pixel values loaded into the cache 220 and the preset weight values in the ROM 310 each time, determine the target weighted set based on the calculation results, and output the digital tag corresponding to the target weighted set as the recognition result. The preset weight values in the multiple weighted sets are determined according to the model parameters of a pre-trained fully connected neural network.
[0064] Specifically, the number of weighted reassemblies is set based on the output of the pre-trained fully connected neural network. The output of the fully connected neural network is determined by the number of digits expected to be predicted. One weighted reassembly corresponds to one network output. The fully connected neural network will have multiple outputs, each corresponding to an expected digit, which is the digit label of the weighted reassembly corresponding to that output. The preset weight values are the weights of the connection layers connected to that output. Let's assume that the image input to the fully connected neural network contains 400 pixels, and the fully connected neural network has 5 expected digits as outputs. Then, the weighted reassembly corresponding to each expected digit will contain 400 preset weight values.
[0065] In this embodiment, the preset weight values of multiple weight groups can be stored in one ROM, or the preset weight values can be stored in multiple ROMs. The number and capacity of the ROMs can be adjusted according to the total amount of preset weight values.
[0066] In this embodiment, the target weighted combination refers to the weighted combination selected from multiple weighted combinations based on the calculation results; typically, there is only one target weighted combination. The numerical label corresponding to the selected weighted combination is the target numerical label.
[0067] A sending module 400 is connected to the inference accelerator module 300 and is used to send the recognition result output by the inference accelerator module.
[0068] In this embodiment, the transmitting module 400 can be constructed using existing or future communication interfaces of the FPGA. The transmitting module 400 can use a communication interface corresponding to the receiving module, such as serial ports for both the transmitting module 400 and the receiving module 100, or it can be adjusted according to the specifications of the host computer interface.
[0069] This embodiment of a handwritten digit image recognition system utilizes a receiving module deployed in an FPGA to receive pixel values of the handwritten digit image to be recognized, and a RAM control module to store the pixel values. An inference accelerator module stores the preset weight values of each weight group determined by the model parameters of a pre-trained fully connected neural network in a ROM. The pixel values are then compared with the preset weight values stored in the ROM to obtain a digit tag corresponding to the target weight group based on the calculation results. Finally, the digit tag is sent as the recognition result through a sending module. This system implements the function of a fully connected neural network using pure hardware, offering advantages such as low cost, low power consumption, high computational efficiency, and good real-time performance.
[0070] In some possible implementations, the preset cache capacity is equal to the data size corresponding to four pixels.
[0071] Among some possible implementations, Figure 2This is a schematic diagram of the internal structure of the inference accelerator module provided in an embodiment of the present invention. Please refer to it. Figure 2 As shown, the inference accelerator module 300 further includes: an address generation module 320, a multiplication module 330 corresponding to each pixel value in the cache 220, an accumulation module 340 corresponding to the weighted reassembly, a comparison module 350, and an output module 360.
[0072] The address generation module 320 is used to generate a ROM access address based on the pixel point corresponding to the pixel value in each cache 220, and to extract a preset weight value corresponding to each pixel value in the cache 220 from each weight set of the ROM based on the ROM access address.
[0073] The multiplication module 330 is used to calculate the product of the pixel values in the cache 220 with the preset weight values taken from each weight set and corresponding to each pixel value in the cache 220;
[0074] The accumulation module 340 is used to accumulate the products belonging to the same weight group to obtain the sum corresponding to each weight group.
[0075] The comparison module 350 is used to compare the sums of each weighted combination after the accumulation is completed, and to take the weighted combination corresponding to the maximum sum as the target weighted combination.
[0076] The output module 360 is used to match the target weighted recombination with the preset mapping relationship to obtain the target digital label and output it, wherein the preset mapping relationship represents the correspondence between the weighted recombination and the digital label.
[0077] In some possible implementations, there are ten weighted reorganizations, with the ten weighted reorganizations corresponding to the numbers 0 to 9.
[0078] In some possible implementations, a ROM 310 of the inference accelerator module 300 is used to store all preset weight values for a weight reassembly.
[0079] Specifically, the ten weight groups are labeled with numbers 0 to 9. All the preset weight values corresponding to each number label are stored in a ROM. That is, ten ROMs are set up to store the preset weight values of the ten weight groups respectively, so that the preset weight values belonging to the same weight group can be stored in the same ROM in sequence, thereby improving the efficiency of the inference accelerator module in retrieving data from the ROM, and thus further improving the computational efficiency.
[0080] In some possible implementations, the pre-trained fully connected neural network is trained according to the following steps:
[0081] Import and load the MNIST dataset;
[0082] A fully connected neural network model is constructed based on a preset mathematical model and the image size in the MNIST dataset. The preset mathematical model is Y = WX, where X represents the input of the fully connected neural network, the dimension of X is equal to the number of pixels corresponding to any image in the MNIST dataset after flattening it from two dimensions to one dimension, Y represents the output of the fully connected neural network, the dimension of Y is equal to the preset total number of expected predicted numbers, and W represents the connection layer weights between the input and output of the fully connected neural network.
[0083] The fully connected neural network model was trained in multiple rounds using the training and test sets in the MNIST dataset.
[0084] The fully connected neural network model trained through multiple rounds is used as the pre-trained fully connected neural network.
[0085] In some possible implementations, the preset weight values in multiple weight reassemblies are calculated according to the following steps:
[0086] Calculate the quantization coefficient according to Formula 1;
[0087]
[0088] Where s is the quantization coefficient, a is the maximum absolute value of the model parameters of the pre-trained fully connected neural network, and b is the logarithm of the maximum range of the quantized data with respect to 2.
[0089] The model parameters of the fully connected neural network are symmetrically quantized according to Formula 2 to obtain the quantized model parameter matrix;
[0090] M q =round(M·s) Formula 2;
[0091] Where M is the model parameter matrix of the fully connected neural network to be quantized, M q M·s is the quantized parameter matrix, s is the quantization coefficient, and the function round() represents rounding all elements in the M·s matrix.
[0092] If the input to a fully connected neural network is a row matrix, then one column of the quantized model parameter matrix corresponds to a preset weight value for a set of weights.
[0093] If the input to a fully connected neural network is a column matrix, then each row of the quantized model parameter matrix corresponds to a preset weight value for a set of weights.
[0094] For some possible implementations, please refer to Figure 2 As shown, the handwritten digit image recognition system also includes a clock generator 500;
[0095] The clock generator 500 provides a first clock signal with a frequency of 100MHz to the multiplication module 330, the accumulation module 340, and the comparison module 350, so that any multiplication operation performed by the multiplication module 330, any addition operation performed by the accumulation module 340, and any comparison operation performed by the comparison module 350 are all completed within the period of the first clock signal.
[0096] Please refer to the following for possible implementations. Figure 2 As shown, the clock generator 500 is also used to provide a second clock signal with a frequency of 50MHz to the receiving module 100, the transmitting module 400 and the RAM control module 200.
[0097] Among some possible implementations, Figure 3 This is a schematic diagram of the structure of the handwritten digit image recognition system provided in an embodiment of the present invention. Please refer to it. Figure 3 As shown, unlike the previous embodiments, the handwritten digit image recognition system also includes a conversion module 600;
[0098] The conversion module 600 is used to convert the handwritten digit image to be recognized into a grayscale image before the receiving module 100 receives the handwritten digit image to be recognized. The conversion module 600 is implemented by software.
[0099] To facilitate understanding of the present invention, the following describes the present invention in detail using a grayscale image recognition scenario with an image format of 28×28 pixels as an example. This embodiment provides the implementation process of deploying the above-mentioned handwritten digit image recognition system, and the specific process is as follows:
[0100] First, determine the fully connected neural network structure that is compatible with the 28×28 pixel grayscale image;
[0101] Figure 4 A schematic diagram of the structure of a fully connected neural network applied to this format is shown, such as... Figure 4 As shown, the handwritten digit image recognition system uses a fully connected neural network structure with 784 input nodes and 10 output nodes, without hidden layers or biases. The input to this fully connected neural network is a flattened 28×28 pixel 8-bit grayscale image provided by the MINST dataset, and the output is a score predicting the digits (0-9) in the image content. The mathematical model corresponding to this fully connected neural network is as follows: Figure 4 The formulas on the lower and right sides are shown. The mathematical model of this fully connected neural network is expressed as:
[0102] Y = WX
[0103] Where Y is a vector with dimension 10. W is a matrix with dimensions 784×10. X is a vector with dimension 784.
[0104] Then, training is performed based on the determined fully connected neural network structure. Please refer to... Figure 5 As shown, the training process of a fully connected neural network is as follows:
[0105] Step 1: Import and load the dataset; In this specific embodiment of the invention, Python is used to implement the specific neural network training algorithm, and the PyTorch open-source machine learning database is used to assist in the construction of the neural network; First, the MNIST dataset needs to be imported from the local machine into the program and converted into a tensor data structure; then, the batch size when loading the dataset, the shuffle attribute settings when importing, etc., are set, and the training set and test set are loaded into the program.
[0106] Step 2: Building the Neural Network; In a specific embodiment of the present invention, the neural network is built according to the mathematical model of the fully connected neural network; First, the tensor of the image data input to the network needs to be flattened, and its size is flattened from 28×28 to 1×784; Then, the torch.nn.Linear(784, 10, bias=False) function is used to implement a fully connected neural network with a structure of 784×10 without bias.
[0107] Step 3: Create the loss function and optimizer, and set the parameters related to neural network training; in a specific embodiment of the present invention, the loss function is the cross-entropy loss function, the optimizer is the stochastic gradient descent optimizer, and the number of training rounds is set.
[0108] Step 4: Determine whether ten rounds of training have been completed; In a specific embodiment of the present invention, the entire neural network training process consists of ten rounds of training, each round of training including two parts: training and testing; The training part adjusts and optimizes the weight parameters of the neural network model using the training dataset, and the testing part performs a verification test on the optimized neural network using the test dataset; After all ten rounds of training are completed, the model data is saved.
[0109] Step 5: Determine whether all image data in the training set has been acquired. The training process is complete once all image data has been used in the training process.
[0110] Step Six: The neural network processes the input image data; In a specific embodiment of the present invention, the neural network processes the input image data according to the weight parameters within the current network to obtain the prediction result.
[0111] Step 7: Calculate the error of the current neural network using the loss function; in a specific embodiment of the present invention, the cross-entropy loss function is used to calculate the error between the predicted value and the true value of the neural network.
[0112] Step 8: Calculate the gradient of each weight parameter using the backpropagation algorithm; In a specific embodiment of the present invention, it is first necessary to clear the gradient of each weight parameter in the model to zero, and then use the backpropagation algorithm to recalculate the gradient of each weight parameter.
[0113] Step 9: Use the optimizer to optimize the weight parameters based on the gradients of each weight parameter. The optimizer is implemented using the stochastic gradient descent algorithm.
[0114] Step 10: Determine whether all image data in the test set has been acquired. The test process is complete once all image data has been used in the test.
[0115] Step 11: The neural network makes predictions on the input image data; In a specific embodiment of the present invention, the neural network makes inference predictions on the input image data based on the weight parameters obtained after current training.
[0116] Step 12: Calculate the error using the loss function and accumulate it to obtain the overall error; In a specific embodiment of the present invention, the cross-entropy loss function is still used to calculate the error between the predicted value and the true value of the neural network, and all errors are accumulated to obtain the overall error of the model prediction after the current round of training.
[0117] Step 13: Calculate the prediction accuracy; In a specific embodiment of the present invention, the number of correctly predicted images is calculated, and the ratio of this number to the total number of images in the test set is used to obtain the prediction accuracy of the model after this round of training.
[0118] Step Fourteen: Complete all ten rounds of training and save the neural network model data to obtain the trained fully connected neural network.
[0119] Next, the trained fully connected neural network is deployed in hardware. According to its mathematical model, the inference and prediction process involves a 1×784-dimensional input data vector cross-multiplying by a 784×10-dimensional weight parameter matrix. This matrix cross-multiplication can be decomposed into element-wise multiplication, where each of the 784 input pixel data points is multiplied sequentially by its corresponding weight parameter and then summed. The data from the 10 output nodes is then compared, and the maximum value y is determined. n The corresponding subscript n is the inference result of the neural network.
[0120] When using trained fully connected neural network model parameters in hardware, it is necessary to quantize the trained model parameters first. The model parameter quantization process in specific implementation is as follows:
[0121] Since all parameters in the neural network model are single-precision floating-point numbers, to simplify computation and adapt to the FPGA's fixed-point computation capability, the parameters in the trained neural network model are quantized, converting them from single-precision floating-point numbers to INT10 fixed-point numbers. Symmetric quantization is used here, utilizing the formula... Obtain the quantization coefficient s, and then use the quantization formula M q =quantize(M,s)=round(M·s) Quantizes the trained parameters, converting them from single-precision floating-point numbers to INT10 fixed-point numbers. Here, s is the quantization coefficient, a is the maximum absolute value of the parameters in the pre-trained fully connected neural network model, and b is the logarithm of the maximum range of quantized data divided by 2. For INT10 quantization, b=10. M is the model parameter matrix of the fully connected neural network to be quantized. q M·s is the quantized parameter matrix, s is the quantization coefficient, and the function round() represents rounding all elements in the M·s matrix.
[0122] For further details, please refer to Figure 6 As shown, the handwritten digit image recognition system first receives image pixel data transmitted from the host computer via a serial port receiving module. Then, the system's internal RAM buffers the input data for subsequent data shaping. Because the neural network inference accelerator module has four degrees of parallelism, meaning it can process four pixel data points per clock cycle, the input data is first shaped after being retrieved from RAM, merging the four 8-bit pixel data points into a single 32-bit data point before transmitting it to the inference accelerator module. The inference accelerator module, upon receiving the four input pixel data points, first multiplies them with their corresponding weight parameters. Then, the 10 output nodes sum the products of the four input pixel data points, and then add this summation to the summation from the previous loop to complete the current loop. RAM continues to provide input pixel data to the inference accelerator module for the same operation until the loop reaches 196 (784 ÷ 4), at which point the loop ends, and all input pixel data points are multiplied by their corresponding weight parameters. The accumulated result at this point is the matrix cross product Y. Compare the data sizes of each element y0 to y9 within Y, i.e., the 10 output nodes, and find the maximum value y. n The corresponding index n represents the inference result of the neural network. Finally, the handwritten digit image recognition system transmits the recognition result to the host computer via a serial port transmission module.
[0123] For further details, please refer to [link / reference] again. Figure 3As shown, taking UART for serial port transmission and reception as an example, this handwritten digit image recognition system consists of five parts: a clock generator, a serial port receiving module, a RAM control module, an inference accelerator module, and a serial port transmitting module. The RAM control module internally instantiates a 784×8-bit RAM. The names and functions of each module are shown in Table 1.
[0124] Table 1. Module Names and Functions of the Handwritten Digital Image Recognition System
[0125]
[0126] For further details, please refer to [link / reference] again. Figure 2 As shown, the inference accelerator module mainly consists of four modules: a neural network module, an accumulation module, a comparison module, and an address generation module. The neural network module internally instantiates four multiplication modules to calculate the product of the four input pixel data and their corresponding weight parameters. It also internally instantiates ten ROM modules to store the weight parameters. The names and functions of each module in the inference accelerator module are shown in Table 2.
[0127] Table 2. Module Names and Functions within the Inference Accelerator Module
[0128]
[0129] at last, Figure 7 The diagram shows an image set from the MNIST dataset used for training a fully connected neural network. It can be seen that the images in this dataset are all square black and white handwritten digit images. Figure 8 This is a schematic diagram illustrating the prediction results of the host computer display system for handwritten digit image recognition provided by this invention. In the specific implementation process, images from the MNIST test set are used as input data for the handwritten digit image recognition system. First, the images need to be preprocessed. MATLAB is used to convert the images into grayscale images, and then the grayscale data of each pixel is converted into hexadecimal code and saved to a .txt file. The host computer establishes a connection with the handwritten digit image recognition system via a serial port debugging assistant, sending the hexadecimal image pixel data to the system through the assistant. After inference, the system obtains the prediction result and sends it to the host computer via serial port. The host computer's serial port debugging assistant receives the system's prediction result. Figure 8 In the serial port debugging assistant window, the area within the box formed by the long and short interval lines is the image pixel data sent by the host computer to the handwritten digit image recognition system, and the data within the dashed box is the system's prediction result.
[0130] The handwritten digit image recognition system of this embodiment has at least the following beneficial technical effects: A fully connected neural network is built using the PyTorch open-source database, trained on the MNIST dataset, and the error between the predicted and actual values is calculated using the cross-entropy loss function. The gradients of each weight parameter within the neural network are calculated using the backpropagation algorithm, and the weight parameters are optimized using the stochastic gradient descent algorithm, training the neural network model to achieve a prediction accuracy of over 90%. Then, symmetric quantization is used to convert the parameters in the model from single-precision floating-point numbers to INT10 fixed-point numbers, which are then shaped and exported. The trained neural network model is mapped to a hardware circuit, which adopts a sequential batch processing architecture, implementing the neural network inference operation process through multiplication, accumulation, and comparison modules. Peripheral circuitry is then designed for the inference accelerator module, using RAM and its controller to cache and reshape externally input data. Data interaction between the system and the outside world is achieved through UART serial port receiving and transmitting modules. Finally, the entire system is deployed on an FPGA development board, forming a handwritten digit image recognition system. This system is entirely implemented using FPGAs, featuring high processing speed, low power consumption, and low overall embedded system cost, while ensuring high recognition accuracy. It has successfully implemented a handwritten digit image recognition system. The entire inference accelerator module requires only 200 clock cycles to obtain prediction results. With a clock cycle of 100MHz, theoretically, this inference accelerator can process 5,000 28×28 pixel black and white handwritten digit images per second.
[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A handwritten digit image recognition system, characterized in that, The system is deployed on an FPGA and includes: The receiving module is used to receive the pixel values corresponding to the pixels of the handwritten digit image to be recognized; The RAM control module, which is connected to the receiving module, is used to store the pixel values corresponding to each pixel in the handwritten digit image to be recognized into RAM, and to load the pixel values in RAM into the cache sequentially with a preset cache capacity. An inference accelerator module is connected to the RAM control module. The inference accelerator module includes a ROM, which stores multiple weighted sets with different digital labels. Each weighted set includes a preset weight value corresponding to each pixel. The inference accelerator module is used to perform calculations on the pixel values loaded into the cache and the preset weight values in the ROM each time, determine the target weighted set based on the calculation results, and output the digital label corresponding to the target weighted set as the recognition result. The preset weight values in the multiple weighted sets are determined according to the model parameters of a pre-trained fully connected neural network. A sending module, which is connected to the inference accelerator module, is used to send the recognition result output by the inference accelerator module; The inference accelerator module further includes: an address generation module, a multiplication module corresponding to each pixel value in the cache, an accumulation module corresponding to the weighted reassembly, a comparison module, and an output module; The address generation module is used to generate a ROM access address based on the pixel corresponding to the pixel value in each cache, and to extract a preset weight value corresponding to each pixel value in the cache from each weight set of the ROM based on the ROM access address. The multiplication module is used to calculate the product of the pixel value in the cache with the preset weight value taken from each weight set and corresponding to each pixel value in the cache. The accumulation module is used to accumulate the products belonging to the same weight group to obtain the sum corresponding to each weight group. The comparison module is used to compare the sums of each weighted combination after the accumulation is completed, and to take the weighted combination with the largest sum as the target weighted combination. The output module is used to match the target weighted reorganization with the preset mapping relationship to obtain the target digital label and output it. The preset mapping relationship represents the correspondence between the weighted reorganization and the digital label.
2. The handwritten digit image recognition system according to claim 1, characterized in that, The preset cache capacity is equal to the data size corresponding to four pixels.
3. The handwritten digit image recognition system according to claim 1, characterized in that, There are ten weighted reorganizations, and the numerical labels corresponding to the ten weighted reorganizations are the numbers 0 to 9.
4. The handwritten digital image recognition system according to claim 1, characterized in that, One ROM of the inference accelerator module is used to store all preset weight values for a weight reassembly.
5. The handwritten digit image recognition system according to claim 1, characterized in that, The pre-trained fully connected neural network is trained according to the following steps: Import and load the MNIST dataset; A fully connected neural network model is constructed based on a preset mathematical model and the image size in the MNIST dataset. The preset mathematical model is Y=WX, where X represents the input of the fully connected neural network, the dimension of X is equal to the number of pixels corresponding to any image in the MNIST dataset after flattening it from two dimensions to one dimension, Y represents the output of the fully connected neural network, the dimension of Y is equal to the preset total number of expected predicted numbers, and W represents the connection layer weights between the input and output of the fully connected neural network. The fully connected neural network model was trained in multiple rounds using the training and test sets in the MNIST dataset. The fully connected neural network model trained through multiple rounds is used as the pre-trained fully connected neural network.
6. The handwritten digit image recognition system according to claim 5, characterized in that, The preset weight values in multiple weight reassemblies are calculated according to the following steps: Calculate the quantization coefficient according to Formula 1; Formula 1; in, For quantization coefficients, This represents the maximum absolute value of the parameters in a pre-trained fully connected neural network model. This is the logarithm of the maximum range of the quantized data with respect to 2. The model parameters of the fully connected neural network are symmetrically quantized according to Formula 2 to obtain the quantized model parameter matrix. Formula 2; in, Let be the model parameter matrix of the fully connected neural network to be quantized. The quantized parameter matrix, For quantization coefficients, the function Representative to Round all elements in the matrix to the nearest whole number. If the input to a fully connected neural network is a row matrix, then one column of the quantized model parameter matrix corresponds to a preset weight value for a set of weights. If the input to a fully connected neural network is a column matrix, then each row of the quantized model parameter matrix corresponds to a preset weight value for a set of weights.
7. The handwritten digit image recognition system according to claim 1, characterized in that, The system also includes a clock generator; The clock generator is used to provide a first clock signal with a frequency of 100MHz to the multiplication module, the accumulation module and the comparison module, so that any multiplication operation performed by the multiplication module, any addition operation performed by the accumulation module and any comparison operation performed by the comparison module are all completed within the period of the first clock signal.
8. The handwritten digit image recognition system according to claim 7, characterized in that, The clock generator is also used to provide a second clock signal with a frequency of 50MHz to the receiving module, the transmitting module and the RAM control module.
9. The handwritten digit image recognition system according to any one of claims 1-8, characterized in that, The system also includes a conversion module; The conversion module is used to convert the handwritten digit image to be recognized into a grayscale image before the receiving module receives the handwritten digit image to be recognized.