Convolutional neural network handwritten number recognition teaching system based on ZYNQ

Abstract

The invention relates to the field of artificial intelligence teaching and embedded systems, and discloses a convolutional neural network handwritten number recognition teaching system based on ZYNQ. The system supports two operation modes: in a pure PS (Procesing System) mode, preprocessing, convolution, activation, pooling and classification operations are all completed by an ARM (Advanced RISC Machines) processor; in a PS + PL (Program Logic) cooperative acceleration mode, a part of the neural network layer is deployed in programmable logic, and the recognition frame rate is improved and the power consumption is reduced through fixed-point quantization and a pipeline structure. The system adopts a PCIe channel to transmit identification data to an upper computer, frame rate changes, feature maps and resource use differences in different modes are displayed in real time in a visual interface, students are helped to understand the influence of hardware acceleration on CNN performance, and the system is suitable for artificial intelligence and embedded teaching experiments.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence teaching and embedded systems technology, and more particularly to a Zynq-based convolutional neural network handwritten digit recognition teaching system. This system achieves efficient recognition of handwritten digit images and provides teaching demonstrations through the collaborative operation of a processing system (PS) and programmable logic (PL), combined with fixed-point quantization and pipelined parallel computing architecture. Background Art

[0002] With the widespread application of deep learning technology in image recognition, convolutional neural networks have become the mainstream method for tasks such as handwritten digit recognition due to their excellent feature extraction and classification capabilities. However, while traditional CPU-based software implementations are easy to teach and verify algorithms, they suffer from significant processing speed bottlenecks, making them unsuitable for real-time interactive experimental teaching.

[0003] For example, existing teaching platforms only support PS-side operations, making it difficult to directly demonstrate the advantages of hardware acceleration and lacking comparative teaching and visual feedback. In contrast, FPGA platforms offer highly parallel computing capabilities and customizable hardware resources. However, existing CNN teaching systems based on FPGAs often use a single operating mode or implement acceleration only on the programmable logic side. This lacks comparative verification and visualization methods, making it difficult to directly demonstrate the impact of hardware acceleration on CNN performance. Therefore, an embedded teaching system is needed that supports pure software verification, flexibly switches to hardware acceleration, and displays performance differences through a visual interface to meet the needs of artificial intelligence practice and embedded experimental teaching. To meet these needs, a ZYNQ-based convolutional neural network handwritten digit recognition teaching system is proposed. Summary of the Invention

[0004] In order to solve the problems existing in the background technology, the present invention provides a ZYNQ-based convolutional neural network handwritten digit recognition teaching system.

[0005] A ZYNQ-based convolutional neural network handwritten digit recognition teaching system, characterized by comprising the following steps:

[0006] S1: System initialization and parameter configuration, camera module driver loading and image acquisition parameter setting to ensure the hardware platform is in an operational state;

[0007] S2: Use the camera to collect handwritten digital image data in real time and perform standardized preprocessing operations on specific areas of the original image, including grayscale conversion, binarization, and converting the RGB888 image data into AXI4Memory Map standard format data.

[0008] S3: Select the image data flow running path according to the CNN handwritten digit recognition mode set by the host computer;

[0009] S4: In pure PS mode, the data collected and pre-processed by the camera is input into DDR3 storage through the AXI4 read-write driver controller, and the ARM processor completes convolution operations, activation function calculations, pooling calculations, full connection calculations, and classification and recognition tasks;

[0010] S5: In the PS+PL collaborative mode, the convolution layer, activation function, and pooling layer computing tasks are deployed on the PL side to implement hardware acceleration of some CNN layers. The pooled data is then input into the DDR3 storage through the AXI4 read / write driver controller, and subsequent calculations are completed on the PS side.

[0011] S6: Fixed-point quantization technology is used on the PL side to quantize floating-point weights and bias parameters, and parallel computing of convolutional layers and pooling layers is implemented through a pipelined parallel structure.

[0012] S7: The image data and recognition results of the computation process are uploaded to the host computer via the PCIe bus using the Riffa open source framework. A visual interface displays a real-time comparison of the frame rates of the two modes, visualization of the feature maps of each layer, CNN convolutional recognition results, and a comparison of the time required to recognize a single handwritten digit.

[0013] S8: Dynamically switch the operating mode and repeat steps S2-S7 to achieve comparative demonstration in different modes.

[0014] Furthermore, all initialization parameters, including camera parameter configuration, digit recognition area, and CNN weight bias parameters, are loaded after the system is powered on and cannot be changed to ensure stable system operation.

[0015] Furthermore, in the pure PS mode of S1, the processed handwritten digital image data collected by the camera is stored in the DDR3 of the PS side through the AXI4 read-write driver controller from the AXI-HP bus channel, and then the PS side reads the corresponding area data of DDR3 for CNN calculation processing.

[0016] Furthermore, in the PS+PL collaborative acceleration mode of S5, RTL code is written to implement the convolution layer, activation function, and pooling layer calculations, and data interaction between the PL and PS ends is completed through the AXI4 read-write driver controller.

[0017] Furthermore, S6's fixed-point quantization performs 16-bit fixed-point quantization on weight parameters. The pipeline parallel structure consists of four stages: input feature map buffering and data alignment, parallel multiplication and accumulation calculation of convolution kernels, ReLU activation, and pooling operation.

[0018] Furthermore, the S7's computer visualization interface is only used to configure the current handwritten digit recognition operating mode, display images during CNN processing, and compare frame rates between the two modes. The camera captures image data, the CNN recognizes handwritten digits, and the execution of different operating modes is all performed on the Zynq hardware system.

[0019] Furthermore, the implementation formula of the CNN convolutional layer in the system is:

[0020]

[0021] Furthermore, the activation function implementation formula in the system is:

[0022] a(i,j)=max(0,y(i,j))

[0023] Furthermore, the pooling layer implementation formula in the system is:

[0024]

[0025] Furthermore, the fully connected layer implementation formula in the system is:

[0026] z=W·x+b

[0027] Among them, y(i,j) is the calculated image coordinate, w(p,q) is the weight parameter of the corresponding coordinate, and b(p,q) is the bias parameter of the corresponding coordinate. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is the system structure diagram of the teaching system in the present invention

[0029] Figure 2 This is the system flow chart of the teaching system in the present invention DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described below with reference to the accompanying drawings and embodiments.

[0031] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention.

[0032] Example 1.

[0033] like Figure 1 As shown in the teaching system structure diagram, this embodiment provides a ZYNQ-based convolutional neural network handwritten digit recognition teaching system, which includes a camera, an image preprocessing module, an AXI4 read-write driver controller, an external DDR3 memory, a PS-side CNN recognition module, a PL-side CNN acceleration module, a Riffa-PCIe driver module, and PC software.

[0034] The camera is used for:

[0035] Collect external handwritten digital images and transmit image data to the ZYNQ hardware board.

[0036] The image preprocessing module is used to:

[0037] The handwritten digital image data collected by the camera is preprocessed in the recognition area, first converted to grayscale and then binarized.

[0038] The AXI4 read / write driver controller is used to:

[0039] The processed image data is written into DDR3 via the AXI4 bus for subsequent modules to perform recognition and processing.

[0040] The external DDR3 memory is used for:

[0041] Store large amounts of image data for subsequent modules to process.

[0042] The PS-side CNN recognition module is used to:

[0043] Complete the entire process of CNN image recognition on the PS side, including convolution, activation, pooling, full connection, and output recognition results.

[0044] The PL-side CNN acceleration module is used to:

[0045] The convolutional layer and pooling layer of CNN are implemented in hardware on the PL side to accelerate the speed of CNN image recognition.

[0046] The Riffa-PCIe driver module is used to:

[0047] Transmit the processed image data of each CNN layer, transmit the final image recognition result, transmit the operating mode of the current CNN process set by the PC host software, and transmit the CNN image recognition time under different operating modes.

[0048] The PC software is used to:

[0049] Real-time display of the frame rate comparison of the two modes, visualization of feature maps of each layer, CNN convolutional recognition results, and time comparison of single handwritten digit recognition.

[0050] Example 2.

[0051] like Figure 2 As shown in the teaching system flow chart, this embodiment provides a ZYNQ-based convolutional neural network handwritten digit recognition teaching system, which includes:

[0052] S1: System initialization and parameter configuration, camera module driver loading and image acquisition parameter setting to ensure that the hardware platform is in an operational state.

[0053] The initialization parameters of the camera module are stored in the external DDR3 memory and only need to be configured once when the system is running. The weight bias parameters required to run CNN are stored in the internal BRAM of the PL side, so that they can be quickly read when running CNN.

[0054] S2: The camera collects handwritten digital image data in real time and performs standardized preprocessing operations on specific areas of the original image, including grayscale conversion, binarization, and conversion and output of RGB888 image data into AXI4Memory Map standard format data.

[0055] S3: Select the image data flow running path according to the CNN handwritten digit recognition mode set by the host computer.

[0056] The data interaction between the host computer and the ZYNQ board is all completed via the PCIe bus.

[0057] S4: In pure PS mode, the data collected and pre-processed by the camera is input into DDR3 for storage through the AXI4 read-write driver controller, and the ARM processor completes convolution operations, activation function calculations, pooling calculations, full-connection calculations, and classification and recognition tasks.

[0058] S5: In the PS+PL collaborative mode, the convolution layer, activation function, and pooling layer computing tasks are deployed on the PL side to achieve hardware acceleration of some CNN layers. The pooled data is then input into the DDR3 storage through the AXI4 read / write driver controller, and subsequent calculations are completed on the PS side.

[0059] S6: Fixed-point quantization technology is used on the PL side to quantize floating-point weights and bias parameters, and parallel computing of convolutional layers and pooling layers is implemented through a pipeline parallel structure.

[0060] S7: The image data and recognition results of the operation process are uploaded to the host computer using the Riffa open source framework via the PCIe bus. The visual interface displays in real time the frame rate comparison between the two modes, the visualization results of the feature maps of each layer, the CNN convolution recognition results, and the time comparison of a single handwritten digit recognition.

[0061] S8: Dynamically switch the operating mode and repeat steps S2-S7 to achieve comparative demonstration in different modes.

[0062] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A ZYNQ-based convolutional neural network handwritten digit recognition teaching system, characterized by: The following steps are involved: S1: System initialization and parameter configuration, camera module driver loading and image acquisition parameter setting to ensure the hardware platform is in an operational state; S2: Use the camera to collect handwritten digital image data in real time and perform standardized preprocessing operations on specific areas of the original image, including grayscale conversion, binarization, and converting the RGB888 image data into AXI4Memory Map standard format data. S3: Select the image data flow running path according to the CNN handwritten digit recognition mode set by the host computer; S4: In pure PS mode, the data collected and pre-processed by the camera is input into DDR3 storage through the AXI4 read-write driver controller, and the ARM processor completes convolution operations, activation function calculations, pooling calculations, full connection calculations, and classification and recognition tasks; S5: In the PS+PL collaborative mode, the convolution layer, activation function, and pooling layer computing tasks are deployed on the PL side to implement hardware acceleration of some CNN layers. The pooled data is then input into the DDR3 storage through the AXI4 read / write driver controller, and subsequent calculations are completed on the PS side. S6: Fixed-point quantization technology is used on the PL side to quantize floating-point weights and bias parameters, and parallel computing of convolutional layers and pooling layers is implemented through a pipelined parallel structure. S7: The image data and recognition results of the computation process are uploaded to the host computer via the PCIe bus using the Riffa open source framework. A visual interface displays a real-time comparison of the frame rates of the two modes, visualization of the feature maps of each layer, CNN convolutional recognition results, and a comparison of the time required to recognize a single handwritten digit. S8: Dynamically switch the operating mode and repeat steps S2-S7 to achieve comparative demonstration in different modes.

2. The system according to claim 1, wherein: All initialization parameters, including camera parameter configuration, digit recognition area, and CNN weight bias parameters, are loaded after the system is powered on and cannot be changed to ensure stable system operation.

3. The system according to claim 1, wherein: In the pure PS mode of S1, the processed handwritten digital image data collected by the camera is stored in the DDR3 of the PS end through the AXI4 read-write driver controller from the AXI-HP bus channel, and then the PS end reads the corresponding area data of DDR3 for CNN calculation processing.

4. The system according to claim 1, wherein: In the PS+PL collaborative acceleration mode of the S5, RTL code is written to implement the convolution layer, activation function, and pooling layer calculations, and data interaction between the PL and PS ends is completed through the AXI4 read-write driver controller.

5. The system according to claim 1, wherein: The fixed-point quantization of S6 performs 16-bit fixed-point quantization on the weight parameters. The pipeline parallel structure includes four stages of pipeline, which are: input feature map buffer and data alignment, parallel multiplication and accumulation calculation of convolution kernel, ReLU activation, and pooling operation.

6. The system according to claim 1, wherein: The S7's computer visualization interface is used only to configure the current handwritten digit recognition operating mode, display images during CNN processing, and compare frame rates between the two modes. The camera captures image data, the CNN recognizes handwritten digits, and the execution of the different operating modes is all performed on the Zynq hardware system.

7. The system according to claim 1, wherein: The CNN convolutional layer, activation function, pooling layer, and fully connected layer in the system satisfy the following mathematical definitions: The implementation formula of the convolution layer is: The activation function implementation formula is: a(i,j)=max(0,y(i,j)) The implementation formula of the pooling layer is: The implementation formula of the fully connected layer is: z=W·x+b Among them, y(i,j) is the calculated image coordinate, w(p,q) is the weight parameter of the corresponding coordinate, and b(p,q) is the bias parameter of the corresponding coordinate.

Images