FPGA-based neural network accelerator and application thereof

Abstract

The application relates to an FPGA-based neural network accelerator and application thereof, which comprises a controller, a data input cache unit, a weight input cache unit, an operation unit and an output cache unit; the controller controls the cycle boundary and convolution step length of the neural network accelerator to match the layer structure of a quantized residual neural network; the data input cache unit splits and stores input data; the weight input cache unit splits and stores the weight of the quantized residual neural network; the operation unit performs convolution operation on the input data and weight data; and the output cache unit stores the output result and transmits the output result to a PS end through AXI4 bus burst transmission. The application realizes a method for applying a convolutional neural network to iris recognition, greatly improves the recognition success rate compared with a traditional algorithm, accelerates the inference process, and makes up for the shortcoming that the recognition method is not suitable for mobile deployment.

Description

Technical Field

[0001] This invention relates to an FPGA-based neural network accelerator and its applications, belonging to the field of neural network model technology, and particularly to deep learning hardware acceleration and its application in biometrics. Background Technology

[0002] Currently, GPUs are the mainstream choice for accelerating model training and applications. However, due to their high power consumption, GPUs are unsuitable for embedded devices and other terminal applications with high requirements for power consumption, efficiency, and real-time performance. In contrast, FPGAs are more efficient and adaptable, making them more suitable for mobile acceleration. FPGAs excel at parallelizing computations in the spatial dimension, allowing for flexible configuration of highly parallel architectures to achieve high-speed computation.

[0003] As people move into the digital age, their demands for the accuracy and convenience of identity verification methods are constantly increasing. Traditional identity verification methods mainly include identity tokens (such as keys, certificates, etc.) along with identity information (such as account numbers, passwords, etc.), or a combination of the two (such as bank cards). In practice, these methods all share the same drawback: they are easily lost and forged. Furthermore, traditional identity verification systems cannot effectively verify whether the person holding these identity tokens is the true owner. Therefore, once impersonated, the true owner will suffer significant losses. With the rapid development of optoelectronic technology, microcomputer technology, image processing technology, and pattern recognition technology, a new biometric identification technology has emerged. This is a high-tech identification method that relies on human biometrics for identity verification. Biometric identification methods refer to identity verification techniques that utilize unique physiological and behavioral characteristics of individuals. Currently, commonly used biometric identification technologies employ biometrics based on physiological features such as face, fingerprints, and iris scans, as well as behavioral features such as handwriting and voice.

[0004] There are many methods for identifying human characteristics. Among all biometrics, fingerprints are relatively stable, but fingerprint acquisition is not non-invasive. Facial features have many advantages (such as initiative, non-invasiveness, and user-friendliness), but facial features change with age, are not very reliable, and are easily forged. Voice features have similar advantages to facial features, but they change with age, health status, and environment, and speaker recognition systems are easily fooled by recordings and can be forged. Iris recognition solves these problems and possesses some characteristics that the other biometrics mentioned above do not have. Therefore, in recent years, iris recognition technology has been considered one of the most promising biometric technologies.

[0005] The main application areas of iris recognition include: (1) High-end access control: access control for enterprises, research institutions, high-end residential buildings, bank vaults, safes, armories, archives, nuclear power plants, airports, confidential departments, computer rooms, etc. (2) Public security criminal investigation: management of floating population, entry and exit management, ID card management, driver's license management, etc. (3) Medical and social security: identity verification of blood donors, social welfare recipients, labor insurance personnel, etc. (4) Network security: e-commerce, network access, computer login, etc. (5) Other applications: attendance, identity verification of examination personnel, information security, etc. With the maturity of long-distance iris image acquisition technology and the further improvement of recognition algorithm performance, iris recognition has become possible in daily life.

[0006] Convolutional neural networks (CNNs) are widely used in computer vision, particularly in object detection and image recognition, demonstrating promising application prospects. This aligns perfectly with the characteristics of iris recognition. As the number of layers in a deep CNN increases, the recognition and feature extraction performance improves, especially with the advent of residual neural networks, which addresses the issue of shallow feature degradation as the network deepens. However, VggNet-16, proposed in 2014, has a Caffe model size exceeding 500MB, requiring 15 billion floating-point calculations for a single forward inference operation. This makes deploying commonly used CNNs on hardware devices extremely difficult, rendering it unsuitable for embedded devices. If deployment is attempted, the inference process must be accelerated. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention proposes an FPGA-based neural network accelerator. Its key feature is the use of a trained, high-accuracy residual neural network. By removing the average pooling layer and fully connected layers, the output feature map is expanded, and the resulting sequence is used as the encoding of the iris. The recognition result is obtained by calculating the feature encoding of the image to be recognized and comparing it with existing feature encodings.

[0008] The present invention also proposes an embedded iris recognition system with an FPGA-based neural network accelerator and an iris recognition method, which achieves high-speed recognition on a mobile platform.

[0009] Terminology Explanation:

[0010] 1. Daugman's rubber strip model, an iris preprocessing model, unfolds the iris ring into a rectangular matrix through polar coordinate transformation, and performs proportional compensation for the deformation and distortion of the iris texture caused by this.

[0011] 2. Stride, step length;

[0012] 3. Padding: Fills pixels;

[0013] 4. Zynq chip, a multi-core heterogeneous processor SoC launched by Xilinx;

[0014] 5. DDR4 memory module, fourth-generation memory chip;

[0015] 6. Qt interactive interface, a graphical interactive interface developed by Qt.

[0016] The technical solution of this invention is as follows:

[0017] An FPGA-based neural network accelerator includes a controller, a data input buffer unit, a weight input buffer unit, a computation unit, and an output buffer unit.

[0018] The controller is used to: control the loop boundaries and convolution stride of the neural network accelerator to match the structure of each layer of the quantized residual neural network;

[0019] The data input buffer unit is used to: split and store the input data, and transmit the obtained data to the processing unit;

[0020] The weight input buffer unit is used to: split and store the weights of the quantized residual neural network, and transmit the obtained weight data to the computing unit;

[0021] The computation unit is a parallel multiply-accumulator, used to: perform convolution operations on the input data and weight data;

[0022] The output buffer unit is used to: store the output results and transmit them to the PS terminal in bursts using the AXI4 bus.

[0023] According to a preferred embodiment of the present invention, the input data is split and stored, including: the data stream, i.e., the input data, flows sequentially into the linear buffer under the control of a clock, at which time a new set of convolutional data will be generated in the convolution window.

[0024] According to a preferred embodiment of the present invention, before splitting and storing the weights of the quantized residual neural network, the weights of the quantized residual neural network are pre-divided at the PS end.

[0025] According to a preferred embodiment of the present invention, the neural network accelerator includes N operation units, each operation unit being a multiply-accumulate tree with M convolutional kernels, and the N operation units performing parallel operations on M×N convolutional kernels.

[0026] An embedded iris recognition system with an FPGA-based neural network accelerator includes the neural network accelerator, a quantized residual neural network and a Zynq chip, a DDR4 memory module, a touch screen, and an infrared camera; the touch screen and the infrared camera are connected to the Zynq chip, and the DDR4 chip is mounted on a Zynq development board and connected to the Zynq chip;

[0027] The ddr4 memory module is used to: cache the operating system and run the entire embedded iris recognition system;

[0028] The touchscreen is used to: display the Qt interactive interface;

[0029] The infrared camera: captures grayscale images of the iris and sends them to the Zynq chip for preprocessing;

[0030] The Zynq chip includes an ARM core, also known as the PS terminal, which is used to capture and preprocess the grayscale image of the iris, and to feed the preprocessed iris image into the trained and quantized residual neural network.

[0031] The trained and quantized residual neural network is used for iris recognition.

[0032] Programmable logic circuits, also known as PL terminals, are used in integrated neural network accelerators to effectively shorten the time for convolution calculations.

[0033] An iris recognition method, implemented through the embedded iris recognition system with an FPGA-based neural network accelerator, includes:

[0034] Capture a grayscale image of the iris and preprocess it;

[0035] The preprocessed iris image is fed into the trained and quantized residual neural network for iris recognition.

[0036] The feature code of the iris image is calculated by the quantized residual neural network after training. Then, the similarity is calculated with the code in the database. If the similarity reaches the threshold, the identity authentication is successful; otherwise, the identity authentication fails.

[0037] According to a preferred embodiment of the present invention, the pretreatment process is as follows:

[0038] (1) Image segmentation: The specific process is as follows:

[0039] Detecting a point within the pupil: First, remove reflected light spots within the pupil; then, binarize the grayscale image of the iris; finally, find the coordinates of the centroid within the pupil as the initial point.

[0040] Inner boundary localization: Starting from the pupil, i.e. the initial point, proceed in three directions: left, right, and down. Use the gradient difference at the pupil-iris boundary to detect points in the three directions. Then, calculate the radius and center coordinates of the inner boundary circle from these three non-collinear points.

[0041] Outer boundary localization: The gradient difference at the iris-eyeball boundary is used to detect points in three directions, and the radius and center coordinates of the outer circle are calculated from these three non-collinear points; once the boundary is found, the search stops and the radius and center coordinates of the outer circle are calculated.

[0042] (2) Image unfolding: By transforming the polar coordinate system and the rectangular coordinate system, the iris ring located between the sclera and the pupil in the rectangular coordinate system is normalized to the standard rectangle in the rectangular coordinate system;

[0043] (3) Image enhancement increases the contrast of the image and improves the dynamic range, resulting in a clear iris image.

[0044] Further preferred, before the outer boundary localization process, the image is processed using Gaussian filtering.

[0045] A further preferred method is to use Daugman's rubber strip model to convert the circular shape of the iris in the Cartesian coordinate system into a 56*224 rectangular template in the polar coordinate system, copy it four times, and then stitch it together to form a square.

[0046] According to a preferred embodiment of the present invention, the residual neural network includes an input layer and a plurality of residual blocks;

[0047] In the input layer, the input image is convolved by 64 large 7*7 convolutional kernels, then subjected to max pooling and ReLU layers to obtain a feature map of 56*56 pixels with 64 channels.

[0048] The residual blocks include residual blocks with the same image size and residual blocks with a reduced image size. The residual blocks with the same image size have two convolutional layers with stride=1 and padding=1. The convolutional layers use small 3*3 convolutional kernels, and the number of channels is equal to the number of convolutional kernels. The residual connection is implemented by direct addition. The residual blocks with a reduced image size have the first convolutional layer with stride=2, use small 3*3 convolutional kernels, and double the number of convolutional kernels, which means the number of channels in the output image is doubled. The second convolutional layer has stride=1, and the number of channels is equal to the number of convolutional kernels. The residual connection is implemented by sampling each channel at intervals in the row and column directions, then expanding it to the same number of channels as the output by zero padding, and then adding it to the output.

[0049] The beneficial effects of this invention are as follows:

[0050] This invention realizes a method for applying convolutional neural networks to iris recognition, which greatly improves the success rate of recognition compared with traditional algorithms, while accelerating the inference process. It also makes up for the shortcomings of the traditional recognition method, such as the huge increase in computation and unsuitability for mobile deployment, and provides a new iris recognition solution. Attached Figure Description

[0051] Figure 1 This is an architecture diagram of an FPGA-based neural network accelerator.

[0052] Figure 2 A schematic diagram of linear storage of the input image;

[0053] Figure 3 This is a schematic diagram illustrating the pre-blocking of weights;

[0054] Figure 4 This is a schematic diagram illustrating the computational functions of the arithmetic unit.

[0055] Figure 5 This is a structural block diagram of the embedded iris recognition system with a convolution accelerator according to the present invention;

[0056] Figure 6 This is a schematic diagram illustrating the transformation of the circular iris in Cartesian coordinates into a rectangular template in polar coordinates.

[0057] Figure 7 A schematic diagram illustrating the process of determining the intersection point of any ray emanating from the center of the circle with the boundaries of the eyeball and the pupil.

[0058] Figure 8 This is a diagram illustrating the process of copying a rectangular template four times and then stitching them together to form a square template.

[0059] Figure 9 This is a schematic diagram of the residual block structure;

[0060] Figure 10 This is a flowchart illustrating the quantification process. Detailed Implementation

[0061] The present invention will be further defined below with reference to the accompanying drawings and embodiments, but is not limited thereto.

[0062] Example 1

[0063] An FPGA-based neural network accelerator, such as Figure 1 As shown, it includes a controller, a data input buffer unit, a weight input buffer unit, a computation unit, and an output buffer unit;

[0064] The controller is used to: control the loop boundary and convolution stride of the neural network accelerator to match the structure of each layer of the quantized residual neural network; and dynamically configure the loop boundary register and convolution stride register of the neural network accelerator (i.e., the convolution accelerator) according to the configured convolutional layers to achieve the control of the above process. The data input buffer unit is used to: split and store the input data, and transmit the obtained data to the computation unit; the weight input buffer unit has the same function as the data input buffer unit, used to: split and store the weights of the quantized residual neural network, and transmit the obtained weight data to the computation unit; the computation unit is a parallel multiply-accumulator used to: perform convolution operations on the input data and weight data; the output buffer unit is used to: store the output results and transmit them in bursts to the PS terminal via the AXI4 bus.

[0065] The data input buffer unit, weight input buffer unit, and output buffer unit integrate the functions of burst control, data segmentation and merging, data rearrangement, and block storage of the AXI4 bus.

[0066] Example 2

[0067] The difference between the FPGA-based neural network accelerator described in Example 1 and the one in Example 1 is that:

[0068] The input data is split and stored, such as Figure 2 As shown, the process includes: the data stream, i.e., the input data, flows sequentially into the linear buffer under the control of the clock, at which point a new set of convolutional data is generated in the convolution window. This fully reuses the convolutional image, greatly reducing bandwidth pressure. Specifically, for a common 3x3 convolution, it mainly consists of 3 rows of shift registers connected end to end, with the structure as follows: Figure 2 As shown. Figure 2 Each square in the diagram represents a data register, part of the same row of linear buffers, and is shown separately here. The linear buffer is a set of shift registers. If the image width is M, then the depths of the three rows of shift registers are 3, M, and M, respectively. The data stream passes through these three rows of shift registers sequentially under clock. When the first image data reaches the last position, the 3x3 data within the dashed box in the diagram becomes the data in the convolution window, which can be used for convolution with the 3x3 quantized weight matrix. This structure avoids repeatedly fetching data from discontinuous memory addresses and fully reuses the convolutional image, reducing system bandwidth and power consumption.

[0069] Before splitting and storing the weights of the quantized residual neural network, the weights are pre-divided at the PS (Power Sequence Pointer). Based on the number of input and output channels, they are divided into M×N blocks, which facilitates the use of the neural network accelerator. For example... Figure 3 As shown, specifically for a common 3x3 convolution, Figure 3 The middle row represents the number of input channels for a certain convolutional layer, and the columns represent the number of output channels for the convolutional layer. Each element is a 3x3 weight matrix. Figure 3 The weights are concatenated sequentially into a 1x9 vector for easy representation. When the parallel capabilities of the deployed system are limited, a pre-block structure as shown in the figure can be used for each layer's weight matrix. That is, rows are grouped into N elements and columns into M elements, which are then rearranged in storage to facilitate subsequent continuous block-by-block memory reads, improve read and write efficiency, and speed up system operation.

[0070] The neural network accelerator comprises N operational units, each of which is a multiply-accumulate tree with M convolutional kernels. The N operational units perform parallel operations on M×N convolutional kernels. Due to the use of techniques such as pipeline, unroll, and dataflow, this allows a single operational unit (e.g., ...) to perform parallel operations on M×N convolutional kernels. Figure 4 As shown, the convolution process with multiple inputs and multiple outputs can be achieved in a single clock cycle, maximizing the acceleration of the AI ​​inference process.

[0071] When using the aforementioned neural network accelerator, firstly, the PS sends configuration information via the AXI4 bus. After receiving the configuration information, the controller initializes the convolution accelerator and notifies the PS, while simultaneously sending data and weight data. Then, the transmitted data and weight data are split and stored by the data input buffer unit and weight input buffer unit, respectively. After temporary storage, they are sent to the computation unit. Finally, the computation unit performs highly parallel convolution operations. The results are received by the output buffer unit, converted in format, and transmitted to the PS via the AXI4 bus, completing one convolution call.

[0072] Example 3

[0073] An embedded iris recognition system with an FPGA-based neural network accelerator, such as Figure 5 As shown, it includes the neural network accelerator described in Embodiment 1 or 2, the quantized residual neural network and Zynq chip, DDR4 memory module, touch screen, and infrared camera; the touch screen and infrared camera are connected to the Zynq chip via a USB interface, and the DDR4 chip is mounted on the Zynq development board and connected to the Zynq chip;

[0074] The DDR4 memory module is used to cache the operating system and run the entire embedded iris recognition system. The touchscreen is used to display the Qt interactive interface, which includes a real-time display window for images captured by the infrared camera and touch-sensitive function keys. The infrared camera captures grayscale images of the iris and sends them to the Zynq chip for preprocessing. The infrared camera has LED auxiliary illumination, and a light-shielding material is used to shield the path between the camera and the eye, thus blocking ambient light. The Zynq chip internally includes an ARM core, also known as the PS (Power Sequencer), which performs grayscale image capture and preprocessing of the iris, and feeds the preprocessed iris image into the trained, quantized residual neural network. Preprocessing refers to obtaining the grayscale image from the infrared camera; this preprocessing enriches the image information and improves the recognition success rate. The trained, quantized residual neural network performs iris recognition. The programmable logic circuit, also known as the PL (Programmable Logic) terminal, is used to integrate the neural network accelerator, effectively shortening the convolution calculation time.

[0075] Figure 5 In this invention, the neural network inference program is the implementation program of the embedded iris recognition method proposed in this invention, and the convolution part is the calling program of the neural network accelerator proposed in this invention.

[0076] Example 4

[0077] An iris recognition method, implemented using the embedded iris recognition system with an FPGA-based neural network accelerator as described in Example 3, includes:

[0078] Capture a grayscale image of the iris and preprocess it;

[0079] The preprocessed iris image is fed into the trained and quantized residual neural network for iris recognition.

[0080] The feature code of the iris image is calculated using a trained, quantized residual neural network. This feature code is then compared to codes in the database. If the similarity reaches a threshold, authentication is successful; otherwise, authentication fails. The final convolution results in a 7x7x512 image, which is unfolded in a specific order to obtain the feature code of the iris image. The feature code is then compared to other codes in the database using a plaintext distance calculation. If the distance is less than a certain threshold, authentication is successful; otherwise, recognition fails. Finally, the result is displayed on a touchscreen via an interactive interface.

[0081] The preprocessing process is as follows:

[0082] (1) Image segmentation: Based on the principle that "three non-collinear points determine a circle", the specific process is as follows:

[0083] Detecting a point within the pupil: The pupil reflects infrared light from the acquisition device, forming a bright spot (large grayscale), which affects the overall grayscale distribution of the eye image. First, the reflected light spot within the pupil is removed. Then, an appropriate grayscale threshold is selected to binarize the grayscale image of the iris; the grayscale value of the pupil region is set to 0 (black), and the grayscale value of the sclera region is set to 1 (white). Finally, the coordinates of the centroid within the pupil are found as the initial point; the coordinates of the centroid of the binarized image can be obtained by directly calculating the average coordinates of the pixels (pixel value multiplied by the horizontal and vertical coordinates and divided by the number of pixels in the horizontal and vertical directions).

[0084] Inner boundary localization: Due to the large gradient of the inner boundary of the iris image, the pupil and iris will appear as peaks and troughs respectively on the grayscale gradient map. Starting from the pupil (the initial point), points are detected in three directions: left, right, and down, using the gradient difference at the pupil-iris boundary. The radius and center coordinates of the inner boundary circle are then calculated from these three non-collinear points. The search stops once the boundary is found. Therefore, the localization range is not the entire eye image, resulting in a relatively fast speed.

[0085] Outer boundary localization: The method for localizing the outer boundary of the iris is similar to that for the inner boundary, continuing the search outward based on the previous step, expanding the grayscale search range. Points in three directions are detected using the gradient difference at the iris-eyeball boundary, and the radius and center coordinates of the outer circle are calculated from these three non-collinear points. Once the boundary is found, the search stops, and the radius and center coordinates of the outer circle are calculated.

[0086] (2) Image unfolding: The polar coordinate system and the rectangular coordinate system are transformed to normalize the iris ring located between the sclera and the pupil in the rectangular coordinate system to a standard rectangle in the rectangular coordinate system; in order to facilitate subsequent feature extraction.

[0087] (3) Image enhancement increases the contrast of the image and improves the dynamic range, resulting in a clear iris image.

[0088] Before outer boundary localization, the image is processed using Gaussian filtering to minimize interference from eyelids and eyelashes.

[0089] Using Daugman's rubber strip model, the circular shape of the iris in Cartesian coordinates is converted into a 56*224 rectangular template in polar coordinates using a formula. This template is then copied four times and stitched together to form a square. Figure 6 As shown, the specific implementation process is as follows:

[0090] Daugman's rubber strip model transforms the ring into a rectangular matrix using polar coordinates and then compensates for the resulting distortion of the iris texture.

[0091] Normally, the outer circle of the iris and the inner circle of the pupil are not concentric, but the difference between their centers is not significant. Therefore, to simplify the algorithm, we approximate the centers of the outer and inner edges of the iris as coincident. Here, the outer edge of the iris is defined as a circle centered at coordinates (x0, y0). Therefore, the intersection points (Q and P) of any ray emanating from the center (x0, y0) with the outer circle and the inner circle of the pupil can also be determined, such as... Figure 7 As shown.

[0092] [x inner =x0+r inner ×cosθy inner =y0+r inner [×sinθ] (1)

[0093] [x outer =x0+r outer ×cosθy outer =y0+r outer [×sinθ] (2)

[0094] In the formula, θ is the angle between the ray and the horizontal line.

[0095] Any point on line segment PQ can be represented by a linear combination of P and Q, thus transforming the iris image from the Cartesian coordinate system (x,y) to the (r,θ) space.

[0096] The circular shape of the iris in Cartesian coordinates is converted into a 56*224 rectangular template in polar coordinates. This rectangle is then copied four times and stitched together to form a 224*224 square template. Figure 8 As shown.

[0097] A residual neural network consists of an input layer and several residual blocks;

[0098] In the input layer, the input image is convolved by 64 large 7*7 convolutional kernels (stride=2, padding=3), then max pooling and ReLU layers are applied to obtain a 56*56 pixel feature map with 64 channels.

[0099] In this embodiment, the residual blocks adopt the classic ResNet18 structure, which includes eight residual blocks. Each residual block consists of two convolutional layers. Convolutional layers, batch normalization (BN) layers, and ReLU activation are commonly used operations in deep neural network structures, and will not be elaborated here. Max pooling layers and the eight residual blocks are linearly connected to form the structure of the neural network. The specific structure is shown in Table 1.

[0100] Table 1

[0101]

[0102]

[0103] The structure of the residual block is as follows Figure 9 As shown, the residual blocks include residual blocks with the same image size and residual blocks with a reduced image size. The residual blocks with the same image size have two convolutional layers with stride=1 and padding=1. The convolutional layers use small 3*3 convolutional kernels, and the number of channels is equal to the number of convolutional kernels. The residual connection is implemented by direct addition. The residual blocks with a reduced image size have the first convolutional layer with stride=2, use small 3*3 convolutional kernels, and double the number of convolutional kernels, which means the number of channels in the output image is doubled. The second convolutional layer has stride=1, and the number of channels is equal to the number of convolutional kernels. The residual connection is implemented by sampling each channel at intervals in the row and column directions, then expanding it to the same number of channels as the output by zero padding, and then adding it to the output.

[0104] In the quantized residual neural network, the quantization process includes:

[0105] Performing 8-bit quantization on the convolution process is beneficial for accelerating the FPGA process. The specific quantization method in this example is as follows: Figure 10 As shown:

[0106] First, perform forward fusion of the BN layers:

[0107] The expression for a convolutional layer is y = w*x + b, where x is the input image, y is the output image, w represents the weights, b represents the bias, and * represents the dot product.

[0108] Normalization: u and α 2 These are the mean and variance of each batch, respectively; ζ is a small value added to the denominator to prevent the denominator from being zero; γ represents the scaling factor, and β represents the offset factor; these are two parameters that the model automatically learns during training.

[0109] Substituting y = wx + b, we get:

[0110] By separating the variables and parameters, we get: w' and b' refer to the weights and biases calculated after the BN layer fusion, respectively;

[0111] Based on the above reasoning, the BN layer is fully integrated into the convolution calculation during the inference stage without any loss of precision. This approach can effectively reduce the amount of computation, reduce resource consumption, and speed up the inference process.

[0112] Then, weight quantization and activation quantization:

[0113] Weight quantization refers to: dividing the weight w' by the parameter s1, rounding down, truncating to obtain an 8-bit result, multiplying the result by the parameter s1 to obtain the inverse quantization result, calculating the squared mean of the difference between the result and the original data, and taking the s1 value with the smallest mean square error; s1 refers to the weight quantization parameter, which is obtained through multiple tests.

[0114] Activation quantization: Divide the preprocessed image (i.e., the feature map) by the parameter s2, round down, truncate to get an 8-bit result, then multiply the result by the parameter s2 to get the dequantization result. Calculate the mean squared difference between the result and the original data, and take the s2 value with the smallest mean squared error. s2 refers to the activation quantization parameter, which is obtained from multiple tests. However, since the feature map is the result of ReLU, it is always greater than zero, so the sign bit is omitted.

[0115] Meanwhile, s1*s2 is used as the quantization parameter for the bias, and the result of each convolution layer and the bias are preserved as 32-bit signed numbers.

[0116] After obtaining the quantization parameters, the actual process of activating quantization in the program is as follows: calculate the result by multiplying s1*s2 and dividing by s3, where s1 is the quantization coefficient of the upper-layer feature map, s2 is the weighted quantization coefficient, and s3 is the quantization coefficient of the next-layer feature map, and the closest 2 is taken for each. n The value q allows us to use bit shifting instead of dequantization and weighting. That is, shifting the binary data left by one bit achieves multiplication by 2, and shifting it right by one bit achieves division by 2, thus improving computational efficiency.

[0117] The residual connection should be multiplied by q and divided by the quantization coefficients of the two convolution layers it spans to ensure mathematical equivalence.

[0118] The training process of the quantized residual neural network is as follows: A large number of pupil images are acquired in the target environment using similar equipment, with several images for each pupil. The images are classified and preprocessed according to the acquired pupils, serving as training data. The training data is then used to obtain predicted values ​​through forward propagation, and the parameters of the neural network are updated through backpropagation.

[0119] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An FPGA-based neural network accelerator, characterized in that, It includes a controller, a data input buffer unit, a weight input buffer unit, a computation unit, and an output buffer unit; The controller is used to: control the loop boundaries and convolution stride of the neural network accelerator to match the structure of each layer of the quantized residual neural network; The data input buffer unit is used to: split and store the input data, and transmit the obtained data to the processing unit; The weight input buffer unit is used to: split and store the weights of the quantized residual neural network, and transmit the obtained weight data to the computing unit; The computation unit is a parallel multiply-accumulator, used to: perform convolution operations on the input data and weight data; The output buffer unit is used to: store the output results and transmit them to the PS terminal in bursts using the AXI4 bus.

2. The FPGA-based neural network accelerator according to claim 1, characterized in that, The input data is split and stored, including: the data stream, i.e., the input data flows into the linear buffer sequentially under the control of the clock, at which time a new set of convolutional data will be generated in the convolution window.

3. The FPGA-based neural network accelerator according to claim 1, characterized in that, Before splitting and storing the weights of the quantized residual neural network, the weights of the quantized residual neural network are pre-divided at the PS end.

4. The FPGA-based neural network accelerator according to claim 1, characterized in that, The neural network accelerator includes N There are 10 operational units, each operational unit is... M A multiply-add tree with convolutional kernels, N Each processing unit implements M × N Parallel computation of convolution kernels.

5. An embedded iris recognition system with an FPGA-based neural network accelerator, characterized in that, Includes a neural network accelerator as described in any one of claims 1-4, a quantized residual neural network and a Zynq chip, a DDR4 memory module, a touch screen, and an infrared camera; the touch screen and the infrared camera are connected to the Zynq chip, and the DDR4 chip is mounted on a Zynq development board and connected to the Zynq chip; The ddr4 memory module is used to: cache the operating system and run the entire embedded iris recognition system; The touchscreen is used to: display an interactive interface; The infrared camera: captures grayscale images of the iris and sends them to the Zynq chip for preprocessing; The Zynq chip includes an ARM core, also known as the PS terminal, which is used to capture and preprocess the grayscale image of the iris, and to feed the preprocessed iris image into the trained and quantized residual neural network. The trained and quantized residual neural network is used for iris recognition. Programmable logic circuits, also known as PL terminals, are used in integrated neural network accelerators to effectively shorten the time for convolution calculations.

Images