Floating-point number convolutional neural network reasoning deployment method based on FPGA
By implementing floating-point number calculations on FPGAs according to IEEE754 standard, the problem of accuracy loss when transplanting algorithms on FPGAs is solved, and efficient floating-point number calculations and accurate inference results are achieved.
Patent Information
- Application Number
- CN202411767700.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-04
- Publication Date
- 2025-03-25
AI Technical Summary
When porting the host computer algorithm to the FPGA, we face the problem of accuracy loss, especially because the decimal point position in the FPGA needs to be artificially defined as a fixed point number, resulting in changes in accuracy error.
By determining the data storage, expression and calculation methods according to the IEEE754 standard, and selecting the floating point digital width that the board can deploy based on the board resources, a more accurate calculation method is achieved. Specific methods include deploying floating point multiplication and addition modules on FPGAs, supporting half-precision, single-precision, double-precision, and custom-precision floating point calculations.
It realizes efficient floating-point number calculation on FPGA, significantly improving the inference efficiency, the system can complete inference within the calculation cycle, and supports continuous image input and recognition, and the accuracy error is controlled within an acceptable range.
Smart Images
Figure CN119692474A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of convolutional neural networks, and in particular to an FPGA-based floating-point convolutional neural network reasoning and deployment method. Background Art
[0002] FPGAs have been widely used in circuit function verification due to their flexibility, high parallelism, and programmability. Through the programmable characteristics of FPGAs, circuit functions can be quickly verified on hardware after minor modifications to the host computer. With its high parallelism, linear operations can be decomposed into multiple parallel operations, thereby shortening the overall operation time. In addition, FPGA products integrate programmable digital signal processing modules. For example, AMD's XCKU060-FFIVA1156-2-I model contains 2760 DSP48E2 modules, which can support mathematical operations such as multiplication and accumulation. Compared with CPLDs, FPGAs have richer register resources, so the maximum clock frequency of the module can be increased by increasing clock latency. As long as the board resources are sufficient, FPGAs can achieve mathematical calculations with high bit widths and high clock rates. However, when porting host computer algorithms to FPGAs, there is a problem of precision loss. The host computer usually uses floating-point numbers that comply with the IEEE 754 standard, while the decimal point position in the FPGA needs to be manually defined as a fixed-point number. Different decimal point positions will lead to changes in precision errors. If you want to implement more accurate calculations on FPGA, you need to determine the data storage, expression, and calculation methods based on the IEEE754 standard, and select the floating-point bit width that the board can deploy based on the board resources.
[0003] Convolutional Neural Networks (CNN) are a type of feedforward neural network with deep structure and convolutional computing. They first extract features from input data through convolutional layers to identify local patterns such as edges and textures. Then, the feature map is downsampled through pooling layers to reduce data dimensions and computational burden, and enhance the robustness of the network to prevent overfitting. Introducing activation functions between layers to map linear results to nonlinear space enables the network to learn more complex features and relationships, thereby improving the performance of the model in tasks such as classification and regression. Through these combinations, CNN can effectively process high-dimensional and complex data. However, with the rapid evolution of convolutional neural network architecture, it is challenging to efficiently deploy network models in real applications. To cope with this change, it is recommended to use FPGA to deploy CNN. FPGA has high parallel computing capabilities, programmability, and hardware acceleration features, and can flexibly adjust the network structure according to needs while maintaining efficient computing performance. Compared with GPUs, FPGAs not only perform well in terms of power consumption and resource usage, but also can achieve customized optimization, which is particularly suitable for edge computing and low-latency scenarios. Therefore, FPGA is an ideal choice for deploying CNN networks. Summary of the invention
[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and propose a floating-point convolutional neural network reasoning deployment method based on FPGA, which can support half-precision, single-precision, double-precision and custom-precision floating-point calculations according to the amount of FPGA board resources.
[0005] In order to achieve the above object, the technical solution specifically adopted by the present invention is as follows:
[0006] A floating-point convolutional neural network reasoning deployment method based on FPGA includes the following steps: the original data input by the host computer is converted into floating-point numbers by the integer-to-floating-point module and then enters the handwritten digit recognition reasoning network, and the result is obtained after being processed by the floating-point multiplication module, the floating-point addition module, and the floating-point comparison module; wherein the handwritten digit recognition reasoning network includes 2 convolutional layers, 1 fully connected layer, and 1 decision layer, and both convolutional layers are customized for module reuse and corresponding control signals to realize the reuse of system convolution modules and unit convolution modules. Specifically, the following steps are included:
[0007] S1, convolutional layer
[0008] (1) After the FPGA is powered on, (CI×CO×((K×K)+1)) stored weight values and bias values are read from weight_bias_buf, and the valid signal controls the entry of the weight bias registers in CO conv_sys_pe and CI conv_unit_pe; where CI represents the number of input channels of the convolution layer; CO represents the number of output channels of the convolution layer; and K represents the size of the convolution kernel.
[0009] (2) The input CI channel serial data is converted into parallel data through conv_data_buffer; the input serial data will be stored in a register array of size CI×I×I×K; in each parallel array, the input serial data row counter r_raw_cnt==K-1, the column counter r_col_cnt==K-1, the state machine in the module will start to jump from P_ST_IDLE to P_ST_ONE, when the column counter r_col_cnt==I-1, the state machine will jump to the next state, if the row counter r_raw_cnt==I-1, and the column counter r_col_cnt==I-1, it means that an image has been completely input and processed, the state machine jumps back to P_ST_IDLE, waiting for the next image to be recognized; where I represents the size of the image to be recognized;
[0010] (3) CI×K×K data captured by the conv_data_buffer module enter C0 conv_sys_pe modules in parallel. Each conv_sys_pe obtains CI results, which are then passed through an addition tree composed of 2^($clog(CI)) adders to obtain CO outputs.
[0011] (4) conv_sys_pe receives and processes the data, inputs it into conv_unit_pe, performs convolution on the data of each channel, and sums and adds the bias through the addition tree;
[0012] (5) The convolved data is subjected to maximum pooling and ReLU activation;
[0013] S2, fully connected layer
[0014] (1) After the FPGA is powered on, FO×((FI×(IM×IM))+1) stored weight values and bias values are read from weight_bias_buf and controlled by the valid signal to enter the weight bias registers in FO fc_sys_pe; where FI represents the number of input channels of the fully connected layer, FO represents the number of output channels of the fully connected layer, and IM represents the size of the data input into the fully connected layer;
[0015] (2) Input data enters fc_data_buffer for straightening (multi-dimensional tensors are straightened into one-dimensional vectors); FI+1 synchronous FIFOs are used to complete parallel-to-serial conversion. After the conversion is completed, the data is input into FO fc_sys_pe for multi-channel parallel computing;
[0016] (3) After receiving the data, fc_sys_pe reads the continuous weight data from DUAL_RAM and sends the two streams of data to the floating-point multiplication module. The output result then enters the floating-point addition array composed of multiple fc_unit_pe (containing floating-point addition modules);
[0017] (4) fc_unit_pe adds the product of data and weight and processes the bias;
[0018] (5) Convert the results obtained by parallel calculation of F0 fc_sys_pe into serial and parallel form and input them to the decision layer;
[0019] S3, decision-making level;
[0020] (1) After separating the input data, it is serially converted according to the D recognition types defined in the top-level parameters, where D is the number of output channels of the fully connected layer before the decision layer;
[0021] (2) Use a floating-point comparison array composed of multiple floating-point comparison modules to compare the parallel data, obtain the largest data and its position, output the corresponding result, and obtain the recognition data.
[0022] Furthermore, the convolution structures of the two convolutional layers are the same basic architecture, and the serial data is converted into parallel data matching the convolution kernel size through the data cache module, so that K×K floating point number calculations are completed in one clock cycle, where K represents the convolution kernel size;
[0023] The convolution completed data passes through a redundant addition tree (based on powers of 2) to implement the accumulation process of K×K->1. Because the basis is a power of 2, the intermediate data does not need additional control and can be directly built according to the needs using the pipeline. If the current data is missing, just input 0.
[0024] At the end of each output data, the maximum pooling layer and ReLU activation layer are connected to realize the downsampling function and reduce the subsequent data transmission consumption;
[0025] Customize module multiplexing and corresponding control signals to realize the multiplexing of system convolution modules and unit convolution modules.
[0026] Furthermore, the fully connected layer builds the required parallel fully connected layer calculation channels according to the number of output channels. If the previous module is a convolutional layer parallel output, a parallel-to-serial conversion module is required to straighten the multi-dimensional tensor into a one-dimensional vector.
[0027] In each fully connected layer calculation channel, data and fully connected layer weights are processed in fc_unit_pe, and the product results are accumulated using an addition array composed of multiple floating-point addition modules. Unlike the addition tree in the convolutional layer, there are too many parameters in the fully connected layer and they are not fully expanded, so a serial addition tree is used;
[0028] The data at the parallel output end will be converted from parallel to serial and input to the next fully connected layer or decision layer.
[0029] Furthermore, the decision layer performs serial-to-parallel conversion on the input data and stores it in a register matrix for temporary storage, inputs it into a floating point comparison module, compares the size, compares the largest data with the data in the register matrix, and outputs the equal matrix number, which is the final recognition result.
[0030] Furthermore, the integer-to-floating-point module is used to convert the externally input image data, which is 0-255 integer data, into a data format required by the current precision, and the data format is a floating point number that complies with the IEEE 754 standard.
[0031] Furthermore, the floating-point multiplication module needs to go through three steps to obtain the multiplication result: first, the sign bit, exponent bit, and mantissa bit of the input data are extracted using a register, then the exponents are added, the mantissas are multiplied, and finally the mantissa result is normalized, including LZD leading zero detection and mantissa truncation.
[0032] Furthermore, the floating-point number addition module needs to go through six steps to obtain the addition result: first, use the register to extract the sign bit, exponent bit, and mantissa bit of the input data, compare the sizes, distinguish the larger number from the smaller number, obtain the order difference between the larger number and the smaller number, and re-judge the order difference, complete the order and mantissa addition, complete the normalization of the mantissa addition result through the LZD circuit, and process the exponent bit according to the mantissa normalization result; finally, use the register to obtain the result according to the flag signal in the previous steps.
[0033] The present invention has the following characteristics and beneficial effects:
[0034] In the multi-channel input and single-channel input convolution layers, the system convolution module and the unit convolution module are reused, the data flow is the same, and the only difference is the control signal, which effectively improves the development efficiency.
[0035] Based on FP32 floating-point precision, floating-point addition and multiplication are used as core computing units to achieve efficient hardware inference. At the same time, the basic computing unit adopts a floating-point format that complies with the IEEE754 standard and is parameterized to achieve a balance between precision and FPGA resources.
[0036] The main network contains several convolutional layers and fully connected layers. The accuracy verification results show that the maximum relative cumulative error between the inference results of FPGA implementation and Python implementation occurs in the last fully connected layer, and the relative error does not exceed 0.02%, which meets the expected accuracy requirements.
[0037] The present invention gives full play to the parallel computing advantages of FPGA and significantly improves the reasoning efficiency. The system can complete reasoning within the calculation cycle and realize pipeline processing to support continuous image input and recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 The overall block diagram of the convolutional neural network for handwritten digit recognition;
[0039] Figure 2 This is the state transition diagram of data_buffer;
[0040] Figure 3 The data_buffer state machine control diagram for an input image of size 28x28;
[0041] Figure 4 This is a diagram of how to grab the parameters in the matrix in data_buffer;
[0042] Figure 5 This is the specific convolution operation flow chart of the conv_pe module;
[0043] Figure 6 This is the maxpool_ReLU structure diagram;
[0044] Figure 7 This is the IEEE 754 multiplication flowchart;
[0045] Figure 8 This is the IEEE 754 addition flow chart;
[0046] Fig. 9 Compare flowcharts for ieee754;
[0047] Fig.10 This is a comparison chart of the error between the python middle layer data and the vivado simulation middle layer data under FP32 conditions;
[0048] In the figure: (a) is the result data comparison diagram of the first layer convolution channel one; (b) is the result comparison diagram of the second layer convolution channel one; (c) is the result comparison diagram of the fully connected layer.
[0049] Fig.11 This is the accuracy chart of vivado simulation of one thousand images under FP32 conditions;
[0050] Fig.12 It is a detailed diagram of FPGA board resource consumption;
[0051] Fig.13 This is a detailed diagram of FPGA power consumption;
[0052] Fig.14 Identify the number '7' on the FPGA board and return the result image. DETAILED DESCRIPTION
[0053] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other.
[0054] The present invention is based on the handwritten digit recognition convolutional neural network, and changes the network architecture and reduces parameters, so that the FPGA has the resources to realize all its computing operations and data storage. The specific block diagram of the handwritten digit recognition convolutional neural network of the present invention is as follows Figure 1 As shown: it includes INT2FLOAT layer, the first convolution layer (CONV1), the second convolution layer (CONV2), the fully connected layer (FC) and the decision layer (DIR), and its floating point number format is FP32; among them:
[0055] The function of the INT2FLOAT layer is to convert the external input 8-bit image data into FP32. The specific measures are as follows: Use the LZD circuit to find the position A where the highest bit in the input data is 1. Because the output image data are all integers, the sign bit remains 0. For the exponent bit, and the mantissa bit. After obtaining A, normalize it, use the exponent offset of the current precision plus 8-A, and consider the hidden bit of the mantissa, that is, 127+(8-A-1), to get the exponent bit; right shift the input data by (23-A-1) bits and logically OR it with 23'h7fffff to get the mantissa bit.
[0056] The convolution kernel size of the first convolution layer (CONV1) is 5×5×3, with no padding and a stride of 1. The input image size is 28×28, and the conv1_data_buffer state machine jump diagram is as follows Figure 2 As shown, the state machine controls the serial data to be converted into matrix parallel data. Figure 3As shown. Because the size of the convolution kernel of this layer is 5×5, there are 5 states in total, and the number of states can be adjusted with the size. For example, when IDLE does not perform data capture, when the fourth data of the fifth line is input serially, the state jumps to P_ST_ONE to perform effective data capture. In P_ST_ONE, 1 to 5 lines of data are captured, in P_ST_TWO, 2 to 6 lines of data are captured, and so on. In the P_ST_FIRE state, it jumps back to P_ST_ONE. At this time, after one round, 6 to 10 lines of data are captured. The relative position of the first P_ST_ONE and the second P_ST_ONE differs by 5 in the number of input data lines, that is, the number of states or the size of the convolution kernel. When the state machine is not in the IDLE state, that is, it enters the data capture state, each time the state machine jumps, it is necessary to determine whether the current input data is the last line of the complete image. If it is the last line, the overall data capture is completed, jump back to IDLE, and wait for the next condition that satisfies the IDLE jump into P_ST_ONE.
[0057] After the data is captured, it is sent to conv1_pe for convolution processing, such as Figure 5 As shown in the figure, after the conditions are met, data_buffer captures 25 data in each cycle, that is, 5×5 in size, which matches the size of the first layer convolution kernel. The product with 25 weights is obtained through 32 parallel ieee_754_mult units. The 32 multiplication units are set for the subsequent addition tree to achieve two-by-two pipeline addition. If 25 multiplication units are used, part of the data in the subsequent accumulation link (25->1) needs to be controlled and stored separately to achieve synchronization, which reduces the portability of the module. After the 32 data are accumulated into a parameter through the addition tree, the parameter needs to be added with the bias of the current channel.
[0058] Therefore, the output process of the first convolutional layer (CONV1) is:
[0059]
[0060] Where D represents depth and d is a variable of depth. If the image RGB has three depth channels, the present invention uses single-channel data, and the input image is processed in black and white on the host computer. d，p，q Represents the weight of the pth row and qth column of the convolution kernel. d,i+p，i+q represents the data of the dth layer, i-th row to i+p-th row, and j-th column to j+q-th column of the input data. That is, it matches the size of the convolution kernel. f is the activation function, which is the ReLU function in the present invention, that is, f(x) = max{0, x}. b represents the bias value of the current channel.
[0061] After accumulation, the first convolution layer has 3 processing channels. The results obtained by parallel operation enter the maxpool_ReLU layer. For the process of the maximum pooling layer, refer to the convolution layer data_buffer. When the "convolution kernel size" is 2×2, it is not convolved after extraction, but enters the ieee_754_compare module comparison tree for comparison to obtain the maximum value of the 4 data. Figure 6 As shown in the figure, you can modify the "convolution kernel size" according to the pooling requirements, such as 3×3, 4×4, and modify the corresponding number of state machines to achieve fast modification of the maximum pooling layer.
[0062] maxpool_ReLU layer structure: Because floating point numbers are used, the highest sign bit can determine whether it is positive or negative, which helps us process the input data quickly. A highest bit judgment is performed at the output of the maxpool module. If it is 0, the output is maintained; if it is 1, the output is 0.
[0063] The processing process of the second convolutional layer (C0NV2) is:
[0064]
[0065] Where m represents the number of output channels of the second convolutional layer, which can also be called the number of convolution groups of the second layer. n represents the number of input channels of the second convolutional layer, that is, the number of output channels of the first convolutional layer. d represents the image depth, ω m,n,d,p,q Indicates the weight corresponding to the dth image depth of the mth output channel and the nth input channel data. b m Represents the bias of the current group. f represents the ReLU function.
[0066] Therefore, the structure of the second convolutional layer (CONV2) is as follows:
[0067] The data input to the second convolutional layer is the output of the first convolutional layer, with a total of three input channels. In order to better extract higher-level feature information, it is necessary to use different groups of convolution kernels to extract features for the three inputs at the same time, accumulate them, and finally add them to the bias. The number of output channels of the second convolutional layer is determined according to the number of groups.
[0068] The second convolution layer (CONV2) is set to 3 input channels, 3 output channels, and a convolution kernel size of 5×5. There are 9 convolution kernels in total, divided into three groups (i.e. the number of output channels). Each group processes 3 input channel data in parallel. Therefore, the entire second layer architecture is: 3 data_buffers are used at the input port to perform convolution extraction on the 3 input channels. The extracted data is sent to 3 parallel groups respectively.
[0069] (conv2_sys_pe), each group has 3 parallel convolution units (conv2_unit_pe) corresponding to the number of input channels. The convolution units refer to the processing units of the first convolution layer. In summary, 9 parallel convolution processes are performed simultaneously. After the three parallel convolutions in each group obtain the results, they will enter a 4->2->1 addition tree. The three results are accumulated and added with the bias of the current group to complete the high-level feature extraction of a group. The second convolution layer is a multiple reuse of the first layer and the addition of external control signals, which reflects the reusability of the convolution layer and speeds up the development progress.
[0070] The output process of the fully connected layer (FC) is:
[0071]
[0072] Among them, i represents the number of output channels of the fully connected layer, that is, the recognition type of the entire network. The fully connected layer straightens the multi-dimensional tensor of the previous layer of data (the second convolutional layer) into a one-dimensional vector, and n represents the lower n input values of the one-dimensional vector. i Indicates the offset of the current output channel.
[0073] Thus, the overall structure of the fully connected layer (FC) is obtained:
[0074] After the input of the previous layer of data (the second convolutional layer), it needs to be converted into a one-dimensional vector and sent to K parallel fc_sys_pe modules for convolution. K represents the number of output channels of the fully connected layer, and each fc_sys_pe contains several fc_unit_pe. For example, the second convolutional layer inputs 3-channel 4×4 data, which is converted into a one-dimensional vector 48×1. The number of output channels for handwritten digit recognition is 10, so 10 fc_sys_pe are processed in parallel. The weights and biases in each fc_sys_pe are input into RAM and stored through the control signal in the fc_weight_bias_buf module. When the data to be processed and its valid signal are input into fc_sys_pe, the weights are read from RAM. The two signals enter ieee754_mult and are multiplied. The output result pipeline enters the fc_unit_pe module. The main function of the fc_unit_pe module is to add the odd and even positions of the 2N consecutive input data. For example, input a, b, c, d, e, f. The output is a+b, c+d, e+f. Multiple fc_unit_pe combinations can complete 2N->1, for example 48->24->12->6->3->2->1. Use 7 fc_unit_pe units to complete pipeline accumulation. Note that the fourth missing data in 3->2 (actually 4->2) is the bias value of the channel. The final result (2N->1) is the final output value of the channel. Compared with the previous convolution layer, in order to save resources and the power consumption of a large number of multiplication calculations at the same time is too high, this time-for-space approach is adopted. After the final data of the ten channels are obtained at the same time, they are converted into serial data through PISO (parallel in serial out) and input into the decision layer (DIR).
[0075] The decision layer (DIR) includes a serial-to-parallel conversion module and a floating-point comparison module. The overall structure is a comparison tree, that is, the maximum value of M data is obtained by comparing two by two. For example, the ten data input by the full connection layer will go through 10->5->3->2->1, and the data in stages 5 and 3 will be left blank and stored, which is synchronized with the subsequent steps, and all use the ieee754_compare module. This layer needs to be adjusted in real time according to actual requirements.
[0076] The workflow of the handwritten digit recognition convolutional neural network of the present invention is as follows:
[0077] S1, convolutional layer
[0078] (1) The host computer starts up and reads the weight bias from the weight_bias_buf module; when the FPGA is powered on, it reads (CI×CO×((K×K)+1)) stored weight values and bias values from weight_bias_buf, and controls the entry of the weight bias registers in CO conv_sys_pe and CI conv_unit_pe by the valid signal. CI represents the number of input channels of the convolution layer; CO represents the number of output channels of the convolution layer; and K represents the size of the convolution kernel.
[0079] (2) The input CI channel serial data is converted into parallel data through conv_data_buffer; the input serial data will be stored in a register array of size CI×I×I×K; in each parallel array, the input serial data row counter r_raw_cnt==K-1, the column counter r_col_cnt==K-1, the state machine in the module will start to jump from P_ST_IDLE to P_ST_ONE, when the column counter r_col_cnt==I-1, the state machine will jump to the next state, if at this time the row counter r_raw_cnt==I-1, and the column counter r_col_cnt==I-1, it means that at this time an image has been completely input and processed, the state machine jumps back to P_ST_IDLE, waiting for the next image to be recognized; where I represents the size of the image to be recognized.
[0080] The number of states of the state machine is related to K, specifically K+1. Depending on the different states of the state machine, the location and method of grabbing data from the register array are correspondingly different, such as Figure 4 In one cycle, K×K data are captured, i.e., the matrix data required for one convolution, and are input in parallel into the corresponding conv_sys_pe module for convolution processing.
[0081] (3) The Cl×K×K data captured by the conv_data_buffer module enter CO conv_sys_pe modules in parallel. Each conv_sys_pe obtains CI results, which are then passed through an addition tree composed of 2^($clog(C1)) adders to obtain C0 outputs.
[0082] (4) conv_sys_pe receives and processes data, and inputs it into conv_unit_pe. conv_unit_pe performs convolution on each channel data, and sums and adds bias through the addition tree. Specifically: convolve the conv_sys_pe data input, and the multiplication parties are the conv_data_weight initialization weight and the parallel input data respectively. Use $clog(K×K) to obtain the overall size of the convolution kernel, and define an addition tree with AT=2^($clog(K×K)-1) as the base. The base is a power of 2, which can make the parallel processing of the addition tree more efficient, reduce intermediate logic control, and facilitate subsequent transplantation. Then, after ($clog(K×K)-1)×add latency Time gets the K2×K2 accumulated values of the entire convolution cross-correlation, where add latency Represents the delay time of a floating point adder. The final result of the addition tree passes through an addition unit again and is compared with the bias value of the channel. i (i=1,2,…,C0) add to get the final value of the module. There are C0×CI conv_unit_pe modules in parallel, AT multipliers in each conv_unit_pe module are in parallel, and AT adders form an addition tree. The total delay of a single conv_unit_pe calculation is mult. latency +$clog(K×K)×add latency , where mult latency Indicates the delay time of a floating point multiplier. A total of (I-K+1)×(I-K+1) calculations are performed. This module can perform pipeline convolution, input (I-K+1)×(I-K+1) convolution requests, and the overall time consumption is: mult latency +$clog(K×K)×add latency +(I-K+1)×(I-K+1). Where $clog() is the logarithmic function with base 2 in Verilog.
[0083] (5) The convolved data is max pooled and ReLU activated through the maxpool_ReLU layer. maxpool can be regarded as a single-input single-output convolution layer with a convolution kernel size of 2. The 4 extracted data are not convolved, but enter the comparison circuit to find the maximum value. The sign bit of the maximum value is judged. If SIGN = 0, it is a non-negative number and the output is maintained. If SIGN = 1, it is a negative number and the output is 0.
[0084] S2. The data and the weight of the fully connected layer are processed in fc_unit_pe through the fully connected layer, and the product results are accumulated based on the addition array; specifically:
[0085] (1) The host computer starts up and reads the weight bias from the weight_bias_buf module; when the FPGA is powered on, FO×((FI×(IM×IM))+1) stored weight values and bias values are read from weight_bias_buf and controlled by the valid signal to enter the weight bias registers in FO fc_sys_pe. Among them, FI represents the number of input channels of the fully connected layer, FO represents the number of output channels of the fully connected layer, and IM represents the size of the data input into the fully connected layer.
[0086] (2) The input data enters fc_data_buffer for straightening (multi-dimensional tensors are straightened into one-dimensional vectors); FI+1 synchronous FIFOs are used to complete the parallel-to-serial conversion. After the conversion is completed, the data is input into FO fc_sys_pe for multi-channel parallel computing.
[0087] (3) fc_sys_pe receives data and processes it. Unlike the convolutional layer, if the calculation of the fully connected layer is expanded and parallelized, it will consume a lot of resources. A method of exchanging time for area is adopted, as follows: Compared with the convolutional layer storing weight bias in the register array, the weights and biases of the fc_sys_pe modules of each channel of the fully connected layer are stored in DUAL_RAM (dual-port RAM). At the fc_data_buffer output port, output buffer processing is performed, and the output data is continuous (FI×(IM×IM)) data. Entering fc_sys_pe, continuous weight data will be read from DUAL_RAM. The two streams of data are sent together to the floating-point multiplication module, and the output result is also continuous (FI×(IM×IM)) data. Then it enters the addition array composed of multiple (need to be determined according to the specific number of product results) fc_unit_pe.
[0088] (4) fc_unit_pe adds the product of data and weight based on the floating point addition module and processes the bias; the input data is tapped by the register to obtain data
[0089] i_data{d 1, ,d2,…,d i; i<(FI×(IM×IM))} and i_data_1d, so that one cycle is staggered. When i_valid is valid, r_flag=~r_flag (negated). When r_flag and i_valid_1d are valid, i_data will be added to i_data_1d. In fact, d1 is added to d2, d3 is added to d4, and so on. If there are N data at the input end, the output end will be N / 2 data, and o_valid controls whether it is valid. Perform pipeline assembly according to the specific situation. Realize the accumulation of (FI×(IM×IM)) data to 1 parameter. Then compare this parameter with bias i (i=0,1,…,FO) are added to get the fully connected layer result.
[0090] (5) The results obtained by parallel calculation of F0 fc_sys_pe are converted into serial and input to the decision layer.
[0091] S3, the decision layer converts the input data into serial-to-parallel data and stores it in the register matrix for temporary storage, inputs the floating point comparison module, compares the size, compares the largest data with the data in the register matrix, and outputs the equal matrix number, which is the final recognition result; specifically:
[0092] (1) Separate the input data and convert serial to parallel; convert the input serial data into serial-to-parallel data according to the D recognition types defined in the top-level parameters. D is the number of output channels of the fully connected layer before the decision layer. It is also the number of types recognized at the end. For example, for handwritten digit recognition, D is 10.
[0093] (2) Compare the parallel data to obtain the largest data and its position, output the corresponding result, and obtain the identification data; the data after serial conversion is stored in D registers. And input it to the compare module for pairwise comparison. The comparison module uses P_MOD = "BIG". comparison cycles, where Indicates the rounding symbol. Finally, the final result is searched with the data in the register. The corresponding position is output, which is the final result of the entire network recognition.
[0094] Let's take FP32 as an example (sign 1, exponent 8, mantissa 23).
[0095] The floating point multiplication module in this embodiment is as follows:
[0096] The floating point multiplication module has input channels a, b, i_valid and output channels c, o_valid. Figure 6 As shown, there are three steps:
[0097] (1) Use reg to store the input data to extract the sign bit, exponent bit, and mantissa bit. Using registers can reduce timing issues.
[0098] (2) XOR the sign bit, add the exponents, multiply the mantissas, and check for overflow. XOR the sign bit to directly get the sign bit of the final output. The result of exponent addition is stored in r_eyp[9:0]. The result of mantissa multiplication is stored in r_mult_fra[47:0]. Overflow check is divided into overflow and underflow.
[0099] Overflow: 1, the exponent bits of input a or b are all 1; 2, ! r_eyp[9]&& r_eyp[8]. The result of exponent addition overflows, and the final result exceeds the number represented by the 8-bit mantissa. Overflow; 3, | r_eyp[7:0]. The result of exponent addition is a special number.
[0100] Underflow: 1, r_eyp[9]&&! r_eyp[8] means the exponents of a and b are too small, and the addition result is not as large as the currently defined exponent bias (127). It is close to 0+0. The final result is directly 0; 2, one of the exponent bits of input a or b is all zero, that is, a non-normal number, underflow.
[0101] (3) The output data uses reg output to reduce timing problems.
[0102] The floating point addition module in this embodiment is as follows:
[0103] The floating point multiplication module has input channels a, b, i_valid and output channels c, o_valid. Figure 7 As shown, a total of 6 steps are required:
[0104] (1) Use reg to store the input data to extract the sign bit, exponent bit, and mantissa bit. Using registers can reduce timing issues.
[0105] (2) Compare the size; exclude the sign bit, that is, compare the exponent and the mantissa. Because the decimal point positions of the input addend and the augend are not necessarily corresponding to each other, that is, the exponent bits are equal. It is necessary to distinguish which number is larger between a and b. Therefore, the smaller number is aligned to the larger number. (If the larger number is aligned to the smaller number, the left side of the mantissa of the larger number will be lost, resulting in a large weight data error. If the smaller number is moved to the larger number, only the right side of the mantissa of the smaller number will be lost, resulting in a small weight data error, with little loss of precision.) The larger number is stored in big_data, and the smaller number is stored in small_data.
[0106] (3) Obtain the order difference; use big_data_exp-small_data_exp to get the original order difference. Make a judgment: Because the mantissa bits of FP32 plus the hidden bit are 24 bits in total. If the order difference between the two is greater than this number, we believe that the larger number is too different from the smaller number. Even if small_data is enlarged by 2^24 times, it is still not as good as the minimum precision that FP32 can represent under the current exponent of the larger number. At this time, the order difference is fixed to 24.
[0107] (4) Align the exponents and add the mantissas; get the order difference redef_exp_diff after judgment. Shift the mantissa of the smaller number to the left by the corresponding number of bits, and add the exponent bits to the corresponding number of bits. Make the exponent bits of the larger number equal to the smaller number. After the order is completed, add the mantissa bits. If the sign bits of the larger number and the smaller number are equal, add them directly, r_add_fra[24:0]={1'b1,r_big_fra[23:0]+{1'b1,w_small_fra_shift[23:0]}. If the signs are different, r_add_fra[24:0]={1'b1,r_big_fra[23:0]–{1'b1,w_small_fra_shift[23:0]}; where w_small_fra_shift is the mantissa of the smaller number after order alignment, and r_big_fra is the mantissa of the larger number. In step (1), the way we compare the size does not include the sign bit. Therefore, at this time, the big number and the small number are distinguished in the form of absolute value. Therefore, after the order, r_big_fra must be greater than or equal to w_small_fra_shift.
[0108] (5) Normalize the added mantissa; With the r_add_fra obtained in step (4), use LZD (leading zero detection circuit) to obtain the first high-order bit address A of r_add_fra. Shift r_add_fra left by A. At the same time, subtract A from r_big_exp_r3 to obtain the normalized exponent bit r_exp_shift[8:0], so that the data size remains unchanged after the normalization result.
[0109] (6) Output: Sign bit: r_o_c_sign = r_big_sign_r3; Exponent bit: r_o_c_exp; 1. If r_exp_shift[8] in step (5) is high, it means that if normalized, the exponent bit will become a negative number, which can be understood as a small sum. At this time, the overall output is a non-normalized number; 2. If r_add_fra
[24] in step (4), r_o_c_exp will be increased by one. If it overflows, that is, r_o_c_exp[8] is high, it overflows and outputs a special number. Mantissa bit: If r_o_c_exp is not a special case, r_o_c_fra is determined by r_add_fra
[24] . If it is high, it is r_add_fra[23:1], otherwise r_add_fta[22:0].
[0110] The floating point comparison module in this embodiment is as follows:
[0111] The floating point multiplication module has input channels a, b, i_valid and output channels c, o_valid. It also defines a constant P_MOD, which can be used to determine whether to output a larger number or a smaller number. Figure 7 As shown, there are three steps:
[0112] (1) Use reg to extract the sign bit, exponent bit, and mantissa bit of the input data. Using registers can reduce timing issues.
[0113] (2) Determine the size: By default, a is determined to be a large number. First determine the sign. If the signs are different, determine the size based on the positive and negative signs.
[0114] If they have the same sign and are positive numbers, determine the exponents. If they are different, distinguish the sizes directly, and the larger number is the larger number. If they are the same, determine the mantissa. If they are different, distinguish the sizes directly, and the larger number is the larger number. If they are the same, then a and b are equal.
[0115] If they have the same sign and are negative numbers, determine the exponents. If they are different, distinguish the sizes directly and the larger number is the smaller number. If they are the same, determine the mantissa. If they are different, distinguish the sizes directly and the larger number is the smaller number. If they are the same, then a and b are equal.
[0116] (3) Output: Use registers to reduce timing violations. If P_MOD = "BIG", c = max{a,b}; if P_MOD = "SMALL", c = min{a,b}.
[0117] Example 1: FPGA-based FP32 precision convolutional neural network simulation
[0118] The development platform of this embodiment uses Vivado 2020.2, and the simulation platform is Modelsim 20.4. The overall architecture implemented is as follows Figure 1As shown. First, in Python Figure 2 The handwritten digit recognition network in is modeled and trained, and the accuracy of the model can reach 96.10%. Save the parameters of each layer, that is, the weights and biases of the convolutional layer and the fully connected layer, export them in the form of "%f1.6", convert them into ".coe" format using Matlab, and initialize the ROM (read-only memory) in the weight bias storage of each module.
[0119] In this embodiment, the entire reasoning process of the handwritten digit recognition network consumes a total of 312 clock cycles (from the last valid falling edge of the input data to the last valid output of the recognition result). Fig.10 , under the condition that the overall network accuracy is set to FP32, the comparison of the Vivado simulation intermediate values of channel 1 of the first convolution layer, channel 2 of the second convolution layer, and channel 1 of the fully connected layer with the intermediate values extracted in Python. The absolute difference between the two and the Python exported value are used to perform the following operation to obtain the relative error p%:
[0120]
[0121] The relative error will reach the maximum in the last layer (the fully connected layer in this embodiment) after accumulation. Fig.10 The precision error is within an acceptable range of 0.02%. The precision error is mainly affected by the following: the form of extracting network parameters and intermediate values from Python, and the number of subsequent decimal places. This embodiment uses the "%f1.6" form to extract.
[0122] The network is tested with 1000 consecutive images, where the images are arranged in a cyclic pattern of 0-1-2-3-4-5-6-7-8-9. The simulation results are as follows: Fig.11 As shown in the figure, the final recognition accuracy of 1000 images is 97.00%. This is because of the number of test images selected. If the number of test images is large enough, it will eventually fluctuate around 96.10%, which is consistent with the python result.
[0123] Example 2: FPGA-based FP32 precision convolutional neural network deployment
[0124] The development platform of this embodiment uses Vivado2020.2, and the development board is based on
[0125] XCKU060-ffva1156-2-i is the core of AXKU062. The deployment clock is 200MHZ, and the timing test is passed without timing violation. Because the FP32 format is used for deployment, the pursuit of accuracy consumes more resources, and the data is all 32bit. The overall resource consumption is as follows Fig.12 As shown. The power consumption is 1.75W, Fig.13shown.
[0126] In the reasoning process of a single image, the processing time is 312 clock cycles. After analysis, the floating point operations (including only floating point addition and floating point multiplication, excluding comparison and conversion) in a single process of this embodiment are: 50122 times. Therefore, under the 200M clock, the number of floating point operations per second FLOPS is 50122×200×10 6 / 312=32.129G.
[0127] In fact, the board effect is as Fig.14 As shown, the result is correct (recognizing the number 7).
[0128] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.
Claims
1. A floating point convolutional neural network reasoning deployment method based on FPGA, characterized in that: The method comprises the following steps: the original data input by the host computer is converted into floating-point numbers by an integer-to-floating-point module and then enters a handwritten digit recognition inference network, and is processed by a floating-point multiplication module, a floating-point addition module, and a floating-point comparison module to obtain a result; wherein the handwritten digit recognition inference network comprises two convolutional layers, one fully connected layer, and one decision layer, and both of the two convolutional layers are customized for module multiplexing and corresponding control signals to realize the multiplexing of the system convolutional module and the unit convolutional module.
2. The method for deploying floating-point convolutional neural network reasoning based on FPGA as described in claim 1, characterized in that: The convolution structures of the two convolutional layers are the same basic architecture. The serial data is converted into parallel data matching the convolution kernel size through the data cache module, so that K×K floating point number calculations are completed in one clock cycle, where K represents the convolution kernel size. The convolution completed data passes through a redundant addition tree to realize the accumulation process of K×K->1. If the current data is missing, just input 0; At the end of each output data, the maximum pooling layer and ReLU activation layer are connected to realize the downsampling function and reduce the subsequent data transmission consumption; Customize module multiplexing and corresponding control signals to realize the multiplexing of system convolution modules and unit convolution modules.
3. The method for deploying floating-point convolutional neural network reasoning based on FPGA as described in claim 1, characterized in that: The fully connected layer builds the required parallel fully connected layer calculation channels according to the number of output channels. If the previous module is a convolutional layer parallel output, a parallel-to-serial conversion module is required to straighten the multi-dimensional tensor into a one-dimensional vector. The fully connected layer uses a serial addition tree. In each fully connected layer calculation channel, the data and the fully connected layer weights are processed in fc_unit_pe, and the product results are accumulated using a floating-point addition array composed of multiple floating-point addition modules; The data at the parallel output end will be converted from parallel to serial and input to the next fully connected layer or decision layer.
4. The method for deploying floating-point convolutional neural network reasoning based on FPGA as claimed in claim 1, characterized in that: The decision layer converts the input data into serial-to-parallel data and stores it in the register matrix for temporary storage, inputs it into the floating point comparison module, compares the size, compares the largest data with the data in the register matrix, and outputs the equal matrix number, which is the final recognition result.
5. The method for deploying floating-point convolutional neural network reasoning based on FPGA as claimed in claim 1, characterized in that: The integer to floating point module is used to convert the external input image data, which is 0-255 integer data, into the data format required by the current precision. The data format is a floating point number and complies with the IEEE 754 standard.
6. The method for deploying floating-point convolutional neural network reasoning based on FPGA as claimed in claim 1, characterized in that: The floating-point multiplication module needs to go through three steps to obtain the multiplication result: first, use the register to extract the sign bit, exponent bit, and mantissa bit of the input data, then add the exponents, multiply the mantissas, and finally normalize the mantissa result, including LZD leading zero detection and mantissa truncation.
7. The method for deploying floating-point convolutional neural network reasoning based on FPGA as claimed in claim 1, characterized in that: The floating-point number addition module needs to go through six steps to obtain the addition result: first, use the register to extract the sign bit, exponent bit, and mantissa bit of the input data, compare the size, distinguish the larger number from the smaller number, obtain the order difference between the larger number and the smaller number, and re-judge the order difference, complete the order and mantissa addition, complete the mantissa addition result normalization through the LZD circuit, and process the exponent bit according to the mantissa normalization result; finally, use the register to obtain the result according to the flag signal in the previous steps.
8. The method for deploying floating-point convolutional neural network reasoning based on FPGA as claimed in claim 1, characterized in that: The specific steps include: S1, convolutional layer (1) After the FPGA is powered on, (CI×CO×((K×K)+1)) stored weight values and bias values are read from weight_bias_buf, and the valid signal controls the entry of the weight bias registers in CO conv_sys_pe and CI conv_unit_pe; where CI represents the number of input channels of the convolution layer; CO represents the number of output channels of the convolution layer; and K represents the size of the convolution kernel. (2) The input CI channel serial data is converted into parallel data through conv_data_buffer; the input serial data will be stored in a register array of size CI×I×I×K; in each parallel array, the input serial data row counter r_raw_cnt==K-1, the column counter r_col_cnt==K-1, the state machine in the module will start to jump from P_ST_IDLE to P_ST_ONE, when the column counter r_col_cnt==I-1, the state machine will jump to the next state, if the row counter r_raw_cnt==I-1, and the column counter r_col_cnt==I-1, it means that an image has been completely input and processed, the state machine jumps back to P_ST_IDLE, waiting for the next image to be recognized; where I represents the size of the image to be recognized; (3) CI×K×K data captured by the conv_data_buffer module enter C0 conv_sys_pe modules in parallel. Each conv_sys_pe obtains CI results, which are then passed through an addition tree composed of 2^($clog(CI)) adders to obtain CO outputs. (4) conv_sys_pe receives and processes the data, inputs it into conv_unit_pe, performs convolution on the data of each channel, and sums and adds the bias through the addition tree; (5) The convolved data is subjected to maximum pooling and ReLU activation; S2, fully connected layer (1) After the FPGA is powered on, FO×((FI×(IM×IM))+1) stored weight values and bias values are read from weight_bias_buf and entered into FO weight bias registers in fc_sys_pe controlled by the val id signal; where FI represents the number of input channels of the fully connected layer, FO represents the number of output channels of the fully connected layer, and IM represents the size of the data input into the fully connected layer; (2) Input data enters fc_data_buffer for straightening processing; FI+1 synchronous FIFO is used to complete the parallel-to-serial conversion. After the conversion is completed, the data is input into FO fc_sys_pe for multi-channel parallel calculation; (3) After receiving the data, fc_sys_pe reads the continuous weight data from DUAL_RAM and sends the two streams of data to the floating-point multiplication module. The output result then enters the floating-point addition array composed of multiple fc_unit_pe; (4) fc_unit_pe adds the product of data and weight and processes the bias; (5) Convert the results obtained by parallel calculation of F0 fc_sys_pe into serial and parallel form and input them to the decision layer; S3, decision-making level; (1) After separating the input data, it is serially converted according to the D recognition types defined in the top-level parameters, where D is the number of output channels of the fully connected layer before the decision layer; (2) Use a floating-point comparison array composed of multiple floating-point comparison modules to compare the parallel data, obtain the largest data and its position, output the corresponding result, and obtain the recognition data.