FPGA-based target detection model acceleration design method

Abstract

The invention discloses an FPGA-based target detection model acceleration design method, and relates to the technical field of computer vision. The acceleration design method comprises the steps that a block RAM is adopted as a feature map cache unit, DDR4 data provided by peripheral equipment are converted into stream data at the input end of the feature map cache unit, and the stream data are stored in a line scanning mode; before the feature map is input into convolution processing, the feature map is processed through a quantization operation unit, and floating-point number data is converted into fixed-point data; and inputting the quantized feature map and the model weight into a convolution operation unit to execute convolution operation, caching the feature map into a row buffer area in rows in each convolution in the convolution operation unit, and multiplying the feature map with a weight matrix position by position and adding the feature map. According to the method, ZYNQ is adopted as a hardware platform, the target detection algorithm is accelerated and optimized through a software and hardware collaborative design method, the operation speed of the target detection algorithm is improved, and the accuracy of target detection is ensured.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an FPGA-based target detection model acceleration design method. Background Art

[0002] Object detection technology plays an important role in all areas of social life. In the field of intelligent monitoring, efficient object detection can monitor and identify objects such as people and vehicles in real time, providing a safer and more intelligent monitoring system. In the field of autonomous driving, fast and accurate object detection is the key to achieving reliable traffic environment perception. In the field of industrial automation and robotics, object detection can be used in applications such as object recognition and logistics management to improve production efficiency and quality. However, the complexity of object detection algorithms, large amount of calculations, and high real-time requirements have brought higher challenges to traditional computing platforms. In order to meet these needs and promote the further implementation of object detection technology, FPGA (Field-Programmable Gate Array) technology has received widespread attention. FPGA has the advantages of high performance, low power consumption and flexible configuration, making object detection applications faster, smarter and more efficient.

[0003] The parallel processing capability of FPGA is particularly suitable for rapid verification and testing of algorithms. Although FPGA provides many advantages, it still has some limitations in practical applications. First, the hardware acceleration design of FPGA requires professional hardware description language programming knowledge, which increases the complexity of the design and the difficulty of development; second, although the programmability of FPGA provides flexibility, there may be gaps in performance and energy efficiency compared with dedicated ASICs (application-specific integrated circuits). In addition, the resource utilization and clock frequency of FPGA may be limited by its physical characteristics and design constraints, which may affect the performance of the target detection model when running on FPGA. Finally, the hardware acceleration solution of FPGA may need to be customized for different algorithms or models, which increases the development cost and cycle.

[0004] In summary, although FPGA has significant advantages in improving AI reasoning performance and hardware resource utilization efficiency, it still faces challenges in design professionalism, performance limits, resource utilization, and customized development. In order to further improve the performance of artificial intelligence reasoning chips and reduce the complexity of computing units, this paper proposes a hardware acceleration design scheme for deploying target detection models on FPGA. Summary of the invention

[0005] The present invention provides an FPGA-based target detection model acceleration design method, which reduces the complexity of the calculation unit and improves the algorithm running speed, so as to solve the problems of insufficient performance and resource utilization in the application of the target detection model on the FPGA.

[0006] The present invention is achieved through the following technical solutions:

[0007] The present invention provides an FPGA-based target detection model acceleration design method, which deploys the target detection model in a ZYNQ chip board. The acceleration design method includes:

[0008] A block RAM is used as a feature map buffer unit, and DDR4 data provided by a peripheral device is converted into stream data at an input end of the feature map buffer unit, and stored in a row scanning manner;

[0009] Before the feature map is input into the convolution process, the feature map is processed by a quantization operation unit to convert the floating point data into fixed point data;

[0010] The quantized feature map and the model weight are input into a convolution operation unit to perform a convolution operation. In the convolution operation unit, each convolution caches the feature map into a row buffer by row grouping, and then multiplies and adds the weight matrix position by position;

[0011] The convolution result is output to the RAM storage unit.

[0012] The present invention improves the running speed of the YOLOv5s algorithm and ensures the accuracy of target detection through the method of software and hardware collaborative optimization. The software level uses the ARM processor to undertake data preprocessing tasks, and prepares the data in the memory into the format required for the convolution operation to adapt to the needs of the hardware accelerator. The hardware part configures an accelerator that specifically performs convolution operations in the FPGA, caches the image data in a row buffer pixel matrix composed of a register stack by row group each time, and then multiplies and adds the weight matrix position by position, simulates the sliding window effect during the convolution operation, and performs fixed-point quantization processing on the model, reducing the computational complexity and memory requirements of the model, so that the quantized model can be efficiently deployed on the ZYNQ platform to perform computing tasks. After completing the convolution calculation, the result data is transmitted back to the memory of the ARM processor, and the ARM processor completes subsequent data processing and caching operations. Through the method of software and hardware collaborative optimization, the running speed of the target detection algorithm is improved, the accuracy of target detection is guaranteed, and a high-performance, low-power intelligent processing system is realized.

[0013] Furthermore, the step of inputting the quantized feature map and the model weight into a convolution operation unit to perform a convolution operation includes:

[0014] Get the number of network layers according to network configuration parameters and initialize the network layer;

[0015] Traverse the network layer through forward reasoning of the target detection model to determine whether the current network layer is a convolutional layer;

[0016] If it is a convolutional layer, initialize the convolution scheduling state machine, send the feature map, weight and bias quantization parameters required for a convolution operation to the convolution operation unit to perform the convolution operation; if it is not a convolutional layer, determine whether the input layer data has been calculated. After the input layer data has been calculated, load the input layer data and calculate;

[0017] Determine whether all network layers have been calculated. If so, end the calculation; if not, calculate the next network layer.

[0018] Furthermore, before performing the convolution operation, the target detection model is deployed on the ZYNQ chip board, including:

[0019] Training the target detection model to obtain the number of network layers and the weight of each layer of the target detection model;

[0020] The weight values ​​and bias parameter values ​​are fixed-pointed to obtain the fixed-point model parameters, and a binary weight bias file is generated and deployed to the ZYNQ chip board.

[0021] Furthermore, the process of processing the feature map by a quantization operation unit to convert the floating-point data into fixed-point data includes: decomposing the floating-point data into a mantissa M0 and an exponent n by a frexp function.

[0022] Furthermore, in the convolution operation unit, multiplication is performed with low bit width, using INT18 as the accumulator bit width;

[0023] Saturation processing is performed on the INT18 data generated after all accumulations are completed, INT18 is truncated to INT16, and then multiplied with the mantissa M0 of UINT16 to obtain INT32 data;

[0024] If the [15+Shift_Fator-1]th bit of the INT32 data is 1, the data is shifted right and 1 is added to obtain the calculation result. If the [15+Shift_Fator-1]th bit is 0, the calculation result remains unchanged and Shift_Fator is the offset value.

[0025] The result of the operation is shifted right by 15 bits plus Shift_Fator to obtain INT16 data, and saturation processing is performed again to convert the INT16 data into INT18 data.

[0026] Furthermore, the convolution operation unit uses DSP48 as a multiplier, and the accumulator uses a ternary tree adder.

[0027] Furthermore, DSP48 is used to implement two multiplication parallel calculations, the method is as follows:

[0028] The two INT8 multiplications A×C and B×C are equivalent to (A+B)×C, the upper 9 bits of the A data are input to the 27-bit first input port of DSP48, and the sign bit of A is extended from 8 bits to 9 bits, and the remaining bits are filled with zeros;

[0029] Input the high 9 bits of data B into the 27-bit second input port of DSP48, and perform sign bit extension on B, expanding it from 8 bits to 9 bits, and fill the remaining bits with zeros;

[0030] Input the lower 8 bits of data C into the 18-bit third input port of DSP48, and perform sign bit extension on C, expanding it from 8 bits to 9 bits, and fill the remaining bits with zeros;

[0031] The 48-bit output port of DSP48 outputs the results of the two multiplication operations.

[0032] Furthermore, the length of the line buffer is configurable, and the length of each line is configured according to the image size of the corresponding network layer.

[0033] Furthermore, a ping-pong buffer mechanism is introduced into the feature map buffer unit.

[0034] Furthermore, the target detection model is YOLOv5s, and the board is a ZYNQ Ultrascale+ chip board.

[0035] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0036] The present invention adopts ZYNQ as the hardware platform and accelerates and optimizes the target detection algorithm through a software-hardware collaborative design method, thereby improving the running speed of the target detection algorithm, ensuring the accuracy of target detection, and realizing high-efficiency, low-power intelligent processing.

[0037] The model is trained using open source data sets, and weight bias files are generated and stored in the system memory. The ARM processor is responsible for data preprocessing, preparing the data in the memory into the format required for convolution operations, and splitting and scheduling the weight parameters of the convolution kernel in batches to adapt to the needs of the hardware accelerator. This not only improves resource utilization efficiency, but also ensures good compatibility with the ZYNQ chip.

[0038] The convolutional neural network accelerator is designed to perform convolution operations specifically to improve the algorithm's operational efficiency. After completing the accelerated calculation, the result data is transferred back to the memory of the ARM processor, which then completes subsequent data processing and caching operations, making full use of FPGA resources.

[0039] Through a single DSP48, dual INT8 multiplication is performed in parallel, and the multiplication and addition operations in the convolution operation are implemented simultaneously in multiple dimensions. For each operation of 3×3 convolution, an operation module will have 9 values ​​added through the addition tree. Finally, the results of the multiplication operation are added by using the "ternary tree" adder architecture to finally obtain the result of the convolution operation, which improves the speed of the addition operation.

[0040] The convolution accelerator is designed with a deeply configurable line cache architecture, which dynamically adjusts the length of the line buffer according to the image size requirements of different levels to ensure that it is compatible with the current feature map size, thereby improving performance while reducing resource consumption.

[0041] Since the convolution accelerator is designed with a deeply configurable row cache architecture, there is no need to divide the feature map into fixed-size blocks. For larger feature maps, we can perform "rectangular blocking" and perform convolution on only a few rows at a time to increase the speed of convolution operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without creative work. In the drawings:

[0043] Figure 1 It is a schematic diagram of a convolution operation hardware design structure of an embodiment of the present invention;

[0044] Figure 2 is a quantization processing flow chart of an embodiment of the present invention;

[0045] Figure 3 It is a convolution operation circuit design structure diagram of an embodiment of the present invention;

[0046] Figure 4 is a schematic diagram of a pyramid row buffer mechanism with dynamically configurable depth according to an embodiment of the present invention;

[0047] Figure 5 This is an ARM-side scheduling flow chart of an embodiment of the present invention;

[0048] Figure 6 It is a schematic diagram of an FPGA-based target detection model acceleration design solution according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0050] It should be noted that the terms "including" and "having" and any variations thereof in the specification and claims of the present invention and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to or inherent to other steps or units of the device.

[0051] The terms used in the various embodiments of the application are only used to describe the purpose of specific embodiments and are not intended to limit the various embodiments of the application. As used herein, the singular form is intended to also include the plural form, unless the context clearly indicates otherwise. Unless otherwise limited, all terms used here (including technical terms and scientific terms) have the same meaning as the meanings commonly understood by ordinary technicians in the field of the various embodiments of the application. The terms (such as the terms defined in the dictionary generally used) will be interpreted as having the same meaning as the contextual meaning in the relevant technical field and will not be interpreted as having an idealized meaning or an overly formal meaning, unless clearly defined in the various embodiments of the application.

[0052] The embodiments of the present invention provide an FPGA-based target detection model acceleration design method, which is beneficial to improving the AI ​​reasoning performance and hardware resource utilization of the FPGA, improving the running speed and accuracy of the target detection algorithm deployed on the FPGA, and realizing high-efficiency and low-power intelligent processing.

[0053] The present invention aims to improve the performance of hardware acceleration design by deploying target detection models on existing FPGA boards. To achieve this goal, the present invention deploys the target detection model to the ZYNQ chip board to build a heterogeneous system architecture for software and hardware collaborative acceleration. During the research process, the system is divided into two main parts: software and hardware, which are independently developed and work together. The software part runs the Linux Ubuntu 18.04LTS operating system on the ARM processor to implement the software logic of the algorithm. The hardware part involves the configuration of the FPGA and the implementation of the self-designed convolutional neural network accelerator module. The design and integration of the hardware IP core are completed on the programmable logic (PL) side to ensure that the designed IP core can be efficiently called by the processor system (PS) side.

[0054] like Figure 1 As shown in the schematic diagram of the convolution operation hardware design structure, the hardware accelerator part of the present invention mainly includes a feature map buffer unit, a quantization operation unit and a convolution operation unit.

[0055] In the feature map buffer unit, a block RAM storage unit is used, and a row scanning method is used in the feature map buffer unit to store and arrange data, ensuring that the data stored in the dual-port RAM memory and register meet the sliding window mechanism of the convolution operation.

[0056] At the input end of the cache module, the DDR4 data provided by the peripheral device is converted into stream data. Specifically, the DDR4 data is converted into stream data via the AXI4 Stream protocol, and then queued in the feature map buffer and injected into the convolution acceleration module for processing. At the input end of the cache module, it refers to the stream data exported by the convolution acceleration module after the calculation is completed. It is moved back to the peripheral RAM storage space through the AXIDMAIP core module, and the ARM processor completes subsequent data processing and cache operations. The data stream exported by the convolution acceleration module is correspondingly processed by dequantization and pooling, and then stored in the output buffer area. The system design ensures that in each timing cycle, hundreds of bits of data are able to complete input and output operations.

[0057] Furthermore, in order to optimize the overall throughput performance of the system, a ping-pong buffer mechanism is introduced in the buffer unit.

[0058] The quantization operation unit module is used to reduce the model size. Before the feature map is input into the convolution process, the feature map is processed by the quantization operation unit to convert the floating-point data into fixed-point data, thereby converting the floating-point operation into the fixed-point operation.

[0059] The convolution accelerator constructed in this embodiment is based on the Pytorch framework and relies on the Post Training Static Quantization (PTSQ) model optimization technology to quantize the 32-bit floating-point parameters in the trained neural network model into 8-bit integer representation. Specifically, before performing the convolution operation, the feature map data and weight data will be subtracted from their respective zero points, that is, according to the convolution operation The floating point numbers of the convolution operation are replaced into quantized integers and then properly transformed mathematically using the following formula:

[0060]

[0061] In the above formula, variable r is a floating point number. represents the element in the i-th row and j-th column of the input feature map, represents the element in the jth row and kth column of the convolution kernel, Represents the element in the i-th row and k-th column of the output feature map; N represents the width or height of the input feature map; after the quantization operation, S1, S2, and S3 represent the scaling factors; Respectively The integer result after quantization; Z is the position of the zero point after quantization, and Z1, Z2, and Z3 represent The corresponding deviation value.

[0062] It can be seen that except Except for the floating point number, the rest is INT8 operation. According to the storage characteristics of the floating point number itself, the factor M can be decomposed into the mantissa M0 and the exponent n using the frexp() function: M = 2 -n M0, so the entire process of model reasoning after quantization can be converted into a fixed-point operation process.

[0063] Furthermore, when performing convolution operations, multiplication operations between integer values ​​are performed with a lower bit width, such as the result of multiplying two INT8 type values ​​is INT16 type. Considering the saving of FPGA resources, in actual situations, INT8 and INT8 multiplication uses INT18 as the accumulator bit width, which can effectively avoid data overflow.

[0064] Specifically, see Figure 2 The quantization process flow chart shown includes the following steps:

[0065] S101, performing saturation processing on the INT18 data generated after all accumulation operations are completed, truncating the INT18 data to INT16, and then performing multiplication operation with the mantissa M0 of the UINT16 type, thereby obtaining INT32 data;

[0066] S102, then obtain the numerical state of the [15+S_Factor-1]th bit in the shifter; if the value of this bit is 1, the data is right-shifted and 1 is added to the right-shifted value accordingly; if the value of this bit is 0, the operation result remains unchanged. S_Fator is the offset value of the shifter.

[0067] S103, perform a right shift operation of 15 bits plus S_Factor (ie, n) on the calculation result of S102 to obtain INT16 data again, and finally perform saturation processing again to convert the INT16 data into INT8.

[0068] In the convolution operation unit module, the image data is cached in a pixel matrix composed of register stacks (row buffer) by row group each time, and then multiplied and added with the weight matrix position by position to simulate the sliding window effect during convolution operation.

[0069] Furthermore, the convolution operation unit uses DSP48 as a multiplier, and the accumulator uses a ternary tree adder.

[0070] There are a lot of underutilized bit width resources when a single DSP48 executes a single INT8 multiplication. To solve this problem, this section fully exploits the structural characteristics of DSP48 to implement parallel execution of dual INT8 multiplication on a single DSP48 resource.

[0071] Combination Figure 1 and Figure 3 The convolution operation circuit design structure diagram shown in the figure. If two INT8 multiplications A×C and B×C are to be performed, the two are equivalent to (A+B)×C. The upper 9 bits of the A data are input to the 27-bit A port of DSP48, and the sign bit of A is extended from 8 bits to 9 bits, and the remaining 18 bits are filled with 0. Similarly, for the 27-bit B port, the sign bit of data B is extended from 8 bits to 9 bits, the remaining bits are filled with zero, and the B data is input in the lower 9 bits of the B port. The C port is 18 bits wide, and the sign bit of data C is also extended from 8 bits to 9 bits, the remaining bits are filled with zero, and the data C is input in the lower 8 bits of the C port. After completing the above configuration, the 48-bit P output port of DSP48 can correctly output the required two products (A+B)×C results.

[0072] like Figure 3 As shown in the figure, the multiplication and addition operations in the convolution operation are implemented simultaneously in multiple dimensions by utilizing the parallelism of the FPGA and the limited DSP resources in the convolution operation. In each operation, a calculation module will have 9 values ​​added through the addition tree, including intra-channel addition and inter-channel addition. For the sum of the product of the elements in the 3×3 matrix and the accumulation operation in the 8-way input channel containing the bias term, the "ternary tree" adder architecture is used to add the results of the multiplication operation to finally obtain the result of the convolution operation.

[0073] Furthermore, the length of the line buffer is configurable, and the length of each line is configured according to the image size of the corresponding network layer.

[0074] See also Figure 4 The schematic diagram of the pyramid row buffer mechanism with dynamically configurable depth is shown. In a deep neural network convolution operation, there are multiple feature maps of different sizes. Usually, the feature map of the next layer is half the spatial size of the previous layer. In terms of the channel dimension, the next layer is usually twice the size of the previous layer. In order to further optimize the utilization of hardware resources, this embodiment proposes a length-configurable row buffer, which dynamically adjusts the length of the row buffer according to the image size requirements of different levels. The maximum depth of the row buffer corresponds to the input size. Taking this as a reference, a step-by-step tapping method is adopted. Through the mechanism of dynamic configuration of the multiplexer, the dynamic adjustment of the row buffer depth required for the current layer image is realized to ensure that it is compatible with the current feature map size.

[0075] After the feature map is input from the off-chip DDR memory, it needs to be cached to the FPGA on-chip memory resources. Since this convolution accelerator is designed with a deeply configurable row cache architecture, there is no need to block the feature map into fixed sizes. For larger feature maps, "rectangular blocking" can be performed, and only a few rows of convolution are performed each time. For example, the on-chip memory can only accommodate feature maps of size 52×52, and when a feature map of size 104×104 needs to be convolved, the original feature map is divided into four 104×26 rectangular blocks, which are input in sequence and processed in FPGA mode.

[0076] Furthermore, at the software level, the ARM processor is used to split the weight parameters of the convolution kernel and schedule them in batches to adapt to the needs of the hardware accelerator.

[0077] First, load the weight file and network layer parameter file. This step completes the deployment of the target detection model in the ZYNQ chip board, which occurs before image detection. The target detection model is deployed on the ZYNQ chip board, specifically including: training the target detection model, obtaining the number of network layers and the weight of each layer of the target detection model; fixing the weight value and bias parameter value to obtain the fixed-point model parameters, generating a binary weight bias file, and deploying it to the ZYNQ chip board.

[0078] The model training can use open source data sets, such as VisDrone, or collect sample data in specific fields. The model training method uses a known training method. The fixed-point processing of weight values ​​and bias parameter values ​​is the same as the fixed-point processing of feature images, converting floating-point data into INT8 data.

[0079] After deploying the target detection model, before each convolution task, the ARM processor splits the weight parameters of the convolution kernel and schedules them in batches. Figure 5 The ARM-side scheduling flowchart shown includes the following steps.

[0080] S110, obtaining the number of network layers according to network configuration parameters, and initializing the network layers.

[0081] S120, traversing the network layers through forward reasoning of the target detection model to determine whether the current network layer is a convolutional layer.

[0082] S130, if it is a convolution layer, then initialize the convolution scheduling state machine, i.e. reset it, and start from the initial state, send the feature map, weight and bias quantization parameters required for a convolution operation to the convolution operation unit to perform the convolution operation;

[0083] If it is not a convolutional layer, determine whether the input layer data has been calculated. When the input layer data has been calculated, load the input layer data and calculate it.

[0084] S140, determining whether all network layers have been traversed, if so, terminating the calculation; if not, calculating the next network layer.

[0085] When the original image is divided into rectangular blocks as a whole, in the above step S130, the feature map, weight and bias quantization parameters required for the convolution operation of a convolution block are sent to the convolution operation unit each time to perform the convolution operation, and it is determined whether the convolution layer has processed all the blocks. If not, the corresponding offset address is incremented and the next block is calculated until all blocks are processed.

[0086] Finally, a complete FPGA-based target detection model acceleration design solution is as follows Figure 6 As shown, in one embodiment of the present invention, the target detection model uses YOLOv5s, and the FPGA board uses the ZYNQ Ultrascale+ chip board. First, the YOLOv5s target detection model is trained by collecting data to generate a binary model weight bias file. At the software level, the above-mentioned method of splitting and batch scheduling the weight parameters of the convolution kernel through the ARM processor is used to form a binary code bit control file. At the hardware acceleration level, the RTL design of the convolution accelerator is encapsulated as an IP core to form a VIVADO hardware path of the hardware accelerator, and generate a bit stream and hwh hardware description file. The model weight bias file, batch scheduling control file, and hardware description file are deployed to the ZYNQ Ultrascale+ chip board.

[0087] The present invention adopts ZYNQ Ultrascale+MPSoC ZCU104 as the experimental hardware platform, accelerates and optimizes the YOLOv5s target detection algorithm through the software-hardware collaborative design method, proposes a method for optimizing convolution multiplication and addition operations, makes full use of hardware resources, and then explores the parallelization of convolution blocks, thereby achieving performance improvement while reducing resource consumption. The convolution layer is batch-scheduled by the software part, and the circuit structure reuse strategy is adopted in the hardware part. The energy consumption of ZCU104 is 3.915W, which is 1.56 times the energy consumption of directly using the ARM core. However, the computing processing rate of ZCU104 is 20.04GOPs, which is 557.22 times that of directly using the ARM core. These optimization measures not only improve the resource utilization efficiency, but also ensure good compatibility with the ZYNQUltrascale+ chip.

[0088] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. The object detection model acceleration design method based on FPGA is characterized by: The target detection model is deployed in the ZYNQ chip board. The acceleration design method includes: A block RAM is used as a feature map buffer unit, and DDR4 data provided by a peripheral device is converted into stream data at an input end of the feature map buffer unit, and stored in a row scanning manner; Before the feature map is input into the convolution process, the feature map is processed by a quantization operation unit to convert the floating point data into fixed point data; The quantized feature map and the model weight are input into a convolution operation unit to perform a convolution operation. In the convolution operation unit, each convolution caches the feature map into a row buffer by row grouping, and then multiplies and adds the weight matrix position by position; The convolution result is output to the RAM storage unit.

2. The FPGA-based target detection model acceleration design method according to claim 1, characterized in that: The step of inputting the quantized feature map and the model weight into a convolution operation unit to perform a convolution operation includes: Get the number of network layers according to network configuration parameters and initialize the network layer; Traverse the network layer through forward reasoning of the target detection model to determine whether the current network layer is a convolutional layer; If it is a convolutional layer, initialize the convolution scheduling state machine, send the feature map, weight and bias quantization parameters required for a convolution operation to the convolution operation unit to perform the convolution operation; if it is not a convolutional layer, determine whether the input layer data has been calculated. After the input layer data has been calculated, load the input layer data and calculate; Determine whether all network layers have been calculated. If so, end the calculation; if not, calculate the next network layer.

3. The FPGA-based target detection model acceleration design method according to claim 2, characterized in that: Before performing the convolution operation, the target detection model is deployed on the Zynq chip board, including: Training the target detection model to obtain the number of network layers and the weight of each layer of the target detection model; The weight values ​​and bias parameter values ​​are fixed-pointed to obtain the fixed-point model parameters, and a binary weight bias file is generated and deployed to the ZYNQ chip board.

4. The FPGA-based target detection model acceleration design method according to claim 1, characterized in that: The step of processing the feature map by a quantization operation unit to convert the floating-point data into fixed-point data includes: decomposing the floating-point data into a mantissa M0 and an exponent n by a frexp function.

5. The FPGA-based target detection model acceleration design method according to claim 4, characterized in that: In the convolution operation unit, multiplication is performed with low bit width, and INT18 is used as the accumulator bit width; Saturation processing is performed on the INT18 data generated after all accumulations are completed, INT18 is truncated to INT16, and then multiplied with the mantissa M0 of UINT16 to obtain INT32 data; If the [15+S_Fator-1]th bit of the INT32 data is 1, the data is right-shifted and added by 1 to obtain the operation result. If the [15+S_Fator-1]th bit is 0, the operation result remains unchanged and S_Fator is the offset value. The operation result is shifted right by 15 bits plus S_Fator to obtain INT16 data, and saturation processing is performed again to convert the INT16 data to INT18 data.

6. The FPGA-based target detection model acceleration design method according to claim 5, characterized in that: The convolution operation unit uses DSP48 as a multiplier, and the accumulator uses a ternary tree adder.

7. The FPGA-based target detection model acceleration design method according to claim 6, characterized in that: Use DSP48 to implement two multiplication parallel calculations, the method is: The two INT8 multiplications A×C and B×C are equivalent to (A+B)×C, the upper 9 bits of the A data are input to the 27-bit first input port of DSP48, and the sign bit of A is extended from 8 bits to 9 bits, and the remaining bits are filled with zeros; Input the high 9 bits of data B into the 27-bit second input port of DSP48, and perform sign bit extension on B, expanding it from 8 bits to 9 bits, and fill the remaining bits with zeros; Input the lower 8 bits of data C into the 18-bit third input port of DSP48, and perform sign bit extension on C, expanding it from 8 bits to 9 bits, and fill the remaining bits with zeros; The 48-bit output port of DSP48 outputs the results of the two multiplication operations.

8. The FPGA-based target detection model acceleration design method according to claim 1, characterized in that: The length of the line buffer is configurable, and the length of each line is configured according to the image size of the corresponding network layer.

9. The FPGA-based target detection model acceleration design method according to claim 1, characterized in that: A ping-pong buffer mechanism is introduced into the feature map buffer unit.

10. The FPGA-based target detection model acceleration design method according to claim 1, characterized in that: The target detection model is YOLOv5s, and the board is a ZYNQ Ultrascale+ chip board.

Images