Unmanned aerial vehicle target detection system based on FPGA
By adopting FPGA data multiplexing and parallel processing design in the UAV target detection system, the problems of long detection time and high power consumption of traditional systems are solved, and faster detection speed and higher energy efficiency are achieved. .
Patent Information
- Application Number
- CN202310519681.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-10
- Publication Date
- 2024-11-12
AI Technical Summary
Traditional drone target recognition systems need to send image information to the cloud for processing, resulting in a long detection time, unable to achieve real-time detection, and the power consumption of the CPU and GPU is high and the computing performance is insufficient.
Design a drone target detection system based on FPGA, adopts a neural network acceleration module designed with data multiplexing and parallel processing, and reduces the number of input feature maps and weight data on FPGA, and adopts data streaming and parallel processing technology. External storage access and resource consumption of computing units.
It achieves faster detection speed and higher energy efficiency ratio, improves the flexibility and versatility of the system, and can perform a variety of neural network object detection algorithms in real time on the drone side.
Smart Images

Figure CN118941981A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an FPGA-based unmanned aerial vehicle target detection system. The system utilizes the parallel processing advantages of FPGA, adopts data reuse and parallel processing design, realizes the deployment and acceleration of lightweight neural networks, improves the detection speed and energy efficiency, and can realize a variety of neural network target detection algorithms. Background Art
[0002] Traditional drone target recognition is to transmit the image information taken by the drone to the cloud server. However, since the data information collected by the drone needs to be sent to the cloud, it takes a long time to infer and cannot meet the requirements of real-time detection, which limits its actual application scenarios. In recent years, research on specific scenarios, specific target detection tasks and specific embedded platforms has become a hot topic. As a semi-custom circuit in the field of application-specific integrated circuits (ASICs), FPGA can change the hardware functions of the system through programming, thereby greatly improving the flexibility and versatility of system design. Compared with CPUs and GPUs that need to frequently access DRAM, resulting in high power consumption, FPGAs rely on their own rich on-chip storage resources to greatly reduce the access delay of off-chip storage, while improving computing performance with lower power consumption, and FPGAs also support parallel computing. Summary of the invention
[0003] A drone target detection system, characterized in that the system comprises a camera module, an image data acquisition module, a preprocessing module, a neural network acceleration module, an airborne control module, a display module and a power module.
[0004] First, the camera module on the drone collects image data. Since the size format of the collected image is not the preset 640×640 size, a series of image preprocessing operations such as stretching and compression are required. Then, the neural network accelerator system is used to perform feature extraction, classification, regression and other operations, and finally the detection results are transmitted to the display module. The overall task scheduling and working status acquisition of the system are controlled by the onboard control module.
[0005] The neural network acceleration module is characterized by including two major parts: data multiplexing and parallel processing design, and performs network operations by reasonably allocating FPGA resource designed TGU acceleration IP.
[0006] The data reuse of the neural network acceleration module is characterized by the fact that, in view of the limited on-chip storage resources of the FPGA, the strategy adopted is to pre-store the input feature map data and the weight data used by each convolutional layer in the off-chip resource DDR before the computing unit starts working. After the computing unit starts working, the parameters stored in the DDR are transferred to the BRAM in batches, and the transmission process is implemented in the form of data stream. Secondly, the data that has been loaded into the on-chip storage resources is fully reused, and after the data is used up, it is directly discarded and new data is written to continue reuse, thereby improving the running speed of the accelerator.
[0007] The parallel processing of the neural network acceleration module is characterized by the parallel processing design of the computing units, input channels and output channels of the convolution operation, which accounts for the largest proportion of the computational workload.
[0008] Compared with the existing technology, the system has faster detection speed and higher energy efficiency ratio. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] Figure 1 This is the structure diagram of the UAV target detection system;
[0010] Figure 2 It is the access method of data in DDR;
[0011] Figure 3 The parallel multiplication-addition tree design diagram for the Conv_3×3 convolution computing unit;
[0012] Figure 4 This is a diagram of a parallel computing unit with 8 input channels;
[0013] Figure 5 Design graphs for output channel parallelism. DETAILED DESCRIPTION
[0014] The data access method in the DDR of the present invention is as follows Figure 2 As shown. Where H represents the number of rows of input feature map data, W represents the number of columns of input feature map data, and C represents the number of channels of input feature map data. The input feature map data is stored pixel by pixel in DDR, and each pixel corresponds to several channels. Therefore, it is necessary to design three counters for rows, columns, and channels to control the calculation process of each row and the entire input feature map. The input feature map data is stored in batches by channel during the storage process of DDR.
[0015] When the convolution operation is designed for parallel processing, in order to reduce frequent off-chip memory access, data can be reused between multiple convolution windows. In the convolution operation, there are three parallel methods: parallel multiplication and addition operations within the convolution, parallel input channels, and parallel output channels. Among them, the parallel multiplication and addition operations within the convolution are for the 3x3 convolution module. A 3x3 convolution requires nine-point multiplication and addition operations. Finally, the nine-point multiplication and addition results are added together to obtain the result of a convolution window. This result is only the pixel value of an output channel on the output feature map. The parallel architecture design realizes the convolution internal multiplication and addition operation with a parallelism of 9, such as Figure 3 shown.
[0016] The parallel processing design of 8 input channels is as follows Figure 4 As shown. The parallel processing architecture of the input channel completes the convolution operation of the 8 input channels in the corresponding 8 computing units PE, and in the same clock cycle. The result after the feature map data and its corresponding weight data are calculated is output after a multiplication-add tree operation, that is, the results of the 8 convolutions are accumulated. If the number of channels of the input feature map is greater than 8, the result after accumulation is not the final result, but a part of it. At this time, it is necessary to calculate how many times the 8 input channel parallel processing operations need to be performed based on the total number of channels of the input feature map, and accumulate the output results of each time. The final accumulated output result is the final value of an output channel.
[0017] The parallel processing design of the output channel is as follows Figure 5 As shown. The 8-input and 8-output block design is adopted, that is, 8 input channels and 8 output channels are processed at a time. Each corresponding calculation will first read the data of the 8 input channels in the feature map and the data of the 8 channels in the 8 convolution kernels, and then load them into the computing unit (PE) for calculation. The result obtained is only a part of the output channel value. At this time, the 8-channel data calculated is the gray part in the figure. Out_channel_x_sum is the partial result obtained for each channel. Next, the feature map data and weight data of the last 8 channels will be loaded. According to this operation, the data of 8 channels are taken in turn for calculation until all 256 channel data are calculated, and the intermediate results are accumulated to obtain the final result of each channel of the first 8 output channels. After the calculation of the first 8 output channels is completed, the data in the last 8 convolution kernels are taken according to the same operation for calculation with the feature map data. Here, the feature map data does not need to be changed, but it is just restarted and calculated with the new weight data. Through such data reuse, the loss of on-chip resources is also greatly reduced.
Claims
1. A drone target detection system, characterized in that: The system includes a camera module, an image data acquisition module, a preprocessing module, a neural network acceleration module, an airborne control module, a display module and a power module.
2. Neural network acceleration module, characterized by It includes two major parts: data multiplexing and parallel processing design. The network computing is performed by rationally allocating FPGA resources to design the TGU acceleration IP.
3. The data reuse of the neural network acceleration module is characterized by: In view of the limited on-chip storage resources of FPGA, the strategy adopted is to store the input feature map data and the weight data used by each convolution layer in the off-chip resource DDR before the computing unit starts working. After the computing unit starts working, the parameters stored in DDR are transferred to BRAM in batches, and the transmission process is implemented in the form of data stream. Secondly, the data that has been loaded into the on-chip storage resources is fully reused. After the data is used up, it is directly discarded and new data is written for continued reuse, thereby improving the running speed of the accelerator.
4. The parallel processing of the neural network acceleration module is characterized by: The computing units, input channels, and output channels of the convolution operation, which accounts for the largest proportion of the computational effort, are designed for parallel processing.
