An air target detection method based on FPGA

Abstract

The present invention relates to an air target detection method based on FPGA, belonging to the technical field of real-time target detection and tracking. The method divides the target detection area of ​​the input image into nine squares on average; then calculates the extreme value image of the entire nine-square grid area image; calculates the maximum value, minimum value and corresponding position coordinates of each sub-area in the nine-square grid based on the extreme value image; and finally determines the position of the target in combination with the target polarity. The present invention combines the complex extreme value image calculation process into a template filtering process, which is easy to implement on FPGA and greatly reduces the calculation time. The entire algorithm uses the standard AXI4-Stream video stream as the interface for parallel pipeline processing, and the PL and PS ends of the FPGA work together to complete the implementation of the entire algorithm.

Description

Technical Field

[0001] The invention belongs to the technical field of real-time target detection and tracking, and in particular relates to an air target detection method based on FPGA. Background Art

[0002] Against a sky background, the background information is relatively simple, and the grayscale values ​​of the target and background differ significantly. Therefore, methods for detecting sky targets based on the contrast between the target and background are the most common. Meanwhile, inter-frame difference algorithms for moving targets, due to their low computational complexity, have attracted considerable attention in systems with high real-time requirements. Due to the speed of moving targets, inter-frame difference algorithms are prone to producing "holes" and "ghosting" when detecting foreground targets. The "Target Detection and Tracking Algorithm Based on Sky Background" (CN112270657A) algorithm calculates a visual saliency map and morphological gradients for each grayscale frame to obtain target edge contour information, calculates the difference between three adjacent frames, and performs target detection through morphological denoising and target domain merging. While improvements have been made to address these issues, they remain significant for high-speed moving targets, significantly impacting the accuracy and real-time performance of target detection. "A method for detecting aerial targets based on infrared images" (CN111210422A) extracts the contrast features and brightness difference features of the neighborhood of each pixel point in the infrared image respectively, then fuses the two features and performs double threshold processing to extract the target position. It can effectively weaken the interference of noise and the influence of uneven target brightness, reduce the false scene rate, and improve the detection rate. However, for real-time systems with high resolution and high frame rate, this method is difficult to meet the real-time requirements of the system. Summary of the Invention

[0003] (1) Technical issues to be solved

[0004] The technical problem to be solved by the present invention is: how to provide an efficient, flexible and real-time air detection method to improve the accuracy of target detection and reduce the calculation time of extreme value images.

[0005] (2) Technical solution

[0006] In order to solve the above technical problems, the present invention provides an air target detection method based on FPGA, the steps are as follows:

[0007] The first step is to divide the target detection area of ​​the input image ImgB into nine squares on average;

[0008] The second step is to find the extreme value image ImageSum of the entire nine-square grid area image;

[0009] The third step is to find the maximum value max[n], minimum value min[n] and corresponding position coordinates point_max[n] and point_min[n] of the extreme value image of each sub-region in the nine-square grid according to the extreme value image ImageSum, where n∈{1,2,…9};

[0010] The fourth step is to determine the target position according to the target polarity and the data output in the third step, and output the target point position information;

[0011] Steps 1 to 3 are implemented by the PL side of the FPGA, and step 4 is implemented by the PS side of the FPGA.

[0012] The present invention also provides a system for implementing the method.

[0013] The present invention also provides a performance verification method for the detection method.

[0014] (3) Beneficial effects

[0015] (1) The method of the present invention adopts the image block filtering detection method and completes target detection through extreme value images, thereby improving the accuracy of target detection;

[0016] (2) The method of the present invention combines the entire complex extreme value image calculation process into a calculation process of the product of a 7×7 template and the original image. The extreme value image can be obtained by traversing the entire search area with the template. The template traversal process fully utilizes the parallel characteristics of the FPGA and directly processes the collected video images in a pipeline, which greatly reduces the processing time of the entire target detection algorithm and improves the real-time performance of the algorithm.

[0017] (3) The method of the present invention can detect targets in almost full screen real time;

[0018] (4) The method of the present invention makes the entire algorithm processing more efficient and flexible through the collaborative work of the PS side and the PL side of the FPGA. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of the air target detection method based on FPGA of the present invention;

[0020] Figure 2 It is the PL side IP diagram of the FPGA of the present invention;

[0021] Figure 3 This is a single white target detection result image;

[0022] Figure 4 This is a single black target detection result image;

[0023] Figure 5 It is a picture of the detection results of multiple white targets in different regions;

[0024] Figure 6 This is a picture of the detection results of multiple black targets in different regions;

[0025] Figure 7 This is a diagram of the hardware resource usage of the FPGA algorithm IP. DETAILED DESCRIPTION

[0026] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below with reference to the accompanying drawings and examples.

[0027] Combined with attachment Figure 1 The FPGA-based air target detection method of the present invention divides the target detection area of ​​the input image into nine squares on average; then calculates the extreme value image of the entire nine-square area image; based on the extreme value image, calculates the maximum and minimum values ​​of each sub-area in the nine-square grid and their corresponding position coordinates; finally, determines the position of the target based on the target polarity. The specific steps are as follows:

[0028] The first step is to divide the target detection area of ​​the input image ImgB into nine squares on average;

[0029] The target detection area is located at any position of the input image ImgB, and its size is adjustable, with a maximum size of (col-3)×(row-3), where col and row represent the columns and rows of the input image ImgB, respectively. The search area is evenly divided into nine-square grids, and each sub-region in the grid has the same size. w and h represent the width and height of the grid area, respectively.

[0030] The second step is to find the extreme value image ImageSum of the entire nine-square grid area image. The process is:

[0031] 2.1 Taking the target point (i, j) as the center, take a 7×7 grayscale data block P(i, j) in the input image ImgB, and let I(i, j) be the grayscale value of each pixel in the grayscale data block P(i, j), then

[0032] The weight template T is used to calculate the mean value Avg(i, j) of the grayscale data block P(i, j). The calculation formula is shown in formula (1):

[0033]

[0034] in,

[0035] 2.2 Perform mean filtering on the target point (i, j) and calculate the filtering result I E(i,j) and the average gray value Avg(i,j), and the formula is shown in formula (2):

[0036] diff(i,j) = I E (i,j) - Avg(i,j) i∈1,2...,h; j∈1,2...,w (2)

[0037] 2.3 The diff(i,j) value of the target point (i,j) is subjected to mean filtering, and the extreme value of the target point (i,j) is calculated, and the formula is shown in formula (3):

[0038] sum(i,j) = 4*E diff (i,j) + 2*I,j)

[0039] i∈1,2...,h; j∈1,2...,w (3)

[0040] Wherein, E diff (i,j) is the diff(i,j) mean filtering result, and I(i,j) is the gray value of the target point (i,j) in the input image block P(i,j).

[0041] According to the calculation method of the extreme value sum(i,j), the extreme value of each pixel in the entire nine-square region image is obtained, and the extreme value image ImageSum is composed of all the extreme values.

[0042] The entire calculation process of the extreme value of each pixel in the entire nine-square region image is extracted as the result of formula (4) after mathematical calculation:

[0043]

[0044] Wherein,

[0045] The entire nine-square region image is traversed by using the template T2, the extreme value of each pixel in the image is obtained, and the extreme value image ImageSum is composed of all the extreme values.

[0046] Thirdly, according to the extreme value image ImageSum, the maximum value max[i] and the minimum value min[j] of the extreme value image of each sub-region in the corresponding nine-square region and the position coordinates point_max[i] and point_min[j] corresponding thereto are obtained, wherein i,j∈{1,2,...,9};

[0047] The above three steps use the HLS tool to complete the C code design and the conversion of the VHDL / Verilog language, and generate a FPGA-based air target detection method PL (Programmable Logic) IP core (Intellectual Property Core). The video input and output interface of the IP core is an AXI4-Stream interface, and the communication interface with the PS end is an AXI4-Lite interface. The IP core is as shown in Figure 2 The AXI4-Stream format video stream enters the air target detection IP for target detection, and the detected target information is transmitted to the PS (Processing System) end through the AXI4-Lite interface. Meanwhile, the original video is output to the next stage through the AXI4-Stream interface.

[0048] In the fourth step, the target position is determined according to the target polarity and the output data in the third step, and the target point position information is output. This part is implemented by the PS end of the FPGA, and can be implemented in the following several cases:

[0049] The first case is single target detection, and the process is as follows:

[0050] 4.1 If the target is a white target, find the maximum value max[m] from the maximum values max[n] output in the third step, find the corresponding point_max[m] according to the maximum value max[m], and output the target point position point_max[m], where m∈{1, 2, … 9};

[0051] 4.2 If the target is a black target, find the minimum value min[m] from the minimum values min[n] output in the third step, find the corresponding point_min[m] according to the minimum value min[[m], and output the target point position point_min[m], where m∈{1, 2, … 9};

[0052] The second case is multi-target regional detection, and each region detects at most one target. The process is as follows:

[0053] 4.3 According to the actual target brightness and the requirements of the system on the target, the system target threshold is given, the white target threshold is max_T, the black target threshold is min_T, and then all the maximum values and minimum values output in the third step are normalized. The normalization coefficient k is as shown in formula (5):

[0054]

[0055] The normalized maximum value set Max[n] and the minimum value set Min[n] are obtained;

[0056] 4.4 If the target is a white target, find all Max[m'] greater than the white target threshold max_T from the maximum value set Max[n], find the corresponding point_max[m'] according to Max[m'] in turn, output the target point position point_max[m'], wherein m'∈{1,2,…9};

[0057] 4.5 If the target is a black target, find all Min[m''] less than the black target threshold min_T from the minimum value set Min[n], find the corresponding point_min[m''] according to Min[m''] in turn, output the target point position point_min[m''], wherein m''∈{1,2,…9}.

[0058] In order to verify the performance of the method of the present application, the following experiment is made: on the hardware platform with Xilinx ZYNQ-7035 as the main processing chip, input a video with a resolution of 1280*1024, the search gate size is (1280-3)*(1024-3), and the gate area is averagely divided into nine grids. The video stream link clock is 200MHz, and single white target and single black target detection are respectively carried out, and the detection results are shown in Figs. 2 and 3 respectively. After the black and white target thresholds are set according to experience, multi-target regional detection is respectively carried out, Figure 3 and Figure 4 The multi-target regional detection results of the white target are shown in Fig. 4, Figure 5 The multi-target regional detection results of the black target are shown in Fig. 5. The time consumption of the method of the present application for completing one frame of image target detection in the experiment is 6.655 milliseconds, and the resource occupancy rate is extremely small, wherein the most used LUT resource is only 4%, and the specific situation is shown in Fig. 6. With the improvement of the FPGA device process and the improvement of the video stream link clock synchronization, the time consumption of the algorithm is also reduced accordingly, and the real-time advantage of the present algorithm will be more prominent. Figure 6 Figure 7 It can be seen that the present application combines the complex extreme value image process into a template filtering process, which is easy to implement in FPGA and greatly reduces the calculation time. The entire algorithm uses a standard AXI4-Stream video stream as an interface for parallel pipeline processing, and the PL end and the PS end of the FPGA work together to complete the implementation of the entire algorithm.

[0059] It can be seen that the present application combines the complex extreme value image process into a template filtering process, which is easy to implement in FPGA and greatly reduces the calculation time. The entire algorithm uses a standard AXI4-Stream video stream as an interface for parallel pipeline processing, and the PL end and the PS end of the FPGA work together to complete the implementation of the entire algorithm.

[0060] The above only describes the preferred embodiments of the present application, and it should be pointed out that for ordinary skilled persons in the art, some improvements and modifications can be made without departing from the technical principles of the present application, and these improvements and modifications should also be regarded as the protection scope of the present application.​

Claims

1. A method for detecting air targets based on FPGA, characterized in that: Here are the steps: The first step is to divide the target detection area of ​​the input image ImgB into nine squares on average; The second step is to find the extreme value image ImageSum of the entire nine-square grid area image; The third step is to find the maximum value max[n], minimum value min[n] and corresponding position coordinates point_max[n] and point_min[n] of the extreme value image of each sub-region in the nine-square grid according to the extreme value image ImageSum, where n∈{1,2,…9}; The fourth step is to determine the target position according to the target polarity and the data output in the third step, and output the target point position information; Steps 1 to 3 are implemented by the PL side of the FPGA, and step 4 is implemented by the PS side of the FPGA; In the first step, the target detection area is located at any position in the input image ImgB. The size is adjustable, and the maximum size is (col-3)×(row-3), where col and row represent the column and row of the input image ImgB respectively. The search area is evenly divided into nine-square grids. Each sub-region in the nine-square grid has the same size. w and h represent the width and height of the nine-square grid area respectively. In the second step, the process of finding the extreme value image ImageSum of the entire nine-square grid area image is: 2.1 Taking the target point (i, j) as the center, take a 7×7 grayscale data block P(i, j) in the input image ImgB, and let I(i, j) be the grayscale value of each pixel in the grayscale data block P(i, j), then The weight template T is used to calculate the mean value Avg(i, j) of the grayscale data block P(i, j). The calculation formula is shown in formula (1): in, 2.2 Perform mean filtering on the target point (i, j) and calculate the filtering result I E The difference between (i, j) and Avg(i, j) is diff(i, j), which is calculated as follows: diff(i,j)=I E (i,j)-Avg(i,j) (2) 2.3 Perform mean filtering on the difference diff(i,j) of the target point (i,j), and then calculate the extreme value sum(i,j) of the target point (i,j). The calculation formula is shown in formula (3): sum(i,j)=4*E diff (i,j)+2*I(i,j) (3) Among them, E diff (i, j) is the mean filtering result of diff(i, j), I(i, j) is the grayscale value of the target point (i, j) in the grayscale image block P(i, j); According to the calculation method of extreme value sum(i,j), the entire nine-square grid area image is traversed to obtain the extreme value of each pixel in the image, and the extreme value image ImageSum is composed of all extreme values.

2. The method for detecting an air target based on FPGA according to claim 1, wherein: The second step is to find the entire calculation process of the extreme value of each pixel in the entire nine-square grid area image, which is refined into the result of formula (4) after mathematical calculation: in, Use template T2 to traverse the entire nine-square grid area image, find the extreme value of each pixel in the image, and compose its extreme value image ImageSum from all extreme values.

3. The method for detecting an air target based on FPGA according to claim 1, wherein: The fourth step is to determine the target position according to the target polarity and the data output in the third step, and output the target point position information, which can be implemented in the following situations: The first one is single target detection. The process is as follows: 4.1 If the target is a white target, find the maximum value max[m] from the maximum values ​​max[n] of each sub-region output in the third step, find the corresponding point_max[m] based on the maximum value max[m], and output the target point position point_max[m], where m∈{1,2,…9}; 4.2 If the target is a black target, find the minimum value min[m] from the minimum values ​​min[n] of each sub-region output in the third step, find the corresponding point_min[m] based on the minimum value min[[m], and output the target point position point_min[m], where m∈{1,2,…9}; The second method is to detect multiple targets in different regions. Each region can detect at most one target. The process is as follows: 4.3 According to the brightness of the actual target and the system's requirements for the target situation, the system target threshold is given. The white target threshold is max_T and the black target threshold is min_T. Then, all the maximum and minimum values ​​output in the third step are normalized. The normalization coefficient k is shown in formula (5): Get the normalized maximum value set Max[n] and minimum value set Min[n]; 4.4 If the target is a white target, find all Max[m'] values ​​greater than the white target threshold max_T from the maximum value set Max[n], find the corresponding point_max[m'] according to Max[m'], and output the target point position point_max[m'], where m'∈{1,2,…9}; 4.5 If the target is a black target, find all Min[m”] that are less than the black target threshold min_T from the minimum value set Min[n], find the corresponding point_min[m”] according to Min[m”] in turn, and output the target point position point_min[m”], where m”∈{1,2,…9}.

4. The method for detecting an air target based on FPGA according to claim 1, wherein: The method is deployed on the FPGAPL side in the form of an IP core.

5. The method for detecting air targets based on FPGA according to claim 4, wherein: The video input and output interface of this IP core is in AXI4-Stream format, and the communication interface with the PS end is in AXI4-Lite format.

6. The method for detecting an air target based on FPGA according to claim 4, wherein: The IP core uses C / C++ language to complete the functional code writing, and is converted into FPGA-runnable VHDL or Verilog code through the HLS high-level synthesis tool.

Images