A method for detecting and tracking drone targets in the sky under complex background

Abstract

The invention discloses a method for detecting and tracking sky UAV targets under complex backgrounds. The method comprises the following steps: using a related edge detection method to extract obvious edges in an image, extracting contours of non-sky background edges in the image, and using the contours as non-target suppression area templates; extracting high-frequency significant area targets in the image by using a wavelet transform method, and suppressing low-frequency background targets in the image; performing preliminary screening on the extracted target detection frame by using target prior information, retaining strong prior targets, and then performing target judgment by using a deep learning model to obtain the screened UAV target; initializing a tracker by using the coordinates of the center point of the target frame, the width of the target frame, and the height of the target frame of the extracted UAV target; and obtaining the target position information of the current frame by using a Kalman filter to predict the target position and speed information of the previous frame; and removing background target interference and effectively tracking the target.

Description

Technical Field

[0001] The invention belongs to the technical field of digital image processing, and in particular relates to a method for detecting and tracking a sky unmanned aerial vehicle target under a complex background. Background Art

[0002] In recent years, with the rapid development and widespread application of UAV technology, UAV target detection technology has become a hot research topic. The trend of unmanned and information-based warfare in modern warfare makes it essential to have an early warning system that can effectively detect low-altitude UAV targets. The research on target detection and tracking technology under complex sky background has important applications in civil, aerospace and military, and is an important research field for key technologies such as image processing, target recognition, navigation and guidance. Small target detection and tracking is an important branch in the field of computer vision. It mainly focuses on detecting and tracking targets with small size, low contrast and possible background interference in images or video sequences. Due to the long range, the target size is usually small, and it is even strongly interfered by clouds and clutter, making target detection difficult.

[0003] The current target detection algorithms are mainly divided into single-frame based detection and multi-frame based detection. The multi-frame based detection algorithm can effectively improve the accuracy of target detection by continuously tracking the target between multiple adjacent frames. However, due to the high complexity of the algorithm and the difficulty of continuous tracking when the target is small, it is not suitable for small target detection in complex backgrounds. The single-frame based target detection algorithms are mainly divided into the following three categories: background suppression, visual saliency, and target-background separation.

[0004] The main target tracking methods are: (1) template-based tracking, which tracks the target by establishing relevant template information about the target's geometric shape, texture features, etc., but the template needs to be updated as the target's appearance changes and is not universal; (2) feature point-based tracking, which is achieved by extracting key feature points on the target, such as SIFT, SURF or ORB, and tracking these points in the video sequence, but feature points do not have target recognition and will be disturbed by the background; (3) correlation filter-based tracking, which trains a filter on the target and then uses the filter in each frame to find the new position of the target, but the correlation filter algorithm has difficulty in solving the situation where the target imaging size changes greatly, especially when the target moves from near to far, and the correlation filter algorithm needs to continuously update the target template, which has a high time complexity; (4) tracking based on motion model and data association, which mainly includes motion model prediction based on Kalman filter or particle filter and data association based on Hungarian algorithm.

[0005] The target detection and tracking technology of visible light sequence images under complex sky background is mainly limited by the elimination of background clutter targets, rapid capture of target detection, data association of multiple targets and stable tracking. Summary of the invention

[0006] In view of this, the present invention provides a method for detecting and tracking drone targets in the sky under a complex background, which can solve the problem of drone target detection and tracking in visible light sequence images under a complex sky background. When using a video multi-frame detection method, the algorithm complexity is high and it is difficult to track the target. When using a single-frame based detection method, it is easily interfered by background targets and is difficult to apply in scenes with low signal-to-noise ratio.

[0007] The technical solution for implementing the present invention is as follows:

[0008] A method for detecting and tracking a sky drone target under a complex background comprises the following steps:

[0009] S1. Use the correlation edge detection method to extract the obvious edges in the image and the contours of the non-sky background edges in the image as the non-target suppression area template.

[0010] S2, by using the wavelet transform method, the high-frequency salient area targets in the image are extracted, and the low-frequency background targets in the image are effectively suppressed;

[0011] S3. Preliminarily screen the detection frame extracted in the previous step based on the target’s size, shape, color and other prior information, retain the strong prior targets, and then use the deep learning model to make target judgments to obtain the selected drone serialized target B. t ;

[0012] S4. Extract the drone target B t , with the coordinates of the center point of the target frame (x, y), the width of the target frame W t , and the target box height H t Initialize the tracker;

[0013] S5. The target position information of the current frame is obtained by predicting the target position and speed information of the previous frame through a Kalman filter.

[0014] Furthermore, before S1, the image sequence frames are filtered by using a median filter to filter the high-frequency noise in the image while retaining the texture and edge information of the original image, and the image pixel value is converted from the three-channel color domain to the single-channel grayscale domain.

[0015] Furthermore, S1 specifically includes:

[0016] S11: performing a histogram equalization operation on the image to increase the amount of edge information in the image;

[0017] S12: using an edge detection algorithm to obtain edge features of the grayscale image and performing Gaussian feathering processing;

[0018] S13: Perform morphological dilation and erosion operations on the image after edge detection to obtain a binary image, and extract the sky area and the non-sky area from the binary image based on the assumption that the sky area is on the upper side of the image frame.

[0019] Furthermore, in S2, low-frequency background targets mainly refer to cloud targets that are relatively prominent in the sky but change slowly in the spatial domain.

[0020] Furthermore, the Hungarian algorithm is used to associate and match the existing M tracker targets with the center points of the N detected targets, and three matching states are obtained: the detected target is matched to the tracker at the same time, the current frame is assigned a tracker but the detector does not detect the target, and a new target is detected but the tracker is not assigned. According to the matching state, when the state is that the detected target is matched to the tracker at the same time, the position of the matched tracker is reinitialized with the detected target center as the initial value, and the position of the Kalman filter is corrected at the same time.

[0021] Beneficial effects:

[0022] 1. The method of the present invention is applied to the problem of UAV target detection and tracking in visible light sequence images under complex sky background, which can remove background target interference and effectively track the target.

[0023] 2. In the preprocessing stage of the present invention, the image sequence frames are filtered by using a median filter to remove noise from the image and convert the image into a grayscale mode, which is beneficial for subsequent processing.

[0024] 3. The present invention uses the wavelet transform method to extract high-frequency significant area targets in the image and effectively suppress low-frequency background targets in the image, thereby effectively solving the problem of slow recognition speed using full-size images. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0026] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0027] The present invention provides a method for detecting and tracking a target of a sky drone under a complex background. The process of the present invention is as follows: Figure 1 The specific implementation details of each part are as follows:

[0028] 1. Image preprocessing: By replacing the pixel value at the center of a window with the median value of all pixel values ​​in a window, that is, by using the median filter to process the image sequence frame F t Filtering is performed to remove high-frequency noise in the image while retaining the texture and edge information of the original image, so as to remove noise from the image and convert the image into grayscale mode, which is conducive to the effective output of target detection.

[0029] 2. Background extraction: Initialize the image frame F 0 Use the correlation edge detection method to extract the obvious edges in the image, and extract the contours of the non-sky background edges in the image as the non-target suppression area template. Specifically:

[0030] Step 1: Perform histogram equalization on the image to increase the amount of edge information in the image;

[0031] Step 2: Use edge detection algorithm to obtain edge features of grayscale image and perform Gaussian feathering processing;

[0032] Step 3: Apply morphological dilation and erosion operations to the image after edge detection to obtain a binary image. Based on the assumption that the sky area is on the upper side of the image frame, the sky area and non-sky area are extracted from it.

[0033] 3. Target detection: By using the wavelet transform method, high-frequency salient area targets in the image are extracted, and low-frequency background targets in the image are effectively suppressed, mainly referring to cloud targets that are relatively prominent in the sky but slowly change in the spatial domain. This method can effectively solve the problem of slow recognition speed using full-size images.

[0034] The detection of small targets in the sky is intuitively to determine the isolated singular points in the image, and wavelet transform can well distinguish the targets (singular points) located in the high-frequency part from the background located in the low-frequency part. The target detection algorithm based on wavelet transform is as follows:

[0035] Step 1: Select a suitable wavelet function, such as the Haar wavelet function, based on the characteristics of the image and the required transformation features.

[0036] Step 2: Perform a two-dimensional discrete wavelet transform on the image signal f(x, y) and decompose it into an approximate subband cA and three detail subbands: horizontal detail subband cH, vertical detail subband cV and diagonal detail subband cD.

[0037] Step 3: The coefficients of each scale and direction are processed as follows:

[0038] cA=(cA LL ,cA LH ,cA HLcA HH )

[0039] cH=(cH LL ,cH LH ,cH HL H HH )

[0040] cV=(cV LL ,cV LH ,cV HL cV HH )

[0041] cD=(cD LL ,cD LH ,cD HL cD HH )

[0042]

[0043] Step 4: Use the normalized correlation coefficient to distinguish the target coefficient, background coefficient and noise coefficient. If |coor(j,d,m,n)|<|w(j,d,m,n)|, it is considered to be a noise or background coefficient and is set to zero to achieve suppression; otherwise, it is retained as the target coefficient.

[0044] Step 5: Perform inverse transform on the processed wavelet transform coefficients to obtain the salient image, and set the segmentation threshold to extract the target.

[0045] 4. Target screening: Preliminary screening of the detection frames extracted in the previous step is performed based on the target’s size, shape, color and other prior information, retaining strong prior targets, and then using the trained deep learning recognition model to identify the target frames, outputting the drone category target frames B with high scores t .

[0046] 5. Tracker initialization: Initialization frame F 0 All target boxes detected in B t Initialize the Kalman filter based tracker, where the tracking state is They are the center coordinate point (x, y) of the target frame and the movement speed of the coordinate point Initialized to 0, and the width and height of the target box (w, h)

[0047] 6. Tracking target prediction: According to the target state X in the previous frame image t-1 To predict the target state of the current frame Extract the center position coordinates of the target from the predicted target state;

[0048]

[0049] Among them, A is the target state transfer matrix, w t-1 is process noise and obeys Gaussian distribution.

[0050] 7. Target data association: The Hungarian algorithm is used to associate and match the existing M tracker targets with the center points of the N detected targets, and three matching states are obtained: the detected target is matched with the tracker at the same time, the tracker is assigned to the current frame but the detector does not detect the target, and the new target is detected but the tracker is not assigned, reducing the target ID switching.

[0051] 8. Observation target correction: According to the matching state obtained above, when the state is to detect the target and match the tracker at the same time, the position of the matched tracker is reinitialized with the detected target center as the initial value, and the position of the Kalman filter is corrected at the same time.

[0052] 9. Tracker post-processing: This includes identifying and handling abnormal situations, such as when the target disappears from view, when the target moves abnormally, and when to terminate tracking of the target.

[0053] 10. Algorithm output: Output target tracking result D t , and convert the trace information into the required format.

[0054] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting and tracking drone targets in the sky under complex backgrounds, characterized in that: The following steps are involved: S1. Use the correlation edge detection method to extract the obvious edges in the image and the contours of the non-sky background edges in the image as the non-target suppression area template. S2, by using the wavelet transform method, the high-frequency salient area targets in the image are extracted, and the low-frequency background targets in the image are suppressed; S3. Preliminarily screen the extracted target detection frame based on the size, shape, and color prior information of the target, retain the strong prior targets, and then use the deep learning model to make target judgments to obtain the selected drone serialization target B. t ; S4. Extract the drone target B t , with the coordinates of the center point of the target frame (x, y), the width of the target frame W t , and the target box height H t Initialize the tracker; S5. The target position information of the current frame is obtained by predicting the target position and speed information of the previous frame through a Kalman filter.

2. The target detection and tracking method according to claim 1, characterized in that: Before S1, the image sequence frames are filtered by using a median filter to remove high-frequency noise in the image while retaining the texture and edge information of the original image, and the image pixel value is converted from the three-channel color domain to the single-channel grayscale domain.

3. The target detection and tracking method according to claim 1, characterized in that: S1 specifically includes: S11: performing a histogram equalization operation on the image to increase the amount of edge information in the image; S12: using an edge detection algorithm to obtain edge features of the grayscale image and performing Gaussian feathering processing; S13: Perform morphological dilation and erosion operations on the image after edge detection to obtain a binary image, and extract the sky area and the non-sky area from the binary image based on the assumption that the sky area is on the upper side of the image frame.

4. The target detection and tracking method according to claim 3, characterized in that: In S2, low-frequency background targets mainly refer to cloud targets that are relatively prominent in the sky but change slowly in the spatial domain.

5. The target detection and tracking method according to claim 1 or 4, characterized in that: The Hungarian algorithm is used to associate and match the existing M tracker targets with the center points of the N detected targets, and three matching states are obtained: the detected target is matched with the tracker at the same time, the current frame is assigned a tracker but the detector does not detect the target, and a new target is detected but the tracker is not assigned. According to the matching state, when the state is that the detected target is matched with the tracker at the same time, the position of the matched tracker is reinitialized with the detected target center as the initial value, and the position of the Kalman filter is corrected at the same time.

Images