A method for detecting and tracking an aerial infrared unmanned aerial vehicle target

Abstract

The application discloses a kind of over-the-air infrared unmanned aerial vehicle target detection tracking methods, the edge extraction of pre-acquired over-the-air infrared image is carried out, and black-and-white binary edge image is obtained;Contour search algorithm is used to carry out connection operation to black-and-white binary edge image, adjacent edge points are connected into line segment, and closed contour information is obtained;Corner detection is carried out to over-the-air infrared image, and Fast corner detection algorithm is used to extract corner point information in over-the-air infrared image;Closed contour information and corner point information are fused, and target information is obtained;Lightweight deep learning model VoVNet is used to identify target information, delete non-unmanned aerial vehicle target, and obtain unmanned aerial vehicle target information including unmanned aerial vehicle target;Unmanned aerial vehicle target information is tracked, and the trajectory information of unmanned aerial vehicle is obtained.It can realize the accurate detection and tracking of infrared image unmanned aerial vehicle target, greatly reduce the false detection rate of algorithm, and improve the identification accuracy of algorithm.

Description

Technical Field

[0001] This invention relates to a method for detecting and tracking aerial infrared unmanned aerial vehicle (UAV) targets, belonging to the field of UAV target detection and tracking technology. Background Technology

[0002] In recent years, with the development of technology, the number of civilian drones has been increasing. These drones are characterized by being "low, slow, and small," making them difficult to detect and identify, and posing potential security risks. Therefore, the detection and identification of aerial drone targets has high research value and application prospects.

[0003] Because drone targets are typically small, they are difficult to detect and identify, and the false detection rate is high. To address this issue, current methods mainly employ traditional image detection algorithms or convolutional neural network (CNN) algorithms. Traditional image detection algorithms have a high false detection rate, mistaking birds and clouds for drone targets. CNN algorithms, on the other hand, are not ideal for detecting small targets, resulting in a low detection rate. Therefore, both algorithms have their own shortcomings. In summary, a method that can accurately detect and identify drone targets is needed. To achieve this, the following key issues need to be addressed: (1) How to reduce the false detection rate of the algorithm. (2) How to improve the detection rate of the algorithm. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to overcome the defects of the prior art and provide a method for detecting and tracking aerial infrared UAV targets. The present invention provides a processing flow that combines image detection algorithm and convolutional neural network algorithm, which can realize accurate detection and tracking of UAV targets in infrared images, greatly reduce the false detection rate of the algorithm and improve the recognition accuracy of the algorithm.

[0005] Prior to this invention, a method for detecting and tracking aerial infrared unmanned aerial vehicle (UAV) targets is provided, comprising: extracting edges from a pre-acquired aerial infrared image using the Canny edge detection algorithm to obtain a black-and-white binary edge image; performing a contour search algorithm to connect adjacent edge points in the black-and-white binary edge image into line segments to obtain closed contour information; performing corner detection on the pre-acquired aerial infrared image and extracting corner information from the aerial infrared image using the Fast corner detection algorithm; fusing the closed contour information and corner information to obtain target information; identifying the target information using a pre-trained lightweight deep learning model VoVNet, deleting non-UAV targets, and obtaining UAV target information including the UAV target; and tracking the UAV target information to obtain the UAV's trajectory information.

[0006] Prior to this, a contour finding algorithm is used to perform a connection operation on the black and white binary edge image, connecting adjacent edge points in the black and white binary edge image into line segments to obtain closed contour information. This includes: using a contour finding algorithm to perform a connection operation on the edge image, connecting adjacent edge points in the black and white binary edge image into line segments, retaining closed contours in the black and white binary edge image, deleting non-closed contours in the black and white binary edge image, and obtaining closed contour information.

[0007] Prioritize fusing closed contour information and corner information to obtain target information, including: fusing closed contour information and extracted corner information, retaining closed contour information containing corners, deleting closed contour information not containing corners, and obtaining remaining closed contour information; extracting the remaining closed contour information, retaining the minimum bounding rectangle target box of the contour, and obtaining target information including the minimum bounding rectangle target box.

[0008] Prioritize the pre-training of a lightweight deep learning model VoVNet, including: establishing an infrared drone target sample set; labeling the infrared drone target samples in the randomly selected N or more infrared drone target sample set with labels to obtain labels including drone targets and non-drone targets, where N is a preset positive integer; training the lightweight deep learning model VoVNet using the labeled infrared drone target samples until the lightweight deep learning model VoVNet converges to a certain value, thus obtaining the trained lightweight deep learning model VoVNet.

[0009] First, the drone target information is tracked to obtain the drone trajectory information, including: using the IOU matching method to match the drone target information in frame K with the drone target information in frame K-1, where K is a positive integer. Assuming that there are M trajectories in the drone target information in frame K-1, where M is a positive integer, if the intersection-union ratio (IOU) of the latest minimum bounding rectangle target box retained by the drone target rectangle R in frame K and the trajectory numbered G in frame K-1 is greater than 0.6, then the drone target rectangle R is associated with the trajectory G, and the drone target information corresponding to the drone target rectangle R is added to the drone target information corresponding to the trajectory G to obtain the updated drone trajectory information.

[0010] Prioritize the use of the minimum bounding rectangle target bounding box if there is an unassociated minimum bounding rectangle target bounding box in the UAV target information of frame K.

[0011] Preferably, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method described in any of the first aspects.

[0012] Preferably, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any one of the first aspects.

[0013] The beneficial effects achieved by this invention are as follows:

[0014] This invention provides a processing flow combining image detection algorithms and convolutional neural network algorithms. It employs the Canny edge detection algorithm to extract edges from pre-acquired aerial infrared images, obtaining a black-and-white binary edge image. A contour finding algorithm is then used to connect adjacent edge points in the black-and-white binary edge image, obtaining closed contour information. Corner detection is performed on the pre-acquired aerial infrared image using the Fast corner detection algorithm to extract corner information. The closed contour information and corner information are fused to obtain target information. A pre-trained lightweight deep learning model, VoVNet, is used to identify the target information, removing non-UAV targets to obtain UAV target information including the UAV target. Finally, the UAV target information is tracked to obtain the UAV's trajectory information. This process enables accurate detection and tracking of UAV targets in infrared images, significantly reducing the false detection rate and improving the algorithm's recognition accuracy. Attached Figure Description

[0015] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 These are schematic diagrams of some embodiments of this application. Detailed Implementation

[0017] To facilitate the explanation of the technical solution of this application, some concepts involved in this application will be explained first below.

[0018] See Figure 1 This application discloses a method for detecting and tracking aerial infrared unmanned aerial vehicle targets, comprising the following steps:

[0019] Step 1: Use the Canny edge detection algorithm to extract edges from the pre-acquired aerial infrared image to obtain a black and white binary edge image;

[0020] Step 2: Use a contour search algorithm to perform a connection operation on the black and white binary edge image, connecting adjacent edge points in the black and white binary edge image into line segments to obtain closed contour information;

[0021] Step 3: Perform corner detection on the pre-acquired aerial infrared image. Use the Fast (Features from Accelerated Segment Test) corner detection algorithm to extract corner information from the aerial infrared image.

[0022] Step 4: Fuse the closed contour information and corner point information to obtain the target information;

[0023] Step 5: Use the pre-trained lightweight deep learning model VoVNet to identify target information, delete non-drone targets, and obtain drone target information including drone targets.

[0024] Step 6: Track the drone target information to obtain the drone's trajectory information.

[0025] In this embodiment of the application, step 2 includes: using a contour search algorithm to perform a connection operation on the black and white binary edge image, connecting adjacent edge points in the black and white binary edge image into line segments to obtain closed contour information, including: using a contour search algorithm to perform a connection operation on the edge image, connecting adjacent edge points in the black and white binary edge image into line segments, retaining the closed contours in the black and white binary edge image, deleting the non-closed contours in the black and white binary edge image, and obtaining closed contour information.

[0026] In this embodiment of the application, closed contour information and corner point information are fused to obtain target information, including:

[0027] The closed contour information and the extracted corner point information are merged. The closed contour information containing corner points is retained, and the closed contour information without corner points is deleted to obtain the remaining closed contour information.

[0028] Extract the remaining closed contour information, retain the minimum bounding rectangle target box of the contour, and obtain the target information including the minimum bounding rectangle target box.

[0029] In this embodiment of the application, the lightweight deep learning model VoVNet is pre-trained, including:

[0030] Establish an infrared drone target sample set, and use tags to label the infrared drone target samples in the infrared drone target sample set. Set two tags to obtain tags including drone targets and non-drone targets. Label more than N infrared drone target samples, where N is a preset positive integer. The labeled non-drone targets include various non-drone targets such as bird targets and cloud targets.

[0031] The lightweight deep learning model VoVNet is trained using labeled infrared drone target samples until it converges to a certain value, resulting in the trained infrared drone target recognition model—the lightweight deep learning model VoVNet. The lightweight deep learning model VoVNet can be used to identify infrared targets and determine whether they are drone targets.

[0032] In this embodiment of the application, a pre-trained lightweight deep learning model VoVNet is used to identify target information, delete non-drone targets, and obtain drone target information including drone targets. This includes: using the lightweight deep learning model VoVNet to sequentially identify the minimum bounding rectangle target box after fusion in step 4, deleting non-drone targets, and retaining the information of the minimum bounding rectangle target box including drone targets.

[0033] In this embodiment of the application, tracking the drone target information to obtain the drone's trajectory information includes:

[0034] When the UAV target information of frame K is obtained, K is a positive integer. The IOU (Intersection over Union) matching method is used to match it with the UAV target information of frame K-1. Assuming that there are M trajectories in the UAV target information of frame K-1, if the intersection over union ratio (IOU) of the latest minimum bounding rectangle of the UAV target rectangle R in the UAV target information of frame K and the trajectory numbered G in the UAV target information of frame K-1 is greater than 0.6, then the UAV target rectangle R is associated with the trajectory G, and the UAV target information corresponding to the UAV target rectangle R is added to the UAV target information corresponding to the trajectory G to obtain the updated UAV trajectory information.

[0035] If there is an unassociated minimum bounding rectangle target box in the UAV target information of frame K, then the UAV trajectory information is created using the minimum bounding rectangle target box.

[0036] In this embodiment of the application, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the methods described above.

[0037] In this application embodiment, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described above.

[0038] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0039] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention described herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not invented herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.

[0040] The above specific embodiments further illustrate the purpose, technical solution and beneficial effects of this application. It should be understood that the above are only specific embodiments of this application and are not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of this application should be included within the scope of protection of this application.

Claims

1. A method for detecting and tracking aerial infrared unmanned aerial vehicle targets, characterized in that, include: The Canny edge detection algorithm is used to extract edges from the pre-acquired aerial infrared image to obtain a black-and-white binary edge image; A contour search algorithm is used to perform a connection operation on the black and white binary edge image. Adjacent edge points in the black and white binary edge image are connected into line segments, closed contours in the black and white binary edge image are retained, and non-closed contours in the black and white binary edge image are deleted to obtain closed contour information. Corner detection is performed on the pre-acquired aerial infrared images. The Fast corner detection algorithm is used to extract corner information from the aerial infrared images. The closed contour information and the extracted corner information are fused together. The closed contour information containing corners is retained, and the closed contour information without corners is deleted to obtain the remaining closed contour information. The remaining closed contour information is extracted, and the minimum bounding rectangle of the contour is retained to obtain the target information including the minimum bounding rectangle. Establish an infrared drone target sample set, and use labels to annotate the infrared drone target samples in the randomly selected N or more infrared drone target sample set to obtain labels including drone targets and non-drone targets, where N is a preset positive integer; The lightweight deep learning model VoVNet is trained using labeled infrared drone target samples until it converges to a certain value, thus obtaining the trained lightweight deep learning model VoVNet. The pre-trained lightweight deep learning model VoVNet is used to identify target information, remove non-drone targets, and obtain drone target information including drone targets. The IOU matching method is used to match the UAV target information in frame K with the UAV target information in frame K-1, where K is a positive integer. Assuming that there are M trajectories in the UAV target information in frame K-1, where M is a positive integer, if the intersection-union ratio (IOU) of the latest minimum bounding rectangle of the UAV target bounding rectangle R in frame K and the trajectory numbered G in frame K-1 is greater than 0.6, then the UAV target bounding rectangle R is associated with the trajectory G, and the UAV target information corresponding to the UAV target bounding rectangle R is added to the UAV target information corresponding to the trajectory G to obtain the updated UAV trajectory information. If there is an unassociated minimum bounding rectangle target box in the UAV target information of frame K, then the UAV trajectory information is created using the minimum bounding rectangle target box.

2. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method of claim 1.

3. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method of claim 1.

Images