A drone identification and video tracking method based on Xueliang Project

Abstract

The present invention provides a method for drone identification and video tracking based on the Xueliang Project, comprising: obtaining a camera input video stream; performing target detection and identification based on the camera input video stream, including target detection and preliminary identification and secondary target identification; and tracking the target based on the identification results. The method for drone identification and video tracking based on the Xueliang Project, provided by the present invention, is based on the Xueliang Project and can achieve drone identification and target tracking, making it easy to use.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles (UAVs), and in particular to a UAV identification and video tracking method based on the Xueliang Project. Background Art

[0002] Many existing anti-drone manufacturers are researching the detection and identification of drone radio signals. Common technologies available include radar, radio, optical, protocol cracking, and messaging. Each type of equipment has limitations, requiring drones to reach a certain altitude for detection and identification. However, there is a lack of effective technical means for monitoring drones below treetops or those that have not yet taken off, enabling key areas of drone surveillance. The Xueliang Project boasts extensive camera coverage. Integrating camera data in key areas to perform image recognition, tracking, and early warning for drones, model aircraft, and FPV drones would significantly improve current drone surveillance blind spots, enabling early detection and response while also providing a reliable basis for subsequent action. Therefore, designing a drone identification and video tracking method based on the Xueliang Project is crucial. Summary of the Invention

[0003] The purpose of the present invention is to provide a UAV identification and video tracking method based on the Xueliang Project. Based on the Xueliang Project, the UAV identification and target tracking can be realized and the method is easy to use.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] A drone identification and video tracking method based on the Xueliang Project includes the following steps:

[0006] Step 1: Get the camera input video stream;

[0007] Step 2: Target detection and recognition are performed based on the camera input video stream, which includes target detection and preliminary recognition as well as target secondary recognition;

[0008] Step 3: Track the target based on the recognition results.

[0009] Optionally, in step 2, target detection and recognition are performed based on the camera input video stream, which includes target detection and preliminary recognition and target secondary recognition, and specifically includes the following steps:

[0010] Step 201: Target detection and preliminary recognition are performed based on the camera input video stream;

[0011] Step 202: Perform secondary target recognition based on the detection results.

[0012] Optionally, in step 201, target detection and preliminary recognition are performed based on the camera input video stream, specifically including the following steps:

[0013] Step 2011: Build the Yolov5 model;

[0014] Step 212: performing image segmentation on the camera input video stream based on a 640-pixel size;

[0015] Step 2013: Improve the backbone network and neck network of the Yolov5 model and add a detection frame;

[0016] Step 2014: Input the segmented image into the improved Yolov5 model and output the detection result.

[0017] Optionally, in step 202, target secondary recognition is performed based on the detection results, specifically including the following steps:

[0018] Step 221: Obtain the size of the detected target based on the detection result. If the detected target is smaller than a preset threshold, perform secondary target recognition. The preset threshold is adjusted based on the data accuracy assessment.

[0019] Step 222: Build the Inception V4 model and concatenate the Yolov5 model with the Inception V4 model;

[0020] Step 2023: With the detection target as the center, extract the 224*224 target original image, perform secondary target recognition based on the Inception V4 model, and obtain the final detection result.

[0021] Optionally, in step 3, target tracking is performed based on the recognition result, specifically:

[0022] Build a DeepSort model and optimize its feature extraction network. Perform cascade matching between the DeepSort model and the InceptionV4 model. Input the final detection results into the DeepSort model to obtain the tracking results.

[0023] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: the drone identification and video tracking method based on the Xueliang Project provided by the present invention is based on the Xueliang Project camera to perform drone identification and tracking. After accessing the video stream of the camera in the key area, early detection and early warning can be achieved, avoiding the need for illegal drone flight, and at the same time, more sufficient time and disposal evidence can be obtained for subsequent disposal. The image cutting algorithm is adopted and the calculation process is optimized. While avoiding the reduction in recognition rate of small targets due to compression distortion, the recognition efficiency is significantly improved by improving the calculation process, achieving a small amount of performance and significantly improving the recognition accuracy. The Yolov5 algorithm is improved. By increasing the number of detection heads, drones with a larger target size span can be identified. By improving the detection layer network, small targets such as drones can be identified more accurately. Based on historical labels, the threshold for whether secondary recognition confirmation is required is adjusted, and secondary recognition confirmation is performed on recognition results with low confidence, thereby improving the overall recognition accuracy. For recognition results with low confidence, the original image of the target center with 224*224 pixels is extracted, and Inception is used. V4 performs secondary recognition confirmation and improves the input size of the DeepSort target tracking model to be more suitable for the target size and shape of the UAV. At the same time, it optimizes the DeepSort feature extraction network to improve feature extraction capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a flow chart of a method for identifying and tracking drones using video based on the Xueliang Project according to an embodiment of the present invention;

[0025] Figure 2 Optimize code diagram for recognition efficiency;

[0026] Figure 3 Schematic diagram of code comparison for detection frame optimization;

[0027] Figure 4 This is a schematic diagram of the detection layer code before optimization;

[0028] Figure 5 This is a schematic diagram of the optimized detection layer code;

[0029] Figure 6 Schematic diagram of the original Yolov5 model structure;

[0030] Figure 7 Schematic diagram of the optimized Yolov5 model structure;

[0031] Figure 8 Update the flow chart for the preset threshold;

[0032] Figure 9A schematic diagram comparing the DeepSort feature extraction network results before and after optimization. DETAILED DESCRIPTION

[0033] The purpose of the present invention is to provide a UAV identification and video tracking method based on the Xueliang Project. Based on the Xueliang Project, the UAV identification and target tracking can be realized and the method is easy to use.

[0034] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0035] like Figure 1 As shown, the drone identification and video tracking method based on the Xueliang Project provided in an embodiment of the present invention includes the following steps:

[0036] Step 1: Get the camera input video stream;

[0037] Step 2: Target detection and recognition are performed based on the camera input video stream, which includes target detection and preliminary recognition as well as target secondary recognition;

[0038] Step 3: Track the target based on the recognition results.

[0039] In step 2, target detection and recognition are performed based on the camera input video stream, which includes target detection and preliminary recognition and target secondary recognition, specifically including the following steps:

[0040] Step 201: Target detection and preliminary recognition are performed based on the camera input video stream;

[0041] Step 202: Perform secondary target recognition based on the detection results.

[0042] Target detection and recognition uses a two-layer model cascade. The first layer uses the Yolov5 s-class model optimized for small targets. This model has fewer parameters and faster computation speed, but lower accuracy than the x-class model. The second layer uses the Inception V4 model for secondary recognition and confirmation of the identified targets.

[0043] Among them, the one-stage yolov5 is used for improvement and optimization. Compared with the two-stage target detection and recognition algorithm, the one-stage recognition efficiency is higher and the more accurate drone recognition is achieved by optimizing the yolov5 algorithm.

[0044] Since the drone is a small target and the camera resolution of the Xueliang Project is low, it is necessary to optimize the first layer of Yolov5 based on these two characteristics, which will be explained in detail later.

[0045] In step 201, target detection and preliminary recognition are performed based on the camera input video stream, which specifically includes the following steps:

[0046] Step 2011: Build the Yolov5 model;

[0047] Step 212: performing image segmentation on the camera input video stream based on a 640-pixel size;

[0048] Detailed description: The resolution of images currently captured by Xueliang Project cameras is mainly 1920*1080 pixels and 2560*1400 pixels, while the default training model input size used by Yolov5 is 640*640 pixels, which directly compresses the 2560-pixel size to 640 pixels, making the target features obvious.

[0049] In order to improve the recognition efficiency, we adopt the method of recognition at the core recognition point, and complete the rest of the preprocessing and NMS (non-maximum suppression) in a unified way. The code is as follows: Figure 2 As shown in the figure, for a 2560-pixel image, cut into 4*4 pieces (a total of 16 images), the CPU takes about 120 milliseconds to recognize the entire image. Compared with the original image, the single image recognition time under the same conditions is about 70 milliseconds. The comparison shows that the recognition time is about 1.7 times longer, and the difference in recognition efficiency is small, which is acceptable.

[0050] When cutting the image, the image size is 640 pixels, but the image extraction is performed at a size of 640*1.1 pixels. That is, based on the original cut image, the edges of all cut images are expanded by 32 pixels. The drone is a small target and is fully covered by 32 pixels. Therefore, after using 1.1 times extraction, the detection and recognition errors caused by cutting the drone into two images can be avoided.

[0051] Step 2013: Improve the backbone network and neck network of the Yolov5 model and add a detection frame;

[0052] Since the individual sizes of drones in the camera are quite different, a detection frame is added, and the three detection frames are optimized to four. The optimized code is as follows: Figure 3 As shown, the left side is the original yolov5 model detection frame configuration, and the right side is the optimized detection frame configuration;

[0053] Modify the detection layer structure. After the 17th layer, continue to upsample the feature map so that the feature map continues to expand. At the same time, at the 20th layer, concat the acquired feature map with the second layer feature map in the backbone network to obtain a larger feature map for small target detection. The original detection layer network structure code is as follows: Figure 4 As shown, the optimized detection layer network structure code is as follows Figure 5As shown, the original model structure diagram is as follows Figure 6 As shown, the optimized Yolov5 model structure is as follows Figure 7 As shown;

[0054] Step 2014: Input the segmented image into the improved Yolov5 model and output the detection result.

[0055] In step 202, target secondary recognition is performed based on the detection results, which specifically includes the following steps:

[0056] Step 221: Obtain the size of the detected target based on the detection result. If the detected target is smaller than a preset threshold, perform secondary target recognition. The preset threshold is adjusted based on the data accuracy assessment.

[0057] The detailed description is as follows: the model used in the first layer is based on the Yolov5 S-level model optimized for small targets. The target size is judged. For targets smaller than the default threshold of 100*100, a 224*224 target original image is extracted with the target center point as the center for target secondary recognition. For large targets, the S-level recognition rate of Yolov5 is high enough, and the secondary recognition effect is limited and time-consuming. Therefore, for targets larger than the default threshold of 100*100, there is no need for secondary recognition. For the default threshold, a method based on data accuracy evaluation is used to adjust the threshold. The update process is as follows Figure 8 As shown;

[0058] Step 222: Build the Inception V4 model and concatenate the Yolov5 model with the Inception V4 model;

[0059] Step 2023: With the detection target as the center, extract the 224*224 target original image, perform secondary target recognition based on the Inception V4 model, and obtain the final detection result.

[0060] In step 3, target tracking is performed based on the recognition results, specifically:

[0061] Build a DeepSort model and optimize its feature extraction network. Perform cascade matching between the DeepSort model and the InceptionV4 model. Input the final detection results into the DeepSort model to obtain the tracking results.

[0062] Detailed description: Since DeepSORT is mainly used for pedestrian tracking, the input size is a rectangular box of 128 (height) x 64 (width). However, the drone target is small, and the height is often smaller than the width. Therefore, a rectangular box of 32 (height) x 32 (width) is used. In order to solve the problem that the feature extraction network in the DeepSORT algorithm is shallow and the extraction ability is relatively limited, two Resdiual modules are added to the original feature extraction network of DeepSORT. By modifying the input size and adding two Resdiual modules, the feature extraction ability of the network is enhanced without increasing the amount of calculation, ensuring the real-time performance of the overall tracking task. Among them, the comparison diagram of the DeepSORT model feature extraction network before and after optimization is shown in the figure below. Figure 9 shown.

[0063] The present invention provides a drone identification and video tracking method based on the Xueliang Project. The method is based on the Xueliang Project camera to perform drone identification and tracking. After accessing the video stream of the camera in the key area, early detection and early warning can be achieved, avoiding the need for illegal drone flight. At the same time, more sufficient time and disposal evidence can be obtained for subsequent disposal. The image cutting algorithm is adopted and the calculation process is optimized. While avoiding the reduction in recognition rate of small targets due to compression distortion, the recognition efficiency is significantly improved by improving the calculation process, achieving a small amount of performance and significantly improving the recognition accuracy. The Yolov5 algorithm is improved. By increasing the number of detection heads, drones with a larger target size span can be identified. By improving the detection layer network, small targets such as drones can be identified more accurately. Based on historical labels, the threshold for whether secondary recognition confirmation is required is adjusted. Secondary recognition confirmation is performed on recognition results with low confidence, thereby improving the overall recognition accuracy. For recognition results with low confidence, the original image of 224*224 pixels of the target center is extracted, and Inception is used. V4 performs secondary recognition confirmation and improves the input size of the DeepSort target tracking model to be more suitable for the target size and shape of the UAV. At the same time, it optimizes the DeepSort feature extraction network to improve feature extraction capabilities.

[0064] The present invention uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A drone identification and video tracking method based on the Xueliang Project, characterized in that: The steps include: Step 1: Get the camera input video stream; Step 2: Target detection and recognition are performed based on the camera input video stream, which includes target detection and preliminary recognition as well as target secondary recognition; Step 3: Track the target based on the recognition results; In step 2, target detection and recognition are performed based on the camera input video stream, which includes target detection and preliminary recognition and target secondary recognition, specifically including the following steps: Step 201: Target detection and preliminary recognition are performed based on the camera input video stream; Step 202: Perform secondary target recognition based on the detection results; In step 201, target detection and preliminary recognition are performed based on the camera input video stream, which specifically includes the following steps: Step 2011: Build the Yolov5 model; Step 212: performing image segmentation on the camera input video stream based on a 640-pixel size; Step 2013: Improve the backbone network and neck network of the Yolov5 model and add a detection frame; Step 2014: Input the segmented image into the improved Yolov5 model and output the detection result; In step 202, target secondary recognition is performed based on the detection results, which specifically includes the following steps: Step 221: Obtain the size of the detected target based on the detection result. If the detected target is smaller than a preset threshold, perform secondary target recognition. The preset threshold is adjusted based on the data accuracy assessment. Step 222: Build the Inception V4 model and concatenate the Yolov5 model with the Inception V4 model; Step 2023: With the detection target as the center, extract the 224*224 target original image, perform secondary target recognition based on the Inception V4 model, and obtain the final detection result.

2. The drone identification and video tracking method based on the Xueliang Project according to claim 1 is characterized in that: In step 3, target tracking is performed based on the recognition results, specifically: Build a DeepSort model and optimize its feature extraction network. Perform cascade matching between the DeepSort model and the Inception V4 model. Input the final detection results into the DeepSort model to obtain the tracking results.

Images