A traffic tracking and detection system from the perspective of drones

Abstract

A traffic tracking and detection system for drones. It belongs to the field of target tracking technology, and specifically to the field of lightweight target detection and tracking technology for intelligent transportation and drones. The system includes a target detection module, a target motion trajectory matching module, and a target motion trajectory output module; the target detection module processes the video frame image to obtain the detection frame corresponding to the tracked target and the confidence score corresponding to each detection frame; the target motion trajectory matching module divides the detection frame according to the confidence score, and matches the detection frame with the predicted trajectory after division; the target motion trajectory output module outputs the predicted trajectory as a confirmed trajectory after the same frame detection frame is successfully matched with the predicted trajectory for three consecutive times in the target motion trajectory matching module. It solves the problem of low tracking accuracy caused by the existence of small-sized targets, complex backgrounds, and occlusion in the previous drone tracking of targets.

Description

Technical Field

[0001] The invention belongs to the technical field of target tracking, and in particular relates to a traffic tracking detection system oriented to the perspective of an unmanned aerial vehicle. Background Art

[0002] With the rapid development of global urbanization, the efficiency and safety of urban transportation systems have become one of the key factors supporting the sustainable development of cities, directly affecting the quality of life of residents. Since the monitoring range of traditional fixed cameras is limited by their installation location and viewing angle, a large number of cameras need to be deployed to achieve wide coverage, which poses a challenge in terms of use cost and subsequent maintenance. With its high flexibility and mobility, drones can quickly adjust their flight routes and reach any location in response to emergencies, temporary events or traffic congestion, covering a wide area in a short time and providing multi-angle and high-altitude viewing angles. Therefore, it is necessary to develop an efficient drone-perspective traffic tracking and detection system.

[0003] In recent years, deep learning-based target detection methods, such as SSD, YOLOv5, and YOLOv8, have been widely used in practical applications due to their excellent detection performance and processing speed. In particular, the application of YOLOv8 has achieved a good balance between detection accuracy and speed, which is significantly better than other methods. However, although YOLOv8 performs well in most scenarios, its performance in dealing with small-sized targets, complex backgrounds, and occlusions is still insufficient, which leads to deformation, occlusion, and changes in viewpoint height and angle in multi-target tracking of drones, thus affecting tracking accuracy. Summary of the invention

[0004] In order to solve the problem of low tracking accuracy caused by small-sized targets, complex backgrounds and occlusion in the previous tracking of targets by drones, the present invention provides a traffic tracking detection system oriented to the perspective of drones.

[0005] The system includes a target detection module, a target motion trajectory matching module and a target motion trajectory output module;

[0006] The target detection module processes the video frame images, uses the YOLO-LF model to derive the detection frame and the confidence score of the detection frame from the video frame images, and then filters out the detection frames that do not meet the requirements;

[0007] The YOLO-LF model is obtained by improving the YOLOv8s model. The improvements are as follows:

[0008] Change the number of channels of the last three layers of the YOLOv8s model backbone to 512;

[0009] After adding upsampling and downsampling parts to the neck of the YOLOv8s model, we continue to add the LSKAttention module and the independently designed fast multi-cross structure FPW to the neck of the YOLOv8s model.

[0010] In the YOLOv8s model head part, the number of detection heads is increased from 3 to 4, and a self-designed MODConv module is added to the initial input part of each detection head;

[0011] The target motion trajectory matching module: divides the detection frame according to the confidence score, and matches the detection frame with the predicted trajectory after the division;

[0012] The target motion trajectory matching module uses the NByteTrack model to implement the division and matching of the detection frame. The NByteTrack model is improved on the basis of the ByteTrack model. The improvements are as follows:

[0013] The Kalman filter used in the ByteTrack model was changed to an adaptive Kalman filter;

[0014] Change the IoU value calculation in the ByteTrack model to GIoU value calculation;

[0015] Introducing video stabilization technology into the ByteTrack model;

[0016] The target motion trajectory output module: when the target motion trajectory matching module successfully matches the same frame detection frame with the predicted trajectory three times in a row, the predicted trajectory is output as a confirmed trajectory.

[0017] The beneficial effects of the system of the present invention are:

[0018] The system described in this paper is based on the YOLOv8+ByteTrack multi-target tracking method and redesigns the scheme. First, by adding a small target detection head and adding a self-designed MODConv module to the head to maintain the original channel information; secondly, LSK Attention is used to adaptively adjust the target receptive field in the downsampling stage, and combined with the Soft-NMS strategy to avoid excessive deletion of prediction boxes, thereby improving the detection accuracy of small targets in occluded scenes. Finally, the self-designed fast multi-cross structure FPW is used to reduce redundant calculations and memory accesses. In terms of target tracking, GIoU replaces IoU, and the aspect ratio of the state space is adjusted to a width-based prediction through an adaptive Kalman filter. In addition, the introduced camera compensation mechanism further improves the accuracy and stability of tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is the structure diagram of the traffic tracking and detection system from the perspective of the drone;

[0020] Figure 2 It is the structure diagram of YOLO-LF model;

[0021] Figure 3 This is the structure diagram of the LSKAttention module;

[0022] Figure 4 It is the structural diagram of the space gating unit LSK;

[0023] Figure 5 The structural diagram of the independently designed fast multi-cross structure FPW;

[0024] Figure 6 This is the structure diagram of the independently designed MODConv module;

[0025] Figure 7 A schematic diagram of the situation where the IoU value is the same but the overlap of the targets is different. DETAILED DESCRIPTION

[0026] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of the present invention.

[0027] Embodiment 1,

[0028] Based on the previous drone target tracking system using YOLOv8+ByteTrack, this embodiment improves YOLOv8 and ByteTrack respectively, thereby effectively optimizing the multi-target tracking problem and improving the tracking accuracy. The trajectory of the occluded target can be better restored, and the accuracy of multi-target trajectory prediction in the drone video can be further improved. Through optimization, more accurate drone multi-target tracking is achieved while ensuring processing efficiency, so that the accuracy of multi-target trajectory prediction in the drone video is further improved.

[0029] like Figure 1 As shown, it is a structural diagram of the traffic tracking detection system for the perspective of unmanned aerial vehicles of the present invention, and the system includes a target detection module, a target motion trajectory matching module and a target motion trajectory output module;

[0030] The target detection module is responsible for processing the video frame images to obtain the detection frame corresponding to the tracked target and the confidence score corresponding to each detection frame;

[0031] The target motion trajectory matching module is responsible for dividing the detection frame according to the confidence score, and then matching the detection frame with the predicted trajectory;

[0032] The target motion trajectory output module is responsible for the output of the motion trajectory. When the target motion trajectory matching module successfully matches the same frame detection frame with the predicted trajectory three times in a row, the predicted trajectory is output as the confirmed trajectory.

[0033] Embodiment 2,

[0034] This embodiment further limits the embodiment 1 and specifically introduces the target detection module.

[0035] The target detection module uses the YOLO-LF model to derive the detection frame of the tracked target from the video frame image, and uses the YOLO-LF model to derive the detection frame and the confidence score of the detection frame from the video frame image, and then filters out the detection frames that do not meet the requirements;

[0036] The filtering out of the detection boxes that do not meet the requirements is specifically as follows: calculating the degree of match between the detection box and the prediction box, expressed as an loU value, adjusting the loU value using the Soft-NMS algorithm to obtain the adjusted loU value, setting a threshold for the adjusted loU value, and filtering out the detection boxes corresponding to the adjusted loU values ​​that do not meet the threshold.

[0037] The system described in the present invention is a traffic tracking and detection system for drones, which is mainly responsible for tracking targets in collected drone aerial videos and outputting confirmed state trajectories of target movement. Figure 1 As shown in the figure, after the video frame image in the drone multi-target tracking dataset is input into the YOLO-LF model, the detection frame corresponding to the tracked target will be obtained, and the intersection over union (IoU) between this detection frame and the prediction frame is calculated. The prediction frame is the next frame detection frame obtained after the motion trajectory of the detection frame is predicted using Kalman filtering.

[0038] In the drone image detection task, the traditional non-maximum suppression (NMS) method has serious false detection and missed detection in complex scenes and when the target is covered and overlapped, thus affecting the detection performance. In this regard, the target detection module of the present invention adopts the Soft-NMS algorithm, which reduces missed detection by reducing the score of the overlapping detection box instead of directly removing it when processing overlapping targets.

[0039] In the standard detection process, the detection frames and scores generated by the sliding window are processed by NMS, the detection frame with the highest score is selected, and other frames that overlap with it by more than a certain threshold are removed. However, this may cause valid detection frames to be mistakenly deleted. Mathematically, NMS can be expressed as:

[0040]

[0041] in For the A detection frame, is the corresponding detection box score, is an artificially set threshold. is the detection box with the highest score, Represents the IoU value. Soft-NMS improves this process by using a linear or Gaussian decay function to gradually reduce the score of the detection box that overlaps the highest scoring box. Linear decay reduces the score, while Gaussian decay smoothly adjusts the confidence score according to the IoU value. The calculation expression of Soft-NMS is:

[0042]

[0043]

[0044] This method not only reduces the information loss caused by the fixed threshold, but is also particularly suitable for the detection of small targets in UAV images, effectively improving the detection accuracy.

[0045] The YOLO-LF model is obtained by improving the YOLOv8s model, and its schematic diagram is as follows: Figure 2 As shown, the improvements are:

[0046] (1) The number of channels of the last three layers of the YOLOv8s model backbone is changed to 512. Taking the input to output of the network as the reference direction, the last three layers of the YOLOv8s model backbone are Conv layer, C2f layer and SPPF layer respectively. In the data of the backbone part in the figure, w represents the width of the model, and r represents the scaling ratio of the model. Here, 512 is the maximum number of channels. There are also cases where the number of channels is less than 512 in actual use, such as Figure 2 The figure shows the number of channels of the last three layers of the backbone of a YOLOv8s model.

[0047] (2) After adding the upsampling part and the downsampling part to the neck part of the YOLOv8s model, continue to add the LSKAttention module to the neck part of the YOLOv8s model ( Figure 2 LSK in the text) and the independently designed fast multi-cross structure FPW ( Figure 2 FPW is used in the calculation.

[0048] The upsampling part and the downsampling part are added to the neck part of the YOLOv8s model specifically as follows: the upsampling part is added to the starting position of the bottom-up route of the neck part, and the downsampling part is added to the end position of the top-down route of the neck part;

[0049] The upsampling part includes the Upsample layer from bottom to top ( Figure 2 In the example above, Up is used to represent the Concat layer. Figure 2 CAT in Chinese);

[0050] The downsampling part includes a C2f layer, a Conv layer and a Concat layer from top to bottom.

[0051] Continue to add the LSKAttention module and the independently designed fast multi-cross structure FPW to the neck part of the YOLOv8s model. Specifically: add the LSKAttention module after each C2f layer in the top-down route of the neck part, add the independently designed fast multi-cross structure FPW after the last Concat layer in the top-down route of the neck part, and add the LSKAttention module after the independently designed fast multi-cross structure FPW.

[0052] like Figure 3 The figure shows the structure of the LSKAttention module. In order to enhance the model's target detection capability in complex scenarios, we introduced some modules in the LSKAttention mechanism as the LSKAttention module in this YOLO-LF model, as shown in Figure 3 As shown, the LSKAttention module includes a fully connected network ( Figure 3 The proposed method combines the LSK (space-based attention unit) with the local and dilated convolutions to capture a wide range of spatial contextual information in key areas, thereby refining the model’s feature processing for different situations. It not only enhances the sensitivity of the LSK Attention module to key features, but also improves the accuracy and reliability of complex scene detection by adjusting the spatial focus.

[0053] The structure of the core module space gating unit LSK is as follows Figure 4As shown in the figure, it uses gradually enlarged depthwise separable convolution Conv (K=5, K represents the convolution kernel) and Conv (K=7) to accelerate the expansion of the receptive field and reduce the gap between feature maps. At the same time, it effectively generates multi-scale feature maps and reduces the complexity of the model. In addition, by introducing a dynamic selection mechanism to optimize the kernel selection according to multi-scale information, the feature expression capability is enhanced. The spatial selection mechanism is used to filter feature maps processed by different large convolution kernels. The spatial relationship between features is extracted by using average pooling (Mean) and maximum pooling (Max) operations, and then the spatial attention map is generated through the Concat operation, convolution operation (Conv, K=7) and Sigmoid function. These attention maps form a spatial selection mask, which is used to weightedly fuse the feature maps of different convolution kernels, thereby generating attention-enhanced features and strengthening the extraction of spatial context information.

[0054] Self-designed fast multi-cross structure FPW Figure 5 As shown, it includes 3 branches. The first branch includes the first Conv layer, PWConv layer, second Conv layer and third Conv layer in sequence; the second branch performs weighted fusion on the output of the first Conv layer and the output of the first branch and inputs them into the third branch. The third branch performs Concat operation on the original input and the input of the second branch and outputs the final result.

[0055] At the hardware level of drones, the amount of computation and the amount of parameters determine the computing time and memory resource consumption respectively. Deep learning research is committed to the continuous optimization of network architecture to meet these challenges. The fast multi-cross structure FPW designed independently in the present invention draws on the design concept of the cross-stage partial network structure (CSP) and FasterNet, and proposes an innovative fast multi-cross structure FPW, which aims to improve the efficiency of the vehicle detection model. The structure adopts a three-branch method for feature sampling: the first branch extracts basic spatial information through conventional convolution to ensure accuracy, and the other two branches use a weighted fusion strategy to enhance deep and shallow feature capture. Information integration is achieved through feature splicing, and context perception is enhanced through cross-channel information transmission. It aims to improve the positioning accuracy and overall recognition rate of the model while reducing the computational complexity and the amount of parameters. At the same time, the weighted fusion strategy and feature splicing not only improve the context perception ability of the model, but also improve the positioning accuracy and recognition rate while reducing the computational complexity and the amount of parameters.

[0056] The improvement in the efficiency of the FPW model comes from the application of partial convolution layers (PWConv), which is different from the traditional full-channel convolution calculation. PWConv reduces the parameters and computational complexity by grouping the input channels and performing independent convolutions within each group, while maintaining the integrity of the number of channels and avoiding redundancy. This strategy is similar to GhostNet, but we chose to retain grouped convolutions instead of using depthwise separable convolutions. This design reduces the amount of computation and memory access requirements between feature map channels, allowing the model to extract spatial features more efficiently.

[0057] In PWConv, only feature channels are grouped for convolution, focusing on spatial feature extraction, while the remaining The channels maintain their original features through translation operations, without the need for additional memory access, thus reducing resource consumption. The amount of calculation can be expressed by the following formula:

[0058]

[0059] In the formula, Represents the width and height of the feature map, is the convolution kernel size, is the number of groups, is the total number of channels involved in group convolution, is the number of individual channels of the convolution. In fact, the computational efficiency of grouped convolution is much higher than that of regular convolution. When , the computational complexity of PWConv is only 1 / 2 of that of conventional convolution. , the memory access amount is approximately:

[0060]

[0061] Through this design, the FPW structure significantly improves performance while maintaining a simple architecture.

[0062] (3) In the head part of the YOLOv8s model, the number of detection heads is increased from 3 to 4. The additional detection heads are specially designed for small targets, and their size is 160*160*64*w, where w represents the width of the network where the detection head is located; and a self-designed MODConv module is added to the initial input part of each detection head;

[0063] The self-designed MODConv module is as follows Figure 6As shown in the figure, it includes two branches. The first branch uses depthwise separable convolution (Depthwise Conv) to process each channel of the input feature map, and the second branch uses residual connection to maintain the original information of the feature map. The outputs of the first branch and the second branch are superimposed, and then pointwise convolution (Pointwise Conv) is performed and output (Output).

[0064] This module extracts spatial features by applying convolution kernels to each input channel separately. At the same time, it uses residual connections to maintain the original information of the feature map, which alleviates the gradient vanishing problem in deep networks and merges the features and channels fused in the previous step through point-by-point convolution. The introduction of the MODConv module not only promotes information interaction between channels in the feature layer, but also retains as much feature information as possible before prediction, thus achieving a lightweight model.

[0065] Embodiment 3,

[0066] This embodiment further limits the first embodiment and further limits the target motion trajectory matching module.

[0067] The target motion trajectory matching module divides the detection frame according to the confidence score, and then matches the detection frame with the predicted trajectory;

[0068] The target motion trajectory matching module uses the NByteTrack model to implement the division and matching of detection boxes, such as Figure 1 As shown, the NByteTrack model is obtained by improving the ByteTrack model, and the improvements are:

[0069] The Kalman filter used in the ByteTrack model was changed to an adaptive Kalman filter;

[0070] Change the IoU value calculation in the ByteTrack model to GIoU value calculation;

[0071] Introducing video stabilization technology into the ByteTrack model;

[0072] In the ByteTrack model, IoU is used to filter target boxes. However, since IoU does not consider the distance between non-overlapping targets, it limits the accurate evaluation of the relative position and overlap degree of the targets. Figure 7 As shown in the figure, although the IoU values ​​are the same, the overlap of the objects is different, which shows the limitation of IoU. The NByteTrack model uses the generalized intersection over union (GIoU) ​​as the similarity metric, which improves IoU by calculating the minimum bounding rectangle of the detection box and combining it with the distance metric. The calculation formula of GIoU is:

[0073]

[0074] Among them, C represents the minimum circumscribed rectangular area of ​​the two detection frames. A and B are two detection frames. GIoU considers overlapping and non-overlapping areas, and its value range is [-1,1]. When they are completely overlapped, GIoU is 1; when the two frames have no intersection and are extremely far away, GIoU is -1, making GIoU a more comprehensive distance metric. The use of the GIoU criterion can reduce non-target frames, reduce misjudgments and missed judgments in high-density small target scenes, and improve detection accuracy and reliability in drone monitoring.

[0075] In the tracking algorithm, the choice of Kalman filter state vector has a significant impact on the accuracy of target size estimation. For example, ByteTrack optimizes width estimation by estimating the aspect ratio of the target box, while the SORT algorithm uses a seven-tuple state vector. DeepSORT further expands it to an eight-tuple, focusing on height prediction. NByteTrack introduces an adaptive Kalman filter, which can dynamically adjust the measurement noise covariance and directly estimate the width and height of the bounding box through the state vector to further optimize performance to better adapt to the complexity of drone scenes.

[0076] In drone tracking tasks, accurately tracking the target is crucial to ensure stability and accuracy. The traditional ByteTrack algorithm predicts the subsequent position of the target through Kalman filtering based on the target motion information, effectively improving the tracking speed. However, in a dynamic drone environment, failure to consider the impact of camera motion may cause the tracking frame position to shift or be lost. To solve this problem, the video stabilization technology is introduced as follows: using the global motion compensation module of OpenCV, tracking feature points through the optical flow method and using the RANSAC algorithm to solve the affine transformation matrix of the background motion, and applying the affine transformation matrix to update the state vector of the adaptive Kalman filter to reduce the tracking error caused by camera movement.

[0077] like Figure 1 As shown, the workflow of NByteTrack is given, specifically:

[0078] Step S1, detection frame confidence distinction: set the detection confidence threshold, put the detection frames with confidence greater than or equal to 0.5 into the high confidence detection frame sample set Hbox, put the detection frames with confidence greater than or equal to 0.1 and less than 0.5 into the low confidence detection frame sample set Lbox, and directly delete the detection frames with confidence less than 0.1.

[0079] Step S2, high confidence detection frame matching: Use the improved Kalman filter to predict the target motion trajectory based on the detection frame input by the target detection module. For each frame detection frame in the high-score frame association sample set Hbox, calculate the GIoU value between the next frame detection frame predicted based on the target motion trajectory and the current frame detection frame; According to the GIoU value, use the Hungarian algorithm to match the target motion trajectory and the current frame detection frame to obtain three results: a trajectory that has been matched with the detection frame, an unmatched trajectory, and an unmatched detection frame; Input the unmatched detection frame into the adaptive Kalman filter to create a new trajectory;

[0080] Step S3, low confidence detection frame matching: for each frame detection frame in the low-frame associated sample set Lbox, calculate the GIoU value between the next frame detection frame predicted according to the unmatched trajectory in step S2 and the current frame detection frame; according to the GIoU value, use the Hungarian algorithm to match the unmatched trajectory with the current frame detection frame, and obtain three results: the trajectory matched with the detection frame, the unmatched trajectory, and the unmatched detection frame; delete the unmatched detection frame, and for the unmatched trajectory, keep the first 30 frames and input them into the adaptive Kalman filter for target motion trajectory prediction;

[0081] Step S4, output of matched trajectory: The trajectory matched with the detection frame obtained in steps S2 and S3 is input into the adaptive Kalman filter until it is successfully matched with the detection frame three times and then output as the confirmed trajectory from the target motion trajectory output module.

[0082] Embodiment 4,

[0083] This embodiment further illustrates the experimental results of the traffic tracking and detection system from the perspective of a drone involved in Embodiments 1, 2 and 3.

[0084] To further demonstrate the performance advantages and effectiveness of the YOLO-LF model, the improved model was compared with four models of different sizes (n, s, m, l) in the YOLO series under the same data set and hardware and software conditions. "Our" represents the YOLO-LF model. The experimental results are shown in Table 1 below, where P stands for precision, which refers to the proportion of all detections predicted by the model as positive samples that are actually positive samples. R stands for recall, which refers to the proportion of actual positive samples that are correctly detected. mAP (mean Average Precision) is a very important metric in the field of object detection in machine learning. It is used to measure the performance of object detection algorithms. mAP0.5 means mAP@0.5, which means the mAP value when IoU is set to 0.5. mAP95 means mAP@0.5:0.95, which means the average mAP value at different IoU thresholds (0.5-0.95, step size 0.05) (0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95). Param means the number of parameters, that is, the total number of all learnable parameters that need to be trained in the model.

[0085] Table 1:

[0086]

[0087] This experiment compares the performance of the improved YOLO-LF model with other advanced YOLO versions on small object detection tasks. YOLOv5 improves the robustness of the network through the Focus structure and Mosaic data enhancement, while YOLOv6 effectively improves the performance of small models through the self-distillation strategy, simplified SPPF module and AAT strategy. YOLOv7 demonstrates its efficiency advantage through the ELAN network architecture. Table 2 shows that the YOLO-LF model surpasses other models in the mAP@0.5:0.95 indicator, especially the medium-sized YOLO-LF-m model outperforms YOLOv7l and YOLOv6m in all key performance indicators. In addition, although the YOLO-LF-n model has only 1.6M parameters and a size of 3.6MB, its performance is comparable to larger models such as YOLOv5n and YOLOv7-Tiny. The YOLO-LF model significantly improves performance while reducing the number of parameters and model size, outperforming other detection models, showing the advantages of parameter efficiency and lightweight design.

[0088] Further analysis of the performance improvement after combining the improved YOLO-LF model with the NByteTrack model. As shown in Table 2, compared with the traditional YOLOv8s and ByteTrack models, the improved combination of the present invention (YOLO-LF + NByteTrack) has achieved significant improvements in multiple key performance indicators: MOTA (Multi-Object Tracking Accuracy), HOTA (Higher Order Tracking Accuracy) and IDF1 (ID F1Score) increased by 5.25%, 5.10% and 9.89% respectively, while IDSW (identity switches) decreased by 73.6%, and MOTP (Multi-Object Tracking Precision) decreased by 1.3%.

[0089] These results highlight the advantages of the improved combination in improving the key performance indicators of multi-target tracking, especially the significant improvement of MOTA and IDF1 indicators and the significant reduction of IDSW, showing the significant effect of the algorithm combination in maintaining the consistency of target identity. In addition, the reduction of misjudgment and tracking loss further proves the effective collaboration between the improved detector and tracking algorithm, ensuring the accuracy and stability of detection and tracking.

[0090] Table 2:

[0091]

Claims

1. A traffic tracking and detection system for drones, characterized in that: The system includes a target detection module, a target motion trajectory matching module and a target motion trajectory output module; The target detection module processes the video frame images, uses the YOLO-LF model to derive the detection frame and the confidence score of the detection frame from the video frame images, and then filters out the detection frames that do not meet the requirements; The YOLO-LF model is obtained by improving the YOLOv8s model. The improvements are as follows: Change the number of channels of the last three layers of the YOLOv8s model backbone to 512; After adding upsampling and downsampling parts to the neck of the YOLOv8s model, we continue to add the LSKAttention module and the independently designed fast multi-cross structure FPW to the neck of the YOLOv8s model. The upsampling part and the downsampling part are added to the neck part of the YOLOv8s model. Specifically, the upsampling part is added at the starting position of the neck part from bottom to top, and the downsampling part is added at the end position of the neck part from top to bottom. The upsampling part includes the Upsample layer and the Concat layer from bottom to top; the downsampling part includes the C2f layer, the Conv layer and the Concat layer from top to bottom. The LSKAttention module and the self-designed fast multi-cross structure FPW are added to the neck part of the YOLOv8s model. Specifically, the LSKAttention module is added after each C2f layer in the top-down route of the neck part, and the self-designed fast multi-cross structure FPW is added after the last Concat layer in the top-down route of the neck part, and the LSKAttention module is added after the self-designed fast multi-cross structure FPW. The self-designed fast multi-cross structure FPW includes three branches, the first branch includes the first Conv layer, the PWConv layer, the second Conv layer and the third Conv layer in sequence; the second branch performs weighted fusion on the result output by the first Conv layer and the result output by the first branch and then inputs the result to the third branch, and the third branch performs Concat operation on the original input and the input of the second branch and then outputs the final result; In the YOLOv8s model head part, the number of detection heads is increased from 3 to 4, and a self-designed MODConv module is added to the initial input part of each detection head; The self-designed MODConv module includes two branches. The first branch processes each channel of the input feature map using depthwise separable convolution, and the second branch uses residual connection to maintain the original information of the feature map. The outputs of the first branch and the second branch are superimposed and then convolved point by point for output. The target motion trajectory matching module: divides the detection frame according to the confidence score, and matches the detection frame with the predicted trajectory after the division; The target motion trajectory matching module uses the NByteTrack model to implement the division and matching of the detection frame. The NByteTrack model is improved on the basis of the ByteTrack model. The improvements are as follows: The Kalman filter used in the ByteTrack model was changed to an adaptive Kalman filter; Change the IoU value calculation in the ByteTrack model to GIoU value calculation; Introducing video stabilization technology into the ByteTrack model; The target motion trajectory output module: when the target motion trajectory matching module successfully matches the same frame detection frame with the predicted trajectory three times in a row, the predicted trajectory is output as a confirmed trajectory.

2. The traffic tracking and detection system for drones according to claim 1 is characterized in that: The filtering out of the detection boxes that do not meet the requirements is specifically as follows: calculating the degree of match between the detection box and the prediction box, expressed as an loU value, adjusting the loU value using the Soft-NMS algorithm to obtain the adjusted loU value, setting a threshold for the adjusted loU value, and filtering out the detection boxes corresponding to the adjusted loU values ​​that do not meet the threshold.

3. The traffic tracking and detection system for drones according to claim 2 is characterized in that: Taking the input to output of the network as the reference direction, the last three layers of the backbone of the YOLOv8s model are the Conv layer, the C2f layer, and the SPPF layer.

4. The traffic tracking and detection system for drones according to claim 3 is characterized in that: When the number of detection heads in the YOLOv8s model head is increased from 3 to 4, the added detection head is a detection head specifically for small targets, and its size is 160*160*64*w, where w represents the width of the network where the detection head is located.

5. The traffic tracking and detection system from the perspective of a drone according to claim 4 is characterized in that: The video stabilization technology is specifically: using the global motion compensation module of OpenCV, tracking feature points through the optical flow method and using the RANSAC algorithm to solve the affine transformation matrix of the background motion, and applying the affine transformation matrix to update the state vector of the adaptive Kalman filter to reduce the tracking error caused by the camera movement.

Images