Aircraft target tracking method based on YOLOv8-TLSA
By introducing the spatiotemporal attention mechanism and dynamic weight adjustment mechanism, and combining the WIoU loss function to optimize the YOLOv8 model, the problem of insufficient robustness of YOLOv8 in small target detection and complex scenarios is solved, and more efficient aircraft target detection and tracking is achieved.
Patent Information
- Application Number
- CN202510565630.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-30
- Publication Date
- 2025-10-03
AI Technical Summary
YOLOv8 is not robust enough in small target detection and complex scenarios, and has high computational overhead, which affects detection accuracy and efficiency.
The spatiotemporal attention mechanism and dynamic weight adjustment mechanism are introduced, combined with the Widget over Union (WIoU) loss function, to optimize the YOLOv8 model, enhance the ability to capture aircraft target features, reduce computational overhead, and improve model adaptability and robustness.
It significantly improves the model's detection accuracy for small targets and its generalization ability in complex scenarios, increases the model's convergence speed and training efficiency, and enhances the continuity and stability of target tracking.
Smart Images
Figure CN120747445A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to an aircraft target tracking method based on YOLOv8-TLSA. Background Art
[0002] Object detection is a core task in computer vision, aiming to automatically identify and locate objects in images or videos. With the rapid development of deep learning technology, convolutional neural networks (CNNs) have become one of the most important methods in object detection. In particular, real-time object detection algorithms based on deep convolutional neural networks (DCNNs), such as YOLO, SSD (Single Shot Multibox Detector), and Faster R-CNN, have been widely used in fields such as autonomous driving, security monitoring, face recognition, and robotic vision.
[0003] Since its introduction, the YOLO family of algorithms has made significant progress in the field of object detection. The YOLO algorithm's core advantage lies in its end-to-end training and inference approach, which enables highly accurate object detection and extremely fast inference speed, making it suitable for real-time detection scenarios. YOLOv4 and YOLOv5 further improve detection accuracy and enhance efficiency and robustness by optimizing the network structure and loss function.
[0004] YOLOv8 is the latest version of the YOLO series, featuring numerous improvements and optimizations compared to previous generations. YOLOv8 utilizes advanced network architectures, data augmentation strategies, loss function optimization, and other techniques to achieve a good balance between object detection accuracy and inference speed. However, despite its impressive performance in many applications, YOLOv8 still suffers from inadequate detection of small objects and significant background noise in some cases. Therefore, further improving YOLOv8's detection accuracy, particularly its robustness against small objects and complex scenarios, has become a research hotspot in the field of object detection.
[0005] In recent years, researchers have begun introducing attention mechanisms to enhance the network's ability to focus on target features. Attention mechanisms dynamically weight different regions in the input feature map, enabling the model to automatically identify important target regions and ignore irrelevant background information. Common attention modules include the Squeeze-and-Excitation (SE) module, the Convolutional Block Attention Module (CBAM), and Non-Local Attention. For example, YOLOv4 introduced the CBAM module. However, due to high computational overhead or insufficient optimization for small objects, these modules are not ideal for certain tasks.
[0006] Based on this, the present invention aims to propose an aircraft target tracking method to improve positioning accuracy and avoid the problem of excessive complexity. Summary of the Invention
[0007] In order to make up for the shortcomings of the existing technical problems, the purpose of the present invention is to provide an aircraft target tracking method based on YOLOv8-TLSA, which introduces a spatiotemporal attention mechanism to fully integrate temporal and spatial information, enhances the model's ability to capture aircraft target features, and improves positioning accuracy; introduces the WIoU loss function to reduce the intervention in model training, improves the model's convergence speed and training efficiency, and the dynamic weight adjustment mechanism can adjust the weight in real time according to the target's motion state, reducing unnecessary computational overhead and improving the overall performance of the model.
[0008] To achieve the above object, the present invention provides the following technical solutions:
[0009] A YOLOv8-TLSA-based aircraft target tracking method, comprising:
[0010] (1) Use the selected dataset to obtain image data, convert it into YOLO training format, and divide it into training set and test set;
[0011] (2) Construct a YOLOv8-TLSA aircraft target detection model. This model uses YOLOv8 as the backbone network. The spatiotemporal attention mechanism and dynamic weight adjustment mechanism are introduced between the backbone and neck of YOLOv8 in sequence. A motion state detection module is added after the head. The motion state detection module identifies the motion state of the aircraft target, and the dynamic weight adjustment mechanism adjusts the attention weights of the time dimension and space dimension in the spatiotemporal attention mechanism according to the motion state of the aircraft target.
[0012] (3) Using the data in step (1) to train the YOLOv8-TLSA aircraft target detection model, during the training process, the WIoU loss function is combined to optimize the YOLOv8-TLSA aircraft target detection model parameters to obtain the trained YOLOv8-TLSA aircraft target detection model;
[0013] (4) Acquire the image data of the aircraft target in real time, and use the YOLOv8-TLSA aircraft target detection model as the target detector of the DeepSORT algorithm to track the aircraft target.
[0014] In the present invention, the UA-DETRAC public dataset is selected in step (1), and Python code is used to convert it into the YOLO training format. The image data is randomly divided into a training set and a test set in a ratio of 8:2 for training the aircraft target detection model of YOLOv8-TLSA.
[0015] In the present invention, the spatiotemporal attention mechanism includes a time dimension attention module and a space dimension attention module; the time dimension attention module adopts a TCN-LSTM fusion module, and the space dimension attention module adopts a SimAM attention mechanism.
[0016] In the present invention, the attention weight adjustment process of the time dimension of the TCN-LSTM fusion module is as follows;
[0017] YOLOv8's backbone network extracts the feature map of each frame to form a feature sequence. This feature sequence is input into the TCN-LSTM fusion module to obtain the initial weights of the time dimension. The dynamic weight adjustment mechanism adjusts the weights output by the TCN-LSTM fusion network based on the motion state of the aircraft target. The adjusted time dimension weights are applied to the feature map of each frame, assigning a weight based on the time context to the aircraft target features in each frame.
[0018] In the present invention, the attention weight adjustment process of the spatial dimension of the SimAM attention mechanism is as follows:
[0019] The feature map of each frame is extracted from the backbone network of YOLOv8, and the feature map is divided into multiple spatial sub-regions through group convolution. The SimAM attention mechanism is applied to the feature map in each spatial sub-region to generate spatial attention. The dynamic weight adjustment mechanism is used to adjust the spatial attention weight generated by SimAM according to the motion state of the aircraft target.
[0020] In the present invention, the dynamic weight adjustment mechanism adjusts the attention weight formula of the time dimension and the space dimension as follows:
[0021] α t =α0+β·Δv
[0022] β t =β0+γ·Δθ
[0023] Among them, α t and β t They represent the attention weights of the time dimension and the spatial dimension respectively, α0 and β0 are the initial weights, Δv is the speed change of the target, Δθ is the direction change of the target, β and γ are the coefficients of the target speed change and direction change, respectively.
[0024] In the present invention, the method for calculating the speed change and direction change of the target is as follows:
[0025] (a) Obtain the bounding box information of the aircraft target from the YOLOv8-TLSA detection head, including the bounding box center position, size and confidence;
[0026] (b) The displacement is calculated by the difference in the coordinates of the center points of the bounding boxes in adjacent frames, and the speed and acceleration are calculated based on the frame rate. The direction change is calculated by the angle between the displacement vectors of adjacent frames.
[0027] Compared with the prior art, the present invention has the following beneficial effects:
[0028] 1. This paper introduces a spatiotemporal attention mechanism, combining the TCN-LSTM fusion structure in the time dimension and the SimAM attention mechanism in the space dimension, which can effectively capture the dynamic characteristics of aircraft targets. It also designs a dynamic weight adjustment mechanism to adaptively adjust the attention weights in the time and space dimensions according to the aircraft's motion state, achieving an effective fusion of time and space information. The full integration of time and space information improves the flexibility and accuracy of feature extraction, enhances the model's ability to capture aircraft target features, significantly improves the model's adaptability and detection accuracy for dynamic targets, and provides higher-quality feature representation for subsequent target detection and tracking.
[0029] 2. This paper introduces the Width Over Union (WIoU) loss function, which comprehensively considers the relative position and size differences between objects and incorporates adaptive weighting coefficients to reduce the excessive penalty imposed by geometric factors on the model. By optimizing the loss function design, it reduces interference with model training and significantly improves the model's generalization and robustness in complex scenarios.
[0030] 3. This paper combines the YOLOv8-TLSA target detection model with the DeepSORT algorithm, using deep learning methods to extract and match target features, improving the continuity and stability of target tracking. The intersection over union (IOU) and Hungarian algorithm are used to match detection results with predicted trajectories, ensuring the accuracy and reliability of tracking results.
[0031] 4. This paper reduces intervention in model training through the design of the WioU loss function, improving the model's convergence speed and training efficiency. The dynamic weight adjustment mechanism can adjust weights in real time based on the target's motion state, reducing unnecessary computational overhead and improving the model's overall performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 This is a schematic diagram of the YOLOv8-TLSA network structure of the present invention. DETAILED DESCRIPTION
[0033] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.
[0034] like Figure 1 As shown, the present invention discloses an aircraft target tracking method based on YOLOv8-TLSA, comprising:
[0035] Step 1: Select a suitable dataset, load the target detection image data, convert it to the YOLO training format, and divide it into training and test sets.
[0036] The UA-DETRAC public dataset was used for object detection image data and converted to the YOLO training format using Python code. To prevent overfitting caused by repeated learning of the same features and to reduce training time, 2,000 images were randomly extracted from the dataset, converted to the YOLO training format, and randomly divided into training and test sets in an 8:2 ratio.
[0037] Step 2: Introduce the spatiotemporal attention mechanism and dynamic weight adjustment mechanism to optimize the YOLOv8 target detection algorithm and build an aircraft target detection model based on YOLOv8-TLSA.
[0038] A spatiotemporal attention mechanism is introduced between the Backbone and neck of YOLOv8. The spatiotemporal attention mechanism includes a temporal dimension attention module and a spatial dimension attention module. The temporal dimension attention module adopts a TCN-LSTM fusion module, and the spatial dimension attention module adopts a SimAM attention mechanism.
[0039] (1) Temporal dimension attention module (TCN-LSTM fusion module): Considering the motion characteristics of aircraft targets in consecutive frames, a temporal dimension attention module is designed to associate and weight the features of aircraft targets in adjacent frames to capture information such as their motion trends and speed changes.
[0040] The feature maps of each frame are extracted from the YOLOv8 backbone network, forming a feature sequence. These feature maps contain the characteristic information of each frame. The feature sequence is then input into the TCN-LSTM fusion network to obtain initial weights for the temporal dimension. A dynamic weight adjustment mechanism adjusts the weights of the TCN-LSTM fusion network output based on the aircraft's motion state (for example, the increased demand for temporal features during acceleration or deceleration), enhancing the model's ability to perceive the temporal characteristics of dynamic targets. The adjusted temporal weights are applied to the feature maps of each frame, assigning weights based on the temporal context to the aircraft's features in each frame.
[0041] (2) Spatial dimension attention module (SimAM attention mechanism):
[0042] The SimAM attention mechanism is introduced to divide the feature map into multiple spatial sub-regions through group convolution, and the attention weight in each sub-region is calculated separately. Then, global fusion is performed to more accurately locate the key parts of the aircraft target, such as the wings and fuselage, and enhance the perception of the target shape and structure.
[0043] The YOLOv8 backbone extracts feature maps for each frame. These feature maps contain the characteristic information of each frame. Grouped convolution is used to divide the feature maps into multiple spatial subregions. Grouped convolution reduces computation while preserving the local correlation of features. The SimAM attention mechanism is applied to the feature maps within each spatial subregion.
[0044] The SimAM attention mechanism consists of a linear transformation and a Softmax function to calculate the correlation between input features and generate attention weights. In order to better achieve attention, the attention mechanism requires important neurons. SimAM defines the minimum energy formula as:
[0045]
[0046] in, represents the minimum energy, t represents the target neuron of the input feature, represents the average value of all neuron activation values, δ 2 represents the variance of all neuron activation values, and λ represents the regularization coefficient.
[0047] According to the characteristics of the attention mechanism, the features are enhanced to obtain:
[0048]
[0049] Among them, E represents the neurons of the feature map Grouping on channels, X represents input features, X∈R C×H×W The purpose of adding sigmoid is to limit the excessively large values in E.
[0050] Through the dynamic weight adjustment mechanism, the spatial attention weights generated by SimAM are adjusted according to the output of the motion state detection module to enhance the model's perception of the target spatial features.
[0051] (3) Dynamic weight adjustment mechanism
[0052] The dynamic weight adjustment mechanism includes:
[0053] (a) Analyze the motion state of the aircraft target in real time through the motion state detection module, including acceleration, speed change and direction change;
[0054] (b) Dynamically adjust the attention weights of the time and space dimensions according to the motion state to enhance the model's adaptability to dynamic targets. The specific adjustment formula is:
[0055] α t =α0+β·Δv
[0056] β t =β0+γ·Δθ
[0057] Among them, α t and β t Represent the attention weights of the time dimension and the spatial dimension respectively, α0 and β0 are the initial weights, Δv is the speed change of the target, Δθ is the direction change of the target, β and γ are the coefficients of the target speed change and direction change respectively;
[0058] Under the dynamic weight adjustment mechanism, the feature sequence processed by the temporal attention module is element-wise multiplied with the feature map processed by the spatial attention module. This process not only integrates temporal and spatial information but also ensures that the weight distribution adaptively matches the target's motion state through the dynamic weight adjustment mechanism. The resulting enhanced feature map incorporates the dynamically adjusted temporal and spatial information, enabling more accurate capture of the aircraft's characteristics. This processed feature map is then fed into the Neck component for multi-scale feature fusion.
[0059] The present invention adds a motion state detection module after the YOLOv8 Head to analyze the motion parameters of the aircraft target in real time; this module is implemented by the following steps:
[0060] (a) Obtain the bounding box information of the aircraft target from the YOLOv8 detection head, including the position (center point coordinates), size (width and height), and confidence level;
[0061] (b) Calculate the displacement and velocity changes of the aircraft target in adjacent frames, and estimate the acceleration and direction changes. Displacement and velocity changes: Calculate the displacement (Δx, Δy) by the difference in the coordinates of the center point of the bounding box in adjacent frames, and calculate the velocity by combining the frame rate (time interval Δt). and acceleration Direction change: Calculate the direction change (Δθ) by the angle between the displacement vectors of adjacent frames, using the inverse tangent function: Then calculate the difference between the angles of adjacent frames.
[0062] Step 3: Use the data in step (1) to train the YOLOv8-TLSA aircraft target detection model. During the training process, combine the WIoU loss function to optimize the YOLOv8-TLSA aircraft target detection model parameters to obtain the trained YOLOv8-TLSA aircraft target detection model; the main parameters during training are: batch size is 16, training rounds are 100 epochs, and image size is 640*640.
[0063] The WIoU (wise intersection over union) loss function is used to enhance the generalization ability of the model. WIoU not only weakens the excessive penalty of geometric factors on the model, but also reduces the intervention in model training.
[0064] Construct distance attention and obtain WIoU with a two-layer attention mechanism:
[0065] L WIoUv1 =R WIoU L IoU
[0066]
[0067] Among them, L WIoUv1 represents the loss of WIoUv1 with two-layer attention mechanism; R WIoU represents the normalized distance between the center points of two anchor boxes; L IoU represents the bounding box regression loss; x represents the horizontal coordinate of the anchor box; x gt represents the horizontal coordinate of the target frame; y represents the vertical coordinate of the anchor frame; gt Represents the vertical coordinate of the target box; W g Represents the width of the minimum bounding rectangle of the prediction box, H g The height of the minimum bounding rectangle representing the ground truth box.
[0068] The WIoU loss function comprehensively considers the relative position and size differences between targets and incorporates adaptive adjustment of weight coefficients, significantly improving the flexibility and robustness of aircraft detection and demonstrating its unique advantages in aircraft detection.
[0069] Step 4: The obtained aircraft target detection model based on YOLOv8-TLSA is combined with the DeepSORT algorithm to obtain the aircraft target tracking model based on YOLOv8-TLSA.
[0070] DeepSORT uses deep learning methods to extract target features through convolutional neural networks and match these features with trackers. DeepSORT takes the detected bounding box, confidence score, and features as input to predict the category and position. After obtaining the predicted trajectory, it combines the intersection over union (IOU) and the Hungarian algorithm to match the target detection results with the predicted trajectory.
[0071] The above is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solutions and inventive concepts of the present invention, should be covered by the scope of protection of the present invention.
Claims
1. An aircraft target tracking method based on YOLOv8-TLSA, characterized in that: include: (1) Use the selected dataset to obtain image data, convert it into YOLO training format, and divide it into training set and test set; (2) Construct a YOLOv8-TLSA aircraft target detection model. This model uses YOLOv8 as the backbone network. The spatiotemporal attention mechanism and dynamic weight adjustment mechanism are introduced between the backbone and neck of YOLOv8 in sequence. A motion state detection module is added after the head. The motion state detection module identifies the motion state of the aircraft target, and the dynamic weight adjustment mechanism adjusts the attention weights of the time dimension and space dimension in the spatiotemporal attention mechanism according to the motion state of the aircraft target. (3) Using the data in step (1) to train the YOLOv8-TLSA aircraft target detection model, during the training process, the WIoU loss function is combined to optimize the YOLOv8-TLSA aircraft target detection model parameters to obtain the trained YOLOv8-TLSA aircraft target detection model; (4) Acquire the image data of the aircraft target in real time, and use the YOLOv8-TLSA aircraft target detection model as the target detector of the DeepSORT algorithm to track the aircraft target.
2. The aircraft target tracking method based on YOLOv8-TLSA according to claim 1, characterized in that In step (1), the UA-DETRAC public dataset is selected and converted into the YOLO training format using Python code. The image data is randomly divided into training and test sets in a ratio of 8:2 for training the YOLOv8-TLSA aircraft target detection model.
3. The aircraft target tracking method based on YOLOv8-TLSA according to claim 1, characterized in that The spatiotemporal attention mechanism includes a time dimension attention module and a space dimension attention module; the time dimension attention module adopts a TCN-LSTM fusion module, and the space dimension attention module adopts a SimAM attention mechanism.
4. The aircraft target tracking method based on YOLOv8-TLSA according to claim 3, characterized in that The attention weight adjustment process of the time dimension of the TCN-LSTM fusion module is as follows; YOLOv8's backbone network extracts the feature map of each frame to form a feature sequence. This feature sequence is input into the TCN-LSTM fusion module to obtain the initial weights of the time dimension. The dynamic weight adjustment mechanism adjusts the weights output by the TCN-LSTM fusion network based on the motion state of the aircraft target. The adjusted time dimension weights are applied to the feature map of each frame, assigning a weight based on the time context to the aircraft target features in each frame.
5. The aircraft target tracking method based on YOLOv8-TLSA according to claim 4, characterized in that The attention weight adjustment process of the spatial dimension of the SimAM attention mechanism is as follows: The feature map of each frame is extracted from the backbone network of YOLOv8, and the feature map is divided into multiple spatial sub-regions through group convolution. The SimAM attention mechanism is applied to the feature map in each spatial sub-region to generate spatial attention. The dynamic weight adjustment mechanism is used to adjust the spatial attention weight generated by SimAM according to the motion state of the aircraft target.
6. The aircraft target tracking method based on YOLOv8-TLSA according to claim 1, characterized in that The dynamic weight adjustment mechanism adjusts the attention weight formula of the time dimension and the spatial dimension as follows: a t =α0+β·Δv b t =β0+γ·Δθ Among them, α t and β t They represent the attention weights of the time dimension and the spatial dimension respectively, α0 and β0 are the initial weights, Δv is the speed change of the target, Δθ is the direction change of the target, β and γ are the coefficients of the target speed change and direction change, respectively.
7. The aircraft target tracking method based on YOLOv8-TLSA according to claim 6, characterized in that The calculation method of the speed change and direction change of the target is as follows: (a) Obtain the bounding box information of the aircraft target from the YOLOv8-TLSA detection head, including the bounding box center position, size and confidence; (b) The displacement is calculated by the difference in the coordinates of the center points of the bounding boxes in adjacent frames, and the speed and acceleration are calculated based on the frame rate. The direction change is calculated by the angle between the displacement vectors of adjacent frames.