Long-tail data distribution-oriented target detection and tracking method
By improving the YOLOv7 model and Sort tracker through a dynamic resampling mechanism and a long-tail loss function, the problem of low detection accuracy of traditional algorithms in complex environments of special unmanned vehicles is solved, and efficient target detection and tracking of long-tail data distribution is achieved.
Patent Information
- Application Number
- CN202510793246.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-13
- Publication Date
- 2025-09-26
AI Technical Summary
Traditional target detection algorithms have low detection accuracy in the complex environment of special unmanned vehicles and are difficult to effectively identify targets in long-tail data distribution.
A dynamic resampling mechanism and long-tail loss function are designed, the YOLOv7 model and Sort tracker are improved, and the detection and tracking effects of tail categories are improved through dynamic threshold control and priority sorting mechanism.
The model's learning efficiency and recognition effect on tail categories are improved, and the accuracy and tracking ability of target detection are enhanced, especially in complex environments of special unmanned vehicles.
Smart Images
Figure CN120708190A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of unmanned vehicle vision and pattern recognition technology, and in particular to a target detection and tracking method for long-tail data distribution. Background Art
[0002] In today's world, science and technology are gaining increasing attention worldwide, and various high-tech technologies are gradually being applied to various industries. The rapid development of artificial intelligence and big data, in particular, has brought about tremendous changes to traditional industrial technologies. Special vehicles are also gradually evolving from traditional mechanization to intelligentization.
[0003] With the development of intelligent specialized vehicles, intelligent driving technology for specialized unmanned vehicles has gradually attracted attention. A key challenge in intelligent driving technology is how to enable unmanned vehicles to automatically identify and track targets within their field of view. Compared to ordinary roads, specialized unmanned vehicles are often used on non-general-purpose roads. The target samples in these application scenarios differ significantly from civilian intelligent driving datasets, and the target sample data exhibits a long-tail distribution.
[0004] Traditional methods for detecting various targets primarily employ algorithms such as HOG+SVM and DPM. However, these traditional target detection algorithms can only detect simple targets with distinct features, and struggle to accurately detect targets in complex environments. Furthermore, traditional target detection algorithms also suffer from low detection accuracy. Consequently, traditional target detection methods struggle to detect targets in specific scenarios. Summary of the Invention
[0005] This application provides a target detection and tracking method for long-tail data distribution, which can be used to solve the technical problem of low accuracy of traditional target detection methods.
[0006] This application provides a target detection and tracking method for long-tail data distribution, which includes:
[0007] Step 1: Collect target image data in the working environment of special unmanned vehicles, annotate and construct a target detection dataset suitable for special unmanned vehicles.
[0008] Step 2: Design a dynamic resampling mechanism to oversample categories with few instances in the training set and undersample categories with many instances.
[0009] Step 3: Improve the YOLOv7 model, add a long-tail loss function, and design a target detection network.
[0010] Step 4: Based on the established image dataset, train the improved YOLOv7 target detection network.
[0011] Step 5: Improve the Sort tracker by adding dynamic threshold control and multi-level matching mechanism to design a target tracker.
[0012] Step 6: Input the trained object detection network into the video stream to be tested, and generate the positions and categories of all objects in the current frame.
[0013] Step 7: Input the current frame target position and category into the tracker, which outputs the ID, category, and position of all targets in the current frame to achieve real-time tracking of all targets in the video stream.
[0014] Compared with the prior art, the present invention has the following beneficial effects:
[0015] Compared with directly using the original dataset for training, this method generates more samples for the tail categories by designing a dynamic resampling mechanism, balancing the data distribution of the training set and improving the model's learning efficiency for the tail categories.
[0016] Compared with existing target tracking algorithms, this method introduces long-tail loss optimization into the YOLOv7 target detection network, improves the classification loss function, and greatly improves the recognition effect of the target detection model on tail category targets.
[0017] Compared with existing trackers, this method improves the tracker's tracking ability for small samples. By introducing dynamic threshold control and priority sorting matching mechanism, it relaxes the discrimination requirements for tail category trajectories and improves the tracking effect of tail category targets. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 An overall flow chart of the method provided in the embodiment of the present application;
[0019] Figure 2 A schematic diagram of dynamic sample ratio calculation provided in an embodiment of the present application;
[0020] Figure 3 A schematic diagram of calculating the classification penalty compensation coefficient provided in an embodiment of the present application;
[0021] Figure 4 This is a flow chart of the target detection and tracking algorithm provided in the embodiments of the present application. DETAILED DESCRIPTION
[0022] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.
[0023] The following first introduces the embodiments of the present application with reference to the accompanying drawings.
[0024] This application provides a target detection and tracking method for long-tail data distribution.
[0025] Step 1: Collect target image data in the special unmanned vehicle working scene, annotate and construct a target detection dataset suitable for special unmanned vehicles.
[0026] Step 1.1: Collect target images of people, transport vehicles, unmanned vehicles, motorcycles, robot dogs, cones, and buildings in the working environment of special unmanned vehicles;
[0027] Step 1.2: Crop the images to a uniform size and manually annotate each image with the object bounding box and object category using the LabelImg tool to generate image labels. Split the annotated images and corresponding labels into training, validation, and test sets in a 7:2:1 ratio.
[0028] Step 2: Design a dynamic resampling mechanism to oversample categories with fewer instances and undersample categories with more instances.
[0029] Step 2.1: Determine a dynamic resampling mechanism to dynamically adjust the sampling probability based on the class frequency; the sampling probability of each class is as follows:
[0030]
[0031] Among them, r i represents the sampling probability of the i-th category; τ s represents the sampling coefficient, which controls the resampling probability; c represents the number of categories; N i Represents the number of instances of the i-th category in the dataset;
[0032] Step 2.2: Traverse all images in the training set, and for the jth image, obtain the category with the smallest number of instances in the dataset, and calculate the sampling probability of the current category as the sampling probability r of the corresponding image. j ; if r j >0, the image is r j The probability of randomly rotating, horizontally flipping and cropping generates new data and adds it to the training set; if r j ≤0, the image is |r j The probability of | being deleted from the training set.
[0033] Step 3: Improve the YOLOv7 model, add a long-tail loss function, and design a target detection network.
[0034] Step 3.1: Introduce dynamic sample ratio to measure the impact of head samples on tail samples. The calculation is as follows:
[0035]
[0036] Among them, M ijrepresents the proportion of samples of the jth category to the ith category; p represents the influence coefficient;
[0037] Step 3.2: Introduce the classification penalty compensation coefficient, which is calculated as follows:
[0038]
[0039] Among them, C ij represents the penalty coefficient of the jth category to the ith category; σ i Represents the probability of the i-th category output by the network; σ j represents the probability of the jth category output by the network; q represents the compensation coefficient;
[0040] Step 3.3: Use the long-tail loss function to replace the cross entropy loss as the classification loss of the target detection network, which is calculated as follows:
[0041]
[0042] Among them, S ij =M ij *C ij ; Represents the characteristics of the i-th category output by the network; y i Indicates the label value of the i-th category.
[0043] Step 4: Train the improved YOLOv7 object detection network on the established image dataset.
[0044] Step 4.1: Train the object detection network on the training set and monitor the loss curve in real time;
[0045] Step 4.2: Save the checkpoint weights every 10 epochs of training, and calculate the mAP evaluation model accuracy on the validation set. If the result is the highest historical mAP value, the corresponding model is used as the best mAP model.
[0046] Step 4.3: After training is completed, save the training results of the last epoch as the final model; calculate the mAP of the final model and the best mAP model on the test set to verify the model accuracy, and use the model with the highest index for the target detection network.
[0047] Step 5: Improve the Sort tracker by adding dynamic threshold control and priority sorting mechanism to determine the target tracker.
[0048] Step 5.1: Set the dynamic threshold coefficient λ, set up a dedicated channel for the tail category trajectories, and appropriately relax the tracker's confidence threshold, matching threshold, and maximum survival time for the tail category; the dynamic threshold coefficient of the i-th category is calculated as follows:
[0049]
[0050] Set the dynamic confidence threshold for category i to:
[0051] T Ci =T C -0.2(1-λ i )
[0052] Among them, T C is the basic confidence threshold, and the dynamic confidence threshold is reduced by 0.2 at most;
[0053] Set the dynamic matching threshold for category i to:
[0054] T Mi λT M (0.9+λ i )
[0055] Among them, T M The dynamic matching threshold is reduced by up to 10% from the basic matching threshold.
[0056] Set the dynamic maximum survival time of the i-th category to:
[0057] T Si =T S (2-λ i )
[0058] Among them, T S The maximum survival time is the basic one, and the dynamic maximum survival time is extended to twice of it;
[0059] Step 5.2: Design a prioritization mechanism to prioritize trajectories with shorter survival times:
[0060] Before matching the target frame of each frame of data, all existing tracks in the tracker are sorted from small to large according to their survival time to form a priority queue; when two tracks have the same survival time, the track with the smaller number of category instances has a higher priority; the tracks are matched with the detection box of the current frame one by one in the order of the priority queue.
[0061] Step 6: The video stream to be tested is input into the trained object detection network to generate the bounding box information and categories of all objects in the current frame.
[0062] Step 7: Input the current frame target position and category into the tracker, which outputs the ID, category, and position of all targets in the current frame to achieve real-time tracking of all targets in the video stream.
[0063] Step 7.1: The tracker receives the target bounding box and category of the current frame, filters the low-confidence detection results according to the dynamic confidence threshold, and removes redundant boxes through non-maximum suppression (NMS).
[0064] Step 7.2: When the first frame image comes in, initialize all target detection boxes to the track state, assign a unique ID, and set the survival time to 0;
[0065] Step 7.3: When a subsequent frame comes in, use the Kalman filter to predict the target state of the current frame based on the trajectory of the previous frame, including the center coordinates, height, width and its velocity components;
[0066] Step 7.4: Use the Hungarian algorithm to find the optimal match and maximize the total IoU value between the predicted track and the detection box. Output three matching results based on the dynamic matching threshold: a successfully matched predicted track and detection box pair; an unmatched predicted track; and an unmatched detection box.
[0067] Step 7.5: For the predicted trajectory and detection box pairs that are successfully matched, use the position information of the detection box to update the Kalman filter state of the corresponding trajectory and reset the survival time; for the unmatched predicted trajectory, add 1 to the survival time of the corresponding trajectory. If the survival time exceeds the dynamic maximum time, the trajectory information is deleted; for the unmatched detection box, initialize a new trajectory, assign a unique ID, and set the survival time to 0;
[0068] Step 7.6: Output the status information of all tracks in the current frame, including ID, category, and bounding box center coordinates, height, and width.
[0069] This application focuses on the problem of target detection and tracking under long-tail data distribution, improves the algorithm model in the public data domain, migrates it to the specific data domain, develops a novel, efficient and integrated target detection and tracking model, and cooperates with unmanned platforms for deployment, testing, joint debugging and demonstration, achieving good results.
[0070] The above-described embodiments of the present application do not constitute a limitation on the scope of protection of the present application.
Claims
1. A target detection and tracking method for long-tail data distribution, characterized in that: The method comprises: Step 1: Collect target image data in the working environment of special unmanned vehicles, annotate and construct a target detection dataset suitable for special unmanned vehicles; Step 2: Design a dynamic resampling mechanism to oversample categories with few instances in the training set and undersample categories with many instances; Step 3: Improve the YOLOv7 model, add a long-tail loss function, and design an object detection network; Step 4: Based on the established image dataset, train the improved YOLOv7 target detection network; Step 5: Improve the Sort tracker by adding dynamic threshold control and multi-level matching mechanism to design a target tracker. Step 6: The video stream to be tested is input into the trained object detection network to generate the positions and categories of all objects in the current frame; Step 7: Input the current frame target position and category into the tracker, which outputs the ID, category, and position of all targets in the current frame to achieve real-time tracking of all targets in the video stream.
2. The method according to claim 1, characterized in that Step 1: Collect target image data in the special unmanned vehicle working scene, annotate and build a target detection dataset suitable for special unmanned vehicles, including: Step 1.1: Collect target images of people, transport vehicles, unmanned vehicles, motorcycles, robot dogs, cones, and buildings in the working environment of special unmanned vehicles; Step 1.2: Crop the images to a uniform size, annotate the object bounding box and object category for each image, and generate image labels; divide the annotated images and corresponding labels into training, validation, and test sets in a ratio of 7:2:
1.
3. The method according to claim 2, characterized in that Step 2: Design a dynamic resampling mechanism to oversample categories with fewer instances and undersample categories with more instances, including: Step 2.1: Determine a dynamic resampling mechanism to dynamically adjust the sampling probability based on the class frequency; the sampling probability of each class is as follows: Among them, r i represents the sampling probability of the i-th category; τ s represents the sampling coefficient, which controls the resampling probability; c represents the number of categories; N i Represents the number of instances of the i-th category in the dataset; Step 2.2: Traverse all images in the training set, and for the jth image, obtain the category with the smallest number of instances in the dataset, and calculate the sampling probability of the current category as the sampling probability r of the corresponding image. j ; if r j >0, the image is r j The probability of randomly rotating, horizontally flipping and cropping generates new data and adds it to the training set; if r j ≤0, the image is |r j The probability of | being deleted from the training set.
4. The method according to claim 3, characterized in that Step 3: Improve the YOLOv7 model, add a long-tail loss function, and design a target detection network, including: Step 3.1: Introduce dynamic sample ratio to measure the impact of head samples on tail samples. The calculation is as follows: Among them, M ij represents the proportion of samples of the jth category to the ith category; p represents the influence coefficient; Step 3.2: Introduce the classification penalty compensation coefficient, which is calculated as follows: Among them, C ij represents the penalty coefficient of the jth category to the ith category; σ i Represents the probability of the i-th category output by the network; σ j represents the probability of the jth category output by the network; q represents the compensation coefficient; Step 3.3: Use the long-tail loss function to replace the cross entropy loss as the classification loss of the target detection network, which is calculated as follows: Among them, S ij =M ij *C ij ; Represents the characteristics of the i-th category output by the network; y i Indicates the label value of the i-th category.
5. The method according to claim 4, characterized in that Step 4: Train the improved YOLOv7 target detection network on the established image dataset, including: Step 4.1: Train the object detection network on the training set and monitor the loss curve in real time; Step 4.2: Save the checkpoint weights every 10 epochs of training and calculate the mAP evaluation model accuracy on the validation set. If the result is the highest historical mAP value, the corresponding model is taken as the best mAP model. Step 4.3: After training is completed, save the training results of the last epoch as the final model; calculate the mAP of the final model and the best mAP model on the test set to verify the model accuracy, and use the model with the highest index for the target detection network.
6. The method according to claim 5, characterized in that Step 5: Improve the Sort tracker by adding dynamic threshold control and priority sorting mechanisms, and determine the target tracker, including: Step 5.1: Set the dynamic threshold coefficient λ, set up a dedicated channel for the tail category trajectories, and appropriately relax the tracker's confidence threshold, matching threshold, and maximum survival time for the tail category; the dynamic threshold coefficient of the i-th category is calculated as follows: Set the dynamic confidence threshold for category i to: T Ci =T C -0.2(1-λ i ) Among them, T C is the basic confidence threshold, and the dynamic confidence threshold is reduced by 0.2 at most; Set the dynamic matching threshold for category i to: T Mi =T M (0.9+λ i ) Among them, T M The dynamic matching threshold is reduced by up to 10% from the basic matching threshold. Set the dynamic maximum survival time of the i-th category to: T Si =T S (2-λ i ) Among them, T S The maximum survival time is the basic one, and the dynamic maximum survival time is extended to twice of it; Step 5.2: Design a prioritization mechanism to prioritize trajectories with shorter survival times: Before matching the target frame of each frame of data, all existing tracks in the tracker are sorted from small to large according to their survival time to form a priority queue; when two tracks have the same survival time, the track with the smaller number of category instances has a higher priority; the tracks are matched with the detection box of the current frame one by one in the order of the priority queue.
7. The method according to claim 6, characterized in that Step 7: Input the current frame target position and category into the tracker, which outputs the ID, category, and position of all targets in the current frame, achieving real-time tracking of all targets in the video stream, including: Step 7.1: The tracker receives the target bounding box and category of the current frame, filters low-confidence detection results based on a dynamic confidence threshold, and removes redundant boxes through non-maximum suppression. Step 7.2: When the first frame image comes in, initialize all target detection boxes to the track state, assign a unique ID, and set the survival time to 0; Step 7.3: When a subsequent frame comes in, use the Kalman filter to predict the target state of the current frame based on the trajectory of the previous frame, including the center coordinates, height, width and its velocity components; Step 7.4: Use the Hungarian algorithm to find the optimal match and maximize the total IoU value between the predicted track and the detection box. Output three matching results based on the dynamic matching threshold: a successfully matched predicted track and detection box pair; an unmatched predicted track; and an unmatched detection box. Step 7.5: For the predicted trajectory and detection box pairs that are successfully matched, use the position information of the detection box to update the Kalman filter state of the corresponding trajectory and reset the survival time; for the unmatched predicted trajectory, add 1 to the survival time of the corresponding trajectory. If the survival time exceeds the dynamic maximum time, the trajectory information is deleted; for the unmatched detection box, initialize a new trajectory, assign a unique ID, and set the survival time to 0; Step 7.6: Output the status information of all tracks in the current frame, including ID, category, and bounding box center coordinates, height, and width.