A street view vehicle target tracking method and device

Abstract

This invention relates to a method and apparatus for street scene vehicle target tracking. The invention first acquires street scene vehicle video; then, it uses a strongsort target detector to detect targets in each video frame; it sets high-confidence thresholds and low-confidence thresholds, and uses these thresholds to filter the targets detected by the strongsort target detector; it performs an IoU operation between the detected targets in the current video frame with a confidence level greater than the low-confidence threshold and the tracked targets in the previous frame, and matches the detected targets that meet the conditions with the tracked targets in the previous frame; the matching results are fused with the detected targets filtered using the high-confidence threshold to obtain the tracked targets in the current video frame. This invention can significantly reduce the problems of missed tracking and false tracking in algorithms, and improve detection accuracy.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method and apparatus for tracking street scene vehicles based on a dynamic confidence threshold using improved strongsort. Background Technology

[0002] In recent years, with the rapid development of intelligent driving technology, street scene vehicle target tracking has made significant progress. However, due to the blurry image quality of street scene videos, the large number of small targets, and severe target occlusion, target tracking in street scene videos presents numerous challenges.

[0003] Current street scene object tracking primarily employs deep learning methods, with algorithms such as deepsort, bytetrack, and strongsort being relatively mature and widely used in multi-object tracking. Among these, strongsort currently boasts the best detection performance. However, the strongsort algorithm still suffers from issues of missed detections and false detections. Summary of the Invention

[0004] This invention addresses the technical problems existing in the prior art by providing a method and apparatus for street scene vehicle target tracking based on an improved strongsort dynamic confidence threshold, which greatly reduces the problems of missed tracking and false tracking in the algorithm and improves the detection accuracy.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0006] In a first aspect, the present invention provides a street scene vehicle target tracking method, comprising:

[0007] Obtain street view vehicle videos;

[0008] A strongsort object detector is used to detect objects in each video frame.

[0009] Set a high confidence threshold and a low confidence threshold, and use the high confidence threshold and low confidence threshold to filter the targets detected by the strongsort target detector;

[0010] The detected targets with a confidence level greater than the low confidence threshold in the current video frame are compared with the tracked targets in the previous frame using an IoU operation to obtain l. iou And match the detected target that satisfies the following formula with the tracked target of the previous frame;

[0011] -ln(1-0.8×l iou )-ln(1-score)≥1.2

[0012] Where score is the confidence level of the detected target in the current video frame image;

[0013] The matching results are fused with the detected targets filtered using a high confidence threshold to obtain the tracking target of the current video frame image.

[0014] Furthermore, the loss function of the strongsort target detector is Hybrid Alpha IOU, adversarial example augmentation is used during model training, and the model structure includes a transformer structure.

[0015] Furthermore, the matching process includes updating, predicting, and matching the tracked target using Kalman filtering and the Hungarian algorithm.

[0016] Furthermore, the fusion output includes:

[0017] The matching results are subjected to IoU and RE-ID operations with the detected targets filtered using a high confidence threshold. The tracked targets and the detected targets are deduplicated and then merged for output.

[0018] Secondly, the present invention also provides a street view vehicle target tracking device, comprising:

[0019] Input module, acquire street view vehicle video;

[0020] The target detection module uses a strongsort target detector to perform target detection on each video frame image;

[0021] The filtering module sets a high confidence threshold and a low confidence threshold, and uses the high confidence threshold and low confidence threshold to filter the targets detected by the strongsort target detector;

[0022] The matching module performs an IoU operation on the detected targets in the current video frame with a confidence level greater than the low confidence threshold and the tracked targets in the previous frame to obtain l. iou And match the detected target that satisfies the following formula with the tracked target of the previous frame;

[0023] -ln(1-0.8×l iou )-ln(1-score)≥1.2

[0024] Where score is the confidence level of the detected target in the current video frame image;

[0025] The fusion module fuses the matching results with the detected targets filtered using a high confidence threshold to output the tracking target of the current video frame image.

[0026] Furthermore, the loss function of the strongsort target detector is Hybrid Alpha IOU, adversarial example augmentation is used during model training, and the model structure includes a transformer structure.

[0027] Furthermore, the matching process includes updating, predicting, and matching the tracked target using Kalman filtering and the Hungarian algorithm.

[0028] Furthermore, the fusion output includes: performing IoU and RE-ID operations on the matching results and the detected targets filtered using a high confidence threshold, and merging the tracked targets and the detected targets after deduplication.

[0029] Thirdly, the present invention provides an electronic device, comprising:

[0030] Memory, used to store computer software programs;

[0031] A processor is used to read and execute the computer software program, thereby implementing the street scene vehicle target tracking method described in the first aspect of the present invention.

[0032] Fourthly, the present invention provides a non-transitory computer-readable storage medium storing a computer software program for implementing a street scene vehicle target tracking method as described in the first aspect of the present invention.

[0033] The beneficial effects of this invention are: addressing the problem of poor detection by the strongsort target detector, this patent changes the IOU loss of the original target detector to hybrid alpha IOU, adds adversarial example enhancement in data preprocessing, and incorporates the latest transformer structure into the model structure to improve basic detection capabilities.

[0034] The issues of missed and false tracking in strongsort are largely related to the confidence threshold of the target detector. When the confidence threshold is too high, some targets will be missed; when the confidence threshold is too low, some targets will be falsely detected. These missed and false detections directly impact the tracker's performance. To address this problem, this patent, through extensive data experiments, found that existing target detectors have virtually no false detections when the confidence level is above a certain threshold, and virtually no missed detections when the confidence level is below a certain threshold. Therefore, the introduction of a dynamic confidence threshold strategy largely solves the problems of false and missed tracking in the tracker. Attached Figure Description

[0035] Figure 1 This is a schematic flowchart of a street scene vehicle target tracking method provided in an embodiment of the present invention;

[0036] Figure 2This is a schematic diagram of a street scene vehicle target tracking device provided in an embodiment of the present invention;

[0037] Figure 3 A schematic diagram of an embodiment of the electronic device provided in this invention;

[0038] Figure 4 This is a schematic diagram of an embodiment of a computer-readable storage medium provided in this invention. Detailed Implementation

[0039] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0040] Figure 1 This is a schematic flowchart of a street scene vehicle target tracking method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes:

[0041] S1, acquire street view vehicle video;

[0042] S2, a strongsort object detector is used to perform object detection on each video frame image;

[0043] S3, set a high confidence threshold and a low confidence threshold, and use the high confidence threshold and the low confidence threshold to filter the detected targets of the strongsort target detector;

[0044] S4, perform an IoU operation on the detected targets in the current video frame with a confidence level greater than the low confidence threshold and the tracked targets in the previous frame to obtain l. iou And match the detected target that satisfies the following formula with the tracked target of the previous frame;

[0045] -ln(1-0.8×l iou )-ln(1-score)≥1.2

[0046] Where score is the confidence level of the detected target in the current video frame image;

[0047] S5, the matching result is fused with the detected target filtered using a high confidence threshold to obtain the tracking target of the current video frame image.

[0048] The issues of missed and false tracking in strongsort are largely related to the confidence threshold of the target detector. When the confidence threshold is too high, some targets will be missed; when the confidence threshold is too low, some targets will be falsely detected. These missed and false detections directly impact the tracker's performance. To address this problem, this patent, through extensive data experiments, found that existing target detectors have virtually no false detections when the confidence level is above a certain threshold, and virtually no missed detections when the confidence level is below a certain threshold. Therefore, the introduction of a dynamic confidence threshold strategy largely solves the problems of false and missed tracking in the tracker.

[0049] In a preferred embodiment, the loss function of the strongsort target detector is hybrid alpha IOU, adversarial example augmentation is used during model training, and the model structure includes a transformer structure.

[0050] To address the issue of poor detection performance of the strongsort object detector, this patent modifies the original object detector's IOU loss to hybrid alpha IOU, incorporates adversarial example enhancement in data preprocessing, and adds the latest transformer structure to the model structure to improve basic detection capabilities.

[0051] In a preferred embodiment, the matching includes: updating, predicting, and matching the tracked target using Kalman filtering and the Hungarian algorithm.

[0052] As a preferred embodiment, the fusion output includes:

[0053] The matching results are subjected to IoU and RE-ID operations with the detected targets filtered using a high confidence threshold. The tracked targets and the detected targets are deduplicated and then merged for output.

[0054] Figure 2 This is a schematic diagram of a street scene vehicle target tracking device provided in an embodiment of the present invention. Figure 2 As shown, the device includes:

[0055] Input module, acquire street view vehicle video;

[0056] The target detection module uses a strongsort target detector to perform target detection on each video frame image;

[0057] The filtering module sets a high confidence threshold and a low confidence threshold, and uses the high confidence threshold and low confidence threshold to filter the targets detected by the strongsort target detector;

[0058] The matching module performs an IoU operation on the detected targets in the current video frame with a confidence level greater than the low confidence threshold and the tracked targets in the previous frame to obtain l. iou And match the detected target that satisfies the following formula with the tracked target of the previous frame;

[0059] -ln(1-0.8×l iou )-ln(1-score)≥1.2

[0060] Where score is the confidence level of the detected target in the current video frame image;

[0061] The fusion module fuses the matching results with the detected targets filtered using a high confidence threshold to output the tracking target of the current video frame image.

[0062] Furthermore, the loss function of the strongsort target detector is Hybrid Alpha IOU, adversarial example augmentation is used during model training, and the model structure includes a transformer structure.

[0063] Furthermore, the matching process includes updating, predicting, and matching the tracked target using Kalman filtering and the Hungarian algorithm.

[0064] Furthermore, the fusion output includes: performing IoU and RE-ID operations on the matching results and the detected targets filtered using a high confidence threshold, and merging the tracked targets and the detected targets after deduplication.

[0065] Please see Figure 3 , Figure 3 This is a schematic diagram illustrating an embodiment of the electronic device provided in this invention. For example... Figure 3 As shown, this embodiment of the invention provides an electronic device 500, including a memory 510, a processor 520, and a computer program 511 stored in the memory 520 and executable on the processor 520. When the processor 520 executes the computer program 511, it performs the following steps:

[0066] S1, acquire street view vehicle video;

[0067] S2, a strongsort object detector is used to perform object detection on each video frame image;

[0068] S3, set a high confidence threshold and a low confidence threshold, and use the high confidence threshold and the low confidence threshold to filter the detected targets of the strongsort target detector;

[0069] S4, perform an IoU operation on the detected targets in the current video frame with a confidence level greater than the low confidence threshold and the tracked targets in the previous frame to obtain l.iou And match the detected target that satisfies the following formula with the tracked target of the previous frame;

[0070] -ln(1-0.8×l iou )-ln(1-score)≥1.2

[0071] Where score is the confidence level of the detected target in the current video frame image;

[0072] S5, the matching result is fused with the detected target filtered using a high confidence threshold to obtain the tracking target of the current video frame image.

[0073] Please see Figure 4 , Figure 4 This is a schematic diagram illustrating an embodiment of a computer-readable storage medium provided by an embodiment of the present invention. For example... Figure 4 As shown, this embodiment provides a computer-readable storage medium 600, on which a computer program 611 is stored. When the computer program 611 is executed by a processor, it performs the following steps:

[0074] S1, acquire street view vehicle video;

[0075] S2, a strongsort object detector is used to perform object detection on each video frame image;

[0076] S3, set a high confidence threshold and a low confidence threshold, and use the high confidence threshold and the low confidence threshold to filter the detected targets of the strongsort target detector;

[0077] S4, perform an IoU operation on the detected targets in the current video frame with a confidence level greater than the low confidence threshold and the tracked targets in the previous frame to obtain l. iou And match the detected target that satisfies the following formula with the tracked target of the previous frame;

[0078] -ln(1-0.8×l iou )-ln(1-score)≥1.2

[0079] Where score is the confidence level of the detected target in the current video frame image;

[0080] S5, the matching result is fused with the detected target filtered using a high confidence threshold to obtain the tracking target of the current video frame image.

[0081] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0082] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0083] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0084] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0085] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0086] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0087] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method of street view vehicle target tracking, the method comprising: The method comprises the following steps: obtaining a street vehicle video; performing target detection on each video frame image by using a strongsort target detector; setting a high confidence threshold and a low confidence threshold, and screening the detected targets of the strongsort target detector by using the high confidence threshold and the low confidence threshold; The detected target in the current video frame image with the confidence greater than the low confidence threshold is subjected to an iou operation with the tracking target in the previous frame to obtain and the detected target satisfying the following formula is matched with the tracking target in the previous frame: wherein score is the confidence of the detected target in the current video frame image; fusing the matching result with the detected target screened by using the high confidence threshold to output a tracking target of the current video frame image. The loss function of the strongsort target detector is a hybrid alpha iou, and the model training adopts adversarial sample enhancement, and the model structure contains a transformer structure.

2. The method of claim 1, wherein, The matching comprises updating, predicting and matching the tracking target by using Kalman filtering and the Hungarian algorithm.

3. The method of claim 1, wherein, The fusion output comprises: performing iou operation and RE-ID operation on the tracking target and the detected target after deduplication, and then merging and outputting.

4. A street view vehicle target tracking apparatus characterized by comprising: The method comprises the following steps: an input module for obtaining a street vehicle video; a target detection module for performing target detection on each video frame image by using a strongsort target detector; a screening module for setting a high confidence threshold and a low confidence threshold, and screening the detected targets of the strongsort target detector by using the high confidence threshold and the low confidence threshold; The matching module performs an iou operation on the detection target in the current video frame image with a confidence greater than a low confidence threshold and the tracking target in the previous frame to obtain a matching result and matches the detection target satisfying the following formula with the tracking target in the previous frame; wherein score is the confidence of the detected target in the current video frame image; a fusion module for fusing the matching result with the detected target screened by using the high confidence threshold to output a tracking target of the current video frame image. The loss function of the strongsort target detector is a hybrid alpha iou, and the model training adopts adversarial sample enhancement, and the model structure contains a transformer structure.

5. The apparatus of claim 4, wherein, The matching comprises updating, predicting and matching the tracking target by using Kalman filtering and the Hungarian algorithm.

6. The apparatus of claim 4, wherein, The fusion output comprises:

7. An electronic device, comprising: performing iou operation and RE-ID operation on the tracking target and the detected target after deduplication, and then merging and outputting. The method comprises the following steps: a memory for storing a computer software program; 8. A non-transitory computer-readable storage medium, comprising: a processor for reading and executing the computer software program, thereby realizing the street vehicle target tracking method according to any one of claims 1-3. The storage medium stores a computer software program for realizing the street vehicle target tracking method according to any one of claims 1-3.

Images