Anchors-free real-time multi-target tracking method based on joint detection and re-identification
By sharing computational resources within a deep learning model, anchorless single-stage multi-target tracking is achieved, solving the problem of slow inference speed in existing methods and improving the real-time performance and robustness of multi-target tracking.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG UNIV OF TECH
- Filing Date
- 2023-03-03
- Publication Date
- 2026-04-28
AI Technical Summary
Existing two-stage multi-target tracking methods suffer from slow inference speed and large model training requirements due to the lack of shared feature maps, making it difficult to achieve real-time multi-target tracking.
An anchorless real-time multi-target tracking method based on joint detection and re-identification is adopted. By constructing a deep learning model, including a feature extraction backbone network, an LSC attention module, a detection branch, and a re-identification branch, the computational load is shared to reduce inference time.
It improves learning efficiency, reduces inference time, and achieves good real-time performance, robustness, stability, and accuracy in multi-target tracking.
Smart Images
Figure CN117437260B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to computer vision multi-object tracking tasks, and more specifically to a single-stage multi-object tracking method based on association detection and re-identification. Background Technology
[0002] Multi-object tracking (MOT) has long been a goal in computer vision, aiming to estimate the trajectories of multiple objects in a video. Successfully solving this task will benefit many applications, such as action recognition, motion video analysis, elderly care, and human-computer interaction. In recent years, most methods have employed a two-stage approach: first, object detection algorithms locate the objects, and then a re-identification model matches and links them to an existing trajectory or generates a new trajectory. Although the two-stage approach has significantly improved multi-object tracking with the development of object detection and re-identification algorithms, it suffers from drawbacks such as slow inference speed and large training models because it does not share feature maps from the object detection and re-identification algorithms. Summary of the Invention
[0003] The purpose of this invention is to provide an anchorless multi-target tracking method based on association detection and re-identification, so as to overcome the technical problems mentioned in the background art.
[0004] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:
[0005] An anchorless real-time multi-target tracking method based on joint detection and re-identification is characterized by constructing a deep learning model for detection and tracking. The deep learning model includes a feature extraction backbone network, an LSC attention module, a detection branch, and a re-identification branch. Specific implementation steps include:
[0006] S1; Take the labeled image as input to the feature extraction backbone network to obtain the image feature pyramid output by the feature extraction backbone network. The feature extraction backbone network includes a deep residual network and a variant DLA parameter aggregation method.
[0007] S2: Upsample and downsample the upper and lower feature layers of the image feature pyramid respectively, and adjust the upper and lower feature layers to the size of the features of the middle layer to form a feature vector F∈R. L×S×C The feature vector F is input into the LSC attention module to obtain the feature vector F′ after feature enhancement.
[0008] S3: Input the feature vector F′ into the detection branch and the re-identification branch to obtain the location information and appearance features respectively;
[0009] S4: Learn based on the obtained prediction results, and perform loss supervision on the detection branch and re-identification branch to update the network parameters of the deep learning model;
[0010] S5: Take the detection results of the image by the best deep learning model after training and put them into the tracker for tracking.
[0011] In step S1, the deep residual network of the feature extraction backbone network is ResNet-34. A variant parameter aggregation method (DLA) is used in the ResNet-34 network to select different layers of the backbone network for different prediction levels. All convolutional layers in the upsampling module are replaced by deformable convolutional layers so that they can dynamically adjust the receptive field according to the size and pose of the object. The input image is propagated through the backbone network to obtain feature pyramids with downsampling rates of 1 / 32, 1 / 16, and 1 / 8.
[0012] The LSC attention module described in step S2 includes three types of attention:
[0013] First, scale-aware attention is used to dynamically fuse different features, as shown in Equation (1):
[0014]
[0015] Where f(·) is a linear function, approximating a 1×1 convolutional layer. It is a hard-sigmoid function;
[0016] Secondly, spatially aware attention based on fused features is used to focus on regions where spatial location and feature level coexist consistently. Considering the high dimensionality of features, it is decomposed into two steps: first, the attention learning is made sparse by compressing deformable convolution, and then features at different levels are aggregated at the same spatial location, as shown in formula (2):
[0017]
[0018] Where K is the number of sparse sampling locations, p k +Δp k To learn the spatial offset Δp k To focus on the shift position of the discrimination region, Δmk is the self-learned position p. k Importance scalar; both are from It is learned from the median level input features;
[0019] Finally, task-aware attention is used to achieve joint learning and inductive representation of different objects, as shown in Equation (3):
[0020]
[0021] in It is the feature patch of the c-th channel, [α] 1 ,α 2 ,β 1 ,β 2 ] T =θ(·) is a hyperfunction that learns to control the threshold of the activation function. θ(·) first performs global average pooling on L×S dimensions to reduce the dimensionality, then uses two fully connected layers and a normalization layer, and finally applies a unique sigmoid function to normalize the output to [-1,1].
[0022] The detection branch in step S3 includes three parallel predictors for the target detection part of the tracking task. Each predictor consists of a 256-channel convolution kernel with a kernel size of 3×3 and a convolution kernel with a kernel size of 1×1. The predictors include: a heatmap predictor, a bounding box size predictor and a center offset predictor.
[0023] The loss function in the heatmap predictor includes:
[0024] The heatmap predictor is responsible for predicting the center position of the object. The output feature map of the heatmap is H×W×1 in size. If the heatmap collapses with the center of the real object, the response of that position in the output feature map will be 1. The response of the heatmap predictor decreases exponentially as the distance between the position in the heatmap and the center of the target increases. For each ground truth (GT) box... Calculate the target center for and The position of the point on the feature map is then obtained by dividing it by the step size, i.e. In the heatmap, the heatmap response at position (x,y) is calculated as shown in formula (4):
[0025]
[0026] Where N represents the number of objects in the image, σ c Standard deviation;
[0027] The loss function is defined as pixel-level logistic regression with focus loss, as shown in Equation (5):
[0028]
[0029] in, This is a predicted heatmap, where α and β are preset loss parameters;
[0030] The loss functions for the frame size predictor and the center offset predictor include:
[0031] The center offset predictor is used to more accurately locate the target's position in the image; since the stride of the feature map is 4, it introduces a non-negligible error of up to 4 pixels; the box offset predictor estimates the continuous offset of each pixel relative to the target center to mitigate the impact on downsampling; the box size predictor is responsible for estimating the size of the target bounding box at each location.
[0032] The outputs of the box offset predictor and the size predictor are represented as and For each GT box Its size can be calculated as Similarly, the offset of GT can be calculated as The offset and size at the corresponding positions are expressed as follows: and Then, l1 loss is added to both predictors, as shown in Equation (6):
[0033]
[0034] Where, λ s The weighting coefficient is set to 0.1.
[0035] In step S3, the re-identification feature extraction in the re-identification branch involves applying a convolutional layer with 128 kernels on top of feature F′ to extract the identity embedding feature at each location. The re-identification loss function includes:
[0036] Re-identification features are learned through a classification task; all objects with the same identity in the training set are considered to be of the same class; the generated feature map is represented as E∈R 128×H×W The target re-identification features extracted from the target whose feature map center is located at (x,y) are:
[0037] For each GT box in the image Obtain the target center A feature vector can be extracted. Then, a fully connected layer and a softmax operation are used to map it to a class distribution vector P = {p(k), k ∈ [1, K]}; the one-hot representation of the GT class label is Li(k), and then the target re-identification loss is calculated as shown in Equation (7):
[0038]
[0039] Where K is the number of categories.
[0040] The loss function in the loss supervision described in step S4 includes:
[0041] The detection and re-identification branches are trained jointly by summing the losses; uncertainty loss is used to automatically balance the detection and re-identification tasks, as shown in equations (8) and (9):
[0042] L detection =L heat +L box (8)
[0043]
[0044] Here, w1 and w2 are learnable parameters that balance the two tasks; specifically, given an image with some objects and their corresponding IDs, heatmaps, bounding box offsets and size maps, and one-hot encoded representations of the objects are generated. These are compared with estimated metrics to obtain a loss for training the entire network.
[0045] The tracker described in step S5 includes an online tracking algorithm based on the Hungarian algorithm, specifically comprising:
[0046] During network inference time, the tracking system performs tracking initialization in the first frame; in subsequent frames, the trajectory set updates the bounding box, the score of the relevant detection, the appearance features, and the "track status" indicating whether the track is active and lost or removed.
[0047] Based on the detection box scores, the detection boxes are divided into high-scoring boxes and low-scoring boxes for separate processing. First, high-scoring boxes are used to detect and calculate a similarity matrix formed by the DIoU distance and inter-frame appearance similarity, thus associating the current frame with the previous trajectory set. Second, low-scoring boxes and tracking trajectories that did not match high-scoring boxes in the first detection are used to calculate a similarity matrix formed by the DIoU distance and inter-frame appearance similarity for matching. The similarity matrix is then globally distributed using the Hungarian algorithm to link the detections to the trajectory set.
[0048] For detection boxes that do not match the tracking trajectory but have a high enough score, a new tracking trajectory is created for them; for tracking trajectories that do not match the detection boxes, they are retained for 30 frames and matched again when they reappear; finally, all unmatched high-confidence detections are initialized as new trajectories.
[0049] This invention improves learning efficiency and reduces inference time by training two tasks in parallel within a single model. The core idea of this method is to perform object detection and identity embedding simultaneously within a single network, thereby reducing inference time by sharing most of the computational resources.
[0050] This invention inputs real-time video into its network, extracts high-resolution feature maps, enhances these features using an LSC attention module, and then applies the obtained feature maps to target detection and re-identification tasks, followed by multi-target tracking. The tracker proposed in this invention employs a two-stage tracking approach and achieves better tracking performance by proposing a novel similarity matrix. The proposed anchorless single-stage multi-target tracking network achieves excellent real-time performance and robustness in multi-target tracking.
[0051] The advantages of this invention are: high learning efficiency, reduced inference time, good real-time performance and robustness, and stability and accuracy when tracking multiple targets. Attached Figure Description
[0052] Figure 1 This is a flowchart of the method of the present invention;
[0053] Figure 2 This is a schematic diagram of the deep learning model of the present invention;
[0054] Figure 3 This is a schematic diagram of the LSC attention module of the present invention. Detailed Implementation
[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.
[0057] In one embodiment, such as Figure 1 As shown, an anchorless multi-object tracking method based on joint detection and re-identification is proposed. A deep learning model is constructed to detect positional information and appearance features. The obtained positional information and appearance features are then tracked by a tracker. The deep learning model includes a feature extraction backbone network, an LSC attention module, a detection branch, and a re-identification branch. The multi-object tracking method based on joint detection and re-identification includes:
[0058] Step S1: Take the labeled tracking image as input to the feature extraction backbone network to obtain the image features output by the backbone network. The feature extraction backbone network includes a deep residual network and a DLA parameter aggregation method.
[0059] For images requiring feature extraction, this implementation first performs random rotation, scaling, translation, and cropping operations on the image and its corresponding annotations before inputting them into the feature extraction backbone network. The backbone network includes a ResNet-34 deep residual network. A variant of DLA is used in the ResNet-34 network as a parameter aggregation method, selecting different layers of the backbone network for different prediction levels. DLA has more skip connections between low-level aggregations, similar to a Feature Pyramid Network (FPN). Furthermore, all convolutional layers in the upsampling module are replaced with deformable convolutional layers so that they can dynamically adjust their receptive fields according to the size and pose of the object. The input image undergoes front propagation through the backbone network to obtain feature pyramids with downsampling rates of 1 / 32, 1 / 16, and 1 / 8.
[0060] Step S2: Upsample and subsample the upper and lower layers of the image feature pyramid respectively, adjust the feature layer to the size of the middle layer, and obtain new image features F∈R. L×S×C The obtained image features F are used as input, such as Figure 3 The diagram shows attention processes involving scale perception, spatial perception, and task perception, with specific operations including:
[0061] First, a scale-aware attention mechanism is introduced to dynamically fuse different features, as shown in Equation (1):
[0062]
[0063] Where f(·) is a linear function, approximating a 1×1 convolutional layer. It is a hard-sigmoid function.
[0064] Another spatially aware attention module based on fusion features is applied to focus on regions where spatial location and feature level coexist consistently. Considering the high dimension of S, this module is decomposed into two steps: first, the attention learning is made sparse by compressing deformable convolutions, and then features at different levels are aggregated at the same spatial location, as shown in Equation (2):
[0065]
[0066] Where K is the number of sparse sampling locations, p k +Δp k To learn the spatial offset Δp k To focus on the shift position of the discrimination region, Δmk is the self-learned position p. k The importance scalar. Both are from It is learned from the median level input features.
[0067] To achieve joint learning and inductive representation of different objects, a task-aware attention is deployed at the end, as shown in formula (3):
[0068]
[0069] in It is the feature patch of the c-th channel, [α] 1 ,α 2 ,β 1 ,β 2 ] T =θ(·) is a hyperfunction that learns to control the threshold of the activation function. θ(·) first performs global average pooling on the L×S dimension to reduce the dimensionality, then uses two fully connected layers and a normalization layer, and finally applies a unique sigmoid function to normalize the output to [-1,1].
[0070] Step S3: The feature vector processed by the LSC module is passed into the detection branch and the re-identification branch. The detection branch includes three parallel predictors for the target detection part of the tracking task. Each predictor consists of a 256-channel convolution with a kernel size of 3×3 and a convolution with a kernel size of 1×1. The three predictors are: a heatmap predictor, a bounding box size predictor, and a center offset predictor. The re-identification branch applies a convolutional layer with 128 kernels on top of the features after feature enhancement in step 2 to extract the identity embedding features at each location.
[0071] Step S4: Utilize the prediction results for learning, and perform loss supervision on the detection branch and the re-identification branch. The loss function for the detection branch includes:
[0072] The heatmap predictor is responsible for predicting the center position of the object. The output feature map of the heatmap is H×W×1 in size. If the heatmap collapses along with the center of the real object, the response at that location in the output feature map will be 1. The response of the heatmap predictor decays exponentially as the distance between the location in the heatmap and the center of the target increases. For each ground truth (GT) box... Calculate the target center for and The position of the point on the feature map is then obtained by dividing it by the step size, i.e. In the heatmap, the heatmap response at position (x,y) is calculated as shown in formula (4):
[0073]
[0074] Where N represents the number of objects in the image, σ c The standard deviation is denoted as .
[0075] The loss function is defined as pixel-level logistic regression with focus loss, as shown in Equation (5):
[0076]
[0077] in, This is a predicted heatmap, where α and β are preset loss parameters;
[0078] The bounding box size predictor and center offset predictor are characterized in that the bounding box offset and size prediction loss function includes: a bounding box offset predictor for more accurately locating the target's position in the image. Since the feature map stride is 4, a non-negligible error of up to 4 pixels is introduced. This predictor estimates the continuous offset of each pixel relative to the target center to mitigate the impact on downsampling. The size predictor is responsible for estimating the size of the target bounding box at each location.
[0079] The outputs of the box offset predictor and the size predictor are represented as and For each GT box Its size can be calculated as Similarly, the offset of GT can be calculated as The offset and size at the corresponding positions are expressed as follows: and Then, l1 loss is added to both predictors, as shown in Equation (6):
[0080]
[0081] Where, λ s The weighting coefficient is set to 0.1.
[0082] The re-identification branch loss function includes: This invention learns re-identification features through a classification task, and all objects with the same identity in the training set are considered to be of the same class. The generated feature map is represented as E∈R 128×H×W The target re-identification features extracted from the target whose feature map center is located at (x,y) are:
[0083] For each GT box in the image Obtain the target center A feature vector can be extracted. A fully connected layer and a softmax operation are used to map it to a class distribution vector P = {p(k), k ∈ [1, K]}. The one-hot representation of the GT class label is L. i (k), and then calculate the target re-identification loss as shown in Equation (7):
[0084]
[0085] Where K is the number of categories.
[0086] The loss function during training includes: jointly training the detection and re-identification branches by summing the losses. An uncertainty loss is used to automatically balance the detection and re-identification tasks, as shown in equations (8) and (9).
[0087] L detection =L heat +L box (8)
[0088]
[0089] Here, w1 and w2 are learnable parameters that balance the two tasks. Specifically, given an image with some objects and their corresponding IDs, heatmaps, bounding boxes, size maps, and single-hot class representations of the objects are generated. These are compared with estimated measures to obtain a loss for training the entire network, achieving the optimal network parameters for the deep learning model.
[0090] Step S5: Input the video frames into the optimal deep learning model trained in the previous steps to obtain positional information and appearance features, which are then used as input to the tracker. The tracker outputs the multi-target tracking results. During network inference time, the tracking system performs tracking initialization in the first frame. In subsequent frames, the trajectory set updates bounding boxes, relevant detection scores, appearance features, and "track states" indicating whether a track is active or missing / removed.
[0091] Based on the detection box scores, the detection boxes are divided into high-scoring boxes and low-scoring boxes, and processed separately. First, high-scoring boxes are used to detect and calculate a similarity matrix formed by DIoU distance and inter-frame appearance similarity, thus associating the current frame with the previous trajectory set. Second, low-scoring boxes are used with tracking trajectories that did not match high-scoring boxes in the first detection (e.g., objects whose scores drop due to severe occlusion in the current frame). A similarity matrix formed by DIoU distance and inter-frame appearance similarity is then calculated for matching. The similarity matrix is globally distributed using the Hungarian algorithm to link the detections to the trajectory set.
[0092] For detection boxes that do not match a tracking trajectory but have a sufficiently high score, a new tracking trajectory is created for them. Tracking trajectories that do not match a detection box are retained for 30 frames and matched again when they reappear. Finally, all unmatched high-confidence detections are initialized as new trajectories.
[0093] By comparing the test results on the MOT Halllenge website with the current state-of-the-art single-stage multi-target tracking algorithm FairMOT, as shown in Table 1, the experimental results show that the present invention has improved to varying degrees in terms of IDF1, MOTA and other indicators, and achieves more stable and accurate multi-target tracking.
[0094] Table 1: Comparison of results between this method and the FairMOT algorithm on the MOTChallenge dataset.
[0095]
[0096] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0097] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.
Claims
1. A real-time, anchorless multi-target tracking method based on joint detection and re-identification, characterized in that, Detection and tracking are performed by constructing a deep learning model. This includes a feature extraction backbone network, an LSC attention module, a detection branch, and a re-identification branch. Specific implementation steps include: S1; Take the labeled image as input to the feature extraction backbone network to obtain the image feature pyramid output by the feature extraction backbone network. The feature extraction backbone network includes a deep residual network and a variant DLA parameter aggregation method. S2: Upsample and downsample the upper and lower feature layers of the image feature pyramid respectively, and adjust the upper and lower feature layers to the size of the middle layer features to form a feature vector. ,in The feature vectors Inputting the LSC attention module yields the enhanced feature vector. The LSC attention module mentioned above includes three types of attention: First, scale-aware attention is used to dynamically fuse different features, as shown in Equation (1): in It is a linear function, approximating a 1×1 convolutional layer. It is a hard-sigmoid function; Secondly, spatially aware attention based on fused features is used to focus on regions where spatial location and feature level coexist consistently. Considering the high dimensionality of features, it is decomposed into two steps: first, the attention learning is made sparse by compressing deformable convolution, and then features at different levels are aggregated at the same spatial location, as shown in formula (2): in, The number of sparse sampling locations. To learn spatial offset To focus on the displacement position of the discrimination region, Position for self-learning Importance scalar; both are from It is learned from the median level input features; Finally, task-aware attention is used to achieve joint learning and inductive representation of different objects, as shown in Equation (3): in It is the feature patch of channel c. It is a hyperfunction that learns to control the threshold of the activation function. First of all Global average pooling is performed on the dimension to reduce the dimensionality, then two fully connected layers and a normalization layer are used, and finally a unique sigmoid function is applied to normalize the output to [-1,1]. S3: Transfer the aforementioned feature vector Input the detection branch and the re-identification branch to obtain the location information and appearance features, respectively; S4: Learn based on the obtained prediction results, and perform loss supervision on the detection branch and re-identification branch to update the network parameters of the deep learning model; S5: Take the detection results of the image by the best deep learning model after training and put them into the tracker for tracking.
2. The anchorless real-time multi-target tracking method based on joint detection and re-identification as described in claim 1, characterized in that, The deep residual network of the feature extraction backbone network described in step S1 is ResNet-34. In the ResNet-34 network, a variant parameter aggregation method (DLA) is used to select different layers of the backbone network for different prediction levels. All convolutional layers in the upsampling module are replaced by deformable convolutional layers so that they can dynamically adjust the receptive field according to the size and pose of the object. The input image is propagated through the backbone network to obtain feature pyramids with downsampling rates of 1 / 32, 1 / 16, and 1 / 8.
3. The anchorless real-time multi-target tracking method based on joint detection and re-identification as described in claim 1, characterized in that, The detection branch described in step S3 includes three parallel predictors for the target detection part of the tracking task. Each predictor consists of a 256-channel convolution with a kernel size of 3×3 and a convolution with a kernel size of 1×1. The predictors include: a heatmap predictor, a bounding box size predictor, and a center offset predictor. The loss function in the heatmap predictor includes: The heatmap predictor is responsible for predicting the center position of the object. The output feature map of the heatmap is H×W×1 in size. If the heatmap collapses with the center of the real object, the response of that position in the output feature map will be 1. The response of the heatmap predictor decreases exponentially as the distance between the position in the heatmap and the center of the target increases. For each ground truth (GT) box... Calculate the target center for and Then the position of the point on the feature map is obtained by dividing it by the step size, i.e. In the heatmap, location The thermal response is calculated as shown in formula (4): in, This indicates the number of objects in the image. Standard deviation; The loss function is defined as pixel-level logistic regression with focus loss, as shown in Equation (5): in, It is a predicted heatmap. These are preset loss parameters; The loss functions for the frame size predictor and the center offset predictor include: The center offset predictor is used to more accurately locate the target's position in the image; since the stride of the feature map is 4, it introduces a non-negligible error of up to 4 pixels; the box offset predictor estimates the continuous offset of each pixel relative to the target center to mitigate the impact on downsampling; the box size predictor is responsible for estimating the size of the target bounding box at each location. The outputs of the box offset predictor and the size predictor are represented as and For each GT box Its size can be calculated as Similarly, the offset of GT can be calculated as The offset and size at the corresponding positions are expressed as follows: and Then add two predictors The loss is shown in formula (6): in, The weighting coefficient is set to 0.
1.
4. The anchorless real-time multi-target tracking method based on joint detection and re-identification as described in claim 3, characterized in that, The re-identification feature extraction in the re-identification branch described in step S3 is based on the feature... A convolutional layer with 128 kernels is applied above to extract identity embedding features at each location, where the re-identification loss function includes: Re-identification features are learned through a classification task; all objects with the same identity in the training set are considered to be of the same class; the generated feature map is represented as... From the center of the feature map located at The target re-identification features extracted from the target are: ; For each GT box in the image Obtain the target center A feature vector can be extracted. It is then mapped to a class distribution vector using a fully connected layer and a softmax operation. ; Represent the one-hot encoding of GT class tags as Then, the target re-identification loss is calculated as shown in Equation (7): Where J is the number of categories.
5. The anchorless real-time multi-target tracking method based on joint detection and re-identification as described in claim 1, characterized in that, The loss function in the loss supervision described in step S4 includes: The detection and re-identification branches are trained jointly by summing the losses; uncertainty loss is used to automatically balance the detection and re-identification tasks, as shown in equations (8) and (9): in, The loss function in the heatmap predictor. The loss function for the box size predictor and the center offset predictor. Loss due to target re-identification and These are learnable parameters that balance the two tasks; specifically, given an image with some objects and their corresponding IDs, heatmaps, box offsets and size maps, and one-hot encoded representations of the objects are generated, which are compared with estimated metrics to obtain a loss to train the entire network.
6. The anchorless real-time multi-target tracking method based on joint detection and re-identification as described in claim 1, characterized in that, The tracker described in step S5 includes an online tracking algorithm based on the Hungarian algorithm, specifically including: During network inference time, the tracking system performs tracking initialization in the first frame; in subsequent frames, the trajectory set updates the bounding box, the score of the relevant detection, the appearance features, and the "track status" indicating whether the track is active and lost or removed. Based on the detection box scores, the detection boxes are divided into high-scoring boxes and low-scoring boxes for separate processing. First, high-scoring boxes are used to detect and calculate a similarity matrix formed by the DIoU distance and inter-frame appearance similarity, thus associating the current frame with the previous trajectory set. Second, low-scoring boxes and tracking trajectories that did not match high-scoring boxes in the first detection are used to calculate a similarity matrix formed by the DIoU distance and inter-frame appearance similarity for matching. The similarity matrix is then globally distributed using the Hungarian algorithm to link the detections to the trajectory set. For detection boxes that do not match the tracking trajectory but have a high enough score, a new tracking trajectory is created for them; for tracking trajectories that do not match the detection boxes, they are retained for 30 frames and matched again when they reappear; finally, all unmatched high-confidence detections are initialized as new trajectories.