Detection-based real-time multi-target ID tracking method, storage medium and equipment

Abstract

The embodiment of the invention provides a detection-based real-time multi-target ID tracking method, a storage medium and equipment. The method comprises the following steps: inputting a to-be-detected data set; obtaining a prediction result of the key positioning point and the anchor frame corresponding to each frame; the prediction result is optimized and adjusted; and for a plurality of continuous frames, completing information matching among the frames, and endowing a corresponding target ID for each detection result on the current frame. According to the embodiment of the invention, the ID tracking algorithm is decomposed into a detection part and a tracking part, in the initial stage of object detection, through the key positioning point and anchor frame integrated detection network, the positions of the key positioning point and the anchor frame are output, position fine adjustment is carried out, the detection precision is improved, and when the frame-frame relevance is established in the object tracking stage, the detection precision is improved. Geometric overlapping degree, appearance feature similarity and motion state consistency between anchor frame detection and trajectory prediction are comprehensively considered, so that matching robustness and trajectory continuity in a shielding scene are improved, and real-time performance is improved.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a detection-based real-time multi-target ID tracking method, storage medium, and device. Background Technology

[0002] Detection-based tracking algorithms are complex tracking methods in computer vision, playing a crucial role in resolving the dynamic positions of objects in video sequences. In the military field, they can be applied to the real-time identification, locking, and tracking of multiple targets in enemy formations on land, continuously providing accurate information on the location and size of enemy targets for operational command.

[0003] Currently, mainstream multi-target tracking algorithms mainly rely on deep learning neural networks. End-to-end network models can take continuous video frames as input and output ID information. However, while these methods excel in accuracy, they suffer from limitations in real-time performance. In military applications, the battlefield situation changes rapidly, making real-time acquisition of enemy equipment and target positions crucial. Therefore, it is necessary to decompose the end-to-end tracking network to improve its real-time performance. Summary of the Invention

[0004] In view of the above-mentioned problems existing in the prior art, the present invention provides a detection-based real-time multi-target ID tracking method, storage medium and device to solve the technical problem of insufficient real-time performance in the prior art.

[0005] This invention provides a detection-based real-time multi-target ID tracking method, comprising the following steps:

[0006] Step S1: Input the dataset to be detected;

[0007] Step S2: For different number of frames in the dataset, process the dataset to obtain the prediction results of key localization points and anchor boxes for each frame;

[0008] Step S3: Optimize and adjust the prediction results so that the anchor frame position is close to the key area to be identified;

[0009] Step S4: For a series of consecutive frames, complete the information matching between each frame and assign the corresponding target ID to each detection result in the current frame.

[0010] In one embodiment, the prediction results of the key positioning points and anchor frames corresponding to each frame in step S2 are synchronously output through the same key positioning point and anchor frame integrated detection network model.

[0011] In one embodiment, the integrated detection network model for key positioning points and anchor frames includes an encoder and a decoder. The encoder includes a backbone network and a neck network. The backbone network adopts a multi-resolution fusion network structure and constructs four pathways in a hierarchical manner. The neck network adopts a network structure in which each layer is interconnected. The four-layer output of the backbone network is linked to the neck network and serves as the input of the neck network. The result of the neck network is input to the decoder. The decoder calculates and iteratively obtains the prediction results of the anchor frame and key positioning points based on the target detection loss function and the key positioning point offset loss function.

[0012] In one embodiment, step S3 includes,

[0013] Step S31: Set weights for each key positioning point;

[0014] Step S32: Calculate and determine the weighted midpoint position of all key positioning points;

[0015] Step S33: Calculate and determine the center point position of the anchor frame prediction result;

[0016] Step S34: Determine the fine-tuning translation vector based on the weighted intermediate point position of all key positioning points and the center point position of the anchor frame prediction result;

[0017] Step S35: Move the anchor frame prediction result along the fine-tuning translation vector to complete the optimization and adjustment of the prediction result.

[0018] In one embodiment, step S4 includes,

[0019] Step S41: Model the initial state of multiple targets;

[0020] Step S42: For each pair of detected targets and trajectories, construct a fusion cost function by integrating their spatial information, appearance information, and motion information;

[0021] Step S43: Use the Hungarian algorithm to complete the globally optimal target matching and assign the corresponding target ID to each detection result in the current frame.

[0022] In addition, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the detection-based real-time multi-target ID tracking method described in any embodiment of the present invention.

[0023] In addition, embodiments of the present invention also provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the detection-based real-time multi-target ID tracking method described in any embodiment of the present invention.

[0024] Compared with existing technologies, the beneficial effects of the detection-based real-time multi-target ID tracking method, storage medium, and device provided by the embodiments of the present invention are as follows: The embodiments of the present invention decompose the ID tracking algorithm into two parts: detection and tracking. In the initial stage of object detection, the key positioning points and anchor frames are output through an integrated detection network of key positioning points and anchor frames. Subsequently, a mathematical model is used to fine-tune the positions to improve detection accuracy. When establishing the correlation between frames in the object tracking stage, the geometric overlap, appearance feature similarity, and motion state consistency between the detected anchor frames and trajectory prediction are comprehensively considered, thereby improving the matching robustness and trajectory continuity in occluded scenes, reducing the number of computational parameters and complexity, and improving real-time performance. Attached Figure Description

[0025] Figure 1 A schematic diagram of the target ID tracking process involved in a detection-based real-time multi-target ID tracking method provided in an embodiment of the present invention;

[0026] Figure 2 A schematic diagram of the backbone network architecture involved in a detection-based real-time multi-target ID tracking method provided in an embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram of the neck network architecture involved in a detection-based real-time multi-target ID tracking method provided in an embodiment of the present invention. Detailed Implementation

[0028] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] Various embodiments and features of this application are described herein with reference to the accompanying drawings.

[0030] These and other features of this application will become apparent from the following description of preferred forms of embodiments given as non-limiting examples, with reference to the accompanying drawings.

[0031] It should also be understood that although this application has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of this application, which have the features described in the claims and are therefore all within the scope of protection defined herein.

[0032] The above and other aspects, features and advantages of this application will become more apparent when taken in conjunction with the accompanying drawings and in view of the following detailed description.

[0033] Specific embodiments of this application are described below with reference to the accompanying drawings; however, it should be understood that the claimed embodiments are merely examples of this application, which can be implemented in various ways. Well-known and / or repeated functions and structures are not described in detail to ascertain the true intent based on the user's historical operations, and to avoid unnecessary or redundant details that would obscure this application. Therefore, the specific structural and functional details claimed herein are not intended to be limiting, but merely serve as the basis and representative basis for the claims to teach those skilled in the art to use this application in various ways with substantially any suitable detailed structure.

[0034] This specification may use the phrases “in one embodiment,” “in another embodiment,” “in yet another embodiment,” or “in other embodiments,” all of which may refer to one or more of the same or different embodiments according to this application.

[0035] The principles and features of the present invention are described below with reference to the accompanying drawings. The embodiments described are for illustrative purposes only and are not intended to limit the scope of the invention. The following description, in conjunction with... Figure 1-3 The preferred embodiments of the present invention will be described in further detail below:

[0036] like Figure 1 As shown, this embodiment of the invention provides a detection-based real-time multi-target ID tracking method, including the following steps:

[0037] Step S1: Input the dataset to be detected;

[0038] Step S2: For different number of frames in the dataset, the dataset is processed to obtain the prediction results of key localization points and anchor boxes for each frame. The prediction results of key localization points and anchor boxes for each frame are synchronously output through the same key localization point and anchor box integrated detection network model. The key localization point and anchor box integrated detection network model includes an encoder and a decoder. The encoder includes a backbone network and a neck network. The backbone network adopts a multi-resolution fusion network structure and constructs four pathways according to the hierarchy. The neck network adopts a network structure in which each layer is interconnected. The four layers of the backbone network are linked to the neck network as the input of the neck network. The result of the neck network is input to the decoder. The decoder calculates and iterates the prediction results of the anchor boxes and key localization points according to the target detection loss function and the key localization point offset loss function.

[0039] In military applications, such as the electro-optical sensors mounted on unmanned surface vessels (USVs), it is necessary to identify incoming targets and classify their payloads. This requires marking bounding boxes on the dataset to select key areas for identification. To aid in the identification of bounding boxes, key areas can be located by marking relevant key positioning points (both manually determined and marked, primarily marking key points of interest for the target to be identified). However, using two separate networks to perform these two tasks would result in wasted time and computing power. Therefore, designing an integrated detection network for key positioning points and bounding boxes can solve these problems.

[0040] The integrated detection network model for key positioning points and anchor frames consists of an encoder and a decoder. The encoder is further divided into a backbone network and a neck network. The backbone network adopts a multi-resolution fusion network structure, constructing four pathways layer by layer: 128*128, 64*64, 32*32, and 16*16, increasing layer by layer. See the appendix for details. Figure 2 The neck network, in particular, constructs an interconnected network structure across all levels; see the appendix for details. Figure 3 The interconnected construction approach enables multi-resolution feature information fusion across multiple network layers, improving prediction and recognition efficiency. The four-layer output of the backbone network is linked to the neck network, serving as its input. The results from the neck network are then fed into the decoder. The decoder has a relatively simple network structure, calculating the iterative parameters based on two loss functions to complete the outputs of two branches (object detection and key point localization).

[0041] The network loss function is designed as follows:

[0042] (1) Target detection loss (L_size)

[0043] The object detection branch is responsible for predicting the width and height of each object. It uses L1 loss (L1 Loss):

[0044]

[0045] in The width and height are obtained from network regression, w n h n This corresponds to the true size. This loss helps the model to more accurately reconstruct the target bounding box.

[0046] (2) Key positioning point offset loss (L_offset)

[0047] To achieve inter-frame learning, the target's displacement vectors Δx and Δy are used between two frames. The supervision label for this branch is the difference in position between the target's center point in the previous and current frames. The loss function also adopts the L1 loss form:

[0048]

[0049] in Let Δx be the motion vector predicted by the network. n Δy n This represents the true inter-frame center point offset. This loss term enables the model to have implicit target tracking capabilities, making it particularly suitable for target recovery after occlusion and short-term loss.

[0050] Step S3 involves optimizing and adjusting the prediction results to ensure the anchor frame position is close to the key area to be identified. Specifically, this includes...

[0051] Step S31: Set weights for each key positioning point;

[0052] Step S32: Calculate and determine the weighted midpoint position of all key positioning points;

[0053] Step S33: Calculate and determine the center point position of the anchor frame prediction result;

[0054] Step S34: Determine the fine-tuning translation vector based on the weighted intermediate point position of all key positioning points and the center point position of the anchor frame prediction result;

[0055] Step S35: Move the anchor frame prediction result along the fine-tuning translation vector to complete the optimization and adjustment of the prediction result.

[0056] The following describes each sub-step of step S3 in detail with an example. Step S2 can obtain the prediction results of key positioning points and anchor frames. These two results are used as input to design an anchor frame detection fine-tuning model. Taking into account the results of anchor frame detection and the position of key positioning points (i.e. key focus points), the target recognition results are optimized and fine-tuned.

[0057] Construct a target detection location optimization model based on the matching of key positioning point location information and anchor frame location information.

[0058] Suppose a total of n key locations are marked: (x1, y1), (x2, y2), ..., (x n ,y n Each key location point can be assigned a weight w based on its distribution. i Calculate the weighted average position of all key positioning points:

[0059]

[0060] Assume the bottom left corner of the anchor box output by the model is (X 1 ,Y 1 The upper right corner is (X) 2 ,Y 2From this, the coordinates of the center point (X) of the anchor frame can be calculated. c ,Y c ):

[0061]

[0062]

[0063] Width and height of the detection frame:

[0064] W=X 2 -X 1

[0065] H = Y 2 -Y 1

[0066] To make the detection box position closer to the required position, the center point of the detection box is moved along the vector... A translation is performed to bring the detection box closer to the weighted midpoint of the target. A translation step size coefficient α∈[0,1] is introduced to control the fine-tuning amplitude (set α=0.5):

[0067]

[0068] Step S4 involves performing information matching between several consecutive frames and assigning a corresponding target ID to each detection result in the current frame. Specifically, this includes...

[0069] Step S41: Model the initial state of multiple targets;

[0070] Step S42: For each pair of detected targets and trajectories, construct a fusion cost function by integrating their spatial information, appearance information, and motion information;

[0071] Step S43: Use the Hungarian algorithm to complete the globally optimal target matching and assign the corresponding target ID to each detection result in the current frame.

[0072] That is, design an improved target matching method to complete information matching between frames and frame stations.

[0073] Steps S2-S3 can complete anchor frame detection, key positioning point detection, and anchor frame fine-tuning for a single image, and finally output the optimized anchor frame detection result, thus completing the target detection for a single frame.

[0074] When the video stream is input, the anchor frame detection and optimization of each frame are completed first. Then, for several consecutive frames (the number of consecutive frames can be manually set in the algorithm), a matching model is established between the anchor frame trajectory of the current frame and the target historical anchor frame position to complete the matching between the current frame and the target historical position.

[0075] Initial state modeling for multiple targets.

[0076] Initial modeling is performed for each dynamic objective, and the state of each objective is modeled as a vector, as shown in the following equation.

[0077]

[0078] Where u and v represent the x and y coordinates of the target center point, respectively, and s and r represent the area and aspect ratio of the target bounding box, respectively. These represent the rates of change of the above variables. During the tracking process, when a detection result is associated with a target, the processed anchor box (observation) is used to update the target's state.

[0079] Design a matching cost fusion algorithm.

[0080] This is achieved through a multimodal matching method that integrates spatial, appearance, and motion information. This method comprehensively considers the geometric overlap, appearance feature similarity, and motion state consistency between the detection box and the trajectory prediction, thereby improving the matching robustness and trajectory continuity in occluded scenarios.

[0081] Specifically, suppose there are N detected targets d1, d2, ..., dn in the current frame. n The previous frame, after filtering and prediction, yielded M trajectory states t1, t2, ..., t3. m For each pair of detected targets and trajectories (t... i ,d j Construct a fusion cost function, expressed as follows:

[0082] Cost(i,j)=α×(1-IoU(i,j))+β×D_app(i,j)+γ×D_mah(i,j)

[0083] Where: IoU(i,j) represents the trajectory t i Predicted bounding box and detection bounding box d j The intersection-union ratio (IU / UU) reflects the degree of spatial overlap; D_app(i,j) represents the distance between appearance features, typically using Euclidean distance or cosine distance, the latter being in the form of:

[0084] D_app(i,j)=1-(fi·fj) / (||fi||×||fj||)

[0085] Where f i and f i Trajectory t i and detection d j eigenvectors.

[0086] D_mah(i,j) represents the Mahalanobis distance, used to measure the error between the predicted trajectory state and the detected observations, and is defined as:

[0087]

[0088] in For trajectory t i The predicted state, where S is the covariance matrix and z is the detection box d. j The observation vector.

[0089] The three weighting coefficients α, β, and γ control the importance of IoU distance, appearance feature distance, and Mahalanobis distance in the matching process, respectively. Based on preliminary experiments, it is recommended to set α = 0.3, β = 0.5, and γ = 0.2, while dynamically adjusting β according to the occlusion situation to enhance the role of occlusion re-identification.

[0090] After constructing the fusion cost function, the Hungarian algorithm is used to achieve globally optimal target matching. After matching is completed, the system updates the state of each trajectory according to the matching results, and sets the trajectory that failed to match to a "temporarily lost" state, allowing it to continue to exist for a certain number of frames (set to 5), waiting for the target to reappear in subsequent frames.

[0091] In addition, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the detection-based real-time multi-target ID tracking method described in any embodiment of the present invention.

[0092] In addition, embodiments of the present invention also provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the detection-based real-time multi-target ID tracking method described in any embodiment of the present invention.

[0093] The above embodiments are merely exemplary embodiments of the present invention and are not intended to limit the present invention. The scope of protection of the present invention is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to the present invention within its spirit and scope of protection, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of the present invention.

Claims

1. A detection-based real-time multi-target ID tracking method, characterized in that, The method comprises the following steps: Step S1, inputting a data set to be detected; Step S2, processing the data set for different frame numbers in the data set to obtain prediction results of key positioning points and anchor boxes corresponding to each frame; Step S3, optimizing and adjusting the prediction results to make the anchor box position close to the key region to be recognized; Step S4, completing information matching between frames for a plurality of continuous frames, and assigning a corresponding target ID to each detection result in the current frame.

2. The method according to claim 1, wherein: The prediction results of the key positioning points and the anchor boxes corresponding to each frame in step S2 are synchronously output by the same key positioning point and anchor box integrated detection network model.

3. The method of claim 2, wherein: The key positioning point and anchor box integrated detection network model comprises an encoder and a decoder, the encoder comprises a backbone network and a neck network, the backbone network adopts a network structure of multi-resolution fusion, four channels are constructed according to layers, the neck network adopts a network structure of interconnection between each level, four outputs of the backbone network are linked to the neck network as inputs of the neck network, and a result of the neck network is input to the decoder, the decoder calculates an iterative prediction result of the anchor box and the key positioning point according to a target detection loss function and a key positioning point offset loss function.

4. The method of claim 1, wherein: The step S3 comprises, Step S31, setting a weight for each key positioning point; Step S32, calculating and determining a weighted intermediate point position of all key positioning points; Step S33, calculating and determining a center point position of the anchor box prediction result; Step S34, determining a fine-tuning translation vector according to the weighted intermediate point position of all key positioning points and the center point position of the anchor box prediction result; Step S35, moving the anchor box prediction result along the fine-tuning translation vector to complete optimization and adjustment of the prediction result.

5. The method of claim 1, wherein: The step S4 comprises, Step S41, modeling a plurality of target initial states; Step S42, constructing a fusion cost function by comprehensively integrating spatial information, appearance information and motion information for each pair of detection targets and trajectories; Step S43, completing global optimal target matching by using a Hungarian algorithm, and assigning a corresponding target ID to each detection result in the current frame.

6. A computer readable storage medium having stored thereon a computer program, characterized in that: The program is executed by the processor to implement the real-time multi-target ID tracking method based on detection according to any one of claims 1-5.

7. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the real-time multi-target ID tracking method based on detection according to any one of claims 1-5.

Images