Open vocabulary multi-target tracking method based on confidence adjustment and wavelet convolution

Abstract

The invention discloses an open vocabulary multi-target tracking method based on confidence coefficient adjustment and wavelet convolution, belongs to the technical field of computer vision and target tracking, and can enhance the adaptive modeling capability of a target motion mode by designing a confidence coefficient weighted Kalman updating mechanism and developing a confidence coefficient noise adaptive Kalman filtering algorithm. And meanwhile, the Haar wavelet transform is introduced, so that the receptive field of convolution operation is remarkably expanded, and the capturing capability of high-frequency motion details and the maintaining capability of low-frequency contour information are effectively improved. And in combination with the joint cost matrix of the motion features and the appearance features, a more adaptive target tracking model is formed, and the tracking accuracy and stability are effectively improved. The method can effectively deal with rapid movement and temporary shielding of the target, accurately tracks various types of targets with undefined types, and is suitable for open vocabulary multi-target tracking tasks in various complex environments.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and target tracking, and particularly relates to an open-vocabulary multi-target tracking method based on confidence adjustment and wavelet convolution. BACKGROUND

[0002] With the rapid development of deep learning, this technology has been widely applied to various tasks in the field of computer vision, including image classification, target detection and target tracking, etc. In recent years, multi-target tracking has become increasingly important in computer vision research and has been widely used in practical scenarios such as autonomous driving and security monitoring. Traditional multi-target tracking methods usually rely on pre-defined and limited target categories, such as specific types of vehicles or pedestrians. Such methods have good performance in dealing with known category targets, but as technology advances and application demands continue to expand, real-world scenarios present higher complexity and diversity, including a large number of unknown or pre-defined targets, such as rare vehicle types, atypical pedestrian behavior, animals and other unexpected entities. In this context, multi-target tracking methods capable of handling open category sets have emerged.

[0003] Open-vocabulary multi-target tracking technology has achieved significant improvements in both functionality and performance compared to traditional multi-target tracking systems. Its core advantage lies in its ability to recognize and track non-predefined categories of objects, thereby significantly enhancing the system's adaptability and flexibility in complex and dynamic environments. Traditional methods are usually limited to known categories such as pedestrians and vehicles, while open-vocabulary multi-target tracking combines deep learning and natural language processing techniques to handle unlabeled or rare targets. When faced with non-standard vehicles, animals or special objects in unexpected events, this technology can still operate effectively. In addition, by continuously learning new data, the system can adapt to environmental changes and improve its understanding and processing capabilities in dynamic scenarios.

[0004] In the field of multi-target tracking, it is a common approach to combine Kalman filters with appearance features. However, due to the fact that detectors often output candidate regions of varying quality, if there is a lack of confidence constraints on detection, the system is prone to interference from low-quality detections during long-term tracking, which can lead to track drift or misassociation. Especially in the context of autonomous driving, target confidence may fluctuate significantly due to factors such as sudden changes in lighting, sensor noise, rain and fog weather, or long-distance shooting. Without proper confidence modeling and screening mechanisms, the tracker is prone to false positives or false negatives, ultimately affecting the safety and stability of vehicles in complex and dynamic environments.

[0005] As an emerging field, the existing methods still have deficiencies in feature learning, especially in considering targets with different scales and complexities, resulting in insufficient extraction of context information. For example, in the intelligent transportation scene, targets often show significant differences due to changes in light, partial occlusion or high-speed motion, affecting the stability and robustness of tracking. Without effective modeling of these context factors, the model often has difficulty in achieving continuous tracking and accurate identification of targets.

[0006] Based on this, the open vocabulary multi-target tracking method based on confidence adjustment and wavelet convolution is designed to solve the above problems. SUMMARY

[0007] In view of the above-mentioned defects of the prior art, the open vocabulary multi-target tracking method based on confidence adjustment and wavelet convolution is provided.

[0008] To achieve the above purpose, the technical scheme is as follows:

[0009] The open vocabulary multi-target tracking method based on confidence adjustment and wavelet convolution comprises the following steps:

[0010] Step one, in the training stage, the current image and the historical image are input, and the main network is processed; the Haar wavelet convolution layer is introduced in the convolution layer to extract the features of the input image, and the feature map of each channel is obtained; then, the Haar wavelet transform is applied to each channel to generate four sub-band feature maps with lower sampling; and the inverse Haar wavelet transform is used to restore the sub-band feature maps to the fusion feature maps with the same dimension as the original input;

[0011] Step two, in the inference stage, the target tracking model is constructed by combining the motion model and the appearance model, the joint cost matrix is constructed by using the appearance features and the motion features, and the target tracking model is obtained for target matching;

[0012] The joint cost matrix adopts formula (4):

[0013] Formula (4)

[0014] Wherein, is the joint cost matrix, represents the similarity measure between the first detection and the second detection based on the motion features; represents the similarity measure between the first detection and the second detection based on the appearance features; represents the similarity measure between the first tracking and the second tracking based on the motion features; represents the similarity measure between the first tracking and the second tracking based on the appearance features;

[0015] ​​Step three, in the tracking stage, the confidence weighted Kalman update module dynamically adjusts the weight of each detection box in the Kalman filter update stage according to the confidence score of each detection box; for the detection box with confidence lower than the threshold value , the original detection box is replaced by a more close to the predicted box position, and is weighted according to the detection confidence; the confidence noise adaptive Kalman filter algorithm is used to dynamically adjust the measurement noise related to the detection box confidence; the trajectory update adopts the hierarchical adaptive feature update strategy.

[0016] Further, the specific steps of step one are as follows:

[0017] 1.1, taking the current image and the historical image as input, processing through the ResNet50 backbone network; in the second convolutional layer of each residual block, a Haar wavelet convolutional layer is introduced to replace the traditional convolutional layer to extract features, so as to obtain the feature map of each channel;

[0018] 1.2, for each channel feature map, four groups of filters defined by formula (1) are used for deep convolution operation to realize two-dimensional Haar wavelet transform; for each input channel of the image , the deep convolution operation will produce four sub-band feature maps as shown in formula (2):

[0019] Formula (1)

[0020] Formula (2)

[0021] Wherein, is a low-pass filter, and , and are a group of high-pass filters, represents the low-frequency component of the image, and , and represent the high-frequency components of the image in the horizontal, vertical and diagonal directions respectively;

[0022] 1.3, inverse Haar wavelet transform is performed through the transpose convolution operation defined in formula (3) to restore the original dimension of the sub-band feature map:

[0023] Formula (3)

[0024] Wherein, represents the transpose convolution operation.

[0025] Furthermore, the Haar wavelet convolution layer is embedded into ResNet50; the input image comes from the TAO dataset.

[0026] Furthermore, in step three, the confidence weighted Kalman update module dynamically adjusts its weight in the Kalman filter update stage by formula (5):

[0027] Formula (5)

[0028] wherein, represents the estimated position of the i-th frame, is the prediction of the position of the i-th frame, is the measurement value after confidence adjustment, is the Kalman gain calculated in the i-th frame, used to update the estimated value, denotes the projection matrix, denotes the target measurement value. Furthermore, in step three, for the detection box with confidence lower than the threshold

[0029] , the original detection box is replaced by a position close to the predicted box by formula (6), and the detection confidence is weighted: Formula (6).

[0030] Furthermore, in step three, the confidence noise adaptive Kalman filter algorithm adopts formula (7):

[0031] Formula (7)

[0032] wherein, is the amplified measurement space noise covariance matrix,

[0033] is the confidence of the detection box, and N is the noise covariance of the measurement space, is the amplification factor. Furthermore, in step three, the detection results are assigned to existing trajectories, and the trajectories are divided into two states: dormant state and active state; when there is no new detection result assigned to the trajectory in a given frame, the trajectory will be set to the dormant state; otherwise, the trajectory remains in the active state; if the trajectory remains in the dormant state for more than a predetermined number of frames, the trajectory is discarded.

[0034] Furthermore, for the trajectory in the dormant state, the trajectory feature is updated by formula (8):

[0035]

[0036] ​​Formula (8)

[0037] wherein, is the updated feature vector, is the feature vector detected in the current frame, is the latest feature vector on the trajectory, is a coefficient for controlling the update proportion.

[0038] Further, for the active state trajectory, the trajectory feature is updated by using formula (9):

[0039] Formula (9)

[0040] wherein, is another coefficient for controlling the update proportion, is a score determined by the detection confidence of the current object and the proportion of all detection confidences in the current frame.

[0041] In order to better achieve the purpose of the present application, the present application further provides a computer storage medium, which is used for storing program data, and the program data is used to realize the open vocabulary multi-object tracking method based on confidence adjustment and wavelet convolution when being executed by a computer.

[0042] Compared with the prior art, the present application has the beneficial effects that: 1) the present application can significantly expand the receptive field of the model by introducing the Haar wavelet transform to process image data. This expansion enables the model to capture more contextual information within a larger spatial range, thereby improving the accuracy of target positioning.

[0043] 2) The present application integrates dynamic motion cues and visual appearance details into a unified and coordinated framework, constructs a joint cost matrix, and forms a more adaptive target tracking model, effectively improving the accuracy and stability of tracking. Unlike the traditional OVTrack model which mainly relies on appearance features for object association, the present application introduces a bipartite graph matching mechanism based on the Hungarian algorithm, optimizes the association process, and ensures that each detection result is associated with the most matching trajectory. This strategy significantly improves the accuracy and robustness of cross-frame tracking, especially in complex scenarios such as rapid target movement and temporary occlusion, it can still maintain stable and accurate tracking performance.

[0044] 3) Due to the wide range and uncertainty of target categories, the detector may produce a large number of detection results with varying quality and confidence in the motion cue part. To deal with this uncertainty and improve tracking robustness, the present invention proposes a confidence-weighted Kalman update module. This module dynamically adjusts the weight of each detection box in the Kalman filter update stage according to its confidence score. When the confidence score of the detection box is high, the target remains the original detection box, which means that the high-confidence detection result is directly used to update the state, ensuring the continuity and accuracy of target tracking. In the case of low confidence score, the module reduces the dependence on these unreliable detection results and relies more on the predicted state of the model, i.e. predicting the possible position of the target through the information of the previous frame. In this way, the system can maintain the continuity and accuracy of tracking even when the quality of the detection box is poor.

[0045] 4) The present invention proposes a confidence noise adaptive Kalman filter algorithm, which not only improves the adaptability to environmental changes, but also reduces tracking errors caused by sudden noise and false detection. Especially in dynamic scenes, such as tracking the nonlinear motion of individuals in a crowd, this method can significantly improve the continuity and reliability of tracking.

[0046] 5) Experimental results show that the overall performance of the present invention on the TAO multi-target tracking dataset is better than that of existing advanced open vocabulary multi-target tracking methods, showing good generalization ability and practical prospects. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0048] Figure 1 Flowchart of the open vocabulary multi-target tracking method based on confidence adjustment and wavelet convolution of the present invention.

[0049] Figure 2 Visualization comparison chart of tracking results of the prior art method and the present invention on the TAO dataset.

[0050] Figure 3 Visualization chart of tracking results of the present invention on network video. DETAILED DESCRIPTION

[0051] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments but not all of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0052] Embodiment one: please refer to the accompanying drawings of the specification Figure 1 , an open vocabulary multi-target tracking method based on confidence adjustment and wavelet convolution, named OVSORTV2, includes the following steps:

[0053] Step one, in the training stage, the current image and the historical image are taken as inputs, and are processed through a ResNet50 backbone network; a Haar wavelet convolution layer (HWTConv) is introduced in the convolution layer to extract features of the input image, and feature maps of each channel are obtained; then, Haar wavelet transform (HWT) is applied to each channel to generate four sub-band feature maps with down-sampling; the inverse Haar wavelet transform (IHWT) is used to restore the four sub-band feature maps to a fusion feature map with the same dimension as the original input; the fusion feature retains the global contour of low frequency and the edge details of high frequency, and can effectively enhance the target recognition and distinguishing ability of the model in a complex scene.

[0054] The application of the Haar wavelet convolution can significantly expand the receptive field of the model, so that the model can capture more context information in the spatial range, thereby improving the accuracy of target positioning.

[0055] The specific steps of step one are as follows:

[0056] 1.1, the current image and the historical image are taken as inputs, and are processed through a ResNet50 backbone network; in the second convolution layer of each residual block (i.e. 3x3 convolution in bottleneck), a Haar wavelet convolution layer is introduced to replace the traditional convolution layer to extract features, so that feature maps of each channel are obtained.

[0057] Further, the Haar wavelet convolution layer is embedded into the ResNet50 to enhance the feature extraction capability.

[0058] Further, the input image comes from the TAO dataset.

[0059] 1.2, for the feature maps of each channel, four groups of filters defined by formula (1) are used to perform depth convolution operation with a step of 2 to realize two-dimensional Haar wavelet transform; for each input channel of the image , the depth convolution operation will produce four sub-band feature maps with down-sampling as shown in formula (2):

[0060] Equation (1)

[0061] Equation (2)

[0062] where, is a low-pass filter, while , and form a set of high-pass filters, denotes the low-frequency components of the image, while , and denote the high-frequency components of the image in horizontal, vertical, and diagonal directions, respectively.

[0063] 1.3. To ensure that these features can be accurately mapped back to the original space for subsequent analysis and processing, the inverse Haar wavelet transform is implemented through the transpose convolution operation defined in Equation (3) using the property that the filter set in Equation (1) forms an orthogonal basis, thereby restoring the original dimensionality of the sub-band feature maps:

[0064] Equation (3)

[0065] where, denotes the transpose convolution operation, which is the inverse process of standard convolution, used to gradually restore the sub-band feature maps to the resolution of the original image.

[0066] Step 2. In the inference phase, a target tracking model is constructed by combining the motion model and the appearance model, and a joint cost matrix is constructed using appearance features and motion features to obtain the target tracking model for target matching.

[0067] First, the visual features of the target are extracted through the image header, and the semantic features of the category are extracted through the text header. Then, the information of the motion model and the appearance model is fused, and the joint cost matrix is constructed by jointly using the appearance cost (calculated from the appearance features) and the motion cost (generated from the motion features). Based on the bipartite graph matching mechanism of the Hungarian algorithm, bilateral matching between the detection results and the trajectories is realized.

[0068] To improve the performance of motion tracking, the Kalman filter is introduced as the core module of the motion model to dynamically predict the future position of the target.

[0069] The joint cost matrix adopts Equation (4):

[0070] Equation (4)

[0071] where, is the joint cost matrix, represents the motion feature-based the similarity measure between the secondary detection and the secondary tracking; representing the similarity measure of the appearance feature based on the secondary detection and the secondary tracking; the similarity measure of the motion feature and the similarity measure of the appearance feature can be complementary in the association process to make up for the deficiency of using appearance association only.

[0072] The present application builds a joint cost matrix by integrating dynamic motion cues and visual appearance details into a unified and coordinated framework, thereby forming a more adaptive target tracking model, effectively improving the accuracy and stability of tracking.

[0073] Step three, in the tracking stage, the confidence weighted Kalman update module adopts the confidence weighted Kalman update method, dynamically adjusts the update weight of the filter according to the confidence of the detection result, thereby significantly improving the robustness of motion prediction under low-quality detection conditions. At the same time, a confidence noise adaptive Kalman filtering algorithm is proposed, which realizes the enhanced adaptive modeling ability of target motion pattern by constructing a noise and confidence coupled modeling mechanism. For trajectory update, a hierarchical adaptive feature update strategy (HAFU) is adopted, which is consistent with the OVSORT method (paper DOI number: 10.1109 / TMM.2025.3557619, original text: Open-Vocabulary Multi-Object Tracking With Domain Generalized and Temporally Adaptive Features), by analyzing the current state of the target, the feature representation is adjusted in real time, thereby more effectively adapting to the changes of target appearance and occlusion interference.

[0074] The specific steps of step three are as follows:

[0075] 3.1, the confidence weighted Kalman update module dynamically adjusts the weight of each detection box in the Kalman filter update stage according to the confidence score, as shown in formula (5); for the detection box with confidence lower than the threshold , the original detection box is replaced by a position closer to the predicted box, and the detection confidence is weighted to improve the robustness of state estimation under low confidence conditions:

[0076] Formula (5)

[0077] Formula (6)

[0078] wherein, estimated position of the i-th frame, is a prediction of the i-th frame position, is a measurement value adjusted by confidence, is a measurement value adjusted by confidence, is a Kalman gain calculated in the i-th frame for updating the estimated value, denotes a projection matrix, denotes a target measurement value.

[0079] 3.2, Confidence noise adaptive Kalman filtering algorithm uses formula (7) to dynamically adjust the measurement noise related to the detection frame confidence, thereby affecting the calculation process of the Kalman gain, to achieve adaptive control of uncertain observations:

[0080] Formula (7)

[0081] wherein, is the amplified measurement space noise covariance matrix, is the confidence of the detection frame, and N is the noise covariance of the measurement space, is the amplification factor.

[0082] 3.3, In the tracking stage, the detection results will be assigned to existing tracks, which are stored in the tracking memory (stores and manages the state information of each tracking target in the past frame), and the tracks are divided into two states: dormant state and active state. When there is no new detection result assigned to the track in a given frame, the track will be set to the dormant state; otherwise, the track remains in the active state. If the track remains in the dormant state for more than a predetermined number of frames, the track will be discarded.

[0083] For the track in the dormant state, formula (8) is used to update the track features:

[0084] Formula (8)

[0085] wherein, is the updated feature vector, is the feature vector detected in the current frame, is the latest feature vector on the track, is a coefficient controlling the update proportion.

[0086] For the track in the active state, formula (9) is used to update the track features:

[0087] Formula (9)

[0088] wherein, ​is another coefficient for controlling the update ratio, is a score determined by the ratio of the detection confidence of the current object to all detection confidences of the current frame.

[0089] Embodiment Two: A computer storage medium for storing program data, which when executed by a computer, implements the open-vocabulary multi-object tracking method based on confidence adjustment and wavelet convolution in Embodiment One.

[0090] Experimental Example: The performance of the method of the present application (OVSORTV2) is compared with that of various existing open-vocabulary multi-object tracking methods: the tracking effect comparison on the TAO public dataset is shown in Table 1; the tracking effect comparison in the closed set test is shown in Table 2. In addition, Figure 2 The tracking result visualization comparison chart of OVSORTV2 and the prior art method on the TAO dataset is shown.

[0091] Table 1: Tracking effect comparison of OVSORTV2 and prior art methods on the TAO dataset

[0092]

[0093] In the table, LocA is the positioning accuracy, AssocA is the association accuracy, TETA is the overall tracking accuracy, and ClsA is the classification accuracy. The base class refers to the class involved in the training sample, and the novel class refers to the class not involved in the training.

[0094] QDTrack, Teter, DeepSORT (ViLD), Tracktor++ (ViLD), and OVTrack are all selected from the paper "Ovtrack: Open-vocabulary multiple object tracking" and belong to commonly used comparison trackers in the field of open-vocabulary multi-object tracking.

[0095] As can be seen from Table 1, the performance of the method OVSORTv2 of the present application on the base class and the novel class is better than that of other tracking methods. Specifically, the overall tracking accuracy (TETA) on the novel class is 29.0%, and the positioning accuracy (LocA) reaches 49.2%. This is very close to the performance on the validation set, in which the TETA and LocA of the novel class are 31.8% and 54.1%, respectively. This indicates that OVSORTv2 effectively alleviates the inconsistency problem of target detection quality and confidence, and improves the accuracy and stability of the system in dynamic and uncertain environments.

[0096] Table 2: Tracking effect comparison of OVSORTV2 and prior art methods in the closed set test

[0097]

[0098] As can be seen from Table 2, the OVSORTv2 method of the present application performs excellently in the closed set test of the TAO verification set. By comparing the indicators, OVSORTv2 is significantly better than other comparative models in key indicators such as positioning accuracy (LocA), association accuracy (AssocA) and overall tracking accuracy (TETA), and has obvious performance advantages. The results fully verify the effectiveness and practicality of the optimization measures made by the present application in improving model accuracy.

[0099] From Figure 2 It can be seen that OVSORTv2 can not only realize effective tracking of ducks after severe occlusion, but also more accurately distinguish similar targets from real targets. Compared with the traditional OVTrack method, it performs better in tracking and identifying after occlusion and distinguishing similar objects.

[0100] Figure 3 The network video tracking result visualization schematic diagram of the method of the present application. As can be seen, OVSORTv2 also has efficient application ability in the real world, and can achieve excellent identification effect whether facing different categories of targets or similar background targets, and exhibits wide applicability and tracking performance.

[0101] In summary, the present application has the following technical effects:

[0102] 1) The present application can significantly expand the receptive field of the model by introducing Haar wavelet transform to process image data. This expansion enables the model to capture more context information in a larger spatial range, thereby improving the accuracy of target positioning.

[0103] 2) The present application integrates dynamic motion cues and visual appearance details into a unified and coordinated framework, constructs a joint cost matrix, and forms a more adaptive target tracking model, effectively improving the accuracy and stability of tracking. Unlike the traditional OVTrack model which mainly relies on appearance features for object association, the present application introduces a bipartite graph matching mechanism based on the Hungarian algorithm, optimizes the association process, and ensures that each detection result is associated with the most matching track. This strategy significantly improves the accuracy and robustness of cross-frame tracking, especially in complex scenes such as rapid target movement and temporary occlusion, it can still maintain stable and accurate tracking performance.

[0104] 3) Due to the wide range and uncertainty of target categories, the detector may produce a large number of detection results with varying quality and significant fluctuations in confidence in the motion cue part. To deal with this uncertainty and improve tracking robustness, the invention proposes a confidence-weighted Kalman update module. This module dynamically adjusts the weight of each detection box in the Kalman filter update phase according to its confidence score. When the confidence score of the detection box is high, the target remains the original detection box, which means that the high-confidence detection result is directly used to update the state, ensuring the continuity and accuracy of target tracking. In the case of low confidence score, the module reduces the dependence on these unreliable detection results and relies more on the predicted state of the model, i.e. predicting the possible position of the target through the information of the previous frame. In this way, even when the quality of the detection box is poor, the system can still maintain the continuity and accuracy of tracking.

[0105] 4) The invention proposes a confidence noise adaptive Kalman filter algorithm, which not only improves the adaptability to environmental changes, but also reduces tracking errors caused by sudden noise and false detection. Especially in dynamic scenes, such as tracking the nonlinear motion of individuals in a crowd, this method can significantly improve the continuity and reliability of tracking.

[0106] 5) Experimental results show that the overall performance of the invention on the TAO multi-target tracking dataset is better than that of existing advanced open-vocabulary multi-target tracking methods, showing good generalization ability and practical prospects.

[0107] In summary, the invention significantly improves the performance of open-vocabulary multi-target tracking, especially in identifying and tracking unpredefined or unseen targets, showing excellent adaptability and accuracy, and has good application prospects.

[0108] The above examples are only used to illustrate the technical solutions of the invention, and not to limit it; although the invention has been described in detail with reference to the foregoing examples, those skilled in the art should understand that they can modify the technical solutions described in the foregoing examples, or make equivalent substitutions for part of the technical features; and these modifications or substitutions will not cause the essence of the corresponding technical solution to deviate from the spirit and scope of the technical solutions of the embodiments of the invention.

Claims

1. An open vocabulary multi-target tracking method based on confidence adjustment and wavelet convolution, characterized in that: The following steps are involved: Step 1: In the training phase, the current image and historical images are used as input and processed by the backbone network; A Haar wavelet convolution layer is introduced into the convolution layer to extract features from the input image and obtain the feature map of each channel. Subsequently, a Haar wavelet transform is applied to each channel to generate four downsampled sub-band feature maps. The sub-band feature maps are then restored to a fused feature map of the same dimension as the original input through an inverse Haar wavelet transform. Step 2: In the inference phase, the motion model and appearance model are combined to build a target tracking model. The appearance features and motion features are used to build a joint cost matrix to obtain the target tracking model for target matching. The joint cost matrix adopts formula (4): Formula (4) in, is the joint cost matrix, Represents motion-based features Secondary testing and Similarity measure between traces; Represents appearance-based features Secondary testing and Similarity measure between traces; Step 3: In the tracking phase, the confidence weighted Kalman update module dynamically adjusts the weight of each detection frame in the Kalman filter update phase according to the confidence score of each detection frame; Below threshold The detection frame of the original detection frame The detected box is replaced with a position closer to the predicted box, and is weighted according to the detection confidence. The measurement noise related to the detection box confidence is dynamically adjusted through the confidence noise adaptive Kalman filter algorithm. The trajectory update adopts a hierarchical adaptive feature update strategy.

2. The open vocabulary multi-target tracking method based on confidence adjustment and wavelet convolution according to claim 1, characterized in that The specific steps of step one are as follows: 1.

1. The current image and historical images are processed through the ResNet50 backbone network. In the second convolutional layer of each residual block, a Haar wavelet convolution layer is introduced to replace the traditional convolutional layer to extract features, thereby obtaining the feature map of each channel. 1.

2. For each channel’s feature map, a deep convolution operation is performed through the four sets of filters defined by formula (1) to realize the two-dimensional Haar wavelet transform; for the image For each input channel of , the depthwise convolution operation will produce four downsampled sub-band feature maps as shown in formula (2): Formula (1) Formula (2) in, is a low-pass filter, and 、 and A set of high-pass filters is formed. represents the low-frequency component of the image, and 、 and They represent the high-frequency components of the image in the horizontal, vertical and diagonal directions respectively; 1.

3. Perform inverse Haar wavelet transform through the transposed convolution operation defined in formula (3) to restore the original dimension of the subband feature map: Formula (3) in, Represents a transposed convolution operation.

3. The open vocabulary multi-target tracking method based on confidence adjustment and wavelet convolution according to claim 2, characterized in that: The Haar wavelet convolution layer is embedded in ResNet50; the input image is from the TAO dataset.

4. The open vocabulary multi-target tracking method based on confidence adjustment and wavelet convolution according to claim 3, characterized in that: In step 3, the confidence weighted Kalman update module uses formula (5) to dynamically adjust its weight in the Kalman filter update phase: Formula (5) in, represents the estimated position of the i-th frame, It is the Prediction of frame positions, is the confidence-adjusted measurement value, It is in The Kalman gain calculated by the frame is used to update the estimated value, represents the projection matrix, Indicates the target measurement value.

5. The open vocabulary multi-target tracking method based on confidence adjustment and wavelet convolution according to claim 4, characterized in that: In step 3, for the confidence Below threshold The detection frame of the original detection frame is converted into Replace it with a position close to the predicted box and weight it according to the detection confidence: Official (6).

6. The open vocabulary multi-target tracking method based on confidence adjustment and wavelet convolution according to claim 5, characterized in that: In step 3, the confidence noise adaptive Kalman filter algorithm uses formula (7): Formula (7) in, is the amplified measurement space noise covariance matrix, is the confidence of the detection box, N is the noise covariance of the measurement space, is the amplification factor.

7. The open vocabulary multi-target tracking method based on confidence adjustment and wavelet convolution according to claim 6, characterized in that: In step 3, detection results are assigned to existing tracks, and tracks are divided into two states: dormant and active. When no new detection results are assigned to a track in a given frame, the track will be set to dormant; otherwise, the track remains active. If a track remains in dormant state for more than a predetermined number of frames, the track is discarded.

8. The open vocabulary multi-target tracking method based on confidence adjustment and wavelet convolution according to claim 7, characterized in that: For dormant trajectories, the trajectory features are updated using formula (8): Formula (8) in, is the updated feature vector, is the feature vector detected in the current frame, is the latest eigenvector on the trajectory, is a coefficient that controls the update ratio.

9. The open vocabulary multi-target tracking method based on confidence adjustment and wavelet convolution according to claim 8, characterized in that: For active trajectories, the trajectory features are updated using formula (9): Formula (9) in, is another coefficient that controls the update ratio, It is a score determined by the ratio of the detection confidence of the current object to the total detection confidence of the current frame.

10. A computer storage medium, characterized in that The computer storage medium is used to store program data, and when the program data is executed by a computer, it is used to implement the open vocabulary multi-target tracking method based on confidence adjustment and wavelet convolution according to any one of claims 1 to 9.

Images