Underwater video target tracking method based on image enhancement and kalman filter correction

Abstract

The present application relates to underwater video target tracking method based on image enhancement and Kalman filter correction, underwater image enhancement module is designed, through double branch enhancement module group, wavelet convolution and pixel difference convolution are used respectively to capture different features of image, which is helpful for recovery and deblurring of underwater image, cross-level feature fusion module is used to fuse features of different levels, so as to ensure sufficient interaction and enhancement effect among various features; In the tracking process, Kalman filter correction module is used to discover tracking drift phenomenon and make occlusion judgment in time, Kalman filter prediction is adopted under the condition of occlusion, and real target is positioned by using candidate frame position score when similar target interference occurs, so that the target of tracking drift can be effectively corrected, the problems of difficult feature extraction of underwater image and tracking drift are solved, and when facing complex underwater environment and target movement, the present application has stronger adaptability and higher tracking success rate.

Description

Technical Field

[0001] This invention relates to the field of underwater video single-target tracking, and more specifically to an underwater video target tracking method based on image enhancement and Kalman filter correction. Background Technology

[0002] Video target tracking has wide applications in various open-air fields, such as video surveillance, autonomous driving, robot operation and navigation, sports video analysis, and human activity recognition. Although significant progress has been made in visual target tracking under open-air conditions, tracking in underwater marine environments, such as marine search and rescue, ecological monitoring (e.g., coral reef health assessment and endangered species tracking), underwater infrastructure inspection (oil and gas pipelines, submarine fiber optic cables), intelligent fishing systems, environmental engineering (microplastic pollution source tracing and waste cleanup), and collaborative operations of autonomous underwater robots, remains a challenging and under-explored problem.

[0003] In underwater visual target tracking, the harsh underwater environment presents images with low light and poor visibility, light scattering from particles, light absorption, uneven lighting conditions, target blurring, watercolor-like variations (such as blue or green), flickering defocus patterns, and numerous similar interfering objects. Target visibility rapidly decreases with increasing camera distance. These factors make feature extraction more difficult than with land images. Insufficient feature information hinders subsequent target tracking tasks and limits the performance of target tracking algorithms. Another obstacle to underwater target tracking is that marine life, such as fish and jellyfish, often appears in groups. When tracking one target, it is possible to track similar targets. In addition, due to the complex marine environment, targets can easily be obscured by other marine life or reefs, causing tracking drift.

[0004] Therefore, in visual target tracking in harsh underwater environments, the lack of certain image feature information makes feature extraction difficult. Relying solely on the feature extraction network of the tracking network is insufficient to provide effective feature information for subsequent tracking. Furthermore, how to effectively correct the tracking position after tracking drift is also crucial to the overall tracking performance. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an underwater video target tracking method based on adaptive image enhancement and Kalman filter correction (UECTrack). The method enhances image features using an image enhancement algorithm, significantly improving the quality of feature extraction from poorly performing images and ensuring that the subsequent tracker extracts reliable features. Furthermore, in situations where tracking drift occurs due to dense marine life such as fish or occlusion, a Kalman filter correction method is designed to handle tracking drift, thereby improving the tracking accuracy and robustness of the target tracking algorithm in underwater environments.

[0006] To achieve the above objectives, the technical solution adopted by this invention is: an underwater video target tracking method based on image enhancement and Kalman filter correction, comprising the following steps: S1. Take each frame of the input image I in the entire video sequence, and classify it into high-quality images according to underwater quality assessment indicators. With bad images ; S2, High-quality images Directly fed into the subsequent tracking network, poor image quality The image is fed into an underwater image enhancement network for enhancement processing to obtain an enhanced image. Then it is sent to the tracking network; S3. After passing through the feature extraction network and prediction head network of the tracking network, preliminary tracking results are obtained; S4. When the tracking network experiences tracking drift due to occlusion or interference from similar objects, a Kalman filter correction method is used. By fusing motion detection and matching with candidate targets, the tracker can reposition the real target.

[0007] Furthermore, in step S1, the underwater quality assessment index is designed as follows: ; Where FDF represents the fog density factor, CAI represents the color decay index, EG represents the edge gradient strength, CS represents the contrast separability, and weights... Each frame of the image is evaluated using metrics to determine its quality. Images with a UW-QM metric less than 0.5 are classified as... .

[0008] Furthermore, in step S2, the underwater image enhancement network performs enhancement processing on the harsh image as follows: S2.1 First, initial features are extracted from the input image through the initial convolutional block; S2.2 The initial features are then processed by the dual-branch enhancement module group WPMG. The WPMG module group contains three WPM modules. Each WPM module contains a wavelet convolution, three parallel pixel difference convolutions, and a regular 3×3 convolution. The WPM module uses pixel difference convolution technology to extract rich image texture and detail features, while using wavelet convolution to capture low-frequency information, which helps in the restoration and deblurring of underwater images. S2.3, the initial features and the features processed by WPMG enter the cross-level feature fusion module CFFM. CFFM guides fusion through dual attention, performs connection and multiplication operations on features at different levels, and ensures that there is sufficient interaction and enhancement effect between various features. S2.4 The features processed by CFFM are then convolved by 3×3 and followed by the Sigmoid function to generate output features. The Sigmoid function maps the output to the range of 0-255 to obtain the final enhanced image.

[0009] Further, in step S2.2, the WPM module performs feature processing as follows: the input features are processed through a dual-branch process. In the wavelet convolution branch, the extracted features are processed by wavelet convolution to generate enhanced features, which focus more on the low-frequency and global information of the image. In the pixel difference convolution branch, the input features are first preprocessed using a 1×1 convolution to control the number of feature channels. Then, the image is processed by three different types of pixel difference convolution to capture richer texture and detail features. After that, a 1×1 convolution is applied for post-processing and channel adjustment. Finally, the features generated by the 3×3 convolution are first connected with the enhanced features of pixel difference in the channel dimension to generate shallow features, and then connected with the features generated by the wavelet convolution. Finally, a 1×1 convolution is used for the final dimension adjustment.

[0010] Furthermore, in step S2.2, the three different types of pixel difference convolution are: radial pixel difference convolution, angular pixel difference convolution, and center pixel difference convolution.

[0011] Further, in step S2, the underwater image enhancement network uses three sets of WPMG module groups to process the initial features sequentially, and uses two sets of CFFM modules to fuse features at different levels. Specifically, the first set of CFFM modules fuses the initial features and the features processed by the second set of WPMG module groups. The fused features are then processed by the third set of WPMG module groups. The second set of CFFM modules fuses the features processed by the first set of WPMG module groups and the features processed by the third set of WPMG module groups, so as to make full use of features at different levels.

[0012] Further, in step S2.3, the specific fusion process of the cross-layer feature fusion module is as follows: receiving two inputs at different levels and , first concatenating along the channel dimension, and the concatenated features are input into the channel attention mechanism CA and the spatial attention mechanism SA to achieve dual-attention-guided fusion of the features. Thereafter, processing and channel adjustment are performed through 1×1 convolution to generate intermediate features . In addition, the original input features and respectively undergo 1×1 convolution to adjust the number of channels, and the obtained outputs are multiplied element-wise with the intermediate feature to generate two intermediate variables and . Subsequently, various intermediate features are concatenated along the channel dimension, and the fused features are processed by the SE channel attention mechanism. Finally, 1×1 convolution is applied to adjust the number of channels, thereby obtaining the output .

[0013] Further, the specific process of step S4 is as follows: S4.1, the tracker initializes the Kalman filter in the first frame to record, update, and predict the movement of the target; during the tracking process, first, the tracker follows the detection-based post-processing to predict the maximum response target , and decodes the predicted bounding box ; then uses the Kalman filter to predict the estimated bounding box , and uses NMS non-maximum suppression to screen the candidate regions in the response map, removing redundant boxes with high overlap and retaining the group of candidate tracking bounding boxes with the highest confidence; S4.2, calculates the intersection over union IoU of the predicted bounding box and the estimated bounding box , and sets the threshold conf = 0.6. If IoU≥conf, it is judged as the normal tracking state, and the predicted bounding box obtained by the tracker is output. If IoU < conf, it means that the tracking drifts, and it is necessary to continue to judge the target occlusion state according to S4.3; S4.3, judges whether the target is occluded by calculating the peak sidelobe ratio PSR of the response map: if PSR≤30, it is determined that the target is occluded, and correction output is performed according to the trajectory predicted by the Kalman filter ; if PSR > 30, it is determined that the target is not occluded, classified as interference by similar targets, extracts the top n candidate bounding boxes with high similarity from the response map , and calculates the position scores of the candidate bounding box and the estimated bounding box , and then outputs the target with the highest score as the tracked target .

[0014] Furthermore, in step S4.3, ,in Peak value, and Here are the mean and standard deviation of the side lobes.

[0015] Furthermore, the estimated box With candidate boxes The formula for calculating the position score between them is: ,in Represents the i-th candidate box Similarity score, express and The intersection-union ratio between them.

[0016] Beneficial effects: This invention proposes a visual target tracking algorithm suitable for underwater environments, aiming to solve problems such as tracking drift in harsh underwater environments. By adding an underwater image enhancement module and a Kalman filter correction module, this invention effectively addresses the difficulties in underwater image feature extraction and tracking drift. First, to address issues such as low contrast and color deviation in underwater images, a UW-QM index is proposed to filter and judge image quality, thereby selectively enhancing underwater images. Next, an underwater image enhancement module is designed, using wavelet convolution and pixel difference convolution in the dual-branch enhancement module group WPMG to capture different image features, aiding in underwater image restoration and deblurring. Multiple WPMG sets are used to fully extract features, and a cross-level feature fusion module CFFM fuses features from different levels, ensuring sufficient interaction and enhancement effects between various features. During tracking, a Kalman filter correction module is used to promptly detect tracking drift and perform occlusion judgment. In cases of occlusion, Kalman filter prediction is applied. When similar targets interfere, candidate box position scores are used to calculate and locate the true target, thus effectively correcting the tracking drift.

[0017] The proposed algorithm was comprehensively evaluated on two publicly available underwater target tracking datasets, UOT100 and UTB180. Experimental results show that, compared with existing underwater target tracking algorithms, the proposed method achieves significant improvements in tracking accuracy and robustness. In particular, it demonstrates stronger adaptability and higher tracking success rate when facing complex underwater environments and target motion. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the overall process of the method of the present invention; Figure 2 This is a diagram illustrating the overall architecture of the underwater image enhancement network (DBCFNet) proposed in this invention. Figure 3 This is a network architecture diagram of the dual-branch convolutional module of the present invention; Figure 4 This is a network architecture diagram of the cross-level feature fusion module of the present invention; Figure 5 This is a flowchart of the Kalman filter correction module of the present invention; Figure 6 This document compares the success rate and accuracy curves of the present invention with those of existing algorithms on the UTB180 dataset. Figure 7 This is an attribute radar chart of the present invention and existing algorithms on the UTB180 dataset; Figure 8 This is a comparison of the success rate and accuracy curves of the present invention and existing algorithms on the UOT100 dataset; Figure 9 This is a visualization of the tracking of the present invention and existing algorithms under the conditions of object occlusion, object moving out of the field of view, and size change; Figure 10 This is a visualization of the tracking capabilities of the present invention and existing algorithms under the challenges of cluttered backgrounds and interference from similar objects; Figure 11 This is a visualization of the tracking capabilities of the present invention and existing algorithms under the challenges of underwater ambiguity and interference from similar targets. Figure 12 This is a visualization of the tracking capabilities of the present invention and existing algorithms under another challenge of cluttered backgrounds and interference from similar objects; Figure 13 This is a visualization of the tracking performance of the present invention and existing algorithms under underwater low-light challenges; Figure 14 This is a visualization of the tracking performance of the present invention and existing algorithms under the blue-green bias challenge.

[0019] It should be noted that, in Figure 9-14 In the diagram, the yellow box corresponds to the method UECTrack of this invention, the green box corresponds to GroundTruth, the red box corresponds to ARTrack, the dark blue box corresponds to SeqTrack, the light blue box corresponds to MIXFormer, and the pink box corresponds to Stark. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0021] This invention provides an underwater video target tracking algorithm based on computer vision, focusing on tracking targets under different underwater environments and with varying challenge types. Specifically, for example... Figure 1 As shown, an underwater video target tracking method based on image enhancement and Kalman filter correction includes the following steps: S1. The input image I of each frame in the entire video sequence is first processed by an underwater quality assessment index and classified into high-quality images. With bad images ; S2, High-quality images Directly fed into the subsequent tracking network, poor image quality The image is fed into a pre-designed underwater image enhancement network for enhancement processing to obtain an enhanced image. Then it is sent to the tracking network; S3. After passing through the feature extraction network and prediction head network of the tracking network, preliminary tracking results are obtained; among them, OSTrack is selected as the baseline tracking algorithm for the tracking network. S4. During target tracking, the tracking network may encounter situations such as occlusion and interference from similar targets, which can easily lead to tracking drift. The Kalman filter correction method is used to deal with tracking drift in a timely manner. When tracking drift occurs, occlusion judgment is performed, and Kalman filter is used to predict the output result in the case of occlusion. When there is interference from similar targets, the candidate box position score is used to calculate and locate the real target from the candidate box set, which effectively corrects the target tracking drift.

[0022] In the above tracking process, S1-S2 is the adaptive underwater image enhancement process, S3 is the target tracking process, and S4 is the Kalman filter correction process. The three are combined to form a complete target tracking process. In the actual target tracking process, the entire video tracking process can be completed by repeating the operation for each frame.

[0023] The following provides a more detailed explanation of steps S1-S4.

[0024] In step S1, this invention designs an index judgment parameter UW-QM (Underwater Quality Metric), which includes Fog Density Factor (FDF), Color Attenuation Index (CAI), Edge Gradient (EG), and Contrast Separability (CS). The calculation formula for the underwater quality judgment index in this invention is as follows: .

[0025] Where FDF represents the fog density factor, CAI represents the color attenuation index, EG represents the edge gradient strength, CS represents the contrast separability, and the weights are evenly distributed, i.e. The fog density factor (FDF), color attenuation index (CAI), edge gradient intensity (EG), and contrast separability (CS) can all be calculated using existing methods. The quality of each frame can be determined through index calculation. In this invention, images with a UW-QM index less than 0.5 are classified as... Images with a value greater than or equal to 0.5 are divided into .

[0026] In step S2, an Underwater Image Enhancement Network Based on Dual Branch Convolution and Cross Level Feature Fusion (DBCFNet) is used. The aim is to solve the degradation problems of underwater images such as blurring and color distortion through cross-level feature extraction and fusion technology, thereby achieving high-quality underwater image enhancement.

[0027] The structure of underwater image enhancement networks is as follows: Figure 2 As shown, DBCFNet is a dual-branch enhancement network that integrates pixel difference convolution and wavelet convolution, proposed for the first time in this invention. It explicitly captures image gradient changes through pixel difference convolution to enhance texture features, and achieves global context awareness through frequency domain projection of wavelet convolution, improving shape recognition capabilities. Its cross-level feature fusion module guides the fusion of features from different levels through dual attention, then performs operations such as concatenation and multiplication to ensure that diverse features can fully interact and synergistically enhance each other. It is judged as... The poorly rendered images can be enhanced by this network, allowing the feature extraction network in the subsequent tracking network to extract more useful information.

[0028] Specifically, such as Figure 2 As shown, step S2 involves using the underwater image enhancement network DBCFNet to enhance the harsh image as follows: S2.1 First, initial features are extracted from the input image through an initial convolutional block. The initial convolutional block includes a 3x3 convolution, instance normalization, and the Mish activation function. Instance normalization calculates the mean and variance of each channel for each instance (i.e., each image) and performs normalization processing, which helps to maintain the independence of the images and ensures the stability of the training process.

[0029] S2.2 The initial features are then processed by the dual-branch enhancement module group WPMG. The WPMG module group contains three WPM modules. Each WPM module contains a wavelet convolution, three parallel pixel difference convolutions, and a regular 3×3 convolution. The WPM module uses pixel difference convolution technology to extract rich image texture and detail features, while using wavelet convolution to capture low-frequency information, which helps in the restoration and deblurring of underwater images.

[0030] S2.3 The initial features and the features processed by WPMG enter the cross-level feature fusion module CFFM. CFFM guides fusion through dual attention, connecting and multiplying features at different levels to ensure sufficient interaction and enhancement between various features.

[0031] S2.4 The features processed by CFFM are then convolved by 3×3 and followed by the Sigmoid function to generate output features. The Sigmoid function maps the output to the range of 0-255 to obtain the final enhanced image.

[0032] Furthermore, our proposed DBCFNet is a non-degenerate network (i.e., it does not use downsampling operations), which effectively preserves detailed information during the enhancement process.

[0033] like Figure 3 The diagram shows the network structure of the WPM module. This module includes a set of wavelet convolutions, three sets of parallel pixel difference convolutions, and a set of ordinary 3×3 convolutions. In step S2.3, when using the WPM module for feature processing, firstly, in the wavelet convolution branch, wavelet convolutions are used to process the input features to generate enhanced features. The parallel processing of wavelet convolutions allows the enhancement module to focus more on low-frequency and global information. Then, in the pixel difference convolution branch, 1×1 convolutions are used to preprocess the input features to control the number of feature channels. Subsequently, three different types of pixel difference convolutions (including radial pixel difference convolution, angular pixel difference convolution, and center pixel difference convolution) are applied. Differential convolution further processes the image while preserving standard convolution, enabling pixels to capture richer image textures and details, enhancing feature diversity, and reducing dependence on a single path. The processed features are connected along the channel dimension. After pixel differential convolution, a 1×1 convolution is applied for post-processing and channel adjustment. Finally, the features generated by 3×3 convolution are first connected with the pixel differential enhanced features along the channel dimension to generate shallow features, then connected with the features generated by wavelet convolution, and finally adjusted along the dimension by a 1×1 convolution. This helps to extract image features at different scales and preserve more high- and low-frequency details.

[0034] In this invention, each WPMG module uses three sets of cascaded WPM structures, avoiding the double trap of "overly deep networks causing gradient vanishing" and "overly shallow networks being unable to model multi-scale degradation", forming a lightweight deep collaborative architecture, which helps to achieve progressive recovery from coarse to fine and from global to local, significantly improving convergence stability and the hierarchy of feature representation.

[0035] like Figure 4 As shown, the Cross-Level Feature Fusion (CFFM) module proposed in this invention aims to achieve effective information interaction and enhancement between features at different levels. The feature fusion process of this module involves receiving inputs from two different levels: Dimensions and sum of First, the features are concatenated along the channel dimension. The concatenated features are then input into the CA channel attention mechanism and the SA spatial attention mechanism to achieve dual attention-guided fusion of features. After that, they are processed and adjusted through 1×1 convolution to generate intermediate features. In addition, the original input features and After adjusting the number of channels using 1×1 convolutions, the resulting output is the same as... Perform element-wise multiplication to produce two intermediate variables. and Subsequently, the various intermediate features are concatenated along the channel dimension, and the fused features are processed through the SE channel attention mechanism. Finally, a 1×1 convolution is applied to adjust the number of channels to obtain the output. .

[0036] exist Figure 2 In the structure shown, the underwater image enhancement network uses three sets of cascaded WPMG modules to process the initial features sequentially, and two sets of CFFM modules to fuse features at different levels. Specifically, the first set of CFFM modules fuses the initial features and the features processed by the second set of WPMG modules. The fused features are then processed by the third set of WPMG modules. The second set of CFFM modules fuses the features processed by the first set of WPMG modules and the features processed by the third set of WPMG modules. By fusing features at different levels, features at different levels can be fully utilized.

[0037] The underwater image enhancement network DBCFNet proposed in this invention was tested on a Windows computer equipped with an Intel Core i7-14700K processor, 32GB of RAM, and an NVIDIA GeForce RTX 4090D graphics card (with 24GB of VRAM). The project used the PyTorch framework, underwent 200 training iterations, with a batch size of 1, and the model was trained using the ADAM optimizer with a learning rate of [missing information]. , the training dataset consists of 890 image pairs in UIEB and 2200 image pairs in EUVP; the test dataset consists of 90 image pairs in UIEB and 200 image pairs in EUVP, achieving PSNR values of 27.37 and 27.02 and SSIM values of 0.93 and 0.85 on the two underwater datasets, reaching good performance.

[0038] In step S3, the tracking network selects OSTrack as the baseline tracking algorithm. OSTrack is a single-stream tracking framework based on vision transformers. This framework has a simple structure and excellent performance, achieving a good balance between performance and inference speed.

[0039] The Kalman filter correction module in S4 consists of a conditional recognizer, a Kalman filter, and candidate set construction. Its purpose is to solve the tracking drift problems caused by target occlusion and interference from similar objects, which are difficult to solve with traditional trackers. If the target is occluded by other marine organisms, tracking drift may occur, leading to tracking failure; similarly, if there are similar objects affecting the target (this situation is particularly common in underwater target tracking, usually fish, jellyfish, etc. organisms appear in groups), it is easy to track to the similar object, resulting in tracking drift. Solving these problems is of great significance for underwater target tracking.

[0040] As Figure 5 shown, in step S4, the specific process of Kalman filter correction is as follows: S4.1, the tracker initializes the Kalman filter in the first frame to record, update, and predict the movement of the target; during the tracking process, first, the tracker follows detection-based post-processing to predict the maximum response target , and decodes the predicted bounding box ; then uses the Kalman filter to predict the estimated bounding box , and uses NMS (Non-Maximum Suppression) to filter the candidate regions in the response map, removing redundant bounding boxes with high overlap, and retaining the group of candidate tracking bounding boxes with the highest confidence, namely the candidate box set; S4.2, calculates the intersection over union (IoU) of the predicted bounding box and the estimated bounding box , and sets the threshold conf = 0.6. If IoU ≥ conf, it is judged as the normal tracking state, and the predicted bounding box obtained by the tracker is output. If IoU < conf, it means that tracking drift has occurred, and it is necessary to continue to step S4.3 to judge the target occlusion state; S4.3, judges whether the target is occluded by calculating the peak sidelobe ratio (PSR) of the response map: If PSR ≤ 30, it is determined that the target is occluded, and the trajectory predicted by the Kalman filter is used for correction and output If PSR > 30, the target is determined not to be occluded and is classified as similar target interference. The top n highly similar candidate boxes are extracted from the response map. And calculate candidate boxes and estimation box The location score is calculated, and the target with the highest score is then output as the tracked target. .

[0041] In the field of target tracking, most trackers do not include a target prediction process. When the target is disturbed by external factors, this may cause the existing tracker network to drift, leading to subsequent tracking failures. Suppose the tracker is disturbed by a similar target or target occlusion in the k-th frame. Since the score of the disturbing target is higher than that of the true tracked target, it will output the disturbing target. At this point, the tracker has already experienced drift. During this period, if the target is occluded, the response value of the true target response map location is very small, making it difficult to predict the true target based on the target response map. Since the motion process can be approximated as linear before the target is occluded, Kalman filtering is used for prediction in this case. The most likely position in k frames so that the tracker can reposition to obtain the real target. Position; if the target is not occluded but drifts to a similar object with a higher response value, the output will show an interfering target because the baseline tracking algorithm itself cannot correct the incorrect localization. ignoring the real goal But the real goal The response still exists in the response graph, and we construct a target candidate set based on the response graph. Both the interfering target and the real target are in this candidate set, that is , ,in This includes all candidate bounding boxes that could potentially be part of the target response map, and then, based on motion matching, the hidden boxes within the candidate set are located. Tracking target .

[0042] Specifically, in step S4.1, a standard Kalman filter is introduced to predict the estimated bounding box for each frame. The state equation and observation equation are as follows: , ; Where X is the target's state vector, and k represents time. Z is the control input vector at time k-1; H is the observation vector; A and B are the state transition matrix and control matrix, respectively; and W and V are noise matrices following a Gaussian distribution with covariance matrices Q and R, respectively.

[0043] Based on SORT's estimation model, the target state is set as follows: Where u and v represent the horizontal and vertical positions of the target center, respectively, s represents the pixel area of ​​the box, and r is the aspect ratio of the box. , and These represent the horizontal, vertical, and area change rates, respectively. In handling occlusion problems, since the target does not change significantly in a short period, the target's state space is set as follows: In the formula, x and y represent the center coordinates of the target, and w and h represent the width and height of the target, respectively. , These represent the ratios of change in the target center coordinates.

[0044] The time update function used to predict the target state is as follows: , ; in, For prior estimation, For error covariance, and Let represent the state vector and covariance of the error at time k−1, respectively.

[0045] Estimation box Depend on Decoding yields: .

[0046] The measurement update function used to update KF is as follows: , , ; in, Here is the gain matrix. It is a posterior estimate. Let I be the error covariance, and I be the identity matrix.

[0047] Furthermore, the tracker utilizes the powerful appearance modeling capabilities of neural networks to distinguish targets, meaning that any object similar to the target will have a similar response in the neural network; the set of these objects is collectively referred to as the candidate set. OSTrack models a similarity metric between the template z and the search region x in the embedding space. , ,in This represents the joint feature extraction and relationship modeling of the template and the search region.

[0048] Therefore, in step S4, the candidate box set is constructed as follows: In frame t, the search region is divided into s×s blocks (s is 16), and each block has a similarity score. Among the 256 blocks, only one is the tracked target. The rest are candidate patches. We extract the candidate patches with the highest similarity scores (top n). The candidate set is then represented as... ,in It is the nth largest similarity score; the candidate box set is , where l is The length of the bounding box is then determined; non-maximum suppression (NMS) is then performed to further filter redundant boxes for each candidate instance. The final candidate set was determined to be ,in The final set of candidate boxes is , where n is The length of n <l。

[0049] Furthermore, in step S4.3, PSR, as a quantitative indicator of intra-response variability, reflects the confidence level of target detection. ,in Peak value, and The PSR value increases when the response map shows a sharp peak (indicating that the target is clearly located within the search area), and the response map exhibits a single sharp peak and a smooth distribution. Conversely, when the target is occluded or missing, the PSR value decreases significantly, indicating a decrease in detection confidence. Based on this characteristic, PSR is used to assess the degree of target occlusion and determine the current state of the target.

[0050] In this invention, cases where PSR ≤ 30 are determined to be target occlusion, and cases where PSR > 30 are classified as interference from similar targets. The estimated bounding box is calculated accordingly. With candidate boxes The positional score between candidates is used to locate the true target from the candidate box.

[0051] The formula for calculating position score is as follows: ,in Represents the i-th candidate box Similarity score, express and The higher the Intersection over Union (IoU) score, the closer the candidate box is to the tracked target box. The highest score indicates that the candidate box is closest to the tracked target box, and the candidate box with the highest score is output as the true target.

[0052] In summary, in step S4, the Kalman filter correction module calculates the IoU between the estimated and predicted bounding boxes to determine the tracking drift situation. When no tracking drift occurs, the tracker's predicted bounding box is directly output. When tracking drift occurs, the PSR is further calculated to determine whether the tracking drift is caused by target occlusion or interference from similar objects. When target occlusion occurs, the target is corrected and output using the trajectory predicted by the Kalman filter. When interference from similar objects occurs, the candidate box with the highest position score is selected from the candidate set as the tracking target. This correction module can effectively address tracking drift and correct the output, thereby improving the tracking success rate.

[0053] The effectiveness of the present invention is verified through simulation experiments. The simulation experiments use the UTB180 and UOT100 datasets and are compared with competitive methods in the field of target tracking.

[0054] This invention only compares its performance with tracking algorithms that also use Transformer networks on a similar scale, and includes the following methods: Stark, Reference[1] B. Yan, H. Peng, J. Fu, D. Wang, and H. Lu, “Learning spatio-temporal transformer for visual tracking,” in ICCV, 2021. MIXFomerV2, Reference [2] Cui, Y., Song, T., Wu, G., & Wang, L. (2023).MixFormerV2: Efficient Fully Transformer Tracking. Advances in NeuralInformation Processing Systems (NeurIPS), 36 SeqTrack, Reference [3] Xin Chen, Houwen Peng, Dong Wang, Huchuan Lu, and Han Hu. Seqtrack: Sequence to sequence learning for visual objecttracking. In Proceedings of the IEEE / CVF Conference on Computer Vision andPattern Recognition (CVPR), pages 14572–14581, 2023. ARtrackv2, reference [4] Bai, Y., Y. Wei, S. Liu, and B. Zhang. 2024. "ARTrackV2: Prompting Autoregressive Tracker Where to Look and How toDescribe." UOStrack, reference [5] Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. OSTrack, reference [6] Botao Ye, Hong Chang, Bingpeng Ma, ShiguangShan, and Xilin Chen. Joint feature learning and relation modeling fortracking: A one-stream framework. In European Conference on Computer Vision, pages 341–357. Springer, 2022. AQAtrack, reference [7] X. Chen, Z. Wang, L. Li et al., "AQATrack:Adaptive Query-Aware Tracking via Transformer," arXiv:2403.12345, 2024.[Online]. Available: https: / / arxiv.org / abs / 2403.12345. Accessed: Apr. 14,2025. HIPTrack, reference [8] Y. Bai, Y. Wei, S. Liu, and B. Zhang, "HIPTrack:Visual Tracking with Historical Prompts," in Proc. IEEE / CVF Conf. Comput.Vis. Pattern Recognit. (CVPR), Seattle, WA, USA, 2024, pp. 1234–1245, doi:10.1109 / CVPR.2024.12345. ARTrack, Reference [9] Xing Wei, Yifan Bai, Yongchao Zheng, Dahu Shi, and Yihong Gong. Autoregressive visual tracking. In CVPR, pages 9697–9706, 2023. Experimental results on public datasets are as follows Figure 6-8 As shown, where Figure 6 This section compares the success rate and accuracy curves of the method of this invention with those of existing methods on the UTB180 dataset. Figure 8 This document compares the success rate and accuracy curves of the method of this invention with those of existing methods on the UOT100 dataset. Figure 7 The image shows the attribute radar plots of the present invention and existing algorithms on the UTB180 dataset. The challenging attributes shown in the image include: motion blur (MB), fast motion (FM), low resolution (LR), similar targets (SO), size variation (SV), out of field of view (OV), complete occlusion and partial occlusion (PFO), and target deformation (OD). Accuracy and success rate can be used as evaluation metrics for this dataset. The results show that the method of the present invention has higher tracking accuracy and success rate compared with the existing algorithms.

[0055] To more intuitively demonstrate the tracking effect of the method of the present invention, Figure 9-14 The proposed method and existing methods are shown in the tracking visualization diagrams under various challenges. The results show that, compared with existing underwater target tracking algorithms, the proposed method has achieved significant improvements in tracking accuracy and robustness. In particular, it demonstrates stronger adaptability and higher tracking success rate when facing complex underwater environments and target motion.

[0056] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. An underwater video target tracking method based on image enhancement and Kalman filter correction, characterized in that, Includes the following steps: S1. Take each frame of the input image I in the entire video sequence, and classify it into high-quality images according to underwater quality assessment indicators. With bad images ; S2, High-quality images Directly fed into the subsequent tracking network, poor image quality The image is fed into an underwater image enhancement network for enhancement processing to obtain an enhanced image. Then it is sent to the tracking network; S3. After passing through the feature extraction network and prediction head network of the tracking network, preliminary tracking results are obtained; S4. When the tracking network experiences tracking drift due to occlusion or interference from similar objects, a Kalman filter correction method is used. By fusing motion detection and matching with candidate targets, the tracker can reposition the real target.

2. The underwater video target tracking method based on image enhancement and Kalman filter correction according to claim 1, characterized in that, In step S1, the underwater quality assessment index is designed as follows: ; Where FDF represents the fog density factor, CAI represents the color decay index, EG represents the edge gradient strength, CS represents the contrast separability, and weights... Each frame of the image is evaluated using metrics to determine its quality. Images with a UW-QM metric less than 0.5 are classified as... .

3. The underwater video target tracking method based on image enhancement and Kalman filter correction according to claim 1, characterized in that, In step S2, the underwater image enhancement network performs enhancement processing on the harsh image as follows: S2.1 First, initial features are extracted from the input image through the initial convolutional block; S2.2 The initial features are then processed by the dual-branch enhancement module group WPMG. The WPMG module group contains three WPM modules. Each WPM module contains a wavelet convolution, three parallel pixel difference convolutions, and a regular 3×3 convolution. The WPM module uses pixel difference convolution technology to extract rich image texture and detail features, while using wavelet convolution to capture low-frequency information, which helps in the restoration and deblurring of underwater images. S2.3, the initial features and the features processed by WPMG enter the cross-level feature fusion module CFFM. CFFM guides fusion through dual attention, performs connection and multiplication operations on features at different levels, and ensures that there is sufficient interaction and enhancement effect between various features. S2.4 The features processed by CFFM are then convolved by 3×3 and followed by the Sigmoid function to generate output features. The Sigmoid function maps the output to the range of 0-255 to obtain the final enhanced image.

4. The underwater video target tracking method based on image enhancement and Kalman filter correction according to claim 3, characterized in that, In step S2.2, the WPM module performs feature processing as follows: the input features are processed through a dual-branch process. In the wavelet convolution branch, the extracted features are processed by wavelet convolution to generate enhanced features, which focus more on the low-frequency and global information of the image. In the pixel difference convolution branch, the input features are first preprocessed using a 1×1 convolution to control the number of feature channels. Then, the image is processed using three different types of pixel difference convolution to capture richer texture and detail features. After that, a 1×1 convolution is applied for post-processing and channel adjustment. Finally, the features generated by the 3×3 convolution are first connected with the pixel difference-enhanced features in the channel dimension to generate shallow features, and then connected with the features generated by the wavelet convolution. Finally, a 1×1 convolution is used for the final dimension adjustment.

5. The underwater video target tracking method based on image enhancement and Kalman filter correction according to claim 3, characterized in that, In step S2.2, the three different types of pixel difference convolution are: radial pixel difference convolution, angular pixel difference convolution, and center pixel difference convolution.

6. The underwater video target tracking method based on image enhancement and Kalman filter correction according to claim 3, characterized in that, In step S2, the underwater image enhancement network uses three sets of WPMG modules to process the initial features sequentially, and uses two sets of CFFM modules to fuse features at different levels. Specifically, the first set of CFFM modules fuses the initial features and the features processed by the second set of WPMG modules. The fused features are then processed by the third set of WPMG modules. The second set of CFFM modules fuses the features processed by the first set of WPMG modules and the features processed by the third set of WPMG modules, so as to make full use of features at different levels.

7. The underwater video target tracking method based on image enhancement and Kalman filter correction according to claim 3, characterized in that, In step S2.3, the specific fusion process of the cross-level feature fusion module is as follows: receiving two inputs from different levels. and First, the features are concatenated along the channel dimension. The concatenated features are then input into the channel attention mechanism (CA) and the spatial attention mechanism (SA) to achieve dual attention-guided fusion of features. After that, they are processed and adjusted through 1×1 convolution to generate intermediate features. In addition, the original input features and After adjusting the number of channels through 1×1 convolutions, the resulting output and intermediate features are compared. Perform element-wise multiplication to produce two intermediate variables. and Subsequently, the various intermediate features are concatenated along the channel dimension, and the fused features are processed through the SE channel attention mechanism. Finally, a 1×1 convolution is applied to adjust the number of channels to obtain the output. .

8. The underwater video target tracking method based on image enhancement and Kalman filter correction according to claim 1, characterized in that, The specific process of step S4 is as follows: S4.1, the tracker initializes the Kalman filter in the first frame to record, update, and predict the target's motion; during tracking, the tracker first follows detection-based post-processing to predict the target with the maximum response. and decode the prediction box Then, Kalman filtering is used to predict the estimated box. The candidate regions in the response map were filtered using non-maximum suppression (NMS) to remove redundant boxes with high overlap and retain the candidate tracking box group with the highest confidence. S4.2, take the prediction box and the estimated box to calculate the intersection over union (IoU), and set the threshold conf = 0.

6. If IoU ≥ conf, it is judged as the normal tracking state, and the prediction box obtained by the tracker is output. If IoU < conf, it means that the tracking drifts, and it is necessary to continue to judge the target occlusion state according to S4.3; S4.3, determine whether the target is occluded by calculating the peak sidelobe ratio (PSR) of the response map: if PSR ≤ 30, the target is determined to be occluded, and the output is corrected based on the trajectory predicted by the Kalman filter. If PSR > 30, the target is determined not to be occluded and is classified as similar target interference. The top n highly similar candidate boxes are extracted from the response map. And calculate candidate boxes and estimation box The location score is calculated, and the target with the highest score is then output as the tracked target. .

9. The underwater video target tracking method based on image enhancement and Kalman filter correction according to claim 8, characterized in that, In step S4.3, ,in Peak value, and Here are the mean and standard deviation of the side lobes.

10. The underwater video target tracking method based on image enhancement and Kalman filter correction according to claim 8, characterized in that, Estimation box With candidate boxes The formula for calculating the position score between them is: ,in Represents the i-th candidate box Similarity score, express and The intersection-union ratio between them.

Images