RGBT target tracking method based on domain adaptation and spatiotemporal information fusion

Abstract

The present invention provides an RGBT target tracking method based on domain adaptation and spatiotemporal information fusion. It mainly solves the problem that existing target tracking solutions have poor tracking effects in complex scenarios such as target appearance changes and occlusions. It includes: 1) constructing an RGBT data sample set; 2) constructing a dual-branch RGBT target tracking model based on a pre-trained RGB tracker and training it; 3) designing an online module update strategy for dynamically updating the initial template image; 4) using the initial template image and the search image as inputs to the trained model, and using the dynamic template image as an additional input to the model; 5) loading the optimal parameters, using the trained model to predict the motion state of the target in subsequent frames, obtaining the target tracking results and outputting them. The present invention greatly reduces the computational cost and can fully utilize the target's temporal information, thereby effectively improving the performance of the feature extraction network and enhancing the accuracy and robustness of target tracking.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology and further relates to target tracking technology. Specifically, it is an RGBT target tracking method based on domain adaptation and spatiotemporal information fusion, which can be used to track and locate targets in security monitoring, human-computer interaction, and autonomous driving. Background Art

[0002] Visual object tracking, a fundamental task in computer vision, aims to locate an object in subsequent frames based on its initial position in the first frame. In recent years, a large number of benchmarks and outstanding works have emerged, achieving impressive tracking performance. Due to the imaging mechanism of visible light images, RGB images possess rich semantic information. However, the performance of RGB trackers degrades dramatically under conditions such as extreme lighting and inclement weather such as rain, fog, and snow. In contrast, TIR images are insensitive to these factors and, compared to RGB images, have low resolution and insufficient texture information. Therefore, to address the limitations of single-modality tracking methods, visible-light-thermal-infrared RGBT (RGB-Thremal) tracking has emerged, aiming to achieve robust tracking by complementing the information in RGB and TIR images. RGBT object tracking has garnered increasing research attention and has broad application potential in security monitoring, human-computer interaction, and autonomous driving.

[0003] With the advancement of computer vision, RGBT object tracking typically employs deep learning methods. This involves first extracting features from the initial template and search image, then performing a fusion module and prediction head to locate the target. Previous RGBT tracking methods (e.g., mfDiMP, APFNet, and HMFT) used an RGB tracker as the base model and fully fine-tuned the model parameters using RGBT datasets. While effective to a certain extent, this fully fine-tuned approach suffers from significant time costs and low efficiency. Furthermore, due to the limited number of annotated samples in RGBT datasets, it cannot leverage pre-trained knowledge from base models trained on large-scale datasets, resulting in a failure to obtain a generalized representation of the object.

[0004] In addition to the full parameter fine-tuning method, existing RGBT tracking methods (ViPT, SDSTrack, OneTracker, etc.) introduce parameter hint learning into the RGBT tracking method. These trackers freeze the backbone network parameters of the base model, add some additional trainable parameters, and generate modality-related hints, thereby adapting the frozen pre-trained model to the RGBT tracking method. This method reduces the computational burden to a certain extent, and the parameter hint learning-based approach achieves performance comparable to full parameter fine-tuning. However, the above-mentioned RGBT tracker ignores temporal information. During the tracking process, both the target and the background are constantly changing, making it difficult for the RGBT tracker to achieve good performance in complex tracking scenarios. Summary of the Invention

[0005] The present invention aims to address the shortcomings of the above-mentioned prior art and propose an RGBT target tracking method based on domain adaptation and spatiotemporal information fusion to solve the problem of tracking failure in existing target tracking technology under complex scenarios such as target appearance changes and occlusions.

[0006] The basic concept behind this invention is to migrate the model's feature extraction capabilities from the visible light domain to the multimodal domain by adding a small number of trainable parameters, thereby improving the model's domain adaptability. Furthermore, an online template update strategy is designed to obtain a dynamic template with temporal information, which complements the spatial information provided by the initial template, improving the tracker's ability to locate targets in complex environments such as those with changing appearance and occlusion. This invention significantly reduces computational costs and fully utilizes the target's temporal information, effectively improving the performance of the feature extraction network and enhancing the tracker's accuracy and robustness.

[0007] The present invention achieves the above-mentioned purpose by the following specific steps:

[0008] (1) Construct a visible light-thermal infrared RGBT data sample set containing real labels and divide it into a training set and a test set;

[0009] (2) Using the pre-trained RGB tracker OSTrack as a benchmark, a dual-branch RGBT target tracking model is constructed:

[0010] (2.1) Expand the pre-trained RGB tracker OSTrack into a dual-branch RGBT tracker, where the dual branches are the visible light RGB modality branch and the thermal infrared TIR modality branch;

[0011] (2.2) Add an adapter to the transformer encoder of the dual-branch RGBT tracker to construct a fine-tuned transformer encoder, and obtain a dual-branch RGBT object tracking model with domain adaptation capability;

[0012] (3) Import the pre-trained OSTrack model parameters, use the training set to train the dual-branch RGBT target tracking model to obtain the optimal parameters, and obtain the final RGBT target tracking model after training;

[0013] (4) Design an online module update strategy for dynamically updating the initial template image; specifically, set the classification score threshold to τ, and when the maximum classification score of the target is greater than τ, update the initial template image to obtain a dynamic template image and execute step (5); otherwise, do not update the initial template image and execute step (6);

[0014] (5) Using the initial template image, dynamic template image, and search image as inputs to the final RGBT target tracking model, directly execute step (7);

[0015] (6) The initial template image and the search image are used as inputs to the final RGBT target tracking model;

[0016] (7) Load the optimal parameters, use the final RGBT target tracking model to predict the target's motion state in subsequent frames, obtain the target tracking results and output them.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] First, since the present invention replaces the backbone of the benchmark model with a fine-tuned transformer encoder in the feature extraction stage, the feature extraction capability of the benchmark model is migrated from the visible light domain to the multimodal domain by fine-tuning the transformer encoder, thereby effectively improving the domain adaptability of the model.

[0019] Second, the present invention designs an online template update strategy during the tracking process, so that the temporal information provided by the dynamic template and the spatial information provided by the initial template are integrated with each other, thereby enabling the RGBT target tracking model based on domain adaptation and spatiotemporal information fusion to achieve robust tracking in complex scenarios such as target appearance changes and occlusions. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart for realizing the method of the present invention;

[0021] Figure 2 This is a schematic diagram of the RGBT tracker network structure constructed in the present invention;

[0022] Figure 3 It is a structural diagram of the fine-tuning transformer encoder in the present invention. DETAILED DESCRIPTION

[0023] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0024] Example 1: Reference Figure 1 The present invention proposes an RGBT target tracking method based on domain adaptation and spatiotemporal information fusion, which is characterized by comprising the following steps:

[0025] Step 1: Construct a visible light-thermal infrared RGBT data sample set containing real labels and divide it into training and test sets;

[0026] Step 2. Using the pre-trained RGB tracker OSTrack as a baseline, build a dual-branch RGBT object tracking model:

[0027] (2.1) Expand the pre-trained RGB tracker OSTrack into a dual-branch RGBT tracker, where the two branches are the visible light RGB modality branch and the thermal infrared TIR modality branch. The above expansion of the pre-trained RGB tracker OSTrack into a dual-branch RGBT tracker includes the following two stages:

[0028] The first stage is to expand OSTrack into weight-sharing parallel branches to perform feature extraction on the visible light RGB modality and thermal infrared TIR modality respectively;

[0029] The second stage is to add cross-modal interactions between parallel branches for information transfer between RGB and TIR modalities.

[0030] (2.2) Add an adapter to the transformer encoder of the dual-branch RGBT tracker. Specifically, an adapter is embedded in the multi-head attention layer and the fully connected layer of the transformer encoder of different branches in parallel to construct a fine-tuned transformer encoder; a dual-branch RGBT target tracking model with domain adaptation is obtained.

[0031] The core component of the fine-tuning transformer encoder is the adapter; the adapter consists of an hourglass structure consisting of two linear layers and a scaling factor, and the hourglass structure is parallelly connected to the original multi-head attention layer branch and the fully connected layer branch through scaling of the scaling factor; the multi-head attention layer contains the interaction information of paired positions, and the fully connected layer contains the knowledge of specific tasks; the parallel connection is used to retain the original features of the independent branches and the context updated by element-by-element scaling and aggregation.

[0032] Assume that the i-1th encoder of different branches outputs the features of their respective modes as After the fine-tuning transformer encoder, the specific calculation process is expressed as follows:

[0033]

[0034] in, is the output of the i-th layer encoder of the RGB modality branch, is the output of the i-th layer encoder of the TIR modality branch; is the output of the multi-head attention layer of the i-th encoder in the RGB modality, is the output of the multi-head attention layer of the i-th encoder of the TIR modality; MSA represents the multi-head attention layer, LN represents the normalization layer, MLP represents the fully connected layer, and Ada represents the adapter structure; s is the scale factor, which is set to 1 in this embodiment.

[0035] Step 3. Import the pre-trained OSTrack model parameters and use the training set to train the dual-branch RGBT target tracking model to obtain the optimal parameters, and obtain the final trained RGBT target tracking model; the implementation steps are as follows:

[0036] (3.1) Input training frames including RGB and TIR modalities;

[0037] (3.2) Crop the training frames of the two modalities to obtain their corresponding initial template images and search images as the input of the model;

[0038] (3.3) The initial template image and the search image are spliced ​​together and fed into the RGBT target tracking model for joint feature extraction;

[0039] (3.4) Input the extracted features into the bounding box prediction network to obtain the bounding box of the target and the confidence score of the bounding box;

[0040] (3.5) Use OSTrack’s loss function to calculate the tracking loss between the tracking result and the true label;

[0041] (3.6) The model parameters corresponding to the minimum tracking loss are taken as the optimal parameters to obtain the final RGBT target tracking model after training.

[0042] Step 4. Design an online module update strategy for dynamically updating the initial template image. Specifically, set the classification score threshold to τ. When the maximum classification score of the target is greater than τ, update the initial template image to obtain a dynamic template image and execute step 5. Otherwise, do not update the initial template image and execute step 6.

[0043] The dynamic template image is used to provide the time information of the target, and the initial template image is used to provide the spatial information of the target. When the maximum classification score of the target is greater than τ, the dynamic template image is obtained as an additional input of the RGBT target tracking model in step 1, and the dynamic template image and the initial template image are jointly input into the RGBT target tracking model. Otherwise, the initial template image is not updated, and only the initial template image is used as the input of the RGBT target tracking model.

[0044] Step 5. Use the initial template image, dynamic template image, and search image as the input of the final RGBT object tracking model and directly proceed to step 7.

[0045] The initial template image, dynamic template image, and search image are used as inputs to the final RGBT target tracking model to predict the target's motion state in subsequent frames. The input image is first segmented into P×P blocks using patch embedding and flattened into a sequence of 6 patches. The features of the initial template, dynamic template, and search image of the two modalities are then obtained through a linear projection layer. Finally, the labeled tokens of the RGB and TIR modal branches are spliced ​​along the channel dimension. The specific implementation process is as follows; then, the image is sent to the fine-tuned transformer encoder for processing.

[0046] The above method splits the input image into P×P blocks through patch embedding and flattens it into a sequence of 6 patches. Then, the initial template, dynamic template, and search image features of the two modalities are obtained through a linear projection layer. Finally, the label tokens of the RGB and TIR modal branches are spliced ​​along the channel dimension. The specific implementation process is shown as follows:

[0047]

[0048]

[0049] in Represent the learnable parameters of the linear projection layer of the RGB modality search image, the initial template image, and the dynamic template image respectively; Represent the learnable parameters of the linear projection layer of the TIR modality search image, initial template image, and dynamic template image respectively; These are patches of RGB and TIR modality search images respectively; They are patches of the initial template images of RGB modality and TIR modality respectively; They are patches of RGB modality and TIR modality dynamic template images respectively; They are the features of the RGB modality and TIR modality search images respectively; are the features of the initial template images in RGB and TIR modalities respectively; They are the features of RGB modality and TIR modality dynamic template images respectively; represents the input of the first layer fine-tuned transformer encoder of the RGB modality branch, represents the input of the fine-tuned transformer encoder in the first layer of the TIR modality branch, and concat represents the concatenation operation along the channel.

[0050] Step 6. Use the initial template image and the search image as input to the final RGBT object tracking model;

[0051] Step 7. Load the optimal parameters, use the final RGBT target tracking model to predict the target's motion state in subsequent frames, obtain the target tracking results and output them.

[0052] In this embodiment, the final RGBT target tracking model is used to predict the target's motion state in subsequent frames. The specific implementation steps are as follows:

[0053] (7.1) Load the optimal parameters of the trained RGBT target tracking model;

[0054] (7.2) Input test frames including RGB and TIR modalities;

[0055] (7.3) Crop the test frames of the two modalities to obtain their corresponding initial template images and search images as the input of the model;

[0056] (7.4) stitching the initial template image and the search image together to obtain a stitched image;

[0057] (7.5) The spliced ​​image is fed into the final RGBT target tracking model for joint feature extraction;

[0058] (7.6) Input the extracted features into the bounding box prediction network to obtain the bounding box of the target and the confidence score of the bounding box. All confidence scores are judged as follows: if the maximum classification score of the target is lower than the set threshold, execute step (7.8); otherwise, execute step (7.7);

[0059] (7.7) Finally, the RGBT target tracking model updates the initial template based on the online template update strategy to obtain a dynamic template image, which is then spliced ​​together with the initial template image and the search image as a spliced ​​image, and the process returns to step (7.5).

[0060] (7.8) The prediction process ends and the target tracking result is output.

[0061] Example 2: Reference Figure 1-3 The overall steps of the target tracking method in this embodiment are the same as those in embodiment 1. A specific example is now given to further describe the implementation process of the present invention in detail.

[0062] Step S1: Obtain RGBT image pairs containing true labels and aligned images from the training set of the RGBT dataset LasHeR, and divide them into training set and validation set;

[0063] Step S2: Using the pre-trained RGB tracker OSTrack as a benchmark, expand it into the RGBT target tracking method provided by the present invention, such as Figure 1 As shown, the specific method is:

[0064] S2.1: Extend the pre-trained RGB tracker OSTrack into a two-branch RGBT tracker, which consists of two stages: The first stage is to extend the RGB tracker OSTrack into parallel branches with shared weights to extract features from the visible light modality and the thermal infrared modality respectively; The second stage is to add cross-modal interaction between the parallel branches to promote information transfer between the visible light modality and the thermal infrared modality and improve the target representation ability of the RGBT tracker.

[0065] S2.2: Embed a fine-tuning transformer encoding block into the two-branch RGBT tracker to construct an RGBT tracker with domain adaptation ability; As Figure 1 and Figure 2 shown, load the model parameters of the RGB tracker OSTrack and freeze them. The visible light branch and the thermal infrared branch share parameters. Use the LasHeR training set to train the fine-tuning transformer encoding block, transfer the feature extraction ability of the model from the RGB domain to the RGBT domain, and improve the adaptation ability of the baseline model in RGBT tracking.

[0066] In the two-branch RGBT tracker, embed an adapter into the multi-head attention layer and the fully connected layer of the transformer encoding block in different branches to construct a fine-tuning transformer. The core component of the fine-tuning transformer encoder is the adapter, which is an hourglass structure composed of two linear layers and a scaling factor. The parameters of the first linear layer (FC Down) of the adapter are The parameters of the second linear layer (FC Up) are where d’ is the intermediate dimension, where d' << d. This hourglass structure is scaled by the scaling factor and connected in parallel to the original multi-head attention layer branch and the fully connected layer branch. The multi-head attention layer contains pairwise position interaction information, and the fully connected layer contains task-specific knowledge, which is beneficial for the model to adapt to new downstream tasks; The parallel design can retain the original features of the independent branches and update the context through element-wise scaling and aggregation.

[0067] S2.3: During the tracking process, the online module update strategy dynamically updates the initial template. The online template provides temporal information of the target, and the initial template provides spatial information of the target. The two templates are jointly input into the RGBT tracker. The templates provide spatio-temporal information of the target to help the model achieve robust tracking in complex application environments.

[0068] As Figure 1As shown, during the tracking process, the online template update strategy dynamically updates the initial template. The dynamic template provides temporal information of the target, while the initial template provides spatial information. The two templates are fused to provide the spatiotemporal information of the target to be tracked. This invention uses the target's maximum classification score as the basis for our online template update, which reflects the confidence of the tracking result. This online update method maximizes the model's ability to distinguish between the target and the background without requiring additional model training. In this embodiment, a threshold of 0.5 is set. If the target's maximum classification score is below the given threshold, the initial template is not updated. If the target's maximum classification score is above the given threshold, the online template update is performed to update the initial template to obtain the dynamic template. The input is the initial template, dynamic template, and search image. These are first segmented into P×P blocks using patch embedding and flattened into a sequence of six patches. A linear projection layer is then used to obtain features for the search image, initial template, and dynamic template in both modalities. Finally, the label embeddings of the RGB and TIR modal branches are concatenated along the channel dimension and fed into the fine-tuned transformer encoder.

[0069] Step S3: Import the pre-trained OSTrack model parameters and fine-tune the RGBT tracker using the LasHeR training set;

[0070] Step S4: Load the parameters of the trained RGBT tracking model. Given a video sequence, the first frame gives the spatial position of the target. The video sequence is fed into the tracking model frame by frame to predict the motion state of the target in subsequent frames.

[0071] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0072] Portions of the present invention not described in detail are common knowledge to those skilled in the art. The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. It is obvious that, after understanding the content and principles of the present invention, those skilled in the art may make various modifications and changes in form and details without departing from the principles and structure of the present invention. However, such modifications and changes based on the concepts of the present invention shall remain within the scope of protection of the claims of the present invention.

Claims

1. A RGBT target tracking method based on domain adaptation and spatiotemporal information fusion, characterized in that: The steps include: (1) Construct a visible light-thermal infrared RGBT data sample set containing real labels and divide it into a training set and a test set; (2) Using the pre-trained RGB tracker OSTrack as a benchmark, a dual-branch RGBT target tracking model is constructed: (2.1) Expand the pre-trained RGB tracker OSTrack into a dual-branch RGBT tracker, where the dual branches are the visible light RGB modality branch and the thermal infrared TIR modality branch; (2.2) Add an adapter to the transformer encoder of the dual-branch RGBT tracker, construct a fine-tuning transformer encoder, and obtain a dual-branch RGBT target tracking model with domain adaptability; the adapter is added to the transformer encoder of the dual-branch RGBT tracker, specifically, the multi-head attention layer and the fully connected layer of the transformer encoder of different branches are embedded with an adapter in parallel to construct a fine-tuning transformer encoder; the core component of the encoder is the adapter, and the adapter consists of an hourglass structure consisting of two linear layers and a scaling factor, and the hourglass structure is connected in parallel to the original multi-head attention layer branch and the fully connected layer branch through scaling factor scaling; the multi-head attention layer contains the interaction information of paired positions, and the fully connected layer contains the knowledge of specific tasks; the parallel connection is used to retain the original features of the independent branches and the context updated by element-by-element scaling and aggregation; it is assumed that the i-1th encoder of different branches outputs the features of their respective modes After the fine-tuning transformer encoder, the specific calculation process is expressed as follows: in, is the output of the i-th layer encoder of the RGB modality branch, is the output of the i-th layer encoder of the TIR modality branch; is the output of the multi-head attention layer of the i-th encoder in the RGB modality, is the output of the multi-head attention layer of the i-th encoder in the TIR modality; MSA represents the multi-head attention layer, LN represents the normalization layer, MLP represents the fully connected layer, Ada represents the adapter structure; s is the scale factor; (3) Import the pre-trained OSTrack model parameters, use the training set to train the dual-branch RGBT target tracking model to obtain the optimal parameters, and obtain the final RGBT target tracking model after training; (4) Design an online module update strategy for dynamically updating the initial template image; specifically, set the classification score threshold to τ, and when the maximum classification score of the target is greater than τ, update the initial template image to obtain a dynamic template image and execute step (5); otherwise, do not update the initial template image and execute step (6); (5) Using the initial template image, dynamic template image, and search image as inputs to the final RGBT target tracking model, directly execute step (7); (6) The initial template image and the search image are used as inputs to the final RGBT target tracking model; (7) Load the optimal parameters, use the final RGBT target tracking model to predict the target's motion state in subsequent frames, obtain the target tracking results and output them.

2. The method according to claim 1, wherein: Step (2.1) expands the pre-trained RGB tracker OSTrack into a dual-branch RGBT tracker, which consists of the following two stages: The first stage is to expand OSTrack into weight-sharing parallel branches to perform feature extraction on the visible light RGB modality and thermal infrared TIR modality respectively; The second stage is to add cross-modal interactions between parallel branches for information transfer between RGB and TIR modalities.

3. The method according to claim 1, wherein: Step (3) of training the dual-branch RGBT target tracking model is as follows: (3.1) Input training frames including RGB and TIR modalities; (3.2) Crop the training frames of the two modalities to obtain their corresponding initial template images and search images as the input of the model; (3.3) The initial template image and the search image are spliced ​​together and fed into the RGBT target tracking model for joint feature extraction; (3.4) Input the extracted features into the bounding box prediction network to obtain the bounding box of the target and the confidence score of the bounding box; (3.5) Use OSTrack’s loss function to calculate the tracking loss between the tracking result and the true label; (3.6) The model parameters corresponding to the minimum tracking loss are taken as the optimal parameters to obtain the final RGBT target tracking model after training.

4. The method according to claim 1, wherein: The dynamic template image in step (4) is used to provide the time information of the target, and the initial template image is used to provide the spatial information of the target. When the maximum classification score of the target is greater than τ, the dynamic template image is obtained as an additional input of the RGBT target tracking model in step (4), and the dynamic template image and the initial template image are jointly input into the RGBT target tracking model. Otherwise, the initial template image is not updated, and only the initial template image is used as the input of the RGBT target tracking model.

5. The method according to claim 1, wherein: In step (5), the initial template image, dynamic template image and search image are used as the input of the final RGBT target tracking model to predict the motion state of the target in subsequent frames. The input image is first divided into P×P blocks by patch embedding and flattened into a sequence of 6 patches. Then, the features of the initial template, dynamic template and search image of the two modalities are obtained respectively through the linear projection layer. Finally, the label tokens of the RGB and TIR modality branches are spliced ​​along the channel dimension and then sent to the fine-tuned transformer encoder for processing.

6. The method according to claim 5, characterized in that: The input image is divided into P×P blocks through patch embedding and flattened into a sequence of 6 patches. Then, the initial template, dynamic template, and search image features of the two modalities are obtained through a linear projection layer. Finally, the label tokens of the RGB and TIR modality branches are spliced ​​along the channel dimension. The specific implementation process is shown as follows: in Represent the learnable parameters of the linear projection layer of the RGB modality search image, the initial template image, and the dynamic template image respectively; Represent the learnable parameters of the linear projection layer of the TIR modality search image, initial template image, and dynamic template image respectively; These are patches of RGB and TIR modality search images respectively; They are patches of the initial template images of RGB modality and TIR modality respectively; They are patches of RGB modality and TIR modality dynamic template images respectively; They are the features of the RGB modality and TIR modality search images respectively; are the features of the initial template images in RGB and TIR modalities respectively; They are the features of RGB modality and TIR modality dynamic template images respectively; represents the input of the first layer fine-tuned transformer encoder of the RGB modality branch, represents the input of the fine-tuned transformer encoder in the first layer of the TIR modality branch, and concat represents the concatenation operation along the channel.

7. The method according to claim 1, wherein: Step (7) uses the final RGBT target tracking model to predict the target's motion state in subsequent frames. The implementation steps are as follows: (7.1) Load the optimal parameters of the trained RGBT target tracking model; (7.2) Input test frames including RGB and TIR modalities; (7.3) Crop the test frames of the two modalities to obtain their corresponding initial template images and search images as the input of the model; (7.4) stitching the initial template image and the search image together to obtain a stitched image; (7.5) The spliced ​​image is fed into the final RGBT target tracking model for joint feature extraction; (7.6) Input the extracted features into the bounding box prediction network to obtain the bounding box of the target and the confidence score of the bounding box. All confidence scores are judged as follows: if the maximum classification score of the target is lower than the set threshold, execute step (7.8); otherwise, execute step (7.7); (7.7) Finally, the RGBT target tracking model updates the initial template based on the online template update strategy to obtain a dynamic template image, which is then spliced ​​together with the initial template image and the search image as a spliced ​​image, and the process returns to step (7.5). (7.8) The prediction process ends and the target tracking result is output.

Images