Target tracking method, device and equipment

Abstract

The invention provides a target tracking method, device and equipment, belongs to the technical field of target tracking, and aims to solve the problem that a target cannot be effectively tracked by a target tracking method in related technologies. The method comprises the steps that a search area sequence corresponding to a search area image of each mode is acquired, a first target sequence corresponding to each dynamic template and a second target sequence corresponding to each static template are acquired, the static templates represent initial features of a tracking target, and the dynamic templates represent motion features of the tracking target; inputting the search area sequence of each mode, the first target sequence and the second target sequence into a weight sharing coding layer, and performing time sequence information fusion and inter-mode information fusion to obtain a fusion feature sequence of each mode; and according to the fusion feature sequence of each mode, the tracking target is selected from the search area image.

Description

Technical Field

[0001] The present disclosure relates to the technical field of target tracking, and in particular to a target tracking method, device and equipment. Background Art

[0002] Visual object tracking is a basic task in computer vision. Its main research task is to accurately locate and track the target in each subsequent frame of video given the initial state of the current frame. Given its great potential and excellent performance, visual object tracking has been widely used in video surveillance, autonomous driving, and robot navigation. However, due to the limitations of the visible light imaging mechanism, VOT (Visual Object Tracking) still faces challenging problems, especially in complex scenes and environments such as extreme lighting and bad weather. This greatly limits the scope of application of target tracking technology.

[0003] Thermal infrared (TIR) ​​imaging is insensitive to changes in illumination, has a certain degree of penetration, and is complementary to the rich color, detail, and texture information of visible light. Therefore, TIR images can be used as complementary information to visible light images.

[0004] Therefore, among the related technologies, target tracking technology based on visible light and thermal infrared image fusion (RGB-T) has become a hot research topic in recent years, with potential application value in the fields of intelligent monitoring, autonomous driving and robotics.

[0005] However, the existing RGB-T multimodal target tracking methods lack full utilization of temporal information and the fusion between modalities is not sufficient and reasonable, which makes the model unable to track the target more robustly. Summary of the invention

[0006] In order to overcome the problems existing in the related art, the present disclosure provides a target tracking method, device and equipment. The technical solution of the present disclosure is as follows: According to a first aspect of an embodiment of the present disclosure, a target tracking method is provided, including: Acquire a search area sequence corresponding to the search area image of each modality, and acquire a first target sequence corresponding to each dynamic template and a second target sequence corresponding to each static template, wherein the static template represents the initial features of the tracked target and the dynamic template represents the motion features of the tracked target; Inputting the search area sequence, the first target sequence, and the second target sequence of each modality into a weight-sharing encoding layer, performing temporal information fusion and inter-modality information fusion, and obtaining a fusion feature sequence of each modality; The tracking target is framed and selected in the search area image according to the fusion feature sequences of each modality.

[0007] Optionally, the search area sequence, the first target sequence, and the second target sequence of each modality are input into a weight-sharing encoding layer for temporal information fusion and inter-modality information fusion to obtain a fusion feature sequence of each modality, including: Performing time series information fusion on the first target sequence and the second target sequence to obtain a target feature sequence, wherein the target feature sequence is used to capture the behavior of the tracked target; Based on the target feature sequence, feature enhancement is performed on the search area sequence to obtain an enhanced feature sequence, wherein the feature enhancement is used to distinguish background information and tracking target information in the search area sequence; Inter-modal information interaction is performed according to the search area sequences of different modalities to obtain cross-modal prompt information; The enhanced feature sequence and the cross-modal prompt information are fused to form a fused feature sequence.

[0008] Optionally, performing time series information fusion on the first target sequence and the second target sequence to obtain a target feature sequence includes: Performing dimensionality reduction information extraction on the second target sequence to obtain cross-time prompt information; Upgrading the cross-time prompt information to a dimension with the same size as the feature information in the first target sequence, and superimposing it with the feature information in the first target sequence to obtain a first target sequence superimposed with the cross-time prompt information; The first target sequence superimposed with the cross-time prompt information is determined as the target feature sequence.

[0009] Optionally, inter-modal information interaction is performed according to the search area sequences of different modalities to obtain cross-modal prompt information, including: Performing dimensionality reduction information extraction on the search area sequence of each modality to obtain first cross-modal prompt information; The first cross-modal prompt information is upgraded to a dimension having the same size as the feature information in the search area sequence to obtain a first upgraded result; the first upgraded result is used to be superimposed with the enhanced feature sequence of another modality.

[0010] Optionally, after the search area sequence is enhanced based on the target feature sequence to obtain an enhanced feature sequence, the method further includes: receiving a second dimension-raising result obtained by a search region sequence of another modality; The second dimension-upgraded result is superimposed on the enhanced feature sequence to obtain the fused feature sequence after cross-modal fusion.

[0011] Optionally, after the tracking target is framed in the search area image, the following steps are further included: Determine the image to be updated by selecting the framed area in the search area image; Evaluate the image to be updated based on the position information, the discrimination information and the appearance information, and determine whether the image to be updated passes the evaluation; If the image to be updated passes the evaluation, the dynamic template is updated, and the image to be updated is determined as the updated dynamic template; If the image to be updated fails the evaluation, the dynamic template is not updated.

[0012] Optionally, selecting the tracking target in the search area image according to the fusion feature sequences of each modality includes: Determine the classification diagram of the fusion feature sequence corresponding to each modality; According to each of the classification graphs, weighted fusion is performed on the fusion feature sequences of each modality to obtain a weighted fusion feature sequence; Determine the corresponding tracking result through the fusion feature sequence corresponding to each modality and the weighted fusion feature sequence; Through the evaluation indicators, each tracking result is evaluated to obtain the corresponding evaluation results; The tracking result with the best evaluation result is determined as the target tracking result; The target to be tracked is selected in the search area image by using the target tracking result.

[0013] Optionally, the search area sequence, the first target sequence and the second target sequence are determined in the following manner, including: According to the set patch size, the image is segmented and expanded into a sequence of multiple patches; the image is a search area image, a static template image or a dynamic template image; Projecting the sequence of the multiple patches into a high-dimensional space through a convolutional network to obtain a feature sequence; Wherein, when the image is a search area image, after projecting the sequence composed of the multiple patches into a high-dimensional space, the projection result is embedded and added with the position information to obtain a feature sequence; the position information is the position of the patch in the search area image.

[0014] According to a second aspect of an embodiment of the present disclosure, there is provided a target tracking device, comprising: An acquisition module is used to acquire a search area sequence corresponding to the search area image of each modality, and to acquire a first target sequence corresponding to each dynamic template and a second target sequence corresponding to each static template, wherein the static template represents the initial features of the tracked target, and the dynamic template represents the motion features of the tracked target; A fusion module is used to input the search area sequence, the first target sequence, and the second target sequence of each modality into a weight-sharing encoding layer to perform temporal information fusion and inter-modality information fusion to obtain a fusion feature sequence of each modality; The tracking module is used to select the tracking target in the search area image according to the fusion feature sequence of each modality.

[0015] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the target tracking method described in the first aspect are implemented.

[0016] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the target tracking method described in the first aspect are implemented.

[0017] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps of the target tracking method described in the first aspect are implemented.

[0018] The present disclosure can effectively strengthen the information interaction between different modalities by performing time series fusion and inter-modal information fusion, can more comprehensively capture the characteristics of the target, and improve the accuracy and robustness of tracking. The dynamic template can reflect the motion characteristics of the tracked target, while the static template characterizes the initial characteristics of the target. The present disclosure can better adapt to the appearance changes and motion states of the target at different time points, thereby improving the stability of tracking. By processing the fused feature sequences of each modality, the tracked target can be more accurately framed in the search area image, which can effectively reduce false detections and missed detections and improve the overall tracking performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the description of the embodiments of the present disclosure will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 is a schematic diagram of steps of a target tracking method shown in an embodiment of the present disclosure; Figure 2 It is a schematic diagram of the workflow of a spatiotemporal bidirectional fusion multimodal fusion target tracking method based on template updating shown in an embodiment of the present disclosure; Figure 3 is a block diagram of a target tracking device shown in an embodiment of the present disclosure; Figure 4 It is a schematic diagram of an electronic device shown in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0022] The terms "first", "second", etc. in the specification and claims of the present disclosure are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable when appropriate, so that the embodiments of the present disclosure can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0023] The methods of feature extraction and association modeling of images using visual Transformer as the backbone network can be divided into two categories: full fine-tuning methods and hinted fine-tuning methods.

[0024] The full fine-tuning method fully tunes the entire network parameters. This method can deeply interact the feature information between the two modalities to a large extent, improving the robustness of model tracking. However, this method is parameter inefficient and requires sufficient data to optimize all parameters.

[0025] As a new paradigm for efficient parameter tuning, prompt fine-tuning has effectively alleviated the problem of insufficient data in the RGB-T field. This method was first applied in the field of natural language processing (NLP). Some early methods introduced modal complementarity prompters to generate effective visual prompts, simplifying the auxiliary modality into a small amount of prompt information. These methods will pre-set a dominant mode (usually RGB mode), but when the dominant mode switches, the modal complementarity cannot be fully and effectively utilized. In order to solve this problem, subsequent methods often set up bidirectional feature interactors to perform feature interaction through bidirectional prompts.

[0026] However, although cue fine-tuning as a parameter-efficient tuning method has improved the tracking capabilities of RGB-T trackers to a certain extent, their effectiveness is still limited by multiple factors, mainly including two problems.

[0027] The first problem: Existing trackers lack full utilization of temporal information. Current research focuses on further enhancing the interactive fusion of complementary information between visible light modalities and thermal modalities to achieve more efficient tracking. Most methods ignore the use of time series information. For long-term tracking, the template information of the initial frame alone is not enough to cope with complex scenes that the tracker may encounter (target appearance changes, deformation, rapid movement, etc.). Some existing methods use the tracking results of each frame to generate new training samples and train the tracker online during the tracking process. Or the training sample update is divided into a long-term dynamic update that continues to be a positive sample, and a short-term update where the predicted target score is less than a manually set threshold. This update method requires offline training of the tracker and updates the domain-specific layer during tracking. However, there is a common problem with the above methods. Due to the lack of judgment on the reliability of the tracking results, this continuous update can easily introduce cumulative errors and lose the original feature information of the target.

[0028] The second problem: the existing trackers are not sufficiently and reasonably integrated between modalities. Existing tracking methods believe that in the feature interaction process of the backbone network, the feature information of the two modalities has been fully interacted and integrated. In the final classification and regression stage of generating the target bounding box based on the feature information, the feature information of the visible light modality and the thermal infrared modality is often directly added or connected. This method generally does not have a significant impact. However, when a certain modality fails completely, this direct superposition or connection will inevitably introduce the noise of the failed modality into the feature information of the other modality, making the cross-modal interaction rough and insufficient.

[0029] In order to solve the above problems, the present disclosure proposes a target tracking method, which can make good use of time information and enhance the fusion between modalities.

[0030] Figure 1 Schematic diagram of a target tracking method according to an embodiment of the present disclosure. Figure 1 As shown, the method may specifically include the following steps: Step S11: Obtain a search area sequence corresponding to the search area image of each modality, and obtain a first target sequence corresponding to each dynamic template and a second target sequence corresponding to each static template, wherein the static template represents the initial features of the tracked target, and the dynamic template represents the motion features of the tracked target.

[0031] Obtain image data of the tracking area from different sensors or data sources, such as visible light cameras, infrared cameras, radars, etc. Ensure that the timestamps of the search area images of each modality are synchronized.

[0032] And obtain the dynamic template and static template corresponding to each modality. Both the dynamic template and the static template are images. The combination of dynamic template and static template can significantly improve the accuracy and robustness of target tracking.

[0033] A dynamic template refers to a template image that is continuously updated over time during the target tracking process. It can be an image of the target in the search area image determined during the latest target tracking. The dynamic template is usually adjusted based on the appearance and motion state of the target in the latest frame to reflect the real-time changes of the target. The dynamic template can capture the latest motion features of the target, helping the tracker to better adapt to the dynamic changes of the target, thereby improving the accuracy and robustness of tracking. And when the target appearance changes significantly, the dynamic template can help the tracker avoid tracking drift caused by template failure.

[0034] A static template refers to a fixed template image that represents the initial features of a target during target tracking. A static template is usually a target image captured at the beginning of tracking, reflecting the basic appearance features of the target. The static template provides a reference point for target tracking, allowing the tracker to quickly match and identify the target when the target appearance does not change much. In addition, when the dynamic template fails or is not updated in time, the static template can be used as an alternative to help maintain the stability of tracking.

[0035] The feature extraction algorithm is applied to the search area image, dynamic template and static template of each modality to generate respective feature sequences. The search area sequence is obtained by the search area image, the first target sequence is obtained by the dynamic template, and the second target sequence is obtained by the static template.

[0036] Step S12: input the search area sequence, the first target sequence, and the second target sequence of each modality into a weight-sharing encoding layer, perform temporal information fusion and inter-modality information fusion, and obtain a fusion feature sequence of each modality.

[0037] The corresponding branch network in the weight-shared encoding layer processes the search area sequence, the first target sequence, and the second target sequence corresponding to each modality respectively.

[0038] In the weight-sharing encoding layer, the feature sequences of each modality are subjected to temporal information fusion and inter-modal information fusion. Temporal information fusion acts on each modality, and is realized based on the first target sequence and the second target sequence corresponding to the modality itself; inter-modal information fusion acts between modalities, and is realized based on the search area sequence of each modality.

[0039] Temporal information fusion helps the tracker better understand the dynamic changes of the tracked target by integrating the information in the time series. It can provide context about the motion trajectory and appearance changes of the tracked target in the time series, thereby improving the prediction accuracy of the target position.

[0040] Inter-modal information fusion can effectively integrate feature information from different modalities and provide complementary information of another modality for different modalities.

[0041] Step S13: According to the fusion feature sequences of each modality, the tracking target is framed in the search area image.

[0042] The target is tracked based on the fusion feature sequence of each modality, and the corresponding tracking results are determined. The target tracking result is determined according to the tracking results corresponding to each modality. According to the target tracking result, the tracking target is framed in the search area image corresponding to each modality.

[0043] By adopting the embodiments of the present disclosure, the complementarity between different modal data can be fully utilized through inter-modal information fusion to enhance the comprehensive understanding of the tracking target. Data of different modalities can often provide different effective information under different environmental conditions, such as lighting changes, occlusion, complex background, etc., thereby improving the robustness of tracking. Using static templates to characterize the initial features of the tracking target helps to maintain consistent recognition of the target identity during the tracking process, and dynamic templates can capture the appearance changes of the target in a timely manner. Combining static templates and dynamic templates enables the tracking system to adapt to the static properties and dynamic behaviors of the target at the same time.

[0044] Figure 2 FIG. 1 is a schematic diagram of a workflow of a spatiotemporal bidirectional fusion multimodal fusion target tracking method based on template updating according to an embodiment of the present disclosure. Figure 2As shown, the weight-shared encoding layer has two branch networks, which process the data of the RGB modality and the thermal infrared modality respectively. For example, the upper branch network processes the search area sequence, the first target sequence, and the second target sequence corresponding to the RGB modality. Inside the branch network, a bidirectional spatiotemporal fusion module is configured between the Transformer blocks at each level of the branch network to perform temporal information fusion within a single modality, and a bidirectional spatiotemporal fusion module is configured between the Transformer blocks of the same level between the branch networks to perform inter-modal information fusion between different modalities. After being processed by the last Transformer block, each branch network obtains a fusion feature sequence corresponding to each modality; then, the tracking target is framed out in the search area image through the fusion feature sequence of each modality.

[0045] In an optional embodiment, the search area sequence, the first target sequence and the second target sequence are determined in the following manner, including: according to a set patch size, the image is divided and expanded into a sequence consisting of multiple patches; the image is a search area image, a static template image or a dynamic template image; through a convolutional network, the sequence consisting of the multiple patches is projected into a high-dimensional space to obtain a feature sequence; wherein, in the case where the image is a search area image, after projecting the sequence consisting of the multiple patches into a high-dimensional space, the projection result is embedded and added with the position information to obtain a feature sequence; the position information is the position of the patch in the search area image.

[0046] According to the requirements of image processing and the limitations of computing resources, the appropriate patch size is set. The size of the patch can capture the local features in the image while maintaining computational efficiency.

[0047] According to the set patch size, the image is divided into multiple small blocks, each of which is called a patch. The obtained patches are expanded into a sequence in a certain order, such as from left to right or from top to bottom.

[0048] After obtaining the sequence of patches, the convolutional neural network is used to extract the features of the image patches. The convolutional neural network processes layer by layer, extracts key information from the pudding, and maps it to a high-dimensional feature space to obtain a feature sequence.

[0049] When the processed image is a search area image, in order to retain the position information of the patch in the search area image, it is necessary to combine the position information with the projected feature sequence. This can be achieved by encoding the position information into a vector and adding it to the feature vector of each patch. After adding the position information to the feature sequence, the search area sequence corresponding to the search area image is obtained. The search area sequence contains not only the feature information of the image patch, but also the position information of the patch in the search area image.

[0050] By adopting the embodiments of the present disclosure, the image is divided into multiple patches according to the set patch size and expanded into a sequence, which not only simplifies the image processing process, but also makes the subsequent feature extraction more efficient. Through the convolutional network, the patch sequence is projected into a high-dimensional space, thereby extracting rich feature information and being able to capture subtle changes and important information in the image. When the image is a tracking area image, after the patch sequence is projected into the high-dimensional space, the projection result is embedded and added with the position information of the patch in the tracking area image, so that the feature sequence contains not only the feature information of the image itself, but also the position information of the patch. The introduction of position information enhances the accuracy of target positioning.

[0051] Among them, in an optional embodiment, the search area sequence, the first target sequence, and the second target sequence of each modality are input into a weight-shared encoding layer for temporal information fusion and inter-modal information fusion to obtain a fusion feature sequence of each modality, including: performing temporal information fusion on the first target sequence and the second target sequence to obtain a target feature sequence, and the target feature sequence is used to capture the behavior of the tracked target; based on the target feature sequence, performing feature enhancement on the search area sequence to obtain an enhanced feature sequence, and the feature enhancement is used to distinguish background information and tracking target information in the search area sequence; performing inter-modal information interaction according to the search area sequences of different modalities to obtain cross-modal prompt information; fusing the enhanced feature sequence with the cross-modal prompt information to fuse the feature sequence.

[0052] After obtaining the first target sequence and the second target sequence corresponding to each modality, the first target sequence and the second target sequence of a single modality are fused with time series information to obtain a target feature sequence. Figure 2 ,Before using the Transformer block to process the feature data of a single modality, a bidirectional spatiotemporal fusion module is used to fuse the temporal information of the first target sequence and the second target sequence.

[0053] The target feature sequence can effectively fuse feature information at different times. The target feature sequence can capture the changing trend of the tracked target in the time series and help the tracking system better understand the dynamic characteristics of the tracked target. This is because the target feature sequence reflects the state of the tracked target at different time points, thereby enhancing the tracking system's understanding of the tracked target's behavior. After obtaining the target feature sequence, the search area sequence is enhanced based on the target feature sequence. Figure 2 , the target feature sequence and the search area sequence are input into the Transformer block, so as to realize feature enhancement of the search area sequence based on the target feature sequence and obtain an enhanced feature sequence. Based on the target feature sequence, the template search and matching method can be used to enhance the features of the search area sequence. The meaning of template search and matching is to study where the pattern of a specific object is located in the image, and then identify the object. It is achieved by sliding on the collected original image and looking for a target similar to the template image. Through template search and matching, it is possible to effectively refer to the feature information of the tracking target at different times in the target feature sequence, thereby improving the feature enhancement effect of the Transformer block on the target feature sequence, effectively realizing the enhancement of the background features and tracking target features in the target feature sequence, and providing support for the subsequent identification and tracking of the target.

[0054] When the features of the search area sequence of a single modality are enhanced, cross-modal prompt information transmitted by other modalities can be received, and the enhanced feature sequence obtained by enhancement is fused with the cross-modal prompt information to obtain a fused feature sequence corresponding to this modality.

[0055] By adopting the embodiments of the present disclosure, by fusing the time series information of the first target sequence and the second target sequence, the generated target feature sequence can effectively capture the dynamic behavior of the target, so that the changes of the target in the time dimension can be better understood, thereby improving the accuracy and robustness of tracking. Based on the target feature sequence, the search area sequence is feature enhanced, and the system can more effectively distinguish between background information and tracking target information. Thereby, false detections and missed detections can be reduced, and the accuracy of target tracking can be improved. According to the search area sequences of different modalities, information interaction between modalities is performed to generate cross-modal prompt information, which can make full use of the complementarity of different modalities and enhance the system's understanding and recognition capabilities of the target.

[0056] Among them, in an optional embodiment, the first target sequence and the second target sequence are subjected to time series information fusion to obtain a target feature sequence, including: extracting dimensionality reduction information on the second target sequence to obtain cross-time prompt information; upgrading the cross-time prompt information to a dimension of the same size as the feature information in the first target sequence, and superimposing it with the feature information in the first target sequence to obtain a first target sequence with superimposed cross-time prompt information; and determining the first target sequence with superimposed cross-time prompt information as the target feature sequence.

[0057] refer to Figure 2 , the temporal information of the first target sequence and the second target sequence is fused through a bidirectional spatiotemporal fusion module between different levels of Transformer blocks in a single branch network. Each Transformer block consists of multi-head self-attention, a normalization layer, a feedforward neural network, and a residual link.

[0058] The bidirectional spatiotemporal fusion module first receives feature information from the dynamic template and the static template, namely the first target sequence and the second target sequence. The feature information included in the first target sequence is the feature information of the dynamic template; the feature information included in the second target sequence is the feature information of the static template.

[0059] The received feature information is processed by dimensionality reduction, the most important features are extracted, and redundant information is removed, thereby obtaining concise cross-time prompt information. Specifically, methods such as principal component analysis, autoencoder or convolution layer can be used to map high-dimensional features to low-dimensional space. The cross-time prompt information formed by the feature information after dimensionality reduction can reflect the changing trend and important features of the tracking target in the time series.

[0060] The reduced cross-temporal cue information is re-upgraded to the same dimension as the template feature information, i.e., the feature information in the second target sequence, so as to ensure that the cross-temporal cue information can be effectively combined with the feature information of other times. Specifically, the dimensionality increase can be achieved through inverse convolution, fully connected layers, or interpolation methods. In the process of dimensionality increase, the integrity of the cross-temporal cue information needs to be maintained to ensure that important features are not lost during the dimensionality increase.

[0061] The upgraded cross-time prompt information is added to the feature information from another time. Specifically, the upgraded cross-time prompt information is added to the feature information in the second target sequence corresponding to the static template. The meaning of addition is feature fusion, which combines the feature information at different time points to form a richer feature representation. Through the addition operation, a new feature representation is generated, namely the target feature sequence, which contains feature information from different times and enhances the understanding of the tracking target.

[0062] In addition, the generated prompt information of different temporal feature information of the same modality can be used as visual prompts and cross-transferred to another modality, so that the temporal feature information of one modality can be used to guide the feature extraction and processing of another modality. Through this cross-transfer, the time series information can be fully utilized, the complementarity between modalities can be enhanced, and the overall tracking performance can be improved.

[0063] By adopting the embodiments of the present disclosure, the understanding of the tracking target can be enhanced by fusing the time series information of the first target sequence and the second target sequence. By performing dimensionality reduction processing on the second target sequence, more concise and important cross-time prompt information can be extracted, which helps to remove redundant information and retain the most useful features for the tracking task, thereby reducing the computational burden. After obtaining the concise cross-time prompt information, it is upgraded to the same dimension as the feature information of the template image, ensuring that the feature information at different times can be effectively compared and fused in the same space.

[0064] refer to Figure 2 ,By setting up a bidirectional spatiotemporal fusion module between the Transformer blocks at the same level of different branch networks,inter-modal information fusion is achieved.

[0065] The bidirectional spatiotemporal fusion module can receive the search area sequences corresponding to each modality, process them, obtain corresponding cross-modal prompt information, and transmit the processed cross-modal prompt information to other modalities with different sources from the cross-modal prompt information.

[0066] Among them, in an optional embodiment, inter-modal information interaction is performed according to the search area sequences of different modalities to obtain cross-modal prompt information, including: performing dimensionality reduction information extraction on the search area sequence of each modality to obtain first cross-modal prompt information; upgrading the first cross-modal prompt information to a dimension of the same size as the feature information in the search area sequence to obtain a first dimensionality upgrade result; the first dimensionality upgrade result is used to superimpose with the enhanced feature sequence of another modality.

[0067] The bidirectional spatiotemporal fusion module uses dimensionality reduction technology to process the search area sequences corresponding to each modality. Through dimensionality reduction processing, the most representative features are extracted to obtain streamlined cross-modal prompt information.

[0068] Interpolation or other dimensionality increasing techniques are used to increase the dimension of the cross-modal cue information after dimensionality reduction to the same dimension as the feature information in the search area sequence, thereby ensuring that the cross-modal cue information after dimensionality increase is spatially compatible with the feature information in the search area sequence of another modality.

[0069] After obtaining the cross-modal prompt information, the bidirectional spatiotemporal fusion module transmits it to other modalities different from the modality corresponding to the cross-modal prompt information, specifically to the Transformer block of the other modality.

[0070] By adopting the embodiments of the present disclosure, by performing dimensionality reduction processing on the search area sequences of different modalities, extracting the most representative features, and generating cross-modal prompt information, it is possible to effectively exchange information between modalities and enhance understanding of the target. The search area sequences of each modality are processed using dimensionality reduction technology to extract important features, which can remove redundant information and make subsequent processing more efficient. The cross-modal prompt information after dimensionality reduction is upgraded to the same dimension as the original search area sequence, ensuring the spatial compatibility of feature information of different modalities. Cross-modal prompt information of different modalities can reflect the state and feature changes of the target in different modalities.

[0071] Among them, in an optional embodiment, after the search area sequence is feature enhanced based on the target feature sequence to obtain the enhanced feature sequence, it also includes: receiving a second dimensionality upgrade result obtained by a search area sequence of another modality; superimposing the second dimensionality upgrade result with the enhanced feature sequence to obtain the fused feature sequence after cross-modal fusion.

[0072] The dimensionality-enhanced result obtained from the cross-modal prompt information corresponding to one modality is used as a visual cue and cross-transferred to another modality.

[0073] After receiving the second dimension-raising result obtained from the search area sequence of another modality, one modality superimposes it with the enhanced feature sequence, which can be a simple element-by-element addition or a weighted addition according to specific weights to emphasize the characteristics of a certain modality.

[0074] After the superposition operation, a fused feature sequence containing complementary features from two modalities is generated.

[0075] refer to Figure 2 After the Transformer block of a certain level of a certain modality obtains the enhanced feature sequence corresponding to the level, the separate output of the modality of the current level is obtained. In the next step, the enhanced feature sequence is superimposed with the dimensionality increase result corresponding to the cross-modal prompt information of another modality to obtain the output of the entire current level.

[0076] By adopting the embodiments of the present disclosure, the generated prompt information is used as a visual prompt and cross-transferred to another modality, which can fully utilize the complementarity of different modalities. The cross-transfer process enhances the information interaction between modalities, allowing the system to maintain high tracking performance in different environments and conditions.

[0077] Among them, in an optional embodiment, according to the fusion feature sequence of each modality, the tracking target is framed out in the search area image, including: determining a classification map of the fusion feature sequence corresponding to each modality; according to each of the classification maps, weightedly fusing the fusion feature sequence of each modality to obtain a weighted fusion feature sequence; determining the corresponding tracking result by respectively using the fusion feature sequence corresponding to each modality and the weighted fusion feature sequence; evaluating each tracking result by using an evaluation indicator to obtain a corresponding evaluation result; determining the tracking result with the best evaluation result as the target tracking result; and framed out the tracking target in the search area image by using the target tracking result.

[0078] After obtaining the fusion feature sequences corresponding to each modality, the weighted fusion feature sequences corresponding to each modality are weighted fused to obtain a weighted fusion feature sequence, i.e., a feature sequence of RGB-T. Specifically, the classification result diagram can be obtained by classifying the fusion feature sequences of RGB and TIR modalities respectively.

[0079] Determine the classification diagram of the fusion feature sequence corresponding to each modality. The classification diagram refers to the classification result diagram obtained after classifying the fusion feature sequence when performing the classification task. Based on the classification diagram of each modality, calculate the contribution of each modality to the final classification result; determine the weight of each modality based on the contribution. After obtaining the weight, perform weighted fusion on the fusion feature sequences of each modality to obtain a weighted fusion feature sequence.

[0080] In the process of weighted fusion of multi-modal fusion feature sequences, data of different modalities will be interfered by noise to varying degrees. The weights obtained through learning can automatically adjust the contribution of each modality in the fusion process, thereby filtering out those modal data with large noise and small contribution to the final result, thereby improving the accuracy and robustness of the final classification and regression tasks.

[0081] One or more stacked convolutional neural networks are designed for each modality's fusion feature sequence and weighted fusion feature sequence. Each fusion feature sequence and weighted fusion feature sequence is classified by the corresponding convolutional neural network to determine the corresponding tracking results.

[0082] Set appropriate evaluation indicators, which are used to evaluate the accuracy of the tracking results. Use the selected evaluation indicators to evaluate each tracking result to obtain the corresponding evaluation result. The evaluation result can be a numerical value, such as the value of a certain evaluation indicator, or a ranking, such as the evaluation indicator ranking from the best to the worst.

[0083] According to the evaluation results corresponding to the fusion feature sequences and weighted fusion feature sequences of each modality, the tracking result with the best evaluation result is selected as the target tracking result.

[0084] Using the target tracking results, such as the position and size of the tracked target, a bounding box is drawn in the search area image to select the tracked target.

[0085] refer to Figure 2 , the disclosed embodiment can be executed by a bounding box determination module. In the case where the modalities include: visible light modality and thermal infrared modality, determine the first fusion feature sequence corresponding to the visible light modality and the second fusion feature sequence corresponding to the thermal infrared modality, input the first fusion feature sequence and the second fusion feature sequence into the bounding box determination module, the bounding box determination module performs weighted fusion on the first fusion feature sequence and the second fusion feature sequence based on the weighted selection of the classification graph to obtain a weighted fusion feature sequence, respectively determine the corresponding tracking result through the first fusion feature sequence, the second fusion feature sequence and the weighted fusion feature sequence; evaluate each tracking result through the evaluation index to obtain the corresponding evaluation result; determine the tracking result with the best evaluation result as the target tracking result; and select the tracking target in the tracking area image through the target tracking result.

[0086] By adopting the embodiments of the present disclosure, the module can effectively fuse feature information from different modes through a weighted selection mechanism, and assign weights according to the confidence of the classification graph, which can reduce the impact of noise when the mode fails, thereby improving the feature expression ability of the effective mode. In the case of a complete failure of a certain mode, the module can also adaptively select the best tracking result by evaluating the performance of each mode, which can ensure that the system can still maintain good tracking performance under adverse conditions.

[0087] Among them, in an optional embodiment, after the tracking target is framed in the search area image, it also includes: determining the image to be updated through the framed area in the search area image; evaluating the image to be updated through position information, discrimination information and appearance information to determine whether the image to be updated has passed the evaluation; if the image to be updated has passed the evaluation, updating the dynamic template and determining the image to be updated as the updated dynamic template; if the image to be updated has not passed the evaluation, not updating the dynamic template.

[0088] The image to be updated is re-selected in the search area image based on the center point of the framed area in the search area image.

[0089] Whether the dynamic template is updated needs to be determined based on the tracking information corresponding to the search area image of the current frame.

[0090] The tracking information is extracted from the search area image of the current frame, including position information, discrimination information and appearance information.

[0091] The position information is the best predicted bounding box output by the tracking head of the tracker in the current frame, which can reflect the geometric shape of the predicted target in the current frame and the position information in the entire search area image. The predicted bounding box output by the tracking head can be used to obtain the position information of the tracked target in the current frame.

[0092] The classification map matrix output by the tracking head can be used as discriminant information. The classification map can well reflect the degree to which the current frame tracker distinguishes the target from the surrounding background. The recognition accuracy of the target can be evaluated based on the confidence value in the classification map. If the confidence is high, it means that the tracker has recognized the target well; if the confidence is low, there may be misjudgment.

[0093] However, relying solely on the output of the tracker as a condition for determining the dynamic template update will inevitably introduce noise due to misjudgment. This erroneous template update may cause the tracker to drift, so the template appearance information is also referenced as an important objective judgment condition. The appearance information of the current frame can be compared with the appearance information in the dynamic template to evaluate whether the appearance of the current target has changed significantly.

[0094] After obtaining the position information, discrimination information and appearance information, the dynamic template is evaluated whether to be updated. For example, a reasonable threshold can be set to determine whether the confidence of the classification map exceeds the threshold; the similarity between the appearance information of the current frame and the appearance information of the dynamic template is determined and evaluated using a similarity metric.

[0095] If the position information, the discrimination information and the appearance information all meet the update conditions, the update operation of the dynamic template is performed; otherwise, the original dynamic template is retained.

[0096] refer to Figure 2 , the template update module can be used to determine whether the dynamic template is updated.

[0097] By adopting the embodiments of the present disclosure, by integrating the tracking information of the current frame, it is possible to dynamically adapt to changes in the tracking target, thereby ensuring that the tracker can more accurately reflect the state of the tracking target at different time points, thereby improving the overall tracking accuracy. Relying on the comprehensive evaluation of position information, discrimination information and appearance information, the risk of misjudgment that may be caused by a single information source is reduced. By comparing and verifying the mutual consistency of different information, the noise introduced by misjudgment can be effectively reduced, and erroneous template updates can be avoided. Incorrect template updates may cause the tracker to drift and affect subsequent tracking effects. This situation can be effectively avoided by comprehensively considering multiple tracking information, ensuring that the tracker can still maintain accurate tracking when the target appearance changes.

[0098] refer to Figure 2 In order to provide an overall explanation of the target tracking method provided by the present invention, the workflow of the method is now introduced, including steps one to six.

[0099] Step 1: Data processing and embedding For an entire RGB or TIR search area image passed in, it is first split according to the set patch size and expanded into a sequence of multiple patches, then projected into a high-dimensional space through a convolutional network, and finally added to the learnable position embedding to obtain an embedded feature sequence.

[0100] Step 2: Perform bidirectional temporal fusion The bidirectional spatiotemporal fusion module receives feature sequences from dynamic templates or static templates, performs dimensionality reduction on the feature information corresponding to the feature sequences, obtains streamlined cross-temporal prompt information, and then re-upgrades the dimension to the same size as the feature information of the template image, adds it to the feature information of another time, and completes the temporal information fusion.

[0101] Step 3: Perform spatial bidirectional fusion The bidirectional spatiotemporal fusion module inputs the search area sequence of a certain modality, performs dimensionality reduction information extraction on the feature sequence, obtains streamlined cross-modal prompt information, and then re-upgrades it to the same dimension as the feature information corresponding to the search area image, and adds it to the feature information in the search area sequence of another modality to complete the inter-modal information fusion.

[0102] Step 4: Perform Transformer backbone network calculation The target feature sequence obtained by fusing the dynamic template and the static template through temporal information is concatenated with the search area sequence and sent to the Transformer network. After normalization, multi-head self-attention, normalization, and multi-layer perceptron calculation, the separate output of the modality of the current layer is obtained. After adding the cross-modal prompt information provided by the other modality, the complete output of the current layer is obtained, that is, the fused feature sequence.

[0103] Step 5: Classification and regression After obtaining the fusion feature sequences corresponding to the two modalities, the fusion feature sequences corresponding to the RGB modality, the fusion feature sequences corresponding to the TIR modality, and the weighted fusion feature sequences corresponding to RGB-T are classified and regressed respectively. The best result among the three tracking results is selected as the target tracking result.

[0104] Step 6: Template Update The template update module passes in and integrates the tracking information of the current frame to determine whether the dynamic template should be updated. The tracking information includes position information, discrimination information, and appearance information.

[0105] Based on the same technical concept, the present disclosure provides a target tracking device. Figure 3 is a block diagram of a target tracking device shown in an embodiment of the present disclosure. Figure 3 As shown, the device comprises: An acquisition module 310 is used to acquire a search area sequence corresponding to the search area image of each modality, and to acquire a first target sequence corresponding to each dynamic template and a second target sequence corresponding to each static template, wherein the static template represents the initial features of the tracked target, and the dynamic template represents the motion features of the tracked target; A fusion module 320 is used to input the search area sequence, the first target sequence, and the second target sequence of each modality into a weight-sharing encoding layer to perform temporal information fusion and inter-modality information fusion to obtain a fusion feature sequence of each modality; The tracking module 330 is used to select the tracking target in the search area image according to the fusion feature sequences of each modality.

[0106] Optionally, the fusion module is specifically used to execute: Performing time series information fusion on the first target sequence and the second target sequence to obtain a target feature sequence, wherein the target feature sequence is used to capture the behavior of the tracked target; Based on the target feature sequence, feature enhancement is performed on the search area sequence to obtain an enhanced feature sequence, wherein the feature enhancement is used to distinguish background information and tracking target information in the search area sequence; Inter-modal information interaction is performed according to the search area sequences of different modalities to obtain cross-modal prompt information; The enhanced feature sequence and the cross-modal prompt information are fused to form a fused feature sequence.

[0107] Optionally, the fusion module is specifically used to execute: Performing dimensionality reduction information extraction on the second target sequence to obtain cross-time prompt information; Upgrading the cross-time prompt information to a dimension with the same size as the feature information in the first target sequence, and superimposing it with the feature information in the first target sequence to obtain a first target sequence superimposed with the cross-time prompt information; The first target sequence superimposed with the cross-time prompt information is determined as the target feature sequence.

[0108] Optionally, the fusion module is specifically used to execute: Performing dimensionality reduction information extraction on the search area sequence of each modality to obtain first cross-modal prompt information; The first cross-modal prompt information is upgraded to a dimension having the same size as the feature information in the search area sequence to obtain a first upgraded result; the first upgraded result is used to be superimposed with the enhanced feature sequence of another modality.

[0109] Optionally, the fusion module is specifically used to execute: receiving a second dimension-raising result obtained by a search region sequence of another modality; The second dimension-upgraded result is superimposed on the enhanced feature sequence to obtain the fused feature sequence after cross-modal fusion.

[0110] Optionally, the device further comprises: A determination module, used to determine the image to be updated by selecting a frame area in the search area image; An evaluation module, used to evaluate the image to be updated based on the position information, the discrimination information and the appearance information, and determine whether the image to be updated passes the evaluation; The updating module is used to update the dynamic template and determine the image to be updated as the updated dynamic template if the image to be updated passes the evaluation; and not update the dynamic template if the image to be updated fails the evaluation.

[0111] Optionally, the tracking module is specifically used to execute: Determine the classification diagram of the fusion feature sequence corresponding to each modality; According to each of the classification graphs, weighted fusion is performed on the fusion feature sequences of each modality to obtain a weighted fusion feature sequence; Determine the corresponding tracking result through the fusion feature sequence corresponding to each modality and the weighted fusion feature sequence; Through the evaluation indicators, each tracking result is evaluated to obtain the corresponding evaluation results; The tracking result with the best evaluation result is determined as the target tracking result; The target to be tracked is selected in the search area image by using the target tracking result.

[0112] Optionally, the acquisition module is specifically used to execute: According to the set patch size, the image is segmented and expanded into a sequence of multiple patches; the image is a search area image, a static template image or a dynamic template image; Projecting the sequence of the multiple patches into a high-dimensional space through a convolutional network to obtain a feature sequence; Wherein, when the image is a search area image, after projecting the sequence composed of the multiple patches into a high-dimensional space, the projection result is embedded and added with the position information to obtain a feature sequence; the position information is the position of the patch in the search area image.

[0113] The present disclosure also provides an electronic device, referring to Figure 4 , Figure 4 is a schematic diagram of an electronic device shown in an embodiment of the present disclosure. Figure 4 As shown, the electronic device 400 includes: a memory 410 and a processor 420. The memory 410 and the processor 420 are connected via a bus communication. A computer program is stored in the memory 410. The computer program can be run on the processor 420 to implement the steps in the target tracking method disclosed in the embodiment of the present disclosure.

[0114] The embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the target tracking method disclosed in the embodiment of the present disclosure are implemented.

[0115] The embodiment of the present disclosure further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps in the target tracking method disclosed in the embodiment of the present disclosure are implemented.

[0116] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0117] It should be understood by those skilled in the art that the embodiments of the present disclosure may be provided as methods, devices or computer program products. Therefore, the embodiments of the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0118] The embodiments of the present disclosure are described with reference to the flowcharts and / or block diagrams of the methods, apparatuses, electronic devices, and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0119] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0120] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0121] Although some embodiments of the present disclosure have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the present disclosure.

[0122] The target tracking method, system and device provided by the present invention are introduced in detail above. The principles and implementation methods of the present invention are explained in detail using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A target tracking method, characterized in that: include: Acquire a search area sequence corresponding to the search area image of each modality, and acquire a first target sequence corresponding to each dynamic template and a second target sequence corresponding to each static template, wherein the static template represents the initial features of the tracked target and the dynamic template represents the motion features of the tracked target; Inputting the search area sequence, the first target sequence, and the second target sequence of each modality into a weight-sharing encoding layer, performing temporal information fusion and inter-modality information fusion, and obtaining a fusion feature sequence of each modality; The tracking target is framed and selected in the search area image according to the fusion feature sequences of each modality.

2. The method according to claim 1, characterized in that The search area sequence, the first target sequence, and the second target sequence of each modality are input into the weight-sharing encoding layer for time series information fusion and inter-modality information fusion to obtain a fusion feature sequence of each modality, including: Performing time series information fusion on the first target sequence and the second target sequence to obtain a target feature sequence, wherein the target feature sequence is used to capture the behavior of the tracked target; Based on the target feature sequence, feature enhancement is performed on the search area sequence to obtain an enhanced feature sequence, wherein the feature enhancement is used to distinguish background information and tracking target information in the search area sequence; Inter-modal information interaction is performed according to the search area sequences of different modalities to obtain cross-modal prompt information; The enhanced feature sequence and the cross-modal prompt information are fused to form a fused feature sequence.

3. The method according to claim 2, characterized in that Performing time series information fusion on the first target sequence and the second target sequence to obtain a target feature sequence includes: Performing dimensionality reduction information extraction on the second target sequence to obtain cross-time prompt information; Upgrading the cross-time prompt information to a dimension with the same size as the feature information in the first target sequence, and superimposing it with the feature information in the first target sequence to obtain a first target sequence superimposed with the cross-time prompt information; The first target sequence superimposed with the cross-time prompt information is determined as the target feature sequence.

4. The method according to claim 2, characterized in that: According to the search area sequences of different modalities, information interaction between modalities is performed to obtain cross-modal prompt information, including: Performing dimensionality reduction information extraction on the search area sequence of each modality to obtain first cross-modal prompt information; The first cross-modal prompt information is upgraded to a dimension having the same size as the feature information in the search area sequence to obtain a first upgraded result; the first upgraded result is used to be superimposed with the enhanced feature sequence of another modality.

5. The method according to claim 4, characterized in that After the search area sequence is enhanced based on the target feature sequence to obtain an enhanced feature sequence, the method further includes: receiving a second dimension-raising result obtained by a search region sequence of another modality; The second dimension-upgraded result is superimposed on the enhanced feature sequence to obtain the fused feature sequence after cross-modal fusion.

6. The method according to claim 1, characterized in that After selecting the tracking target in the search area image, it also includes: Determine the image to be updated by selecting the framed area in the search area image; Evaluate the image to be updated based on the position information, the discrimination information and the appearance information, and determine whether the image to be updated passes the evaluation; If the image to be updated passes the evaluation, the dynamic template is updated, and the image to be updated is determined as the updated dynamic template; If the image to be updated fails the evaluation, the dynamic template is not updated.

7. The method according to claim 1, characterized in that According to the fusion feature sequences of each modality, the tracking target is framed in the search area image, including: Determine the classification diagram of the fusion feature sequence corresponding to each modality; According to each of the classification graphs, weighted fusion is performed on the fusion feature sequences of each modality to obtain a weighted fusion feature sequence; Determine the corresponding tracking result through the fusion feature sequence corresponding to each modality and the weighted fusion feature sequence; Through the evaluation indicators, each tracking result is evaluated to obtain the corresponding evaluation results; The tracking result with the best evaluation result is determined as the target tracking result; The target to be tracked is selected in the search area image by using the target tracking result.

8. The method according to any one of claims 1 to 7, characterized in that: The search area sequence, the first target sequence, and the second target sequence are determined in the following manner, including: According to the set patch size, the image is segmented and expanded into a sequence of multiple patches; the image is a search area image, a static template image or a dynamic template image; Projecting the sequence of the multiple patches into a high-dimensional space through a convolutional network to obtain a feature sequence; Wherein, when the image is a search area image, after projecting the sequence composed of the multiple patches into a high-dimensional space, the projection result is embedded and added with the position information to obtain a feature sequence; the position information is the position of the patch in the search area image.

9. A target tracking device, characterized in that: include: An acquisition module is used to acquire a search area sequence corresponding to the search area image of each modality, and to acquire a first target sequence corresponding to each dynamic template and a second target sequence corresponding to each static template, wherein the static template represents the initial features of the tracked target, and the dynamic template represents the motion features of the tracked target; A fusion module is used to input the search area sequence, the first target sequence, and the second target sequence of each modality into a weight-sharing encoding layer to perform temporal information fusion and inter-modality information fusion to obtain a fusion feature sequence of each modality; The tracking module is used to select the tracking target in the search area image according to the fusion feature sequence of each modality.

10. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the target tracking method according to any one of claims 1 to 8 are implemented.

Images