Video target tracking method and device, medium and equipment

Abstract

The invention belongs to the technical field of computer vision and artificial intelligence, and particularly discloses a video target tracking method and device, a medium and equipment, and the method comprises the steps: synchronously obtaining an infrared image and a visible light image which comprise a to-be-detected target; the infrared image and the visible light image are preprocessed; constructing a to-be-detected target tracking model, and training the model; and inputting the preprocessed infrared image and visible light image into a trained to-be-detected target tracking model so as to carry out identification and position tracking on a to-be-detected target. According to the invention, high-precision, high-robustness and high-continuity position tracking of the to-be-measured target in a complex environment can be realized.

Description

Technical Field

[0001] The present application belongs to the field of computer vision and artificial intelligence technology, and specifically relates to a video target tracking method, device, medium and equipment. Background Art

[0002] Existing infrared and visible light image fusion and video target tracking technologies still face numerous challenges in practical applications, primarily due to crude fusion strategies, difficulty in fully aligning inter-modal differences, and insufficient tracking robustness. Traditional methods often rely on simple image-level or feature-level stitching, which struggles to balance the thermal properties of infrared images with the detailed textures of visible light images. This can lead to structural blurring, weakened heat sources, or edge distortion in the fused image. Furthermore, in video tracking, existing methods have limited capabilities for occlusion, adaptive updates, and multi-frame correlation modeling, making them prone to target drift, mismatches, and even tracking failures in complex dynamic scenes. This severely limits their practicality in high-reliability scenarios such as night vision surveillance, intelligent transportation, and border security.

[0003] Therefore, there is an urgent need for an improved method that can achieve precise alignment of multimodal features, adaptive fusion and highly robust tracking to improve image quality and tracking continuity. Summary of the Invention

[0004] In response to the shortcomings of the existing technology, the purpose of this application is to provide a video target tracking method, device, medium and equipment. Through the joint design of multimodal fusion and occlusion-aware tracking strategy, this application can achieve high precision, high robustness and strong continuity in target tracking in complex environments.

[0005] To achieve the above objectives, this application provides the following technical solutions: A video target tracking method comprises: synchronously acquiring an infrared image and a visible light image containing a target to be measured; preprocessing the infrared image and the visible light image; constructing a target tracking model to be measured and training the model; and inputting the preprocessed infrared image and the visible light image into the trained target tracking model to identify and track the position of the target to be measured.

[0006] Optionally, the infrared image is preprocessed, including: unifying the size of the infrared image and normalizing the thermal value; performing local contrast enhancement and nonlinear thermal value mapping on the normalized infrared image; constructing a spatial Gaussian attention map based on thermal intensity and guiding weighted smoothing processing.

[0007] Optionally, the visible light image is preprocessed, including: standardizing the size and brightness of the visible light image; performing illumination correction and local contrast enhancement on the normalized visible light image; performing edge extraction and detail enhancement on the illumination-corrected visible light image; converting the detail-enhanced visible light image to Lab space and enhancing the a / b channel color contrast.

[0008] Optionally, the target tracking model includes: the multimodal fusion module, which is used to fuse the preprocessed infrared image and the visible light image to obtain a fused image; and the video tracking module, which is used to identify and track the position of the target based on the fused image.

[0009] Optionally, the multimodal fusion module includes: a feature extraction and alignment unit, used to extract multi-scale features of the preprocessed infrared image and visible light image and perform spatial alignment; an adaptive feature fusion unit, used to adaptively fuse the aligned features to obtain a fused image; and an optimization unit, used to structurally optimize and reconstruct the fused image.

[0010] Optionally, the video tracking module includes: an initialization tracking unit for constructing a multimodal template to provide tracking start reference information; a lightweight temporal association unit for combining the previous frame residual information with the current frame features to predict the target displacement; an occlusion perception update unit for detecting the occlusion state based on the similarity heat map change, and guiding the template update strategy to maintain tracking stability.

[0011] The present application also provides a video target tracking method device, which includes: an acquisition unit for synchronously acquiring an infrared image and a visible light image containing a target to be measured; a preprocessing unit for preprocessing the infrared image and the visible light image; a model construction and training unit for constructing a target tracking model to be measured and training the model; and an identification unit for inputting the preprocessed infrared image and the visible light image into the trained target tracking model to identify and track the position of the target to be measured.

[0012] The present application also provides a storage medium comprising instructions, which, when executed on a computer, enables the computer to execute the method as described in any of the preceding items.

[0013] The present application also provides an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above methods when executing the program.

[0014] Compared with the prior art, the present invention has the following beneficial effects: This application fuses the thermal information of infrared images with the texture details of visible light images to construct a multimodal feature fusion mechanism guided by structural consistency. Combined with strategies such as previous frame residual modeling, saliency enhancement, and occlusion perception, this significantly improves target recognizability and tracking stability in complex environments. Compared to traditional methods, this application not only maintains accurate and continuous target positioning in low-light, occlusion, and high-speed motion scenarios, but also exhibits strong anti-interference and occlusion recovery capabilities, achieving more robust and precise target tracking results. It has excellent practical value and promotion prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flow chart of a video target tracking method provided by one embodiment of the present application; Figure 2 is a schematic diagram of an original infrared image provided by another embodiment of the present application; Figure 3 is a schematic diagram of an original visible light image provided by another embodiment of the present application; Figure 4 It is a schematic diagram after the integration of traditional methods; Figure 5 This is a schematic diagram after fusion based on the method described in this application; Figure 6 This is a schematic diagram of the tracking effect of the traditional method; Figure 7 This is a schematic diagram of the tracking effect based on the method described in this application; Figure 8 This is a schematic diagram of the tracking accuracy of the traditional method under high-speed motion; Figure 9 This is a schematic diagram of the tracking accuracy of a target under high-speed motion based on the method described in this application; Figure 10 This is a schematic diagram of occlusion recovery using traditional methods; Figure 11 is a schematic diagram of occlusion recovery based on the method described in this application; Figure 12 It is a structural diagram of a video target tracking device provided by another embodiment of the present application. DETAILED DESCRIPTION

[0016] Specific embodiments of the present application will be described in detail below with reference to the accompanying drawings. Although specific embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0017] It should be noted that certain words are used in the specification and claims to refer to specific components. Those skilled in the art should understand that technicians may use different nouns to refer to the same component. This specification and claims do not use the difference in nouns as a way to distinguish components, but use the difference in the functions of the components as the criterion for distinction. As mentioned throughout the specification and claims, "including" or "comprising" is an open term, so it should be interpreted as "including but not limited to". The subsequent description of the specification is a preferred embodiment of the present application, but the description is based on the general principles of the specification and is not intended to limit the scope of the present application. The scope of protection of this application shall be as defined by the attached claims.

[0018] To facilitate understanding of the embodiments of the present application, further explanation will be given below using specific embodiments as examples in conjunction with the accompanying drawings, and the various drawings do not constitute a limitation on the embodiments of the present application.

[0019] Figure 1 FIG. 1 is a flow chart of a video target tracking method provided by an exemplary embodiment of the present application. Figure 1 As shown, the method includes: S100: synchronously acquiring an infrared image and a visible light image containing a target to be tracked; S200: Preprocessing the infrared image and the visible light image; S300: Build a target tracking model and train the model; S400: Inputting the pre-processed infrared image and the visible light image into a trained target tracking model to identify and track the target.

[0020] In another exemplary embodiment, in step S200, preprocessing the infrared image includes: S201: performing size unification and normalization processing on the infrared image; In this step, since the infrared images come from different sensor devices and have inconsistent resolutions between frames, this step first performs scale normalization on the input image, that is, scaling the infrared image to a preset standard size (such as 224×224 or 512×512) to ensure the consistency of the subsequent model input dimensions.

[0021] After resizing, the infrared images are further normalized to eliminate absolute differences in thermal value distribution between different infrared images. This compresses pixel intensities to the standard range [0, 1]. Normalization not only improves the numerical stability of image processing but also provides a unified thermal value reference for the next step of local contrast enhancement.

[0022] S202: performing local contrast enhancement and nonlinear thermal value mapping on the normalized infrared image; In this step, after resizing and thermal value normalization, the infrared image now possesses standardized thermal signal representation capabilities. However, in scenes with low to medium background temperatures or with subtle thermal gradients (such as nighttime long-distance surveillance and complex urban environments), the target heat source region may still be indistinguishable from the background, limiting subsequent target detection and tracking. To enhance the discernibility of the target region, this step introduces the CLAHE (Contrast Limited Adaptive Histogram Equalization) algorithm. By dynamically adjusting the grayscale distribution within a local image window, this algorithm enhances local image contrast, highlighting the edge structure and localized thermal variations in the heat source region, and suppressing the impact of large low-contrast areas on overall visual perception. Furthermore, by setting a contrast-limiting threshold, the CLAHE algorithm effectively avoids noise amplification caused by over-enhancement in high-dynamic-range infrared images.

[0023] Based on the CLAHE enhancement, this embodiment further adopts a nonlinear thermal value mapping method in this step, such as logarithmic transformation (applying a logarithmic function mapping to the normalized thermal value map to increase the response amplitude of low thermal value areas (cold areas, medium and low temperature areas), thereby enhancing the visual significance of weak thermal signals (such as distant pedestrians and low-heating equipment) in the image) or gamma correction (by adjusting the gamma parameter (γ∈ [0.4, 1.2]), controlling the brightness enhancement degree of medium and low thermal value areas, taking into account the overall visual effect and detail expression of the image, and further improving the clarity and contrast of heat source edges), to increase the response value of medium and low thermal areas, thereby amplifying weak thermal signals (such as distant pedestrians and vehicles with edge heat sources).

[0024] The above processing method can simulate the human eye's sensitive response to temperature differences, realize the active amplification of weak thermal signals in infrared images, and at the same time keep the details of high-heat areas (such as vehicle engines and core heat areas of the human body) without distortion, thereby effectively improving the saliency and discrimination ability of the target area.

[0025] S203: Construct a spatial Gaussian attention map based on thermal intensity and guide weighted smoothing processing.

[0026] In this step, to further highlight the target heat source area in the infrared image and suppress redundant background signals, a spatial thermal weight map is constructed based on the enhanced thermal value map. The specific method is as follows: First, based on the thermal value map after local contrast enhancement and nonlinear thermal value mapping, high-temperature areas in the image are identified, and these heat source areas are used as thermal center points. On this basis, a two-dimensional Gaussian distribution function is constructed around each heat source center to generate the corresponding spatial thermal attention map. This attention map assigns different response weights to different locations in the image, with the weight value of high-temperature areas approaching 1, and the weight of cold background areas far from the heat source gradually approaching 0, thus forming a continuous and smooth spatial weighted map.

[0027] Subsequently, the attention map is used as a mask to perform weighted smoothing on the original enhanced image: in the heat source concentration area, the edge structure is kept clear to retain the thermal characteristics of the key target; while in the background area where the thermal gradient changes smoothly or there is no significant heat source distribution, blur suppression is performed to reduce the response intensity of redundant noise.

[0028] In another exemplary embodiment, in step S200, preprocessing the visible light image includes the following steps: S2001: Standardize the size and brightness of the visible light image; In this step, to ensure consistent input dimensions and distribution characteristics with the infrared image, the visible light image is first subjected to basic normalization processing. Specifically, it includes: Size normalization: uniformly scale images to a set size (such as 224×224 or 512×512) to facilitate subsequent model alignment and fusion; Brightness normalization: normalize the image pixel values ​​to [0,1] or maintain them in the standard 8-bit RGB range; Dynamic range compression: To address the brightness differences in different shooting scenes, the full image brightness is processed through linear adjustment or gamma correction (γ∈[1.5, 2.2]) to eliminate unevenness between bright and dark areas.

[0029] This step serves as the starting point of preprocessing and builds a unified calculation basis for subsequent lighting enhancement and texture extraction.

[0030] S2002: performing illumination correction and local contrast enhancement on the normalized visible light image; During this step, the standardized visible light image may still experience local underexposure and blurred structures at night or in complex lighting scenarios, affecting the accuracy of subsequent feature extraction and target detection. To improve the illumination robustness of visible light images, this embodiment introduces a multi-scale Retinex algorithm (such as MSRCR) to perform local dynamic compression and global contrast enhancement on the standardized visible light image. This algorithm uses a multi-scale logarithmic transformation to decompose the image's reflective and illumination components, dynamically compressing local areas of strong light while enhancing detail in low-light areas. This algorithm simulates the human eye's visual compensation mechanism under varying brightness conditions. Specifically, the algorithm constructs a Gaussian filter pyramid at different scales to achieve multi-scale modeling of the image's local illumination distribution; applies logarithmic dynamic range compression to suppress local overexposure and underexposure, enhancing detail in dark areas; and introduces a color restoration mechanism to maintain image color naturalness while enhancing brightness, preventing color casts and artifacts.

[0031] After the above processing, the visible light image as a whole can present the effects of enhanced edge contrast, clear object contours, rich details and balanced lighting, which helps to improve image quality under complex lighting conditions.

[0032] S2003: performing edge extraction and detail enhancement on the visible light image after illumination correction; In this step, based on an image with good lighting balance, in order to further explore its structural details, this step uses the Sobel operator or Canny edge detection algorithm to extract the image's significant edge areas (such as window frames, license plate outlines, portrait edges, etc.) and generate an edge mask.

[0033] An unsharp mask is then constructed, overlaying the edge map with the original image to enhance local texture details. This structural enhancement not only helps the model accurately identify boundary information, but also significantly improves the visible light image's ability to express high-frequency features, providing an edge location reference for multimodal feature alignment.

[0034] S2004: Convert the detail-enhanced visible light image to Lab space and enhance the a / b channel color contrast.

[0035] In this step, color and structure in visible light images are often coupled, which affects the accuracy of boundary modeling. To effectively decouple color and structure features, this step converts the image from RGB space to Lab color space, where the L channel represents brightness and the a / b channels represent color components.

[0036] By performing nonlinear stretching and normalization on the a / b channels, we can effectively enhance the color contrast between different object regions in visible light images, strengthening the ability to distinguish between objects and backgrounds in visual perception. This operation is particularly suitable for objects with distinguishable color features (such as vehicle bodies / backgrounds, clothing / roads, etc.), further promoting the complementary fusion of color and texture between modalities.

[0037] In another exemplary embodiment, Figure 2 As shown, the target tracking model includes: a multimodal fusion module, which is used to perform multimodal fusion on the preprocessed infrared image and the visible light image to obtain a multimodal fusion image; and a video tracking module, which is used to identify and track the position of the target based on the multimodal fusion image.

[0038] In this embodiment, the multimodal fusion module includes: a feature extraction and alignment unit, an adaptive feature fusion unit and an optimization unit, wherein the feature extraction and alignment unit is used to extract multi-scale features of the preprocessed infrared image and the visible light image and perform spatial alignment, and the feature extraction and alignment unit includes an infrared branch and a visible light branch, which are respectively used to extract features from the preprocessed infrared image and the visible light image.

[0039] The infrared branch is used to extract thermal feature information from preprocessed infrared images and obtain an infrared feature map. The infrared branch includes a thermally guided attention module, a multi-scale pyramid residual extraction module, and a deformable convolution enhancement layer. First, the thermally guided attention module sequentially performs thermal value normalization (mapping the pixel thermal values ​​in the infrared image to the standard interval [0, 1] to eliminate thermal value scale differences between different images), Gaussian smoothing (applying Gaussian filtering to the normalized image to reduce the impact of local noise and highlight the overall trend of thermal distribution), and nonlinear stretching (using nonlinear mapping methods such as logarithmic transformation or gamma correction to dynamically stretch the smoothed thermal value map to further improve the response value of medium and low thermal areas) to generate a thermal weight map w(x, y) (high temperature areas are close to 1, and background areas are close to 0). This thermal weight map not only serves as a spatial attention mask and is element-wise multiplied with the original infrared image to form a forward attention map to spatially enhance the feature expression of the high-heat target area, but also dynamically adjusts the weight distribution of each channel of the infrared image feature map by embedding a channel attention mechanism (CBAM), thereby guiding the model to focus on high-heat target areas (such as human bodies and vehicles) and suppress background interference.

[0040] Based on thermal guidance, the multi-scale pyramid residual extraction module employs a four-branch structure, employing 1×1, 3×3, and 5×5 standard convolutions, as well as a 3×3 dilated convolution to extract features of heat sources at different spatial scales. The 1×1 convolution branch (which reconstructs channel-wise features for each pixel and exhibits strong local responsiveness, helping to preserve critical regions of high thermal intensity but small areas) is used to reconstruct channel features. Combined with batch normalization and ReLU activation, it captures localized high-temperature pixel regions (hotspot centers, such as concentrated heat sources like vehicle engines) in the preprocessed infrared image. This prevents "thermal feature averaging" caused by subsequent convolution or fusion operations, thereby enhancing the model's focus on small objects in the input feature map. The 3×3 convolution branch captures edge contour information of medium-scale thermal targets (such as car windows and tires) in the preprocessed infrared image. Compared to 1×1 convolution, 3×3 convolution has stronger local spatial modeling capabilities and can identify hot-cold boundary transitions within the image. The 5×5 convolution branch is responsible for extracting areas with wide temperature distributions (such as vehicle bodies) in preprocessed infrared images, enhancing the model's perception of low-frequency thermal structures such as background hot spots and hot cluster outlines. When heat sources are distributed over large areas, small-scale convolutions (such as 1×1 and 3×3 convolutions) have difficulty perceiving overall temperature trends, while 5×5 convolutions can provide broader structural capture. The 3×3 dilated convolution branch is responsible for capturing areas with blurred edges and uneven heat diffusion (such as thermal streaks produced by a moving vehicle) in preprocessed infrared images. The 3×3 dilated convolution allows for skip sampling, effectively extracting long-range temperature variation information and enhancing the response to thermal gradient bands. This compensates for the fuzzy areas that the first three branches cannot effectively model, thereby enhancing the model's robustness to continuous thermal distributions and blurred outlines.

[0041] The output features of the four branches are concatenated in the channel dimension and then compressed and fused using a 1×1 convolution to form a unified-scale thermal feature map. This is then residually connected with the preprocessed original infrared image to obtain an enhanced thermal feature map at a unified scale for subsequent infrared-visible light feature alignment and fusion.

[0042] To further improve the model's adaptability to non-rigid structures and blurred edge areas, the infrared branch also introduces a deformable convolution enhancement layer, which includes an offset generation submodule and a deformable convolution submodule. The offset generation submodule takes the enhanced thermal feature map at a uniform scale as input and uses a 3×3 standard convolution to extract the local structure (such as thermal boundaries and texture gradients) in the enhanced thermal feature map to predict a two-dimensional offset vector in the (x, y) direction for each convolution sampling point. The deformable convolution submodule uses a two-dimensional offset vector And the enhanced thermal feature map at a uniform scale is taken as input. At each convolution sampling point, according to the two-dimensional offset vector Dynamically adjust the sampling point position, then perform weighted convolution, and finally output the thermal feature enhancement map :

[0043] in, Represents thermal feature enhancement map At the current output feature point position The response value at reflects the thermal feature expression after deformation sampling and weighting. Infrared branch feature map representing the input; Indicates the total number of sampling points covered by the convolution kernel; Indicates the first sampling locations; Indicates the The learnable offset of points; Indicates the convolution kernel in The weight of the sampling points.

[0044] In summary, the infrared branch constructs a feature extraction path with complete functions, high expression accuracy, and sensitive response to target thermal zones through the combined effects of modules such as thermal attention drive, multi-scale residual modeling, and deformation perception enhancement, providing a high-quality structural foundation and heat source alignment capability for subsequent infrared-visible light feature fusion.

[0045] The visible light branch performs deep feature extraction on preprocessed visible light images, generating visible light feature maps with clear structure, rich texture, illumination adaptability, and modality alignment capabilities. The visible light branch comprises a guided edge enhancement module, a multi-channel frequency perception module, and a color-structure alignment module. First, the guided edge enhancement module calculates the gradient of the preprocessed visible light image using the Sobel operator to extract a preliminary edge image. This edge image effectively highlights the outlines of salient structures in the visible light image, such as details like window frames and license plate edges. This edge image is then converted into a spatial attention mask and embedded as weighted guidance in the backbone convolutional neural network (CNN) before being multiplied with the feature map elements. This guides the backbone network to focus on structurally salient regions during feature extraction and generates an edge attention mask. This improves the model's perception of high-frequency edge details, such as boundaries and textures, while suppressing invalid features in non-target areas. This helps improve the geometric representation accuracy and discriminative power of the visible light feature map in multimodal fusion.

[0046] Building on structural enhancement, the multi-channel frequency perception module incorporates a frequency domain analysis mechanism to exploit discriminative features carried by different frequency components in the image. This module, comprising a frequency domain transformation submodule, a channel attention regulation submodule, and a frequency selectivity enhancement submodule, aims to separate and model different frequency components (low-frequency structure and high-frequency detail) in visible light images and adaptively enhance them, thereby improving texture resolution and boundary definition. The frequency domain transformation submodule first performs a two-dimensional fast Fourier transform (2D FFT) on the input multi-channel visible light feature map, converting each channel from the spatial domain to the frequency domain. This yields a spectrum consisting of both an amplitude spectrum and a phase spectrum. The amplitude spectrum reflects the intensity of local image variations (e.g., texture information), while the phase spectrum preserves structural position information. Furthermore, to enhance frequency representation, the spectrum is centered, and high-pass and low-pass masks are used to extract high-frequency regions (e.g., license plate characters and window frame lines) and low-frequency regions (e.g., road background and vehicle outlines), respectively.

[0047] The channel attention control submodule performs global average pooling and maximum pooling on the spectrum graph in the channel dimension to extract channel response statistics. It then generates a channel attention vector through two fully connected layers with shared parameters, which is used to weight the high-frequency and low-frequency channel feature maps, thereby achieving adaptive enhancement of "high-response frequency channels." Through this collaborative frequency and channel modeling approach, the network can dynamically focus on the frequency bands and channels that contribute most to texture expression, improving the ability to discriminate key texture areas.

[0048] The frequency-selective enhancement submodule is responsible for accurately mining and enhancing discriminative frequency features in the image in the frequency domain. This module first receives a spectrogram processed by a fast Fourier transform (FFT) and introduces a set of learnable two-dimensional complex frequency-domain filters (such as high-pass, bandpass, and band-stop filters) to cover different frequency regions, ranging from high-frequency details (such as license plate characters and window frame outlines) to low-frequency structures (such as vehicle body outlines and background blocks). Unlike traditional static masks, these filters can be dynamically adjusted during training based on the target texture distribution in the specific scene, thereby enhancing key frequency band features in the target area. To avoid missing important information, this module preserves the frequency response characteristics of each filtered output and integrates the full frequency domain information through weighted fusion to enhance the synergistic expression of global and local information. Next, all filtered and enhanced spectrograms are converted back to the spatial domain via an inverse Fourier transform to reconstruct an image feature map containing the enhanced texture. Finally, the image feature map is fused with the original spatial feature map via a residual connection, ensuring that the enhanced features maintain their original structure while improving discriminability. The frequency-selective enhancement submodule can achieve frequency-selective enhancement of image textures, effectively improving the model's sensitivity and ability to distinguish detailed objects in complex scenes.

[0049] The color-structure alignment module is used to achieve decoupled expression of color information and structural information and to achieve cross-modal alignment. The color-structure alignment module includes a feature decoupling submodule, an illumination adaptation recombination submodule and a structural alignment correction submodule, wherein the feature decoupling submodule is used to convert the input visible light image into YUV color space or Lab space, thereby achieving preliminary separation of image color (U / V or a / b channels) and structure (Y or L channel) information. Subsequently, the color component and the brightness component are convolutionally encoded respectively to extract the color feature map and the structural feature map. In order to further strengthen the independence of the two types of features, the module introduces channel separation convolution and orthogonal constraint loss mechanism (in deep networks, for example, two feature vectors 、 Represent different semantic information, but they are highly correlated, then they may not be effectively decoupled, resulting in fuzzy or redundant effects after fusion. Through orthogonal constraints, it is possible to force 、 The two branches are as orthogonal as possible, that is, geometrically the angle between them is 90°, and mathematically the inner product is 0. For this application, the color features and structural features can be made to overlap as little as possible in the feature space, thereby achieving better semantic separation), so that the color branch is mainly responsible for capturing semantic features such as hue and color temperature, while the structural branch focuses on edges, shapes and textures.

[0050] Based on the decoupled features, the illumination-adaptive reconstruction submodule calculates the local brightness gradient of the structural feature map to generate an illumination mask to control the color and structure reconstruction weights. Furthermore, the illumination-adaptive reconstruction submodule introduces a lightweight gated linear unit (GLU) to perform pixel-wise weighted reconstruction of color and structure features. This dynamically suppresses color artifacts in areas with unstable illumination and enhances the structural representation of target boundary areas, thereby generating a fused feature map that is robust to illumination changes.

[0051] The structural alignment correction submodule operates on the structural feature map of the visible light image. Its core goal is to spatially correct regions of offset or inconsistency in the visible light structural features, using the thermal structural features extracted from the infrared image as a guide. This module first feeds the thermal structural features extracted from the infrared image into the lightweight attention guidance module and spatially aligns them with the structural features in the visible light image to form an edge alignment guidance signal. Subsequently, through a residual connection mechanism, it dynamically corrects portions of the visible light structural map that fail to precisely align with the infrared heat source regions. The final output is an aligned structural map with consistent edge positions and clear structural expression, which is then used in the subsequent modal fusion process.

[0052] The adaptive feature fusion unit is used to adaptively fuse the aligned features to obtain a fused image. The adaptive feature fusion unit includes a channel semantic enhancement submodule, a spatial semantic gating submodule, and a structural consistency guidance submodule. The channel semantic enhancement submodule first receives the feature maps output by the infrared branch and the visible light branch respectively, and performs independent global average pooling operations to obtain the global response and peak response information of each channel. Then, a shared two-layer fully connected network (MLP) is used to perform nonlinear mapping on the two sets of pooling results to generate the infrared branch channel weight vector and the visible light branch channel weight vector , thus reflecting the importance of the channels in each mode.

[0053] To further introduce cross-modal dependency modeling, the two sets of channel weights are normalized to enhance complementarity:

[0054]

[0055] in, It represents the attention weight of the infrared channel after difference perception, and represents the importance distribution of infrared in fusion; It represents the attention weight of the visible light channel after difference perception, and represents the importance distribution of visible light in fusion.

[0056] The cross-modal dependency modeling is a complementary modeling strategy that can improve the discrimination ability of the fusion by comparing the channel weights. For example, if Significantly greater than ,but tends to be heavier, on the contrary, if Greater than ,but It will pay more attention to this modality. In addition, through softmax difference modeling, the model can be more inclined to assign larger weights to modalities with significant semantic differences, thereby improving the effect of cross-modal feature fusion.

[0057] The spatial semantic gating submodule uses a shared lightweight convolutional block to generate spatial response maps at position (x, y) for infrared and visible light feature maps respectively. 、 In order to retain key target areas (such as human heat sources and car window edges), the two spatial maps are then subjected to pixel-level complementary weighted fusion:

[0058] in, Represents the response value of the fused spatial attention mask at position (x, y); represents the Sigmoid activation function, It is a trainable weight used to dynamically balance the contribution of infrared and visible light in the spatial domain.

[0059] The final spatial mask It is used as a gating mechanism to adjust the spatial response of the fused feature map, suppressing background noise (such as trees, shadows) and highlighting the target area.

[0060] Considering the natural difference in perception between infrared and visible light, direct fusion often leads to structural inconsistency. To this end, this application designs a structural consistency guidance submodule, which first extracts gradient maps from two branches (using Sobel or Laplace operators) to obtain the structural map. 、 , and then calculate the structural difference map :

[0061] Furthermore, the structural difference map As the guidance information input lightweight residual network, specifically, the structural difference map First, it passes through two consecutive 3×3 convolutional layers, and the convolution operation is followed by a ReLU function to extract local structural change features and enhance nonlinear expression capabilities. Subsequently, a lightweight residual block is introduced. This module contains two consecutive 3×3 convolutional layers, each of which is connected to batch normalization and ReLU activation functions. The input is directly added to the output through skip connections to achieve feature reuse and gradient stabilization, further improving the ability to model fine-grained edge differences. Next, a 1×1 convolutional layer is used to compress the number of channels to 1 to generate a single-channel structural correction map. , which represents the correction factor for the structural difference area to bridge the edge alignment error in the fused result:

[0062] in, It is the fusion map after channel attention and spatial gating, and the final output is This ensures the integrity of the infrared heat source while maintaining the spatial consistency of the structure edge.

[0063] Finally, the output is normalized by the Sigmoid activation function to ensure that the structure correction weight is in the range of [0,1], which can be used to perform local edge enhancement and position fine-tuning of the fusion feature.

[0064] The optimization unit is designed to perform structural optimization and reconstruction on the image features after preliminary fusion, to compensate for the information loss and edge blurring problems that may occur during the modal fusion process, thereby improving the overall quality and expression ability of the fused image. The optimization unit includes a global semantic compensation module, an edge refinement enhancement module, and a multi-scale reconstruction module to achieve multi-level optimization from semantic completion to edge restoration. First, the global semantic compensation module introduces the Transformer encoder structure to model the global context of the preliminary fused image, focusing on capturing long-distance cross-regional dependencies. Since infrared images and visible light images have incomplete information or insufficient semantic complementarity in some areas, direct fusion may lead to problems such as structural disconnection or regional semantic loss. Through the Transformer's multi-head attention mechanism, the global semantic compensation module can dynamically explore the potential semantic associations between long-distance regions in the image features after preliminary fusion, thereby achieving semantic completion of the fused image at the content level and improving overall consistency and expression integrity.

[0065] Secondly, to address issues such as blurred edges and structural fractures that can easily occur during modal fusion, the edge refinement and enhancement module uses an image gradient-guided mechanism to repair structural details. Specifically, the fused image and its Sobel edge map are jointly input into a shallow convolutional network. Feature residual learning is then used to refine and enhance edges to enhance the continuity and recognizability of image structural details, preventing blurred or lost edges after modal fusion. This module focuses on strengthening high-frequency features in the image, such as object outlines and structural boundaries. This ensures that the fusion result maintains the integrity of the heat source information while also having clearer and more continuous edge representation, improving the image's readability and recognizability.

[0066] Finally, the multi-scale reconstruction module introduces a spatial pyramid structure (e.g., Atrous Spatial Pyramid Pooling (ASPP)) to capture image features at different receptive fields, enabling the coordinated modeling of global contours and local textures. This module extracts features from the fused image using multiple sampling scales (scale 1 (rate = 1): equivalent to a standard 3×3 convolution, focusing on fine-grained local texture features; scale 2 (rate = 6): with a wider receptive field, capturing mid-scale structures (such as car windows and human silhouettes); scale 3 (rate = 12): for extracting wide-area background structures (such as the car body and road); and scale 4 (rate = 18): for extracting global semantic information and low-frequency contours). The module then uses a cascaded fusion approach (joining the feature maps extracted at these different sampling scales in the channel dimension and performing subsequent 1×1 convolution for channel compression and information integration, resulting in a unified fused feature map) to reconstruct the feature maps at multiple scales. This results in an optimized image feature map with consistent scale and complete structure. This module can effectively preserve the semantic consistency of images at large scales and edge details at small scales, providing stable and resolution-balanced image input for subsequent high-precision tasks.

[0067] Combining the above three parts, the optimization unit can not only improve the clarity and contrast of the fused image, but also help enhance its structural consistency and semantic expression ability in multi-target scenarios, providing a higher-quality input foundation for subsequent target detection and video tracking tasks.

[0068] In addition, the video tracking module is designed to fuse the multimodal features of infrared images and visible light images to perform real-time recognition and continuous positioning of the dynamic behavior of the target to be tracked in the video. The video tracking module includes: a dynamic initialization unit, a lightweight temporal association unit and an occlusion perception update unit, wherein the dynamic initialization unit is used to construct a multimodal template to provide tracking start reference information to ensure that the system has a multimodal understanding capability of the target at the beginning of the video sequence. The dynamic initialization unit introduces a semantic response template pool (Template Pool), which extracts high semantic features of the target area in the first frame of the video as the main template , used as the overall matching benchmark; at the same time, build an infrared auxiliary template and visible light auxiliary template , among which, infrared auxiliary template Mainly focus on infrared thermal structure (anti-blocking) to improve robustness in blocked environments; visible light auxiliary template Focusing on visible light texture details (to prevent mismatching), strengthening texture and edge features, helps reduce mismatching caused by texture weakening. In subsequent frames, the three types of templates are modally weighted fused to obtain the fused template of the current frame. :

[0069]

[0070] in, Indicates the weighted coefficient of the main template in the current frame, reflecting the credibility of the main template at the current moment; Indicates the weighted coefficient of the infrared template, reflecting the contribution of the infrared feature to the current target; Represents the weighting coefficient of the visible light template, reflecting the contribution of the visible light feature to the current target.

[0071] The lightweight temporal association module aims to establish short-term dependencies between target states and identify motion trends between consecutive frames while maintaining computational efficiency, and predict the possible location of the target in the next frame. The lightweight temporal association module includes a residual update mechanism guided by the previous frame and a dual-channel saliency enhancement mechanism. Specifically, the residual update mechanism guided by the previous frame uses the temporal residual information between the feature map of the current frame and the feature map of the previous frame to guide the update of the current target position, so that the model can better capture motion trends and structural changes. Specifically, first, obtain the feature map of the current frame and the feature map of the previous frame , then, calculate the temporal residual information between the two frames To capture the target's movement direction and change range:

[0072] The dual-channel saliency enhancement mechanism is used to enhance the infrared channel Visible light channel Perform saliency weighting to extract infrared saliency maps and visible light saliency map :

[0073]

[0074] in, Represents the Sigmoid activation function.

[0075] Timing residual information and infrared saliency map and visible light saliency map After cascading, perform 1×1 convolution to obtain the target center displacement vector :

[0076] in, Represents the predicted target center displacement vector.

[0077] The final estimated target search area for the next frame is:

[0078] in, Indicates the estimated target area position of the next frame; Indicates the target area position of the current frame.

[0079] By estimating the target search area for the next frame, the target position can be predicted and updated. This module implements cross-frame dependency modeling through a minimalist structure, effectively improving tracking continuity and anti-interference capabilities, and avoiding the high computational burden of traditional complex time series models in embedded environments.

[0080] The occlusion perception update unit is used to detect the occlusion state based on the similarity heat map change and guide the template update strategy to maintain tracking stability, thereby aiming to improve the robustness and continuity of video target tracking in complex occlusion scenarios. Specifically, this module takes the current candidate region features as the Fusion template with the current frame Calculate chord similarity and generate the current frame similarity heat map :

[0081] in, Indicates the current Frame in candidate region The feature vector of the inner position (x,y); Represents the fusion template vector of the current frame; Represents the L2 norm of the vector, used for normalization operation.

[0082] Get the current frame similarity heat map Afterwards, it is compared and analyzed with the similarity heat map of the previous frame. If a local continuous low-response area (i.e., "low-response hole") is detected in the target area, and its distribution position is significantly offset from the salient area of ​​the previous frame, the system will judge that the target may have been partially occluded or deformed out of the template representation range.

[0083] To enhance the robustness of the tracking system to occlusion, this module introduces an occlusion confidence map As the position weight, it adjusts the participation of each region in the feature map of the current frame. Specifically, the original feature map and occlusion confidence map (The range is between [0,1], 1 means credible, 0 means occluded) and weighted point by point:

[0084] in, Indicates the valid feature value of the current frame at position (x, y).

[0085] By assigning position weights to the current feature map, only high-confidence areas can be retained as valid features to participate in tracking updates, while weakening the impact of occluded areas on position judgment.

[0086] Furthermore, if occlusion is detected, the system will call the infrared auxiliary template Redundant feature compensation is performed to extract effective information from unoccluded areas and guide the model to continuously lock on to the target.

[0087] In summary, the video tracking module, with semantic template initialization, temporal residual modeling, and occlusion-adaptive updating as its core mechanisms, builds a lightweight, efficient, and robust multimodal object tracking solution. Its architecture balances computational efficiency with tracking continuity, adapting to complex environments such as motion, occlusion, and illumination variations, significantly enhancing the system's practicality and stability in multi-target video analysis tasks.

[0088] In another exemplary embodiment, in step S300, the target tracking model to be tested is trained by the following steps: S301: Data preparation and preprocessing: Synchronously collect multiple infrared image and visible light image pairs, preprocess the two types of images separately, and annotate the actual position of the target in the preprocessed infrared image and visible light image to construct a training sample set; S302: Feature extraction and fusion module training: The preprocessed infrared image and visible light are input into the infrared and visible light feature extraction networks to extract thermal and texture structural features respectively; the fusion module introduces channel attention, spatial gating mechanism and structural consistency guidance strategy to achieve adaptive fusion of multimodal features; supervised training is performed through structural reconstruction error with the original image to optimize the fusion effect.

[0089] S303: Template matching and position regression training: Extract candidate region features in each frame and calculate similarity heat maps with the initialized main template, infrared template, and visible light template; construct a supervision heat map based on the annotated positions to optimize matching accuracy, and simultaneously train the position regression branch to predict the target displacement vector and achieve continuous tracking of the target trajectory.

[0090] S304: Occlusion perception module training: Compare the changes in the similarity heat map between the current frame and the previous frame, and construct an occlusion confidence map to suppress low-confidence areas in the feature map; train the model to maintain tracking stability under occlusion conditions.

[0091] S305: End-to-end joint optimization: The fusion module, tracking module, and occlusion perception module are integrated into a unified model, and jointly trained in an end-to-end manner. The model performance is jointly optimized through multiple loss functions (including reconstruction loss, similarity loss, displacement prediction loss, and occlusion suppression loss). When each loss function tends to be stable on the validation set and meets the preset convergence conditions, the training process is considered complete, and finally the optimized model parameters that can be used for multimodal image fusion and robust tracking are obtained.

[0092] In this embodiment, the reconstruction loss function It is expressed as follows:

[0093] in, Represents the weight of the L1 loss term; represents the fused image output by the model; represents the reference image; Represents the weight of the SSIM loss term; Represents the L1 norm, which is used to measure the absolute difference between two image pixels; Represents structural similarity index measure, which is used to measure the similarity of images in structure, brightness, and contrast.

[0094] Similarity loss function It is expressed as follows:

[0095] in, Indicates that the current frame is at position Similarity heat map of ; Represents a supervised heatmap built based on GT (Ground truth) location information; Represents the square of the L2 norm, which is used to measure the difference in heatmaps.

[0096] Displacement prediction loss function It is expressed as follows:

[0097] in, represents the target displacement vector predicted by the model (the change in center position from the current frame to the next frame), represents the actual target displacement vector; represents the smoothed L1 loss.

[0098] Occlusion Suppression Loss Function It is expressed as follows:

[0099] in, Indicates the occlusion confidence value predicted at position (x, y) of the current frame, with a value range of [0, 1]; Indicates the supervision mask of the occluded area, the value range is [0,1], 1 indicates the visible area, and 0 indicates the occluded area; Represents the binary cross entropy loss function, which is used for binary classification probability evaluation.

[0100] Total loss function It is expressed as follows:

[0101] in, 、 、 and They represent the weighted coefficients of reconstruction loss, similarity loss, displacement prediction loss, and occlusion suppression loss respectively.

[0102] Below, this application Figure 2 The original infrared image and Figure 3 Taking the original visible light image shown as an example, the method described in this application is compared with the traditional method.

[0103] Figure 4The images generated by traditional image fusion methods mainly use image-level weighting or feature splicing strategies. Although the effect of modal fusion can be reflected in some areas, the overall image still has obvious structural fuzziness, unclear edges, insufficient thermal expression and other problems. The boundary between the target and the background lacks clear segmentation. Especially in scenes with complex backgrounds or weak target heat sources, it is difficult to form a significant fusion representation. Figure 5 The image presented here is the result of fusion based on the method described in this application. By leveraging multimodal feature alignment, adaptive fusion, and optimized reconstruction mechanisms, this image preserves the thermal response characteristics of the infrared image while also accurately retaining the structural details and texture information in the visible light image. The fusion result exhibits clear boundaries and strong contrast, making the target area more prominent and providing greater perceptual consistency and discriminability, providing a more stable feature foundation for subsequent target detection and tracking.

[0104] Figure 6 The effect of target tracking under the traditional method is demonstrated. It can be seen that although the tracking box (white rectangle) roughly covers the vehicle, there is an obvious offset, slightly biased to one side of the vehicle, indicating that the tracker is prone to error positioning under conditions of background interference or unclear texture, and has poor stability. In addition, the overall grayscale of the image is low, the grayscale values ​​of the trees in the background are close to those of the vehicle, the thermal-texture contrast is not obvious, and the target and background are difficult to distinguish, which interferes with the judgment accuracy of the algorithm; the details of the car body, such as the brand logo and the window outline, are blurred, which can easily lead to the gradual degradation of target features in long-term tracking, and ultimately cause tracking failure. In contrast, Figure 7 The tracking performance of the method of this application is demonstrated. The fused image features are clear, the tracking frame accurately covers the vehicle body, has good symmetry, and has a high degree of edge fit. This fully demonstrates that this method enhances the target saliency and boundary clarity through multimodal fusion, making it easier for the model to extract a stable ROI (Region of Interest). The fusion image combines the thermal response of the infrared image with the detailed texture of the visible light image, making the edges of the vehicle body clear and the lighting soft, forming a good segmentation between the background and the target, and effectively suppressing the interference of irrelevant background information. In addition, this method can also retain high-frequency texture features such as vehicle body reflections and corner contours, providing continuous and identifiable structural information for multi-frame target recognition and relocation, significantly improving the robustness and accuracy of video target tracking.

[0105] Figure 8It demonstrates the tracking effect of traditional methods in scenarios where the target is moving at high speed. Although the tracking frame still covers the main body of the vehicle, there is an obvious delay and position drift. Especially when the vehicle speed is fast or the target turns quickly, the position of the frame cannot be updated in real time, showing a "trailing" effect, and it is difficult to fit the contours of the vehicle body closely. The fundamental reason for this phenomenon is that traditional methods are difficult to resist the image blur caused by high-speed motion, and the edges of the target become blurred, resulting in the model being unable to stably extract discriminant features when the appearance changes rapidly. It is very easy to misjudge the background as the target, showing insufficient fault tolerance. In addition, the image generated by the traditional method is blurred as a whole, the contours of the front and rear of the vehicle are not clear, and the frequency components of the background and the target are highly overlapped, which further aggravates the interference in the tracking process and reduces the positioning accuracy. In contrast, Figure 9 The figure shows the performance of the patented method under the same high-speed motion conditions. The tracking frame always sticks to the edge of the vehicle and remains stably in the center of the motion path without obvious offset or drift, showing good spatiotemporal coherence and dynamic adaptability. This advantage is due to the fact that the method uses infrared images to enhance the boundaries of heat sources and visible light images to enhance high-frequency texture details during the image fusion stage, thereby forming clear and highly recognizable structural features, which improves the model's robustness to motion-blurred scenes. At the same time, the method introduces reinforcement learning strategies and lightweight time series prediction mechanisms, which enable the model to have a certain ability to perceive motion trends and to adaptively adjust tracking strategies. Even if the target undergoes complex changes such as acceleration and deflection, it can continue to stably lock the target position with almost no tracking failure.

[0106] Figure 10 and Figure 11 Both of them show a typical occlusion scenario: a vehicle is partially blocked by a large tree in front of it while driving, and only part of the target is visible. This is a severe test of the tracking system's target integrity judgment and trajectory consistency prediction capabilities. Figure 10 In the traditional tracking method, although the tracking frame roughly covers the main body of the vehicle, after occlusion occurs, especially in the front area, the frame will obviously offset and drift. The target positioning at the occlusion boundary is inaccurate, and even the frame will jump or unstable fluctuation. This problem mainly stems from the fact that traditional methods are highly dependent on the pixel information of the visible area in the current frame, lack the ability to perceive historical trajectories or occlusion completion, and it is difficult to infer from the context whether the occluded area belongs to the entire target. In addition, the difference between the occluded part and the background area at the image grayscale level is small, and the boundary is blurred. The algorithm can easily mistakenly identify it as the background area, resulting in tracking failure or loss of the target.

[0107] In contrast, Figure 11The tracking performance of the patented method under the same occlusion scenario is demonstrated. The tracking frame still stably covers the entire vehicle body. Even if the front of the vehicle is blocked by trees, the frame does not shrink or shift significantly, demonstrating good occlusion tolerance and target consistency judgment capabilities. This robustness is due to the fact that this method integrates the dual-modal information of infrared thermal images and visible light images to construct a fused image with redundant perception capabilities. Combined with the introduced occlusion confidence map and contextual temporal modeling mechanism, the system can still "complete" the spatial position prediction of the occluded area through the semantic clues of historical frames and surrounding frames even when some information is lost. In addition, the occluded area still retains recognizable features such as the heat source of the front of the vehicle and the outline of the tire in the fused image. Even if it is partially invisible visually, the model can still maintain the overall tracking continuity of the target through the temporal features of the Transformer structure modeling, thereby significantly improving the tracking stability and recovery capabilities in occluded environments. In summary, through comparison Figures 4 to 11 It can be clearly seen that the video target tracking method proposed in this application is superior to traditional methods in multiple key performance dimensions. In terms of image fusion quality, this method effectively combines infrared thermal sensing and visible light structural information, and the generated fusion image has clearer boundaries and more prominent target areas; in terms of target tracking performance, the fusion features enhance the target discrimination ability, making the tracking frame more stable and more fitting; under high-speed motion conditions, this method successfully achieves the precise connection of target positions between consecutive frames by relying on temporal residual modeling and motion prediction mechanisms; and in occlusion scenarios, the introduced occlusion perception and confidence weighting strategies significantly improve the adaptability to occluded areas and tracking continuity. Overall, this method shows significant advantages in robustness, accuracy and the ability to adapt to complex environments, and has stronger practical value and promotion prospects.

[0108] In another exemplary embodiment, Figure 12 As shown, the present application also provides a video target tracking device, which includes: an acquisition module 100, used to synchronously acquire an infrared image and a visible light image containing a target to be measured; a preprocessing module 200, used to preprocess the infrared image and the visible light image; a model construction and training module 300, used to construct a target tracking model to be measured and train the model; an identification module 400, used to input the preprocessed infrared image and the visible light image into the trained target tracking model to be measured, so as to identify and track the position of the target to be measured.

[0109] In another exemplary embodiment, the present application further provides a storage medium comprising instructions, which, when executed on a computer, enables the computer to execute any of the methods described above.

[0110] In another exemplary embodiment, the present application also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the methods described above when executing the program.

[0111] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A video target tracking method, characterized in that: The method comprises: Synchronously acquire infrared images and visible light images containing the target to be measured; Preprocessing the infrared image and the visible light image; Build a target tracking model and train the model; The pre-processed infrared image and the visible light image are input into a trained target tracking model to identify and track the target.

2. The video target tracking method according to claim 1, characterized in that: The preprocessing of the infrared image includes: Perform size unification and thermal value normalization on infrared images; Perform local contrast enhancement and nonlinear thermal value mapping on the normalized infrared image; A spatial Gaussian attention map is constructed based on the heat intensity and guided weighted smoothing.

3. The video target tracking method according to claim 1, characterized in that: Preprocessing the visible light image includes: Normalize the size and brightness of visible light images; Perform illumination correction and local contrast enhancement on the standardized visible light image; Perform edge extraction and detail enhancement on visible light images after illumination correction; Convert the detail-enhanced visible light image to Lab space and enhance the a / b channel color contrast.

4. The video target tracking method according to claim 1, characterized in that: The target tracking model to be measured includes: The multimodal fusion module is used to fuse the preprocessed infrared image and the visible light image to obtain a fused image; The video tracking module is used to identify and track the position of the target based on the fused image.

5. The video target tracking method according to claim 4, characterized in that: The multimodal fusion module includes: Feature extraction and alignment unit, used to extract multi-scale features of pre-processed infrared images and visible light images and perform spatial alignment; An adaptive feature fusion unit is used to adaptively fuse the aligned features to obtain a fused image; The optimization unit is used to perform structural optimization and reconstruction on the fused image.

6. The video target tracking method according to claim 4, characterized in that: The video tracking module includes: Initialize the tracking unit to build a multimodal template to provide tracking start reference information; A lightweight temporal association unit that combines the residual information of the previous frame with the features of the current frame to predict the target displacement; Occlusion-aware update unit, which detects occlusion status based on similarity heatmap changes and guides the template update strategy to maintain tracking stability.

7. A video target tracking device, characterized in that: The device comprises: An acquisition module, used for synchronously acquiring an infrared image and a visible light image containing the target to be measured; A preprocessing module, configured to preprocess the infrared image and the visible light image; Model building and training module, used to build the target tracking model to be tested and train the model; The recognition module is used to input the pre-processed infrared image and the visible light image into a trained target tracking model to identify and track the target.

8. A storage medium, characterized in that: The method comprises instructions which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: The electronic device comprises: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

Images