Infrared target tracking method based on multi-task learning
Optimizing the infrared target tracking network through multi-task learning and spatial grouping attention mechanisms has solved the problems of low infrared image resolution, lack of color and low signal-to-noise ratio, and improved the accuracy and positioning ability of infrared target tracking.
Patent Information
- Application Number
- CN202510539338.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-27
- Publication Date
- 2025-08-19
AI Technical Summary
The existing infrared target tracking methods have problems such as low thermal infrared image resolution, lack of color and details, and low signal-to-noise ratio, which leads to difficulty in tracking targets.
The infrared target tracking method based on multi-task learning is adopted to construct feature extraction, feature fusion, segmentation branching and bounding box prediction subnets, combined with two-stage training method, the infrared target tracking network is optimized, and the spatial grouping attention mechanism and Gaussian mask label are introduced to improve the semantic expression and positioning ability of target features.
The accuracy and positioning ability of infrared target tracking are improved, the learning ability of target boundaries and detailed characteristics is enhanced, and the spatial resolution and signal-to-noise ratio of infrared images are improved.
Smart Images
Figure CN120510184A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an infrared target tracking method, in particular to an infrared target tracking method based on multi-task learning. Background Art
[0002] This section merely provides background information related to the present disclosure and is not necessarily prior art.
[0003] In the current field of early warning detection, the sensors of front-end photoelectric detection equipment are mainly visible light sensors and infrared sensors. Compared with visible light sensors, infrared thermal imaging technology based on infrared sensors is not restricted by weather and lighting conditions, has stronger environmental adaptability, and with its unique heat detection capabilities, it has shown significant advantages in distinguishing pedestrians from other inanimate obstacles. By capturing the thermal radiation information emitted by objects, infrared thermal imaging technology can obtain the heat distribution information of the target person in real time, and accurately identify the location of the target person through algorithm processing and analysis. At present, some high-performance infrared target tracking methods have emerged in the field of infrared sensor algorithms, but due to the problem of infrared image imaging quality, these methods still face some problems:
[0004] (1) The resolution of thermal infrared images is low. This is because infrared detector technology and cost factors restrict the degree of pixel miniaturization. Large pixel size results in lower pixel density within a fixed detector area, which in turn affects the spatial resolution of the image. This resolution limitation weakens the ability of thermal infrared images to depict complex details and identify small targets.
[0005] (2) Thermal infrared images are single-channel grayscale images that lack the rich colors and detail levels of RGB images, thus limiting their richness in visual representation and data interpretation. For example, in color images, targets can be tracked based on their color, shape, and texture. However, in thermal infrared images, all targets may appear as similar grayscale outlines, making tracking difficult.
[0006] (3) The signal-to-noise ratio (SNR) of thermal infrared images is generally lower than that of RGB images. Since infrared imaging typically operates under low light conditions, the noise in the image is more significant than in visible light images, especially in the long-wave infrared region. For example, in nighttime urban surveillance, the images captured by infrared cameras may be full of noise, blurring the outlines of pedestrians or vehicles.
[0007] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute prior art known to ordinary technicians in the field. Summary of the Invention
[0008] Purpose of the invention: The technical problem to be solved by the present invention is to provide an infrared target tracking method based on multi-task learning in response to the shortcomings of the existing technology.
[0009] In order to solve the above technical problems, the present invention discloses an infrared target tracking method based on multi-task learning, comprising the following steps:
[0010] Step 1: preprocess the infrared target tracking video sequence;
[0011] Step 2: Build an infrared target tracking network based on multi-task learning;
[0012] Step 3: Generate the Gaussian mask label of the segmentation branch based on the infrared target bounding box obtained in the infrared target tracking network;
[0013] Step 4: Use a two-stage training method to optimize the infrared target tracking network;
[0014] Step 5: Use the trained infrared target tracking model to implement infrared target tracking based on multi-task learning.
[0015] Furthermore, the infrared target tracking dataset is preprocessed as described in step 1, including:
[0016] Randomly select the infrared images of frame i, frame j and frame k from the infrared target tracking video sequence V, and perform cropping and scaling, which are recorded as static template images respectively. Dynamic template images and search images Among them, [H T ,W T ]、[H D ,W D ] and [H S ,W S ] are static template images V i , dynamic template image V j and search image V k The height and width of , C is the number of channels.
[0017] Furthermore, the infrared target tracking network based on multi-task learning described in step 2 includes:
[0018] Feature extraction subnetwork, feature fusion subnetwork, segmentation branch subnetwork, bounding box prediction subnetwork and target state evaluation subnetwork; among them,
[0019] The feature extraction sub-network is based on the input static template image V i , dynamic template image V j and search image V k, extract features and output them to the feature fusion sub-network for feature fusion;
[0020] The feature fusion sub-network outputs the fused features to the segmentation branch sub-network and the bounding box prediction sub-network respectively;
[0021] The segmentation branch subnetwork calculates the loss through the loss function and optimizes the feature extraction subnetwork, the feature fusion subnetwork, the segmentation branch subnetwork and the bounding box prediction subnetwork;
[0022] The bounding box prediction subnetwork performs bounding box prediction on the target and outputs the result to the target state evaluation subnetwork for evaluation.
[0023] Furthermore, the feature extraction sub-network includes:
[0024] Taking the ResNet-50 network as the main body, a spatial grouping enhanced attention module (SGE) is added after the second 1×1 convolutional layer in each bottleneck layer of the ResNet-50 network.
[0025] The feature extraction sub-network respectively processes the input static template image V i , dynamic template image V j and search image V k Perform feature extraction and record the first static template feature map, dynamic template feature map and search feature map output by the third level stage3 in the ResNet-50 network as The second static template feature map output by the second level stage2 is recorded as The third static template feature map output by the first level stage1 is recorded as The fourth static template feature map output by the first convolutional layer conv1 is recorded as
[0026] For the first static template feature map Dynamic template feature map and search feature graph Flatten and concatenate in the spatial dimension, add sinusoidal position encoding, and obtain a mixed feature sequence:
[0027]
[0028] Among them, C M is the number of channels after feature extraction and downsampling, X t 、X D and X S are the sequence lengths of the first static template feature map, dynamic template feature map, and search feature map after expansion in the spatial dimension, respectively, and are expressed as follows:
[0029]
[0030] Where d is the downsampling multiple after feature extraction;
[0031] The spatial grouping enhanced attention module SGE specifically includes:
[0032] Assume that any input feature tensor of the spatial grouping enhanced attention module SGE is:
[0033] The input feature tensor f is grouped along the channel dimension, and the feature tensor of the g-th group is recorded as:
[0034]
[0035] Among them, H f and W f is the height and width of the input feature tensor f, C f is the number of channels of the input feature tensor f, g∈{1,2,…,G}, G is the number of groups;
[0036] For the feature tensor f g Perform global pooling operations along the spatial dimension to compress the global spatial information into weight factors Then use the weight factor α to adjust the feature tensor f g Perform weighted operation to obtain weighted feature subgraph:
[0037]
[0038] For the characteristic subgraph f′ g Sum along the channel dimension to get the feature map of a single channel:
[0039]
[0040] Calculate the mean and standard deviation of the feature map t in the spatial dimension, and then normalize t to obtain the normalized feature map:
[0041]
[0042] Perform a linear mapping operation on the feature map T to obtain the mapped feature map:
[0043] T′=w·T+b
[0044] Among them, the feature map w and b are scaling offset parameters;
[0045] Perform Sigmoid operation on the mapped feature map T′, and convert the obtained spatial weight feature map into the feature tensor f gPerform weighted operations and finally obtain the output of the spatial grouping enhanced attention module SGE.
[0046] Furthermore, the feature fusion sub-network processes the mixed feature sequence f in the feature extraction sub-network. M Processing is performed to obtain the fusion feature map F f ;
[0047] The feature fusion sub-network adopts the codec network of the Detection Transformer architecture.
[0048] Furthermore, the splitting branch sub-network includes:
[0049] The search feature graph in the feature extraction subnetwork Input into the segmentation branch sub-network, first pass through the deconvolution layer Deconv to obtain the first-level mask M1, the first-level mask M1 and the second static template feature map in the feature extraction sub-network The second-level mask M2 is obtained by fusion through the first refinement module R1;
[0050] The second-level mask M2 and the third static template feature map in the feature extraction subnetwork The third-level mask M3 is obtained by fusion through the second refinement module R2;
[0051] The third level mask M3 and the fourth static template feature map in the feature extraction sub-network The final prediction mask is obtained by fusion through the third refinement module R3
[0052] The first refinement module R1, the second refinement module R2 and the third refinement module R3 have the same structure, specifically comprising:
[0053] After the input feature map passes through two 3×3 convolutional layers, it is superimposed with the current level mask, and then passes through a single 3×3 convolutional layer and an upsampling layer in sequence to obtain the next level mask.
[0054] Furthermore, the bounding box prediction subnetwork includes:
[0055] 2 fully convolutional modules, each fully convolutional module includes 4 Conv3×3-BN-ReLU layers and 1 Conv1×1 layer.
[0056] Furthermore, the target state evaluation subnetwork adopts the network structure based on dynamic template updating in the STARK target tracking network.
[0057] Furthermore, the Gaussian mask label of the segmentation branch generated in step 3 includes:
[0058] Take the center point coordinates of the infrared target bounding box to be predicted (C x ,C y ) as the center point coordinates of the two-dimensional Gaussian function, and the width and height of the bounding box are used as the covariance in the horizontal and vertical directions respectively. The confidence of each pixel in the infrared target frame is calculated to obtain the Gaussian mask label, which is as follows:
[0059]
[0060] Among them, Gu(x,y) represents the confidence of the pixel at coordinate (x,y) on the Gaussian mask label, C x and C y represents the coordinates of the center point of the target bounding box, σ h and σ w Represent the height and width of the bounding box respectively, and Exp is the exponential function.
[0061] Furthermore, the two-stage training method described in step 4 is used to optimize the infrared target tracking network, including:
[0062] Step 4-1, in the first stage, the segmentation branch sub-network and the bounding box prediction sub-network are trained using positioning loss and segmentation loss;
[0063] Among them, the positioning loss L loc A weighted combination of L1 loss and GIoU loss is used, as follows:
[0064]
[0065] Where B is the true bounding box coordinate, are the predicted bounding box coordinates, λ l ,λ GIoU are all hyperparameters in the real number category;
[0066] The segmentation loss L m The Dice loss in the target segmentation task is adopted, as follows:
[0067]
[0068] in, Represents the predicted mask label, and M represents the true label;
[0069] The joint loss function L used in the first stage of training M , as follows:
[0070] L M =μL loc +βL m
[0071] Among them, μ and β are hyperparameters in the real number category;
[0072] Step 4-2, the second stage, only the target state evaluation sub-network is trained, and the loss function L IATSE as follows:
[0073]
[0074] Among them, I score Represents the true bounding box B and the predicted bounding box The IoU score, is the IoU score of the target state evaluation subnetwork prediction output, and γ is a hyperparameter in the real number category;
[0075] In step 4-3, based on the loss function of the corresponding training stage, the network parameters of the training target in each stage are updated by the gradient descent method. When the number of training iterations reaches the set number, the training is stopped. After the training of both stages is completed, the trained infrared target tracking model is obtained.
[0076] Beneficial effects:
[0077] 1. This paper designs an infrared target tracking network based on multi-task learning. It introduces an additional segmentation branch on the basis of the existing target tracking model to realize the joint learning of segmentation and tracking positioning networks. The purpose is to enable the segmentation branch to guide the training of the positioning network, thereby further improving the positioning network's ability to learn target boundaries and detailed features.
[0078] 2. The present invention introduces the spatial grouping attention mechanism into the feature extraction network, and uses the attention mechanism to enhance the semantic expression of target features, so that the network can learn target features with strong discriminability that can be used for subsequent matching and positioning operations.
[0079] 3. The present invention combines the target positioning loss with the segmentation loss to guide the training of the segmentation branch and the positioning-related network, aiming to jointly and synchronously optimize the positioning-related networks of the segmentation branch and the tracking branch to improve the feature learning ability of the positioning network. BRIEF DESCRIPTION OF THE DRAWINGS
[0080] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more apparent.
[0081] Figure 1 This is a flow chart of the infrared target tracking method of the present invention.
[0082] Figure 2 This is the overall structure diagram of the infrared target tracking network in the present invention.
[0083] Figure 3This is a structural diagram of the first refinement module in the present invention.
[0084] Figure 4 Schematic diagram of the Gaussian mask label of the segmentation branch in the present invention. DETAILED DESCRIPTION
[0085] This paper proposes an infrared target tracking method based on multi-task learning. Through the joint learning of the segmentation branch and the tracking and positioning network, the segmentation branch aims to guide the training of the positioning network, and uses the attention mechanism to enhance the semantic expression of target features, thereby further improving the positioning network's ability to learn target boundaries and detailed features.
[0086] The specific technical solution is as follows: an infrared target tracking method based on multi-task learning, such as Figure 1 As shown, the steps are as follows:
[0087] Step 1: Preprocess the infrared target tracking dataset;
[0088] Step 1.1: For a given infrared target tracking video sequence V, randomly select the i-th, j-th, and k-th infrared images for cropping and scaling, and record them as static template images respectively. Dynamic template images Search images And V i , V j , V k As the input of infrared target tracking network, [H T ,W T ]、[H D ,W D ] and [H S ,W S ] are static template images V i , dynamic template image V j and search image V k The height and width of , C is the number of channels;
[0089] Step 2: Build an infrared target tracking network based on multi-task learning, such as Figure 2 As shown, it includes: feature extraction subnetwork, feature fusion subnetwork, segmentation branch subnetwork, bounding box prediction subnetwork and target state evaluation subnetwork;
[0090] Step 2.1: The main body of the feature extraction sub-network is ResNet-50, and the spatial grouping enhanced attention module SGE is added after the bottleneck layer of ResNet-50. The model parameters of the feature extraction sub-network are shown in Table 1:
[0091] Table 1 Model parameters of feature extraction sub-network
[0092]
[0093] The feature extraction sub-network performs i , V j , V k Perform feature extraction and record the static template feature map, dynamic template feature map and search feature map output by stage 3 of ResNet-50 as The static template feature map output by stage2 is recorded as The static template feature map output by stage1 is recorded as The static template feature map output by conv1 is recorded as Then the feature map and Flatten and concatenate in the spatial dimension and add sinusoidal position encoding to obtain a mixed feature sequence Among them C M is the number of channels after feature extraction downsampling, d is the downsampling multiple after feature extraction;
[0094] Any input feature tensor of the spatial grouping enhanced attention module SGE is denoted as First, the input is grouped along the channel dimension, and the feature tensor of the g-th group is recorded as Among them, H f With W f is the height and width of f, C f is the number of channels of f, g∈{1,2,…,G}, G is the number of groups;
[0095] Next, the feature tensor f g Perform global pooling operations along the spatial dimension to compress the global spatial information into weight factors Then use this weight factor to adjust the feature tensor f g Perform weighted operation to obtain weighted feature subgraph Afterwards, the feature subgraph Sum along the channel dimension to get the feature map of a single channel And calculate the mean and standard deviation of the feature map t in the spatial dimension, so as to normalize t and obtain the normalized feature map And perform a linear mapping operation on T to obtain the mapped feature map T′=w·T+b, where w and b are scaling offset parameters;
[0096] Finally, the mapped feature map T′ is subjected to Sigmoid operation, and the obtained spatial weight feature map is converted to the feature tensor f gPerform weighted operations to obtain the output of the spatial grouping enhanced attention module SGE;
[0097] Step 2.2: Construct the feature fusion sub-network to process the mixed feature sequence f M Processing is performed to obtain the fusion feature map F f ; The feature fusion sub-network uses the Detection Transformer encoder-decoder network;
[0098] Step 2.3, construct the segmentation branch sub-network; input the search feature map into the segmentation branch sub-network, first pass the deconvolution layer Deconv to obtain the first-level mask M1, M1 and The second-level mask M2 is obtained by fusion through the first refinement module R1; M2 and The third-level mask M3 is obtained by fusion through the second refinement module R2; M3 and The final prediction mask is obtained by fusion through the third refinement module R3 The segmentation branch can pass more target-related spatial information to the positioning branch, thereby enhancing the feature learning ability of the tracking network. By considering the combination of multi-scale features to generate a mask map and fusing shallow spatial information and deep semantic information, the adverse effects of information loss during upsampling can be minimized, and the segmentation branch's ability to perceive details and global contextual information can be improved.
[0099] The first, second and third refinement modules have the same structure. Taking the first refinement module R1 as an example, Figure 3 As shown, the feature map The input to the first refinement module R1 passes through two 3×3 convolutional layers and is then superimposed with the first-level mask M1. It then passes through a single 3×3 convolutional layer and an upsampling layer to produce the second-level mask M2. This series of mask refinement operations aims to use the feature information of the static template to compensate for the information lost in the mask image during upsampling. This process is iterated multiple times until the feature map size matches the template image. Each refinement module has independent parameters, allowing the network to perform specific refinement operations at different learning stages.
[0100] Step 2.4. Construct the bounding box prediction subnetwork. The bounding box prediction subnetwork consists of two fully convolutional modules, each of which includes four Conv3×3-BN-ReLU layers and one Conv1×1 layer.
[0101] Step 2.5: Construct the target state assessment subnetwork. This subnetwork uses the dynamic template-based architecture of the STARK target tracking network. When the first frame of a video sequence is used as a static template, if the target deforms during tracking, the inherent target features in the static template are no longer sufficient to represent the target's changed state in the search image. Therefore, STARK introduces a dynamic template based on the existing model. This template works in conjunction with the static template, enabling the model to capture the target's inherent spatial features while also perceiving changes in the target over time.
[0102] Step 3: Generate Gaussian mask labels for the segmentation branches, such as Figure 4 As shown; take the center point coordinates of the infrared target bounding box (C x ,C y ) as the center point coordinates of the two-dimensional Gaussian function, and the width and height of the bounding box are used as the covariance in the horizontal and vertical directions respectively. The confidence of each pixel in the Gaussian mask label is generated as follows:
[0103]
[0104] Among them, Gu(x,y) represents the pixel value at the coordinate (x,y) on the Gaussian mask label, C x and C y represents the coordinates of the center point of the target bounding box, σ h and σ w Represent the height and width of the bounding box respectively; Figure 4 The segmentation labels generated using formula (1) are shown in Figure 1, where the left column is the real image and the right column is the mask label fused with the original image. It can be seen that the training labels generated based on the Gaussian mask accurately restore the location information of the target on the image, which allows the training of the segmentation branch to proceed smoothly on the mask label.
[0105] Step 4: Optimize the infrared target tracking network using a two-stage training method;
[0106] Step 4.1: The training of the infrared target tracking network based on multi-task learning consists of two stages. In the first stage, the segmentation branch and the bounding box prediction related network are trained using the localization loss and the segmentation loss. The purpose is to improve the localization network's ability to perceive target details and boundary features in the process of refining the segmentation results. The localization loss adopts the weighted combination of L1 loss and GIoU loss, and the formula is as follows:
[0107]
[0108] Where B is the true bounding box coordinate, are the predicted bounding box coordinates, λ l ,λ GIoUAll are real-number hyperparameters. The segmentation loss uses the Dice loss commonly used in target segmentation tasks, and the formula is as follows:
[0109]
[0110] in, Represents the predicted mask label, and M represents the true label. Therefore, the joint loss function used in the first stage of training is defined as follows:
[0111] L M =μL loc +βL m (4)
[0112] μ and β are both hyperparameters in the real number range;
[0113] Step 4.2: In the second stage, only the target state evaluation subnetwork is trained, and the loss function is as follows:
[0114]
[0115] Among them, I score Represents the true bounding box B and the predicted bounding box The IoU score, is the IoU score of the target state evaluation subnetwork prediction output, and γ is a hyperparameter in the real number category;
[0116] Based on the above loss function, the network parameters are updated using the gradient descent method. When the number of training iterations reaches a set number, the training is stopped to obtain a trained infrared target tracking model, which is used to achieve continuous and accurate positioning of infrared targets. The segmentation branch of the present invention only plays a guiding role in the process of joint training with the tracking and positioning network and does not participate in the reasoning process. The segmentation network is a separate branch and needs to be combined with the feature extraction module of the positioning-related network to generate the segmentation results.
[0117] In a specific implementation, the present application provides a computer storage medium and a corresponding data processing unit, wherein the computer storage medium is capable of storing a computer program that, when executed by the data processing unit, can execute the invention content of the infrared target tracking method based on multi-task learning provided by the present invention and some or all of the steps in each embodiment. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).
[0118] Those skilled in the art will clearly understand that the technical solutions in the embodiments of the present invention can be implemented by means of computer programs and their corresponding general hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, in essence or in other words, the part that contributes to the prior art, can be embodied in the form of a computer program, i.e., a software product. The computer program software product can be stored in a storage medium and includes a number of instructions for enabling a device including a data processing unit (which can be a personal computer, server, single-chip microcomputer, MCU, or network device, etc.) to execute the methods described in various embodiments of the present invention or certain parts of the embodiments.
[0119] The present invention provides a method and approach for infrared target tracking based on multi-task learning. There are many methods and approaches for implementing this technical solution. The above is only a preferred embodiment of the present invention. It should be noted that those skilled in the art may make various improvements and modifications without departing from the principles of the present invention, and such improvements and modifications should also be considered within the scope of protection of the present invention. Any components not specified in this embodiment may be implemented using existing technologies.
Claims
1. An infrared target tracking method based on multi-task learning, characterized in that: The steps include: Step 1: preprocess the infrared target tracking video sequence; Step 2: Build an infrared target tracking network based on multi-task learning; Step 3: Generate the Gaussian mask label of the segmentation branch based on the infrared target bounding box obtained in the infrared target tracking network; Step 4: Use a two-stage training method to optimize the infrared target tracking network; Step 5: Use the trained infrared target tracking model to implement infrared target tracking based on multi-task learning.
2. The infrared target tracking method based on multi-task learning according to claim 1, characterized in that: The infrared target tracking dataset is preprocessed as described in step 1, including: Randomly select the infrared images of frame i, frame j and frame k from the infrared target tracking video sequence V, and perform cropping and scaling, which are recorded as static template images respectively. Dynamic template images and search images Among them, [H T ,W T ]、[H D ,W D ] and [H S ,W S ] are static template images V i , dynamic template image V j and search image V k The height and width of , C is the number of channels.
3. The infrared target tracking method based on multi-task learning according to claim 2, characterized in that: The infrared target tracking network based on multi-task learning described in step 2 includes: Feature extraction subnetwork, feature fusion subnetwork, segmentation branch subnetwork, bounding box prediction subnetwork and target state evaluation subnetwork; among them, The feature extraction sub-network is based on the input static template image V i , dynamic template image V j and search image V k , extract features and output them to the feature fusion sub-network for feature fusion; The feature fusion sub-network outputs the fused features to the segmentation branch sub-network and the bounding box prediction sub-network respectively; The segmentation branch subnetwork calculates the loss through the loss function and optimizes the feature extraction subnetwork, the feature fusion subnetwork, the segmentation branch subnetwork and the bounding box prediction subnetwork; The bounding box prediction subnetwork performs bounding box prediction on the target and outputs the result to the target state evaluation subnetwork for evaluation.
4. The infrared target tracking method based on multi-task learning according to claim 3, characterized in that: The feature extraction sub-network includes: Taking the ResNet-50 network as the main body, a spatial grouping enhanced attention module (SGE) is added after the second 1×1 convolutional layer in each bottleneck layer of the ResNet-50 network. The feature extraction sub-network respectively processes the input static template image V i , dynamic template image V j and search image V k Perform feature extraction and record the first static template feature map, dynamic template feature map and search feature map output by the third level stage3 in the ResNet-50 network as The second static template feature map output by the second level stage2 is recorded as The third static template feature map output by the first level stage1 is recorded as The fourth static template feature map output by the first convolutional layer conv1 is recorded as For the first static template feature map Dynamic template feature map and search feature graph Flatten and concatenate in the spatial dimension, add sinusoidal position encoding, and obtain a mixed feature sequence: Among them, C M is the number of channels after feature extraction and downsampling, X T 、X D and X S are the sequence lengths of the first static template feature map, dynamic template feature map, and search feature map after expansion in the spatial dimension, respectively, and are expressed as follows: Where d is the downsampling multiple after feature extraction; The spatial grouping enhanced attention module SGE specifically includes: Assume that any input feature tensor of the spatial grouping enhanced attention module SGE is: The input feature tensor f is grouped along the channel dimension, and the feature tensor of the g-th group is recorded as: Among them, H f and W f is the height and width of the input feature tensor f, C f is the number of channels of the input feature tensor f, g∈{1,2,…,G}, G is the number of groups; For the feature tensor f g Perform global pooling operations along the spatial dimension to compress the global spatial information into weight factors Then use the weight factor α to adjust the feature tensor f g Perform weighted operation to obtain weighted feature subgraph: For the feature subgraph f g ′ Sum along the channel dimension to get the feature map of a single channel: Calculate the mean and standard deviation of the feature map t in the spatial dimension, and then normalize t to obtain the normalized feature map: Perform a linear mapping operation on the feature map T to obtain the mapped feature map: T′=w·T+b Among them, the feature map w and b are scaling offset parameters; For the mapped feature map T ′ Perform Sigmoid operation and convert the obtained spatial weight feature map to the feature tensor f g Perform weighted operations and finally obtain the output of the spatial grouping enhanced attention module SGE.
5. The infrared target tracking method based on multi-task learning according to claim 4, characterized in that: The feature fusion sub-network processes the mixed feature sequence f in the feature extraction sub-network. M Processing is performed to obtain the fusion feature map F f ; The feature fusion sub-network adopts the codec network of the Detection Transformer architecture.
6. The infrared target tracking method based on multi-task learning according to claim 5, characterized in that: The split branch sub-network includes: The search feature graph in the feature extraction subnetwork Input into the segmentation branch sub-network, first pass through the deconvolution layer Deconv to obtain the first-level mask M1, the first-level mask M1 and the second static template feature map in the feature extraction sub-network The second-level mask M2 is obtained by fusion through the first refinement module R1; The second-level mask M2 and the third static template feature map in the feature extraction subnetwork The third-level mask M3 is obtained by fusion through the second refinement module R2; The third level mask M3 and the fourth static template feature map in the feature extraction sub-network The final prediction mask is obtained by fusion through the third refinement module R3 The first refinement module R1, the second refinement module R2 and the third refinement module R3 have the same structure, specifically comprising: After the input feature map passes through two 3×3 convolutional layers, it is superimposed with the current level mask, and then passes through a single 3×3 convolutional layer and an upsampling layer in sequence to obtain the next level mask.
7. The infrared target tracking method based on multi-task learning according to claim 6, characterized in that: The bounding box prediction subnetwork includes: 2 fully convolutional modules, each fully convolutional module includes 4 Conv3×3-BN-ReLU layers and 1 Conv1×1 layer.
8. The infrared target tracking method based on multi-task learning according to claim 7, characterized in that: The target state evaluation subnetwork adopts the network structure based on dynamic template updating in the STARK target tracking network.
9. The infrared target tracking method based on multi-task learning according to claim 8, characterized in that: The Gaussian mask labels for the segmentation branches generated in step 3 include: Take the center point coordinates of the infrared target bounding box to be predicted (C x ,C y ) as the center point coordinates of the two-dimensional Gaussian function, and the width and height of the bounding box are used as the covariance in the horizontal and vertical directions respectively. The confidence of each pixel in the infrared target frame is calculated to obtain the Gaussian mask label, which is as follows: Among them, Gu(x,y) represents the confidence of the pixel at coordinate (x,y) on the Gaussian mask label, C x and C y represents the coordinates of the center point of the target bounding box, σ h and σ w Represent the height and width of the bounding box respectively, and Exp is the exponential function.
10. The infrared target tracking method based on multi-task learning according to claim 9, characterized in that: The two-stage training method described in step 4 is used to optimize the infrared target tracking network, including: Step 4-1, in the first stage, the segmentation branch sub-network and the bounding box prediction sub-network are trained using positioning loss and segmentation loss; Among them, the positioning loss L loc A weighted combination of L1 loss and GIoU loss is used, as follows: Where B is the true bounding box coordinate, are the predicted bounding box coordinates, λ l ,λ GIoU are all hyperparameters in the real number category; The segmentation loss L m The Dice loss in the target segmentation task is adopted, as follows: in, Represents the predicted mask label, and M represents the true label; The joint loss function L used in the first stage of training M , as follows: L M =μL loc +βL m Among them, μ and β are hyperparameters in the real number category; Step 4-2, the second stage, only the target state evaluation sub-network is trained, and the loss function L IATSE as follows: Among them, I score Represents the true bounding box B and the predicted bounding box The IoU score, is the IoU score of the target state evaluation subnetwork prediction output, and γ is a hyperparameter in the real number category; In step 4-3, based on the loss function of the corresponding training stage, the network parameters of the training target in each stage are updated by the gradient descent method. When the number of training iterations reaches the set number, the training is stopped. After the training of both stages is completed, the trained infrared target tracking model is obtained.