Adaptive dark-light enhanced target tracking network structure and target tracking method

Abstract

The present invention relates to an adaptive dark-light enhanced target tracking network structure and a target tracking method. The network mainly includes two modules, namely, illumination enhancement and target tracking. The illumination compensation method is based on illumination estimation and color space changes, and also adaptively enhances the characteristics of dark scenes. In the target tracking module, a network based on the Transformer architecture is mainly adopted, and accurate tracking of the target can be achieved through the self-attention mechanism. Compared with the independent illumination enhancement module, the present invention enhances the image features for the tracking task. The network structure is simple, and the real-time requirements of the target tracking task are maintained while improving the performance. The tracker in this article not only achieves better results in low-light scenes, but also maintains the tracking performance of the tracker in normal light scenes. Compared with the OSTrack algorithm, the accuracy is improved by 8.9% on the UAVDark70 dataset.

Description

Technical Field

[0001] The invention belongs to the field of image processing and relates to an adaptive dark light enhanced target tracking network structure and a target tracking method. Background Art

[0002] During the object imaging process, many environmental factors will affect the scene appearance to varying degrees, including different light intensity, different light incident angles, shadows, and posture changes. These changes will eventually appear in the image. Different images will be collected for the same target object in different environmental conditions, which also brings certain difficulties to the subsequent image understanding tasks.

[0003] As a hot topic in the field of computer vision, target tracking has been successfully applied in many fields. Target tracking involves a wide range of disciplines, including pattern recognition, image processing, computer science, mathematical statistics, etc. Existing target tracking algorithms have excellent performance on classic data sets, but if they are applied to complex scenes, such as scenes with a large number of occlusions and scenes with changing lighting, their performance will decrease to a certain extent. Studying target tracking in occluded scenes can not only promote computer vision research on such problems, but also has greater practical significance for the successful application of target tracking technology in civilian and military scenarios.

[0004] Target tracking methods are mainly divided into the following two categories: the first is based on correlation filtering, and the second is based on deep learning.

[0005] Most algorithms based on correlation filtering use filter templates to record and update the location information of the tracked target, and then calculate the similarity between the template image and each position of each frame image, and use the similarity as the output response map. The position with the highest response value in the output response map is the most similar position and the location of the target. Methods based on deep learning use deep neural networks to extract image features. In real life, due to complex situations such as occlusion, illumination changes, and target deformation, image recognition tasks become more difficult. However, target tracking algorithms based on deep learning can effectively cope with these challenges due to their excellent feature extraction capabilities. Compared with algorithms based on correlation filtering, methods based on deep learning can better generalize these complex situations, so they have become the current mainstream research direction. Summary of the invention

[0006] Technical issues to be solved

[0007] In order to avoid the shortcomings of the prior art, the present invention proposes an adaptive dark-light enhanced target tracking network structure and a target tracking method. In view of the fact that the existing methods cannot extract effective features in scenes with changing lighting, an innovative Transformer layer guided by lighting information is proposed, which makes it possible to use areas with better lighting conditions to help improve the feature representation of areas with poor lighting conditions, thereby improving the final tracking effect.

[0008] Technical Solution

[0009] An adaptive dark light enhanced target tracking network structure, characterized by: comprising two adaptive dark light enhancement modules, a feature extraction and feature fusion module guided by illumination information, and a prediction head;

[0010] The inputs of the two adaptive dark light enhancement modules are the template image and the search image, respectively, and the two images themselves and the image features are brightened; the outputs of the two template image adaptive dark light enhancements are connected to the feature extraction and feature fusion module guided by the illumination information, and the output of the search image adaptive dark light enhancement is connected to the feature extraction and feature fusion module guided by the illumination information;

[0011] The feature extraction and feature fusion module guided by illumination information includes N feature extraction and feature fusion modules, each of which includes an early candidate elimination module and a multi-head attention mechanism guided by illumination information, the feature extraction and feature fusion module of each layer is connected to the feature extraction and feature fusion module of the next layer, and the feature extraction and feature fusion module of the last layer is connected to the prediction head;

[0012] The prediction head includes three different branches, and the three different branches are all composed of multi-layer perceptrons.

[0013] The adaptive dark light enhancement module includes a 1×1 convolution layer, a depth-separable convolution layer and a color preservation module; the module is used to generate brightening features and enhanced images.

[0014] The illumination information guided feature extraction and feature fusion module is based on the self-attention mechanism in Transformer, replaces the multi-head attention with the illumination information guided multi-head attention and adds an early candidate elimination module between the multi-head attention and layer normalization; this module is used to complete the feature extraction and feature fusion of the search image and the template image.

[0015] The prediction head includes three different branches, namely a classification branch for predicting classification scores, a regression branch for predicting normalized bounding box sizes, and an offset compensation branch for predicting offsets to compensate for downsampling quantization losses. The three different branches are all composed of a multi-layer perceptron and are used to generate target tracking results, i.e., bounding box parameters of the target.

[0016] The target tracking network structure uses the Adam optimization method to update the network parameters, and stops training when the loss function value is minimized to obtain an adaptive dark-light enhanced target tracking network.

[0017] The tracking loss function of the target tracking network structure is:

[0018] L track =L cls +L iou +L1

[0019] Among them, L cls represents the classification loss, L iou represents the iou loss between the predicted target bounding box and the true value bounding box, and L1 represents the distance loss between the predicted target bounding box parameters and the true value bounding box parameters.

[0020] In the Adam optimization method, the optimizer parameters betas are set to (0.5, 0.999), the learning rate is 0.0005, the weight_decay is 0.0001, and the batch size is 8.

[0021] A target tracking method is implemented by using the adaptive dark light enhanced target tracking network structure, characterized in that the target tracking steps are as follows:

[0022] Step 1: The template image and the search image are subjected to their respective adaptive dark light enhancement modules to extract the brightened images and brightened features of the template image and the search image, respectively. The formula is as follows:

[0023] (I lu , F lu )=ε(I,L p )

[0024] Among them, ε represents the adaptive dark light enhancement module, which takes an image and its illumination prior map as input; the illumination prior map is obtained by calculating the average value of each pixel of the input image along the channel dimension; the adaptive dark light enhancement network outputs the image I after illumination compensation lu , and the brightening feature F for the next stage lu ;

[0025] The respective adaptive dark light enhancement modules of the template image and the search image are adaptive dark light enhancement modules with weight sharing;

[0026] Step 2: The brightened template image and its brightened features as well as the search image and its brightened features enter the feature extraction and feature fusion module guided by illumination information. The two images are first cut into blocks of a certain size and then the image blocks of the two are spliced ​​and sent to the layer normalization, and then sent to the multi-head attention guided by illumination information. The formula is as follows:

[0027]

[0028] Among them, Y represents the brightened feature vector obtained by concatenating two brightened features, and the query matrix Q, key matrix K and value matrix V are generated by mapping the results of concatenating the search image blocks and the template image blocks; the brightening query matrix Q and the brightening key matrix K calculate the similarity, and put more attention on the darker area in the original image, so that the features of this part of the area get more attention;

[0029] After the multi-head attention guided by the illumination information, it is sent to the early candidate elimination module through the residual connection. The early candidate elimination module scores the search image blocks and discards some search image areas. This reduces the calculation amount of the self-attention mechanism while eliminating the interference of invalid background information on the tracking results, thereby speeding up the operation of the tracker. The output of the early candidate elimination module is sent to the feature extraction and feature fusion module of the next stage after layer normalization and multi-layer perceptron. If it is the last layer, it is output to the prediction head. The calculation formula of the feature extraction and feature fusion module of each layer is as follows:

[0030]

[0031] Among them, LGMSA stands for multi-head attention guided by illumination information, FFN stands for feedforward neural network, LN is the layer normalization operation, and ECE stands for early candidate elimination module;

[0032] Step 3: The result processed in step 2 is used to obtain the final tracking result through the prediction head. The prediction head uses three different branches to obtain the classification score, the normalized target bounding box parameters, and the predicted target position offset compensation to compensate for the downsampling quantization loss. The final target position calculation formula is as follows:

[0033] (x, y, w, h) = (x d +O(0,x d ,y d ), y d +O(1,x d ,y d ), S(0, x d ,y d ), S(1, x d ,y d ))

[0034] Among them, x represents the horizontal coordinate of the target center position, y represents the vertical coordinate of the target center position, w represents the length of the target box, and h represents the width of the target box.

[0035] The specific process of using the prediction head learning to obtain the tracking result in step 3 is:

[0036] Step 3.1: Perform mean subtraction preprocessing on the original template image and the search image, resize the template image to 128*128, and resize the search image to 256*256. When the size exceeds the original image, fill it with the average RGB value; in order to avoid resizing the image during training, each frame of the template image and the search image are extracted offline;

[0037] Step 3.2: Use fine-tuning network training. After the image is mean-reduced and resized, the template image and the search image are sent to the target tracking network for training. During the fine-tuning process, the loss function is defined as the cross entropy function between pixels and the distance function between target positions. The classification result given by the classification head is compared with the corresponding annotation of the image at the pixel level. The cross entropy loss function is:

[0038]

[0039] Among them, y j =1 means that this pixel is a positive sample, p j It represents the probability that the model predicts that it is a positive sample; the target position result given by the regression head is compared with the corresponding annotation position of the image, and the distance loss function is:

[0040]

[0041] It includes L1 loss and GIoU loss;

[0042] Step 3.3: Perform several iterative training on each pair of search images and template images, and judge the model convergence by the change in the value of the loss function until the model reaches a convergence state.

[0043] A computer program product, characterized by comprising computer executable instructions, which are used to implement the method when being executed.

[0044] Beneficial Effects

[0045] The present invention proposes an adaptive dark light enhanced target tracking network structure and target tracking method. For target tracking tasks in scenes with changing illumination, this paper proposes an adaptive dark light enhanced target tracking network. The network mainly includes two modules, illumination enhancement and target tracking, which can realize end-to-end target tracking tasks in scenes with changing illumination. Among them, for the adaptive illumination enhancement task, an illumination compensation method based on illumination estimation and color space change is proposed, which can realize scene adaptive illumination compensation and color preservation of the image, and also complete the adaptive improvement of dark scene features. In the target tracking module, a network based on the Transformer architecture is mainly used, and accurate tracking of the target can be achieved through the self-attention mechanism. Compared with the independent illumination enhancement module, the illumination enhancement module proposed in this paper is integrated with the target tracking task, which can better enhance the image features for the tracking task. The algorithm proposed in this paper has a simple structure, and maintains the real-time requirements of the target tracking task while improving the performance. The tracker in this paper not only achieves good results in low-light scenes, but also maintains the tracking performance of the tracker in normal illumination scenes. Compared with the OSTrack algorithm, the accuracy is improved by 8.9% on the UAVDark70 dataset. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 Schematic diagram of the overall network framework

[0047] Figure 2 Schematic diagram of the adaptive dark light enhancement module

[0048] Figure 3 Schematic diagram of feature extraction and feature fusion module

[0049] Figure 4 Comparison results with and without color preservation mode

[0050] Figure 5 The accuracy and success rate curves of various target tracking algorithms on the UAVDark70 dataset DETAILED DESCRIPTION

[0051] The present invention will now be further described with reference to the embodiments and the accompanying drawings:

[0052] The present invention explores and proposes a new adaptive dark light enhanced target tracking method. In terms of adaptive dark light enhancement, a 1×1 convolution is used to fuse the information of the input image and the illumination prior map, and a large depth-separable convolution kernel is used to simulate the interaction between regions under different illumination, so as to generate a brightened feature map, and finally generate a brightness map. In terms of feature extraction and feature fusion, the present invention uses ViT as the basic framework, and adds a Transformer layer guided by illumination information and an early candidate elimination strategy on the basis of ViT. On the one hand, the Transformer layer guided by illumination information can pay more attention to the areas with low pixel values ​​in the image, that is, insufficient brightness, thereby solving the problem of being unable to extract effective information due to insufficient image brightness in low-light target tracking scenes. On the other hand, the image areas belonging to the background area are gradually identified and then discarded through the early candidate elimination strategy. This strategy can achieve the effect of killing two birds with one stone, that is, discarding some image areas also means reducing the computational amount of the self-attention mechanism while eliminating the interference of invalid background information on the tracking results, and thus speeding up the operation of the tracker. In terms of the prediction head, the center point-based anchor-free prediction method is adopted, which directly selects the model to predict whether each pixel contains the target center point and predicts the size of the bounding box. Compared with the anchor point-based prediction method, the center point-based anchor-free prediction method is more flexible, efficient and more accurate. The center point-based anchor-free prediction method can save some processing of the anchor box in the early stage of the algorithm, reduce unnecessary calculations and thus improve the running speed of the algorithm, which can also make the entire tracking framework neater.

[0053] The present invention proposes an adaptive dark light enhanced target tracking method, whose network structure is as follows Figure 1 As shown, the following steps are included:

[0054] Step 1: Build an adaptive dark-light enhanced target tracking network

[0055] according to Figure 1 , Figure 2 and Figure 3 , Adopt the method in step 1 to construct the adaptive dark light enhancement module, feature extraction and feature fusion module and prediction head. The image is passed through the adaptive dark light enhancement module to obtain the image after illumination compensation and the brightening feature map.

[0056] (1) Adaptive dark light enhancement module

[0057] The adaptive dark light enhancement module will output the image after illumination compensation and the brightening features for the next stage. In this network, a 1×1 convolution is first used to fuse the information of the input image and the illumination prior map. Then, a large depth-separable convolution kernel is used to simulate the interaction between regions under different illumination to generate a brightened feature map and a brightness map. Finally, the color distribution of the original image is maintained by the color preservation module. The adaptive dark light enhancement module is as follows: Figure 2 shown.

[0058] (2) Feature extraction and feature fusion module

[0059] In terms of feature extraction and feature fusion, the present invention uses ViT as the basic framework, and adds a Transformer layer guided by illumination information and an early candidate elimination strategy on the basis of ViT. Figure 3 shown.

[0060] In order to make the Transformer layer used in this paper more suitable for target tracking in scenes with changing lighting, this paper adds the guidance of lighting information to the Transformer layer. The input vector is mapped to generate the query matrix Q, the key matrix K and the value matrix V. In the same image, the lighting conditions in different areas are not the same. The feature representation of the area with poor lighting is not obvious, and the area with better lighting conditions can provide richer semantic information. Therefore, the area with better lighting conditions can be used to help improve the feature representation of the area with poor lighting conditions. In the process of calculating self-similarity in each Transformer layer, the brightening feature map is used as the lighting information for guidance, as shown in formula (1).

[0061]

[0062] Among them, Y represents the brightened feature vector. The brightening query matrix Q and the brightening key matrix K calculate the similarity, which can put more attention on the darker areas in the original image, so that the features of this part of the area get more attention. Then use the value matrix V that is not affected by the brightening features to maintain the original level of attention to the brighter areas in the image. The Transformer layer that uses lighting information as a guide can pay more attention to areas with low pixel values ​​in the image, that is, areas with insufficient brightness, thereby solving the problem of not being able to extract effective information due to insufficient image brightness in low-light target tracking scenarios.

[0063] Early Candidate Elimination Strategy The Transformer layer also uses the early candidate elimination strategy, which is introduced to improve the target tracking performance. The feature extraction and feature fusion module does not use all the image areas for self-attention calculations from beginning to end. It will gradually identify and discard image areas belonging to the background area in some Transformer layers. This strategy can achieve the effect of killing two birds with one stone, that is, discarding some image areas also means that while eliminating invalid background information that interferes with the tracking results, the amount of calculation of the self-attention mechanism is reduced, and therefore the speed of the tracker can be accelerated. Discarding some image areas shortens the length of the vector used by the Transformer, which is why the calculation of the self-attention mechanism is reduced and the speed of the tracker can be accelerated.

[0064] (3) Prediction Head

[0065] This module uses three different branches, namely the classification branch that predicts the classification score, the regression branch that predicts the normalized bounding box size, and the offset compensation branch that predicts the offset to compensate for the downsampling quantization loss. The area with the highest classification score can be considered to be the location of the target, and the final bounding box calculation method is shown in formula (2).

[0066] (x, y, w, h) = (x d +O(0,x d ,y d ), y d +

[0067] O(1, x d ,y d ), S(0, x d ,y d ), S(1, x d ,y d ))(2)

[0068] For the classification branch, regression branch, and offset compensation branch in the prediction head, each branch is composed of five convolutional layers, each of which contains convolution, batch normalization, and activation functions. The center point-based anchor-free prediction method abandons the previous use of anchor points and directly chooses the model to predict whether each pixel contains the target center point and predicts the size of the bounding box. Compared with the anchor point-based prediction method, the center point-based anchor-free prediction method is more flexible, efficient, and more accurate. The center point-based anchor-free prediction method can save some of the processing of the anchor box in the early stage of the algorithm, reduce unnecessary calculations, and thus increase the running speed of the algorithm, which can also make the entire tracking framework neater.

[0069] The image is input to the feature extraction and feature fusion module. The input image of this module is first divided into non-overlapping image blocks, and then each image block is projected into a vector of fixed length and then sent to the Transformer layer. In the present invention, the search image features and template image features are spliced ​​and deformed into a long vector, and the brightening feature map generated by the image enhancement module is also deformed into a long vector, and then self-similarity calculation is performed. Finally, the output of the feature extraction and feature fusion module is sent to the prediction head to obtain the final tracking result.

[0070] Step 2: Define the loss function

[0071] The tracking loss is shown in formula (3):

[0072] L track =L cls +L iou +L1 (3)

[0073] Among them, L cls represents the classification loss, L iou represents the iou loss between the predicted target bounding box and the true value bounding box, and L1 represents the distance loss between the predicted target bounding box parameters and the true value bounding box parameters.

[0074] When calculating the classification loss, the present invention forces the network to update its own parameters so that the tracking head can accurately determine the positive samples and negative samples in the image.

[0075] In order to better reflect the degree of overlap between any two boxes when calculating the IOU loss, the present invention uses the GiOU loss.

[0076] Step 3: Network training

[0077] Use the network constructed in step 1 for training, use the Adam optimization method to update the network parameters, stop training when the loss function value defined in step 2 is the minimum, and obtain an adaptive dark light enhanced target tracking network;

[0078] The samples of the training image sample library are used for training, and the Adam optimization method is used to optimize the network. The optimizer parameters betas are set to (0.5, 0.999), the learning rate is 0.0005, the weight_decay is 0.0001, and the batchsize is 8.

[0079] Step 4: Input the original images to be processed into the adaptive dark light enhancement target tracking network obtained in step 3, and the output bounding box coordinates are the location of the target.

[0080] Experimental results and analysis:

[0081] In order to comprehensively evaluate the effect of the algorithm proposed in the present invention, the present invention first analyzes the effect of the adaptive dark light enhancement module and the Transformer guided by illumination information, and then compares and evaluates them in terms of visualization effect and quantitative indicators on the NAT2021-test-100 and UAVDark70 target tracking datasets.

[0082] (1) Evaluation of the brightening feature map action position

[0083] In order to measure the effectiveness of the Transformer action position guided by lighting information, the present invention is evaluated on the NAT-test-100 dataset and a set of comparative experiments are designed to evaluate the action position of the brightening feature map.

[0084] (a) The brightened feature map is multiplied by the query matrix Q and the key matrix K respectively.

[0085] (b) The brightened feature map is multiplied by the query matrix Q.

[0086] (c) The brightened feature map is multiplied by the key matrix K.

[0087] (d) The brightened feature map is multiplied by the value matrix V.

[0088] (e) The brightened feature map is multiplied by the query matrix Q and the value matrix V respectively.

[0089] With other variables unchanged, the present invention trains the above network, and the final result indicators on the NAT2021-test-100 dataset are shown in Table 1.

[0090] Table 1 Comparison of experimental tracking results of each group of NAT2021-test-100 (the black mark is the best)

[0091]

[0092] It can be seen from Table 1 that the experimental results of group a are the best, that is, the best effect is achieved by multiplying the brightened feature map with the query matrix Q and the key matrix K. The performance of multiplying the brightened feature map with the query matrix Q or the key matrix K alone is not as good as that of group a. This paper believes that this is because the correlation is calculated using the brightened query matrix and the brightened key matrix to pay attention to the features of the dark area. If only one brightened matrix is ​​used in the correlation calculation process, it may not effectively highlight the important features in the dark area.

[0093] (2) Color preservation module evaluation

[0094] In order to measure the effectiveness of the color preservation module, the present invention is evaluated on the NAT2021-test-100 dataset, and a set of comparative experiments are designed to evaluate the color preservation module.

[0095] (a) Without color preservation module.

[0096] (b) Includes the color preservation module.

[0097] Figure 4 For comparison, the left half is the structure diagram without the color preservation module, and the right half is the result diagram with the color preservation module added. It can be seen from the figure that the image on the left has color deviation, while the image on the right does not have this problem, which shows the effectiveness of the color preservation module in alleviating color deviation.

[0098] (3) Analysis of experimental results of UAVDark70 dataset and OTB100 dataset

[0099] In order to verify the versatility of the method proposed in the present invention on different data sets, the present invention is tested on the UAVDark70 data set and the OTB100 data set. The specific quantitative indicators of the UAVDark70 data set are shown in Table 2. It can be seen that the method proposed in the present invention also achieves the best effect on the UAVDark70 data set, proving the effectiveness of the method. Figure 5 The following are the accuracy and success rate curves of various target tracking algorithms on the UAVDark70 dataset.

[0100] Table 2 Comparison of tracking results of target tracking algorithms on the UAVDark70 dataset

[0101]

[0102] The specific quantitative indicators of the OTB100 data set are shown in Table 3. It can be seen that the present invention does not reduce the tracking performance in normal scenarios, and is slightly improved compared to the baseline algorithm.

[0103] Table 3 Comparison of tracking results of OTB100 dataset

[0104]

Claims

1. An adaptive dark light enhanced target tracking network structure and target tracking method, characterized by: It includes two adaptive dark light enhancement modules, a feature extraction and feature fusion module guided by illumination information, and a prediction head; The inputs of the two adaptive dark light enhancement modules are the template image and the search image, respectively, and the two images themselves and the image features are brightened; the outputs of the two template image adaptive dark light enhancements are connected to the feature extraction and feature fusion module guided by the illumination information, and the output of the search image adaptive dark light enhancement is connected to the feature extraction and feature fusion module guided by the illumination information; The feature extraction and feature fusion module guided by illumination information includes N feature extraction and feature fusion modules, each of which includes an early candidate elimination module and a multi-head attention mechanism guided by illumination information, the feature extraction and feature fusion module of each layer is connected to the feature extraction and feature fusion module of the next layer, and the feature extraction and feature fusion module of the last layer is connected to the prediction head; The prediction head includes three different branches, and the three different branches are all composed of multi-layer perceptrons; The target tracking steps are as follows: Step 1: The template image and the search image are subjected to their respective adaptive dark light enhancement modules to extract the brightened images and brightened features of the template image and the search image, respectively. The formula is as follows: (I lu ,F lu )=ε(I,L p ) Among them, ε represents the adaptive dark light enhancement module, which takes an image and its illumination prior map as input; the illumination prior map is obtained by calculating the average value of each pixel of the input image along the channel dimension; the adaptive dark light enhancement network outputs the image I after illumination compensation lu , and the brightening feature F for the next stage lu ; The respective adaptive dark light enhancement modules of the template image and the search image are adaptive dark light enhancement modules with weight sharing; Step 2: The brightened template image and its brightened features as well as the search image and its brightened features enter the feature extraction and feature fusion module guided by illumination information. The two images are first cut into blocks of a certain size and then the image blocks of the two are spliced ​​and sent to the layer normalization, and then sent to the multi-head attention guided by illumination information. The formula is as follows: Among them, Y represents the brightened feature vector obtained by concatenating two brightened features, and the query matrix Q, key matrix K and value matrix V are generated by mapping the results of concatenating the search image blocks and the template image blocks; the brightening query matrix Q and the brightening key matrix K calculate the similarity, and put more attention on the darker area in the original image, so that the features of this part of the area get more attention; After the multi-head attention guided by the illumination information, it is sent to the early candidate elimination module through the residual connection. The early candidate elimination module scores the search image blocks and discards some search image areas. This reduces the calculation amount of the self-attention mechanism while eliminating the interference of invalid background information on the tracking results, thereby speeding up the operation of the tracker. The output of the early candidate elimination module is sent to the feature extraction and feature fusion module of the next stage after layer normalization and multi-layer perceptron. If it is the last layer, it is output to the prediction head. The calculation formula of the feature extraction and feature fusion module of each layer is as follows: Among them, LGMSA stands for multi-head attention guided by illumination information, FFN stands for feedforward neural network, LN is the layer normalization operation, and ECE stands for early candidate elimination module; Step 3: The result processed in step 2 is used to obtain the final tracking result through the prediction head. The prediction head uses three different branches to obtain the classification score, the normalized target bounding box parameters, and the predicted target position offset compensation to compensate for the downsampling quantization loss. The final target position calculation formula is as follows: (x,y,w,h)=(x d +O(0,x d ,y d ),y d +O(1,x d ,y d ),S(0,x d ,y d ),S(1,x d ,y d )) Among them, x represents the horizontal coordinate of the target center position, y represents the vertical coordinate of the target center position, w represents the length of the target box, and h represents the width of the target box.

2. The method according to claim 1, characterized in that: The adaptive dark light enhancement module includes a 1×1 convolution layer, a depth-separable convolution layer and a color preservation module; the module is used to generate brightening features and enhanced images.

3. The method according to claim 1, characterized in that: The illumination information guided feature extraction and feature fusion module is based on the self-attention mechanism in Transformer, replacing the multi-head attention with the illumination information guided multi-head attention and adding an early candidate elimination module between the multi-head attention and layer normalization; This module is used to complete the feature extraction and feature fusion of the search image and the template image.

4. The method according to claim 1, characterized in that: The prediction head includes three different branches, namely a classification branch for predicting classification scores, a regression branch for predicting normalized bounding box sizes, and an offset compensation branch for predicting offsets to compensate for downsampling quantization losses. The three different branches are all composed of a multi-layer perceptron and are used to generate target tracking results, i.e., bounding box parameters of the target.

5. The method according to claim 1, characterized in that: The target tracking network structure uses the Adam optimization method to update the network parameters, and stops training when the loss function value is minimized to obtain an adaptive dark-light enhanced target tracking network.

6. The method according to claim 5, characterized in that: The tracking loss function of the target tracking network structure is: L track =L cls +L iou +L1 Among them, L cls represents the classification loss, L iou represents the iou loss between the predicted target bounding box and the true value bounding box, and L1 represents the distance loss between the predicted target bounding box parameters and the true value bounding box parameters.

7. The method according to claim 5, characterized in that: In the Adam optimization method, the optimizer parameters betas are set to (0.5, 0.999), the learning rate is 0.0005, the weight_decay is 0.0001, and the batch size is 8.

8. The method according to claim 1, characterized in that: The specific process of using the prediction head learning to obtain the tracking result in step 3 is: Step 3.1: Perform mean subtraction preprocessing on the original template image and the search image, resize the template image to 128*128, and resize the search image to 256*256. When the size exceeds the original image, fill it with the average RGB value; in order to avoid resizing the image during training, each frame of the template image and the search image are extracted offline; Step 3.2: Use fine-tuning network training. After the image is mean-reduced and resized, the template image and the search image are sent to the target tracking network for training. During the fine-tuning process, the loss function is defined as the cross entropy function between pixels and the distance function between target positions. The classification result given by the classification head is compared with the corresponding annotation of the image at the pixel level. The cross entropy loss function is: Among them, y j =1 means that this pixel is a positive sample, p j It represents the probability that the model predicts that it is a positive sample; the target position result given by the regression head is compared with the corresponding annotation position of the image, and the distance loss function is: It includes L1 loss and GIoU loss; Step 3.3: Perform several iterative training on each pair of search images and template images, and judge the model convergence by the change in the value of the loss function until the model reaches a convergence state.

9. A computer program product, characterized in that The invention comprises computer executable instructions, which are used to implement the methods of claims 1 to 8 when being executed.

Images