A SiamFC target tracking method combining image preprocessing and triplet attention

Abstract

The application belongs to the field of target tracking, and relates to a SiamFC target tracking method combining image preprocessing and triplet attention, comprising the following steps: 1, obtaining a data set; 2, constructing a SiamFC target tracking network model, the model comprising a first ExCNet module, a first AlexNet module and a triplet attention module connected in sequence, a second ExCNet module and a second AlexNet module connected in sequence, and a cross-correlation algorithm module and a network loss function; the triplet attention module and the second AlexNet module are connected with the cross-correlation algorithm module; step 3, training the SiamFC target tracking network model; 4, performing target tracking. The application can achieve better tracking results on an extremely low-illumination video tracking sequence.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of target tracking, and particularly relates to a SiamFC target tracking method combining image preprocessing and triplet attention. BACKGROUND

[0002] Target detection and tracking is one of the core tasks in the field of computer vision, and is widely used in intelligent monitoring, automatic driving, human-computer interaction, video analysis and other fields. A target tracking algorithm depends on the first frame image in a video sequence frame, and a tracker needs to predict the target size and position of the subsequent frame based on the first frame image. However, in real life, the target may fail to be tracked by the tracker due to factors such as occlusion, scale change, illumination change, rapid motion, etc.

[0003] In recent years, the deep learning technology has developed increasingly mature, and the target tracking technology has gradually shifted from the traditional filtering-based method (such as Kalman filtering, particle filtering) to the deep learning-based method, such as the twin neural network and the Transformer network. These methods perform well in dealing with complex background, target occlusion and deformation, etc. Among them, the twin network tracker is concerned for its balanced accuracy and speed. Bertinetto et al. proposed the SiamFC tracking algorithm (Bertinetto, Luca, et al. "Fully-convolutional siamese networks for object tracking." Computer vision–ECCV 2016 workshops: Amsterdam, the Netherlands, October 8-10 and 15-16, 2016,proceedings, part II 14 . Springer International Publishing, 2016.) which uses a twin neural network and similarity learning to solve the tracking problem. However, in the case of object occlusion and complex environment, the algorithm cannot accurately distinguish the target, resulting in slower tracking speed than some related filter-based tracking algorithms. Based on this, many researchers have proposed methods to improve the SiamFC target tracking network. Li et al. proposed the SiamRPN algorithm (Li, Bo, et al. "High performance visual tracking with siamese region proposal network." Proceedings of the IEEE conference on computer vision and pattern recognition(2018).) which adds a region proposal network to the SiamFC network model to generate candidate regions for extracting more abundant and robust deep information, but the parameter quantity and computational complexity of the network model also increase; Dong et al. proposed a SiamTri network (X. Dong and J. Shen, “Triplet loss in Siamese network for object tracking,” in Proc. Eur. Conf. Comput. Vis. Cham, Switzerland: Springer, 2018, pp. 459-474.) which introduces a triplet loss function into the SiamFC network model, and excavates the potential relationship between the target and positive and negative samples in the iterative training to enhance the representation ability of the network model; Li et al. proposed a SiamVGG network model (Li, Yuhong, Xiaofan Zhang, and Deming Chen. “SiamVGG: Visual tracking using deeper siamese networks.” arXiv preprint arXiv:1902.02804 (2019).) which extracts features through a deeper VGG-16 network model and obtains more robust feature expression using higher-dimensional target features to improve tracking effect; Guo et al. proposed a DSiam network model (Guo, Qing, et al. “Learning dynamic siamese network for visual object tracking.” Proceedings of the IEEE international conference on computer vision . 2017.) which solves target appearance changes and background interference by learning feature transformation.

[0004] The above methods have achieved good results in the field of target tracking, but in natural scenes, the target may be in an extremely low light environment, causing the detection box to drift or even disappear during tracking, and the above network models based on SiamFC rely on feature extraction of the first frame template image, and insufficient feature extraction of the first frame target will lead to poor tracking effect. Therefore, in view of the influence of the light environment and the problem of the first frame target feature extraction, the present application designs a SiamFC target tracking method combining image preprocessing and triplet attention. SUMMARY

[0005] The SiamFC target tracking method combining image preprocessing and triplet attention has the advantages that the SiamFC target tracking method combining image preprocessing and triplet attention can improve the feature extraction of the first frame template image, and the insufficient feature extraction of the first frame target can improve the subsequent tracking effect.

[0006] To achieve the above object, the technical scheme adopted by the present application is:

[0007] A SiamFC target tracking method combining image preprocessing and triplet attention, which is characterized by comprising the following steps:

[0008] Step 1, a data set containing video sequence frames is obtained, and the data set is divided into a training set and a test set;

[0009] Step 2, a SiamFC target tracking network model is constructed; the SiamFC target tracking network model comprises a first ExCNet module, a second ExCNet module, a first AlexNet module, a second AlexNet module, a triplet attention module, a cross-correlation algorithm module, and a network loss function; the first ExCNet module, the first AlexNet module and the triplet attention module are connected in sequence, the second ExCNet module and the second AlexNet module are connected in sequence, the triplet attention module and the second AlexNet module are connected with two input ends of the cross-correlation algorithm module respectively, and an output end of the cross-correlation algorithm module is an output end of the SiamFC target tracking model;

[0010] Step 3, the video sequence frames in the training set are respectively input into the first ExCNet module and the second ExCNet module, a first frame image in the corresponding video sequence frame is recognized by the first ExCNet module and taken as a template image Z, an image after the first frame in the corresponding video sequence frame is recognized by the second ExCNet module and taken as a search image X, and image preprocessing is respectively performed on the template image Z and the search image X to realize the repair of an exposure image and a low-illumination image;

[0011] Step 4, the template image Z and the search image X after completing image preprocessing are input into the first AlexNet module and the second AlexNet module through the first ExCNet module and the second ExCNet module, the template image Z and the search image X are subjected to feature extraction through the first AlexNet module and the second AlexNet module, template deep features and search deep features are obtained, the template deep features are input into the triplet attention module, and the search deep features are input into the cross-correlation algorithm module;

[0012] Step 5, more rich and more distinctive context features in the template deep features are captured by the triplet attention module, and then input into the cross-correlation algorithm module; the context features and the search deep features are calculated by the cross-correlation algorithm module to obtain the target tracking result; and based on the target tracking result and the network loss function, the network parameters are continuously optimized by back propagation to obtain the trained SiamFC target tracking network model;

[0013] Step 6, the video sequence frames in the test set are input into the trained SiamFC target tracking network model to obtain the final target tracking result, and the target tracking is completed.

[0014] Further, in step 2, the first ExCNet module and the second ExCNet module each include a CNN network unit, a brightness value calculation unit, a unary term calculation unit, a binary term calculation unit, an adder unit and a loss calculation unit.

[0015] The input end of the CNN network unit, the first input end of the brightness value calculation unit, the first input end of the unary term calculation unit and the first input end of the binary term calculation unit are respectively used to receive input images I i The output end of the CNN network unit is connected with the second input end of the brightness value calculation unit, and the two output ends of the brightness value calculation unit are respectively connected with the second input end of the unary term calculation unit and the second input end of the binary term calculation unit; the output ends of the unary term calculation unit and the binary term calculation unit are respectively connected with the two input ends of the adder unit; the output end of the adder unit is connected with the input end of the loss calculation unit, and the output end of the loss calculation unit is the output end of the first ExCNet module or the second ExCNet module.

[0016] Step 3 is specifically:

[0017] Step 3.1, the video sequence frames in the training set are respectively input into the CNN network unit in the first ExCNet module and the second ExCNet module, the first frame image in the corresponding video sequence frame is recognized by the CNN network unit in the first ExCNet module as a template image Z, and the image after the first frame in the corresponding video sequence frame is recognized by the second ExCNet module as a search image X.

[0018] Step 3.2, the CNN network unit calculates the shadow amount and the highlight amount according to the input images I i Obtain the intermediate estimate of the shadow amount and the highlight amount , and input it into the brightness value calculation unit.

[0019] Step 3.3, the brightness value calculation unit calculates the brightness value according to the intermediate estimate and input image I i Obtain a brightness-corrected image And input them into the unary term calculation unit and the binary term calculation unit respectively;

[0020] Step 3.4: The univariate calculation unit calculates the image based on the brightness correction. Get unary terms The binary term calculation unit calculates the input image. I i and brightness-corrected images Get the binary term and use an adder to combine unary terms and binary terms The sums are then input into the loss calculation unit;

[0021] Step 3.5: The loss calculation unit calculates the sum of the unary terms. and binary terms Obtain the preprocessed image.

[0022] Furthermore, in step 2, the CNN network unit includes 5 convolutional units and 3 fully connected units connected in sequence;

[0023] Each convolutional unit includes a first 3x3 convolutional layer, a first batch of normalized layers, a first ReLU activation layer, and a first 3x3 max pooling layer connected in sequence; each fully connected unit includes a fully connected layer, a second batch of normalized layers, and a second ReLU activation layer connected in sequence.

[0024] The calculation formula for the brightness value calculation unit is as follows:

[0025] ;

[0026] In the formula: x For the input image I i Brightness value; Output brightness value; This is the increment function corresponding to the shaded area. , , They are constants, ; This is the increment function corresponding to the specular intensity;

[0027] The calculation formula for a single-variable term is:

[0028] ;

[0029] In the formula: and Images after brightness correction The Middle i The average value of the unenhanced and enhanced brightness channels of the image patch; for function;

[0030] The calculation formula for the binary term calculation unit is:

[0031] ;

[0032] In the formula: and They are respectively and the i The adjacent first image block j The average value of the unenhanced and enhanced brightness channels of the image patch;

[0033] The calculation formula for the loss calculation unit is as follows:

[0034] ;

[0035] In the formula: For loss calculation function; i For a given first i Image patch Let be the set of 4 adjacent image blocks of the i-th image block.

[0036] Furthermore, in step 2, both the first AlexNet module and the second AlexNet module include sequentially connected 11*11 convolutional layers, a first batch of normalized + Power_ReLU activation layers, a second 3*3 max pooling layer, a 5*5 convolutional layer, a second batch of normalized + Power_ReLU activation layers, a third 3*3 max pooling layer, a second 3*3 convolutional layer, a third batch of normalized + Power_ReLU activation layers, a third 3*3 convolutional layer, a fourth batch of normalized + Power_ReLU activation layers, and a fourth 3*3 convolutional layer;

[0037] The activation function Power_Relu in the first, second, third, and fourth batches of normalized + Power_Relu activation layers is:

[0038] ;

[0039] In the formula: and They are constants, This is the input to the activation function.

[0040] Further, in step 2, the triplet attention module comprises a first attention branch, a second attention branch, a third attention branch and an integration unit; the input ends of the first attention branch, the second attention branch and the third attention branch are connected with the output end of the first AlexNet module respectively, and the output ends are connected with the input ends of the integration unit respectively; the output end of the integration unit is connected with the first input end of the cross-correlation algorithm module;

[0041] In step 5, the first attention branch is used to obtain the weighted feature map of the interaction between the height dimension and the channel dimension according to the template depth feature; the second attention branch is used to obtain the weighted feature map of the interaction between the width dimension and the channel dimension according to the template depth feature; the third attention branch is used to obtain the weighted feature map of the interaction between the height dimension and the width dimension according to the template depth feature; and then the integration unit is used to integrate the three weighted feature maps according to the weight of 1 / 3, so as to obtain more rich and more distinctive context features.

[0042] Further, in step 2, the first attention branch comprises a height counterclockwise rotation unit, a first attention unit and a height clockwise rotation unit;

[0043] The second attention branch comprises a width counterclockwise rotation unit, a second attention unit and a width clockwise rotation unit;

[0044] The third attention branch comprises an Identity unit and a third attention unit;

[0045] The first attention unit, the second attention unit and the third attention unit all comprise a Z-Pool unit, a Conv unit, a sigmoid function unit and a multiplication unit; the Z-Pool unit, the Conv unit and the sigmoid function unit are connected in sequence, the output end of the sigmoid function unit is connected with the first input end of the multiplication unit, the input end of the Z-Pool unit and the second input end of the multiplication unit are connected with the output ends of the height counterclockwise rotation unit, the width counterclockwise rotation unit or the Identity unit respectively; the output end of the multiplication unit is connected with the input end of the integration unit, and the output end of the integration unit is the output end of the triplet attention module.

[0046] Further, the calculation formula of the cross-correlation algorithm module is:

[0047] ;

[0048] In the formula, R is a response map, that is, a target tracking result; is a context feature; is a search depth feature; is the value of each position on the response map. b

[0049] ​The beneficial effects of the present application are:

[0050] The present application first adopts an image repair network (a first ExCNet module and a second ExCNet module) to perform image preprocessing, realizes repair of an exposure image and a low-illumination image, so as to facilitate subsequent feature extraction; then adopts a first AlexNet module and a second AlexNet module as a backbone network of a SiamFC target tracking network model to perform feature extraction on a template image and a search image; finally adopts a triad attention module to capture more rich and more distinctive context features in a template deep feature, can emphasize the importance of cross-dimension interaction, helps the SiamFC target tracking network model to learn more rich and more distinctive context features, so as to realize better target tracking results than an existing SiamFC network model, and still realize good tracking results on an extremely low-illumination video tracking sequence, to a certain extent, solve the problem of tracking frame drift caused by the influence of light on a target in a target tracking process. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 It is a structural principle schematic diagram of an embodiment of the present application;

[0052] Figure 2 It is a structural principle schematic diagram of a first ExCNet module and a second ExCNet module in an embodiment of the present application;

[0053] Figure 3 It is a structural principle schematic diagram of a first AlexNet module and a second AlexNet module in an embodiment of the present application;

[0054] Figure 4 It is a structural principle schematic diagram of a triad attention module in an embodiment of the present application;

[0055] Figure 5 It is a processing effect diagram of an existing SiamFC network model and a SiamFC target tracking network model in an embodiment of the present application, wherein a, b and c are processing effect diagrams of the existing SiamFC network model, and e, f and g are processing effect diagrams of the SiamFC target tracking network model in the embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the purpose, advantages and characteristics of the present application clearer, the following combines the drawings and specific embodiments to make a further detailed description of a SiamFC target tracking method combining image preprocessing and a triad attention proposed by the present application. The advantages and characteristics of the present application will be clearer according to the following specific embodiments.

[0057] Step 1, obtaining a dataset containing video sequence frames, in this embodiment, the aforementioned dataset is GOT-10k, which contains video sequence frames in extremely low-illumination environments. The dataset is divided into a training set and a test set according to a certain ratio, which is usually 8:2.

[0058] Step 2, constructing a SiamFC target tracking network model

[0059] Referring to Figure 1 , the SiamFC target tracking network model used in this embodiment specifically includes a first ExCNet (image dehazing) module, a second ExCNet module, a first AlexNet (deep convolutional neural network) module, a second AlexNet module, a Convolutional Triplet Attention Module, a cross-correlation algorithm module, and a network loss function. The first ExCNet module, the first AlexNet module, and the Convolutional Triplet Attention Module are connected in sequence, the second ExCNet module and the second AlexNet module are connected in sequence, the Convolutional Triplet Attention Module and the second AlexNet module are connected to the two input ends of the cross-correlation algorithm module, and the output end of the cross-correlation algorithm module is the output end of the SiamFC target tracking model.

[0060] Referring to Figure 2 , the first ExCNet module and the second ExCNet module each include a CNN network unit, a brightness value calculation unit, a unary term calculation unit, a binary term calculation unit, an adder unit, and a loss calculation unit.

[0061] The CNN network unit includes five convolution units and three fully connected units connected in sequence. Figure 2 The first 3*3 convolution layer of each convolution unit includes a first batch normalization layer, a first Relu function activation layer, and a first 3*3 max pooling layer connected in sequence. Each fully connected unit includes a fully connected layer, a second batch normalization layer, and a second Relu function activation layer connected in sequence. The first 3*3 convolution layer of the first convolution unit is the input end of the CNN network unit, and the second Relu function activation layer of the last fully connected unit is the output end of the CNN network unit.

[0062] The input end of the CNN network unit, the first input end of the brightness value calculation unit, the first input end of the unary term calculation unit, and the first input end of the binary term calculation unit are respectively used to receive an input image I i , the input image I iThe template image Z and the search image X are included; the output end of the CNN network unit is connected with the second input end of the brightness value calculation unit, the two output ends of the brightness value calculation unit are respectively connected with the second input end of the unary term calculation unit and the second input end of the binary term calculation unit; the output ends of the unary term calculation unit and the binary term calculation unit are respectively connected with the two input ends of the adder unit; the output end of the adder unit is connected with the input end of the loss calculation unit, and the output end of the loss calculation unit is the output end of the first ExCNet module or the second ExCNet module.

[0063] The calculation formula of the brightness value calculation unit is:

[0064] ;

[0065] In the formula: x is the brightness value of the input image I i ; is the output brightness value; respectively are the intermediate estimates of the shadow amount and the highlight amount; is the incremental function corresponding to the shadow amount, , , respectively are constants, in the embodiment, the values of , are respectively 5, 14 and 1.6, so as to ensure ; is the incremental function corresponding to the highlight amount.

[0066] The calculation formula of the unary term calculation unit is:

[0067] ;

[0068] In the formula: and respectively are the average values of the unenhanced brightness channel and the enhanced brightness channel of the image in the image i if the average value of the enhanced brightness channel is greater than 0.5, the difference is reserved, if it is less than 0, the result of the difference is inverted, so that the difference between the pixels of the input image I i and the pixels of the brightness-corrected image around 0.5 is strengthened, so that the model pays more attention to the difference around the brightness median value; is the function.

[0069] The binary term measures the local consistency of the image processing by calculating the squared difference between the original image block and the processed image block, and the calculation formula of the binary term calculation unit is:

[0070] ;

[0071] wherein: and are the first and the second i neighboring image blocks of the image block. j the average of the unenhanced and the enhanced luminance channels of the image block.

[0072] The loss calculation unit is expressed as an energy minimization problem based on the image block, and the calculation formula of the loss calculation unit is:

[0073] ;

[0074] wherein: is the set of the four neighboring image blocks of the first i image block.

[0075] Referring to Figure 3 , the first AlexNet module and the second AlexNet module each include an 11*11 convolution (Conv) layer, a first batch normalization (BN) + Power_Relu function activation layer, a second 3*3 max-pooling layer (MaxPool), a 5*5 convolution layer, a second batch normalization + Power_Relu function activation layer, a third 3*3 max-pooling layer, a second 3*3 convolution layer, a third batch normalization + Power_Relu function activation layer, a third 3*3 convolution layer, a fourth batch normalization + Power_Relu function activation layer, and a fourth 3*3 convolution layer.

[0076] Among them, the stride of the 11*11 convolution layer, the second 3*3 max-pooling layer, and the third 3*3 max-pooling layer is 2; the stride of the 5*5 convolution layer, the second 3*3 convolution layer, the third 3*3 convolution layer, and the fourth 3*3 convolution layer is 1.

[0077] The nonlinearity of the activation function plays an important role in the training of the neural network, and the Relu activation function is the most common activation function at present, but when the input is negative, the Relu activation function will not activate, which will lead to neuron death. The variant Leaky Relu activation function of the Relu activation function can solve the problem of neuron "death", but will not be saturated at a smaller input. Considering the shortcomings of the commonly used activation function, therefore, in the embodiment, the activation function Power_Relu in the first batch of normalization + Power_Relu function activation layer, the second batch of normalization + Power_Relu function activation layer, the third batch of normalization + Power_Relu function activation layer and the fourth batch of normalization + Power_Relu function activation layer is set as:

[0078] ;

[0079] In the formula: and are constants, which can be obtained by learning, and in the embodiment, the values are 1.05 and 1.67 respectively; is the input of the activation function.

[0080] The Power_Relu activation function will be saturated to a negative value at a smaller input, and can selectively activate a negative value. On the other hand, the new activation function retains the properties of the Softplus function, and according to the related research of neuroscientists, the Softplus function is closer to the activation model of brain neurons, and can be regarded as a smoothing of the Relu activation function.

[0081] Referring to Figure 4 , the triplet attention module includes a first attention branch, a second attention branch, a third attention branch, and an integration unit; the input ends of the first attention branch, the second attention branch, and the third attention branch are connected with the output end of the first AlexNet module respectively, and the output ends are connected with the input ends of the integration unit respectively; the output end of the integration unit is connected with the first input end of the cross-correlation algorithm module.

[0082] The first attention branch includes a high counterclockwise rotation unit, a first attention unit, and a high clockwise rotation unit; the second branch includes a width counterclockwise rotation unit, a second attention unit, and a width clockwise rotation unit; the third branch includes an Identity (identity authentication) unit and a third attention unit.

[0083] The first attention unit, the second attention unit and the third attention unit each comprise a Z-Pool unit, a Conv unit, a sigmoid function unit and a multiplication unit; the Z-Pool unit, the Conv unit and the sigmoid function unit are connected in sequence, an output end of the sigmoid function unit is connected to a first input end of the multiplication unit, and an input end of the Z-Pool unit and a second input end of the multiplication unit are respectively connected to output ends of the height counterclockwise rotation unit, the width counterclockwise rotation unit or the Identity unit; an output end of the multiplication unit is connected to an input end of the integration unit, and an output end of the integration unit is an output end of the triplet attention module.

[0084] Specifically, the calculation formula of the cross-correlation algorithm module is:

[0085] ;

[0086] In the formula, C is a context feature; is a search depth feature; is the value of each position on the response graph; b is a response graph, that is, a target tracking result.

[0087] Step 3, image preprocessing

[0088] Step 3.1, input the video sequence frames in the training set into the CNN network units in the first ExCNet module and the second ExCNet module respectively, identify the first frame image in the corresponding video sequence frame as a template image Z through the CNN network unit in the first ExCNet module, identify the image after the first frame in the corresponding video sequence frame as a search image X through the second ExCNet module; the template image Z and the search image X are the input images mentioned above I i .

[0089] Step 3.2, according to the input images I i obtain the intermediate estimate of the shadow amount and the highlight amount , and input it into the luminance value calculation unit.

[0090] Step 3.3, according to the intermediate estimate and the input image I i obtain the image after luminance correction , and input it into the unary term calculation unit and the binary term calculation unit respectively.

[0091] Step 3.4, according to the image after luminance correction​ obtaining a unary term , by the binary term calculation unit according to the input image I i and the image after brightness correction obtaining a binary term , and adding the unary term and the binary term to the loss calculation unit after the addition.

[0092] Step 3.5, obtaining the preprocessed image by the loss calculation unit according to the added unary term and the binary term .

[0093] Step 4, inputting the template image Z and the search image X after completing the image preprocessing to the first AlexNet module and the second AlexNet module through the first ExCNet module and the second ExCNet module, performing feature extraction on the template image Z and the search image X through the first AlexNet module and the second AlexNet module, obtaining the template deep feature and the search deep feature, and inputting the template deep feature to the triplet attention module and inputting the search deep feature to the cross-correlation algorithm module.

[0094] Step 5, in order to fully extract the feature information of the template image Z and improve the utilization, the extracted template deep feature is added to the triplet attention module to enhance the model's ability to distinguish similar objects, and the triplet attention module is used to capture more rich and distinctive context features in the template deep feature.

[0095] The first attention branch establishes the interaction between the height dimension and the channel dimension. First, the input tensor, i.e., the template deep feature, is rotated counterclockwise by 90 degrees around the H (height) axis through the height counterclockwise rotation unit, and then the average pooling features and the maximum pooling features of the input tensor are connected through the Z-Pool unit to reduce the dimension and depth of the input tensor, further lightening the triplet attention module, and then the Conv unit and the sigmoid function unit are used to obtain the attention weight of the interaction between the height dimension and the channel dimension. After multiplying the input by the attention weight, the result is rotated clockwise by 90 degrees around the H (width) axis through the width clockwise rotation unit to obtain the weighted feature map of the interaction between the height dimension and the channel dimension.

[0096] The second attention branch calculates the interaction between the width dimension and the channel dimension, and the specific implementation steps are the same as those of the first attention branch, except that the input is rotated counterclockwise by 90 degrees around the W (width) axis through the width counterclockwise rotation unit, and the result is rotated clockwise by 90 degrees around the WThe (width) axis is rotated 90 degrees clockwise.

[0097] The third attention branch calculates the interaction of the height dimension and the width dimension, and obtains the weighted feature map of the height dimension and the width dimension according to the template depth feature.

[0098] The three parts of the weighted feature map are integrated by the integration unit, and each part is integrated according to a weight of 1 / 3, so that more rich and more distinctive context features are obtained.

[0099] The context features are input into the cross-correlation algorithm module; the context features and the search depth features are calculated by the cross-correlation algorithm module, and the target tracking result is obtained.

[0100] In summary, the SiamFC target tracking process can be regarded as similarity learning, the template depth feature of 6x6x128 and the search depth feature of 22x22x128 are obtained according to the template image Z and the search image X respectively, the context features are obtained according to the template depth feature of 6x6x128, the context features are regarded as a convolution kernel, and the search depth feature is convolved to obtain a response map of 17x17x1, which is the target tracking result.

[0101] Based on the target tracking result and the network loss function, the network parameters are continuously optimized and iterated by back propagation, and a trained SiamFC target tracking network model is obtained.

[0102] Step 6, input the video sequence frames in the test set into the trained SiamFC target tracking network model, obtain the final target tracking result, and complete the target tracking.

[0103] As shown in Figure 5 The SiamFC target tracking network model of the application can not only correct the brightness of the dark light image, so that the human eye can judge the position of the target faster, but also has only a slight tracking frame drift compared with the existing SiamFC network model, and therefore can be suitable for target detection and tracking in an extremely dark light environment, and has important significance for city environment monitoring systems, field low-illumination target tracking and rapid rescue in a low-illumination environment at night.

Claims

1. A SiamFC object tracking method combining image preprocessing and triplet attention, characterized in that, The method comprises the following steps: Step 1, obtaining a data set containing video sequence frames, the data set being divided into a training set and a test set; Step 2, constructing a SiamFC target tracking network model; the SiamFC target tracking network model comprises a first ExCNet module, a second ExCNet module, a first AlexNet module, a second AlexNet module, a triplet attention module, a cross-correlation algorithm module, and a network loss function; the first ExCNet module, the first AlexNet module, and the triplet attention module are connected in sequence, the second ExCNet module and the second AlexNet module are connected in sequence, the triplet attention module and the second AlexNet module are connected with two input ends of the cross-correlation algorithm module respectively, and an output end of the cross-correlation algorithm module is an output end of the SiamFC target tracking model; Step 3, inputting video sequence frames in the training set into the first ExCNet module and the second ExCNet module respectively, identifying a first frame image in the corresponding video sequence frame as a template image Z through the first ExCNet module, identifying an image after the first frame in the corresponding video sequence frame as a search image X through the second ExCNet module, and respectively performing image preprocessing; Step 4, inputting the template image Z and the search image X after completing image preprocessing into the first AlexNet module and the second AlexNet module through the first ExCNet module and the second ExCNet module, performing feature extraction on the template image Z and the search image X through the first AlexNet module and the second AlexNet module, obtaining template deep features and search deep features, inputting the template deep features into the triplet attention module, and inputting the search deep features into the cross-correlation algorithm module; Step 5, capturing more rich and more distinctive context features in the template deep features through the triplet attention module, inputting the context features into the cross-correlation algorithm module, performing cross-correlation calculation on the context features and the search deep features through the cross-correlation algorithm module to obtain a target tracking result, and based on the target tracking result and the network loss function, constantly optimizing and iterating network parameters through back propagation to obtain a trained SiamFC target tracking network model; Step 6, inputting video sequence frames in the test set into the trained SiamFC target tracking network model to obtain a final target tracking result, and completing target tracking.

2. The SiamFC target tracking method combining image preprocessing and triplet attention according to claim 1, wherein in step 2, the first ExCNet module and the second ExCNet module each comprise a CNN network unit, a luminance value calculation unit, a unary term calculation unit, a binary term calculation unit, an adder unit, and a loss calculation unit; Step 3 is specifically: The input end of the CNN network unit, the first input end of the brightness value calculation unit, the first input end of the unary term calculation unit, and the first input end of the binary term calculation unit are respectively used for receiving an input image I i The output end of the CNN network unit is connected with the second input end of the brightness value calculation unit, two output ends of the brightness value calculation unit are respectively connected with the second input end of the unary term calculation unit and the second input end of the binary term calculation unit, the output ends of the unary term calculation unit and the binary term calculation unit are respectively connected with two input ends of the adder unit, the output end of the adder unit is connected with the input end of the loss calculation unit, and the output end of the loss calculation unit is the output end of the first ExCNet module or the second ExCNet module. ​ Step 3.1, input the video sequence frames in the training set into the CNN network units in the first ExCNet module and the second ExCNet module respectively, identify the first frame image in the corresponding video sequence frame as a template image Z through the CNN network unit in the first ExCNet module, and identify the image after the first frame in the corresponding video sequence frame as a search image X through the second ExCNet module; Step 3.2, obtaining an intermediate estimate of the amount of shadow and highlight by a CNN network unit from the input image I i obtaining an intermediate estimate of the amount of shadow and highlight and inputting it to the luminance value calculation unit; Step 3.

3. The intermediate estimate is inputted to the luminance value calculation unit and the input image I i The luminance-corrected image is obtained and inputted to the monomial calculation unit and the binomial calculation unit, respectively; Step 3.4: The unary term calculation unit calculates the input image... I i and brightness-corrected images Get unary terms The binary term calculation unit calculates the input image. I i and brightness-corrected images Get the binary term and use an adder to combine unary terms and binary terms The sums are then input into the loss calculation unit; Step 3.5, calculating the loss by the loss calculation unit according to the monomial term after addition and the binomial term The preprocessed image is obtained.

3. The SiamFC target tracking method combining image preprocessing and triplet attention according to claim 2, characterized in that: In step 2, the CNN network unit comprises five convolution units and three full connection units connected in sequence; Each of the convolution units comprises a first 3*3 convolution layer, a first batch normalization layer, a first Relu function activation layer and a first 3*3 maximum pooling layer connected in sequence; and each of the full connection units comprises a full connection layer, a second batch normalization layer and a second Relu function activation layer connected in sequence. The calculation formula of the luminance value calculation unit is: ； wherein: x is the luminance value of the input image I i ; is the output luminance value; is the incremental function corresponding to the amount of shadow, , , are constants, ; is the incremental function corresponding to the amount of highlight; The calculation formula of the unary term calculation unit is: ； wherein: and are the luminance corrected images the first i the average of the unenhanced and enhanced luminance channels of the image block; is a function; The calculation formula of the binary term calculation unit is: ； wherein: and are the first i the image block adjacent to the first j the average of the unenhanced and enhanced luminance channels of the image block. The calculation formula of the loss calculation unit is: ； wherein: is a loss computation function; i is a given i-th i image block, is a set of 4 neighboring image blocks of the i-th i image block.

4. The SiamFC target tracking method combining image preprocessing and triplet attention according to any one of claims 1-3, characterized in that: In step 2, the first AlexNet module and the second AlexNet module each comprise an 11*11 convolution layer, a first batch normalization+Power_Relu function activation layer, a second 3*3 maximum pooling layer, a 5*5 convolution layer, a second batch normalization+Power_Relu function activation layer, a third 3*3 maximum pooling layer, a second 3*3 convolution layer, a third batch normalization+Power_Relu function activation layer, a third 3*3 convolution layer, a fourth batch normalization+Power_Relu function activation layer and a fourth 3*3 convolution layer connected in sequence. The activation function Power_Relu in the first batch normalization+Power_Relu function activation layer, the second batch normalization+Power_Relu function activation layer, the third batch normalization+Power_Relu function activation layer and the fourth batch normalization+Power_Relu function activation layer is: ； wherein: and are constants, is the input to the activation function.

5. The SiamFC target tracking method combining image preprocessing and triplet attention according to claim 1, characterized in that: In step 2, the triplet attention module comprises a first attention branch, a second attention branch, a third attention branch and an integration unit; the input ends of the first attention branch, the second attention branch and the third attention branch are connected with the output end of the first AlexNet module respectively, and the output ends are connected with the input ends of the integration unit respectively; and the output end of the integration unit is connected with the first input end of the cross-correlation algorithm module. In step 5, the first attention branch is used to obtain a weighted feature map with height dimension and channel dimension interaction according to the template depth feature; the second attention branch is used to obtain a weighted feature map with width dimension and channel dimension according to the template depth feature; the third attention branch is used to obtain a weighted feature map with height dimension and width dimension according to the template depth feature; and the three weighted feature maps are integrated by the integration unit according to a weight of 1 / 3 to obtain more rich and distinctive context features. 6.The SiamFC target tracking method combining image preprocessing and triplet attention according to claim 5, wherein: In step 2, the first attention branch comprises a height counterclockwise rotation unit, a first attention unit and a height clockwise rotation unit; the second attention branch comprises a width counterclockwise rotation unit, a second attention unit and a width clockwise rotation unit; the third attention branch comprises an Identity unit and a third attention unit; the first attention unit, the second attention unit and the third attention unit each comprise a Z-Pool unit, a Conv unit, a sigmoid function unit and a multiplication unit; the Z-Pool unit, the Conv unit and the sigmoid function unit are connected in sequence, the output end of the sigmoid function unit is connected to the first input end of the multiplication unit, the input end of the Z-Pool unit and the second input end of the multiplication unit are respectively connected to the output end of the height counterclockwise rotation unit, the width counterclockwise rotation unit or the Identity unit, and the output end of the multiplication unit is connected to the input end of the integration unit; and the output end of the integration unit is the output end of the triplet attention module. 7.The SiamFC target tracking method combining image preprocessing and triplet attention according to claim 1, wherein: In step 2, the calculation formula of the cross-correlation algorithm module is: ； In the formula: is a response map, i.e., a target tracking result; is a contextual feature; is a search depth feature; b is the value of each location on the response map.

Images