A multi-template fusion target tracking algorithm based on graph attention network

Abstract

This invention discloses a multi-template fusion target tracking algorithm based on graph attention networks. Specifically, the algorithm implements the following steps: constructing an overall network and training its structure end-to-end; using the trained network to initialize the tracking of the video image sequence to be tracked, obtaining the initial target template and initial target position information for this tracking task; after obtaining the initial target template, the tracking process begins, calculating the position of a target in each frame of the video image sequence and displaying it at the corresponding location in the image, obtaining a tracking result response map; after obtaining the tracking result response map, a two-stage template update threshold judgment mechanism is used to determine whether the current tracking result is reliable. If reliable, the template is updated; if unreliable, the template is not updated; if two reliable tracking results are retained, the latest result replaces the oldest result.

Description

Technical Field

[0001] This invention belongs to the field of video-based single-target tracking technology, specifically relating to a multi-template fusion target tracking algorithm based on graph attention networks. Background Technology

[0002] With the rapid development of fields such as intelligent monitoring, autonomous driving, and human-computer interaction, the demand for computer vision technology is increasing across these sectors. Video object tracking is a hot research branch in computer vision, aiming to predict the position of an arbitrary target manually labeled in the first frame of a video sequence in subsequent video frames using tracking algorithms. Object tracking faces many challenges, mainly categorized into external and internal factors. External factors include cluttered backgrounds, varying lighting, interference from similar objects, and occlusion. Internal factors include non-rigid deformation, motion blur, rotation, and scale changes. The goal of object tracking algorithms is to address these challenges while achieving real-time, robust target tracking.

[0003] With the rapid development of deep learning algorithms, the combination of deep learning and target tracking algorithms has received increasing attention. Among them, algorithms based on Siamese neural network structures are a mainstream direction. By generating a template using the target image given in the first frame, cross-correlation is performed on subsequent images, and the location of the maximum value in the resulting response image is mapped back to the original image as the most likely location of the target. Typically, the target template used in Siamese neural network-based target tracking algorithms remains unchanged, which causes the tracker to be unable to adapt to changes in the target's appearance. Furthermore, many current methods involving template updates lack a sound threshold for template updates, and using unreliable template updates can easily pollute the template. Summary of the Invention

[0004] The purpose of this invention is to provide a multi-template fusion target tracking algorithm based on graph attention networks, which solves the problems of poor robustness to object surface deformation during tracking and template contamination caused by template updates in the existing technology.

[0005] To achieve the above objectives, the technical solution adopted by this invention is: a multi-template fusion target tracking algorithm based on graph attention networks, specifically implemented according to the following steps:

[0006] Step 1: Construct the overall network and train the overall network structure end-to-end;

[0007] Step 2: Use the overall network trained in Step 1 to perform initial tracking processing on the video image sequence to be tracked, and obtain the initial target template and the initial position information of the target for this tracking task;

[0008] Step 3: After obtaining the initial target template from Step 2, the tracking process begins. For each frame of the video image sequence, the position of a target in the image is calculated and displayed at the corresponding location in the image, resulting in a tracking response map.

[0009] Step 4: After obtaining the tracking result response map from Step 3, use a two-stage template update threshold judgment mechanism to determine whether the current tracking result is reliable. If it is reliable, update the template; if it is unreliable, do not update the template. If there are two reliable tracking results retained in Step 3, replace the oldest result with the latest result.

[0010] Step 5: Repeat steps 3 to 4 until all video image sequences have been tracked, thereby obtaining the location of the target in each frame of the video, and the tracking task ends.

[0011] The technical solution of the present invention also has the following characteristics:

[0012] Furthermore, in step 1, the overall network structure is divided into four parts: the first part is a Siamese neural network for deep feature extraction, i.e., a feature extraction network; the second part is a feature embedding module and a template update module based on graph attention network, i.e., a GM module; the third part is a cross-correlation network for accurately locating the target through feature maps, i.e., a target location prediction module; and the fourth part is a two-stage template update threshold judgment mechanism for judging whether the current result is reliable, i.e., a threshold judgment module.

[0013] Furthermore, in step 1, the feature extraction network is: Alex-Net deep feature extraction network.

[0014] Furthermore, in step 1, the end-to-end training of the overall network structure is specifically performed as follows: using meta-learning one-shot learning, four video frames are randomly selected from each dataset video to form an image pair. The first three images are used as the target image, and the latter are used as the search image to train the network. Ten image pairs are selected from each video, and each image pair contains four video frames. The first frame is the first frame of the video, and the last three images are randomly selected from the video. The interval between the second and third images is no more than 15 frames, and the interval between the third and fourth images is no more than 10 frames. The first three frames are used as the search image to synthesize the tracking template, and the last frame is used as the search image. In the search image processing, the three images imported into the 3D convolutional update module are the same, all being the last image of the image pair. The training is performed 50 times, with the batch size set to 8. 32 image pairs are imported at one time, and the loss function is the Logistic loss function.

[0015] Furthermore, in step 1, the generation of training image pairs in step 1 requires data augmentation of the selected images. The data augmentation is specifically implemented according to the following steps:

[0016] Step 1.1: First, perform a random stretch operation on the selected samples in the training set. The size of the stretched samples is set to be between 0.095 and 1.005. Linear interpolation is used to fill the parts that need to be filled after magnification. Then, perform a center crop operation, that is, crop a 263*263 area from the center of the training image pair. Next, perform a random crop operation, that is, crop a 255*255 area from a random position in the training image pair. Finally, perform a crop transformation. The BOX of the original GOT-10K dataset image, that is, the target location box, is given in the form of (left, top, weight, height), that is, the distance of the target box from the left and top edges of the image and the width and height of the target box itself. Through the crop transformation operation, the target box coordinates are transformed into (y, x, h, w), that is, the coordinates of the center point and the height and width of the target box.

[0017] Step 1.2: Calculate the loss:

[0018] The loss function formula used is:

[0019] L(y, v) = log(1 + exp(-yv))

[0020] In the formula, v is the true score of the sample, y∈{+1,-1} is the label of the ground truth, and L is the loss of the sample.

[0021] Step 1.3: After calculating the loss, use gradient descent to optimize the parameters. The formula for calculating the stochastic gradient descent (SGD) method is as follows:

[0022] argmin θ E (z，x，y) L (y，f(z，x；θ))

[0023] The optimal parameter θ can be obtained in this way; after 50 training iterations, the network's final loss value stabilizes below 0.1, and the training process ends.

[0024] Furthermore, step 2 is detailed as follows:

[0025] Step 2.1: Specify the target's location on the first frame of the video image sequence. Extract the target from the image and scale it to obtain a target image of size 127*127*3. Then, input the target image into the feature extraction network of the overall network to obtain features at layers P2, P3, P4, and P5. Use the P5 layer features as high-level features in the regression branch as the initial template for the regression branch. Use the P2 layer features as low-level features in the classification branch as the initial template for the classification branch. The initial template size is 6*6*256 pixels. Where z is the input target image, and the function... Representative feature extraction network.

[0026] Step 2.2: Initialize the parameters:

[0027] In the first frame of a video image sequence, the target position information given by manual calibration is called BOX. BOX has four pieces of information: the target's horizontal coordinate, vertical coordinate, width, and height. Therefore, no tracking calculation is required in the first frame. It is only necessary to set the corresponding initial center coordinates of the target and the initial width and height to the values ​​within the given BOX to complete the target initialization process.

[0028] Furthermore, step 3 is detailed below:

[0029] Step 3.1: First, extract deep features:

[0030] Using the target coordinates in the previous frame of the image / video sequence as the center, the search region is extracted and cropped into a patch, which is then fed into the feature extraction network to extract the depth features of the search region.

[0031] Step 3.2: Feature embedding and template update based on graph attention network:

[0032] Step 3.2.1, Feature Embedding:

[0033] The initial template Z-initial and the current template Z-current are fed into module G. First, the attention matrix f1 is calculated using the following formula:

[0034] f1=(W i Z i ) T ⊙(W c Z c )

[0035] Where ⊙ represents the matrix dot product operation, W i and W cThis represents the linear transformation matrix. After obtaining the attention matrix f1, it is normalized using Softmax to balance the information content. Then, the aggregate feature f2 is calculated, with the specific calculation formula as follows:

[0036] f 2 = Softmax(f1) ⊙ (Relu(W j Z i ))

[0037] Among them W j Let represent the linear transformation matrix. Finally, the latest current template Z_attention, incorporating the attention to the initial template, is obtained using the following formula:

[0038] Z_atteneion=Relu(f2||(Relu(W d Z c )))

[0039] Where || represents the concatnate operation, W j This represents a linear transformation matrix.

[0040] Step 3.2.2, Template Integration:

[0041] Once the current template Z_attention, which has attention to the initial template, is obtained, is passed into the M module for multi-template fusion operation;

[0042] In the first frame template initialization stage, this paper assigns the initial template to the current template Zp, the historical template Zq and the initial template Z-initial, and stacks the three together and passes them through a 3*3*3 3D convolutional layer to obtain the tracking template Z;

[0043] Step 3.2.3: Calculate the tracking results:

[0044] Finally, the response graph is obtained by cross-correlation between the tracking template Z and the search graph X. The specific calculation formula is as follows:

[0045]

[0046] Where * represents the cross-correlation operation, and bII represents the value corresponding to each position.

[0047] Furthermore, the two-stage template update threshold determination mechanism in step 4 is implemented as follows:

[0048] Phase 1: By calculating the Pearson correlation coefficient (PCCs) between the current tracking result and the initial template and comparing it with a pre-set threshold, it is determined whether the tracking result is the target. The formula for calculating PCCs is as follows:

[0049]

[0050] Where i represents the i-th sample. and σ is the sample mean. x and σ y The standard deviation of the sample is used; if both PCCs are greater than the threshold, the second stage begins; APCE and the maximum response value F-max are used as the main criteria to determine whether the current tracking result is reliable; the definition formula for APCE is as follows:

[0051]

[0052] Where R max and R min Let R represent the maximum and minimum values ​​on the score graph, respectively. w, This represents the value in the w-th row and h-th column of the score graph. If both APCE and F-max are lower than the preset threshold, the template update is abandoned and the next tracking is initiated. If both are higher than the threshold, the current tracking result is sent to the GM module for template update.

[0053] Compared with existing technologies, this paper first proposes a multi-template fusion module (GM module) based on graph attention networks. This module propagates the features of the initial template to each reliable template through the graph attention network, suppressing background noise outside the target. Furthermore, a two-stage template update threshold judgment mechanism is proposed, introducing Pearson correlation coefficient and APCE to complement each other. This mechanism can find reliable times to update the tracking template during tracking, helping the tracker continuously adapt to changes in the target's appearance while significantly reducing background noise introduced by template updates and preventing tracking drift. Moreover, these modules are all embeddable, convenient, and quick, improving tracker performance without affecting its real-time performance. Attached Figure Description

[0054] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0055] Figure 1 This is a schematic diagram of the overall framework of the multi-template fusion target tracking algorithm based on graph attention networks of the present invention;

[0056] Figure 2 This is a schematic diagram of network training for the multi-template fusion target tracking algorithm method based on graph attention network of the present invention;

[0057] Figure 3This is a schematic diagram of the feature embedding module (G module) based on graph attention network in the multi-template fusion target tracking algorithm method based on graph attention network of the present invention;

[0058] Figure 4 This is a schematic diagram of the tracking initialization phase of the multi-template fusion target tracking algorithm method based on graph attention network of the present invention;

[0059] Figure 5 This is a schematic diagram of the two-stage template update threshold judgment mechanism of the multi-template fusion target tracking algorithm method based on graph attention network of the present invention;

[0060] Figure 6 This is a schematic diagram of the template update module (M module) of the multi-template fusion target tracking algorithm method based on graph attention network of the present invention;

[0061] Figure 7 This is a tracking accuracy curve of the multi-template fusion target tracking algorithm method based on graph attention network of the present invention;

[0062] Figure 8 This is a tracking success rate curve of the multi-template fusion target tracking algorithm method based on graph attention network of the present invention;

[0063] Figure 9 This diagram illustrates the tracking performance of the multi-template fusion target tracking algorithm based on graph attention networks according to the present invention.

[0064] This invention discloses a multi-template fusion target tracking algorithm based on graph attention networks, the flowchart of which is shown below. Figure 1 As shown, please follow these steps:

[0065] Step 1: Construct the overall network and train the overall network structure end-to-end;

[0066] In step 1, the overall network structure is divided into four parts: the first part is a Siamese neural network for deep feature extraction, i.e., the feature extraction network; the second part is a feature embedding module and a template update module based on graph attention network, i.e., the GM module; the third part is a cross-correlation network for accurately locating the target through the feature map, i.e., the target location prediction module; and the fourth part is a two-stage template update threshold judgment mechanism for judging whether the current result is reliable, i.e., the threshold judgment module.

[0067] The feature extraction network is the Alex-Net deep feature extraction network. End-to-end training of the overall network structure is performed using meta-learning (one-shot learning). Four video frames are randomly selected from each dataset video to form an image pair. The first three images are used as the target image, and the last image is used as the search image to train the network. Ten image pairs are selected from each video, and each image pair contains four video frames. The first frame is the first frame of the video, and the next three images are randomly selected from the video. The interval between the second and third images is no more than 15 frames, and the interval between the third and fourth images is no more than 10 frames. The first three frames are used as the search image to synthesize the tracking template, and the last frame is used as the search image. In the search image processing, the three images imported into the 3D convolutional update module are identical, all being the last image of the image pair. Training is performed 50 times, with a batch size of 8. 32 image pairs are imported at a time, and the loss function is the Logistic loss function.

[0068] In step 1, generating training image pairs requires data augmentation of the selected images. The specific steps for data augmentation are as follows:

[0069] Step 1.1: First, perform a random stretch operation on the selected samples in the training set. The size of the stretched samples is set to be between 0.095 and 1.005. Linear interpolation is used to fill the parts that need to be filled after magnification. Then, perform a center crop operation, that is, crop a 263*263 area from the center of the training image pair. Next, perform a random crop operation, that is, crop a 255*255 area from a random position in the training image pair. Finally, perform a crop transformation. The BOX of the original GOT-10K dataset image, that is, the target location box, is given in the form of (left, top, weight, height), that is, the distance of the target box from the left and top edges of the image and the width and height of the target box itself. Through the crop transformation operation, the target box coordinates are transformed into (y, x, h, w), that is, the coordinates of the center point and the height and width of the target box.

[0070] Step 1.2: Calculate the loss:

[0071] The loss function formula used is:

[0072] L(y, v) = log(1 + exp(-yv))

[0073] In the formula, v is the true score of the sample, y∈{+1,-1} is the label of the ground truth, and L is the loss of the sample.

[0074] Step 1.3: After calculating the loss, use gradient descent to optimize the parameters. The formula for calculating the stochastic gradient descent (SGD) method is as follows:

[0075] argmin θ E (z，x，y) L (y，f(z，x；θ))

[0076] The optimal parameter θ can be obtained in this way; after 50 training iterations, the network's final loss value stabilizes below 0.1, and the training process ends.

[0077] Furthermore, step 2 is detailed as follows:

[0078] Step 2.1: Specify the target's location on the first frame of the video image sequence. Extract the target from the image and scale it to obtain a target image of size 127*127*3. Then, input the target image into the feature extraction network of the overall network to obtain features at layers P2, P3, P4, and P5. Use the P5 layer features as high-level features in the regression branch as the initial template for the regression branch. Use the P2 layer features as low-level features in the classification branch as the initial template for the classification branch. The initial template size is 6*6*256 pixels. Where z is the input target image, and the function... Representative feature extraction network.

[0079] Step 2.2: Initialize the parameters:

[0080] In the first frame of a video image sequence, the target position information given by manual calibration is called BOX. BOX has four pieces of information: the target's horizontal coordinate, vertical coordinate, width, and height. Therefore, no tracking calculation is required in the first frame. It is only necessary to set the corresponding initial center coordinates of the target and the initial width and height to the values ​​within the given BOX to complete the target initialization process.

[0081] Furthermore, step 3 is detailed below:

[0082] Step 3.1: First, extract deep features:

[0083] Using the target coordinates in the previous frame of the image / video sequence as the center, the search region is extracted and cropped into a patch, which is then fed into the feature extraction network to extract the depth features of the search region.

[0084] Step 3.2: Feature embedding and template update based on graph attention network:

[0085] Step 3.2.1, Feature Embedding:

[0086] The initial template Z-initial and the current template Z-current are fed into module G. First, the attention matrix f1 is calculated using the following formula:

[0087] f1=(W i Z i ) T ⊙(W c Z c )

[0088] Where ⊙ represents the matrix dot product operation, W i and W c This represents the linear transformation matrix. After obtaining the attention matrix f1, it is normalized using Softmax to balance the information content. Then, the aggregate feature f2 is calculated, with the specific calculation formula as follows:

[0089] f2 = Softmax(f1) ⊙(Relu(W j Z i ))

[0090] Among them W j Let represent the linear transformation matrix. Finally, the latest current template Z_attention, incorporating the attention to the initial template, is obtained using the following formula:

[0091] Z_atteneion=Relu(f2||(Relu(W d Z c )))

[0092] Where || represents the concatnate operation, W j This represents a linear transformation matrix.

[0093] Step 3.2.2, Template Integration:

[0094] Once the current template Z_attention, which has attention to the initial template, is obtained, is passed into the M module for multi-template fusion operation;

[0095] In the first frame template initialization stage, this paper assigns the initial template to the current template Zp, the historical template Zq and the initial template Z-initial, and stacks the three together and passes them through a 3*3*3 3D convolutional layer to obtain the tracking template Z;

[0096] Step 3.2.3: Calculate the tracking results:

[0097] Finally, the response graph is obtained by cross-correlation between the tracking template Z and the search graph X. The specific calculation formula is as follows:

[0098]

[0099] Where * represents the cross-correlation operation, and bII represents the value corresponding to each position.

[0100] Furthermore, the two-stage template update threshold determination mechanism in step 4 is implemented as follows:

[0101] Phase 1: By calculating the Pearson correlation coefficient (PCCs) between the current tracking result and the initial template and comparing it with a pre-set threshold, it is determined whether the tracking result is the target. The formula for calculating PCCs is as follows:

[0102]

[0103] Where i represents the i-th sample. and σ is the sample mean. x and σ y The standard deviation of the sample is used; if both PCCs are greater than the threshold, the second stage begins; APCE and the maximum response value F-max are used as the main criteria to determine whether the current tracking result is reliable; the definition formula for APCE is as follows:

[0104]

[0105] Where R max and R min Let R represent the maximum and minimum values ​​on the score graph, respectively. w , represents the value in the w-th row and h-th column of the score graph; if both APCE and F-max are lower than the preset threshold, the template update is abandoned and the next tracking is started; if both are higher than the threshold, the current tracking result is sent to the GM module for template update.

[0106] Step 5: Repeat steps 3 to 4 until all video image sequences have been tracked, thereby obtaining the location of the target in each frame of the video, and the tracking task ends.

[0107] Step 5 is as follows:

[0108] For a tracking task, the overall process after initialization is as described in steps 3-4, continuously repeating between template updates and tracking calculations. During this process, the target position for each frame of the video sequence is calculated and represented by a box. Looking at the entire video, the target's motion trajectory is obtained. The tracking task ends when the target positions for all images in the entire video sequence are obtained. The accuracy and success rate of this method on the test set are shown in the appendix. Figure 7 , 8 See attached chart for a comparison of tracking results. Figure 9The results show the tracking results of our method and the benchmark tracker SiamFC on four video sequences. The red box represents the ground truth, the blue box represents the tracking result of our method, and the green box represents the tracking result of SiamFC.

[0109] The foregoing description illustrates and describes several preferred embodiments of the present invention. However, as previously stated, it should be understood that the invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the inventive concept described herein through the foregoing teachings or techniques or knowledge in related fields. Any modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the invention should be within the protection scope of the appended claims.

Claims

1. A multi-template fusion target tracking algorithm based on graph attention networks, characterized in that, The specific steps are as follows: Step 1: Construct the overall network and train the overall network structure end-to-end; Step 2: Use the overall network trained in Step 1 to perform initial tracking processing on the video image sequence to be tracked, and obtain the initial target template and the initial position information of the target for this tracking task; Step 3: After obtaining the initial target template from Step 2, the tracking process begins. For each frame of the video image sequence, the position of a target in the image is calculated and displayed at the corresponding location in the image, resulting in a tracking response map. Step 3 is as follows: Step 3.1: First, extract depth features: Using the target coordinates in the previous frame of the image / video sequence as the center, extract the search area, and crop the search area into a patch and pass it to the feature extraction network to extract the depth features of the search area. Step 3.2: Feature embedding and template update based on graph attention network: Step 3.2.1, Feature Embedding: The initial template Z-initial and the current template Z-current are fed into the G module. First, the attention matrix is ​​calculated. The specific calculation formula is as follows: in This is a matrix dot product operation. and Represent the linear transformation matrix; obtain the attention matrix. Next, the data is normalized using Softmax to balance the information content; then, aggregated features are calculated. ， The specific calculation formula is as follows: in The linear transformation matrix is ​​represented, and the latest current template Z_attention, which incorporates the attention of the initial template, is finally obtained through the following formula: in For concatnate operations, Represents a linear transformation matrix; Step 3.2.2, Template Fusion: After obtaining the current template Z_attention which has attention to the initial template, pass it to the M module for multi-template fusion operation; In the first frame template initialization stage, this paper assigns the initial template to the current template Zp, the historical template Zq and the initial template Z-initial, and stacks the three together and passes them through a 3*3*3 3D convolutional layer to obtain the tracking template Z; Step 3.2.3: Calculate the tracking results: Finally, the response graph is obtained by cross-correlation between the tracking template Z and the search graph X. The specific calculation formula is as follows: in Indicates cross-correlation operation. bII This represents the value corresponding to each position; Step 4: After obtaining the tracking result response map from Step 3, use a two-stage template update threshold judgment mechanism to determine whether the current tracking result is reliable. If it is reliable, update the template; if it is unreliable, do not update the template. If there are two reliable tracking results retained in Step 3, replace the oldest result with the latest result. Step 5: Repeat steps 3 to 4 until all video image sequences have been tracked, thereby obtaining the location of the target in each frame of the video, and the tracking task ends.

2. The multi-template fusion target tracking algorithm based on graph attention network according to claim 1, characterized in that, In step 1, the overall network structure is divided into four parts: the first part is a Siamese neural network for deep feature extraction, i.e., the feature extraction network; The second part is a feature embedding module and a template update module based on graph attention network, namely the GM module; the third part is a cross-correlation network for accurately locating targets through feature maps, namely the target location prediction module; and the fourth part is a two-stage template update threshold judgment mechanism for judging whether the current result is reliable, namely the threshold judgment module.

3. The multi-template fusion target tracking algorithm based on graph attention network according to claim 2, characterized in that, In step 1, the feature extraction network is the Alex-Net deep feature extraction network.

4. The multi-template fusion target tracking algorithm based on graph attention network according to claim 3, characterized in that, In step 1, the end-to-end training of the overall network structure is specifically performed as follows: using meta-learning one-shot learning, four video frames are randomly selected from each dataset video to form an image pair. The first three images are used as the target image, and the latter are used as the search image to train the network. Ten image pairs are selected from each video, and each image pair contains four video frames. The first frame is the first frame of the video, and the last three images are randomly selected from the video. The interval between the second and third images is no more than 15 frames, and the interval between the third and fourth images is no more than 10 frames. The first three frames are used as the search image to synthesize the tracking template, and the last frame is used as the search image. In the search image processing, the three images imported into the 3D convolutional update module are the same, all of which are the last images of the image pair. The training is performed 50 times, the batch size is set to 8, and 32 image pairs are imported at one time. The loss function is the Logistic loss function.

5. The multi-template fusion target tracking algorithm based on graph attention network according to claim 4, characterized in that, In step 1, the training image pairs are generated, and data augmentation needs to be performed on the selected images. The data augmentation is specifically implemented according to the following steps: Step 1.1: First, perform a random stretch operation on the selected samples in the training set. The size of the stretched samples is set to be between 0.095 and 1.

005. Linear interpolation is used to fill the parts that need to be filled after magnification. Then, perform a center crop operation, that is, crop a 263*263 area from the center of the training image pair. Next, perform a random crop operation, that is, crop a 255*255 area from a random position in the training image pair. Finally, perform a crop transformation. The BOX of the original GOT-10K dataset image, that is, the target location box, is given in the form of (left, top, weight, height), that is, the distance of the target box from the left and top edges of the image and the width and height of the target box itself. Through the crop transformation operation, the target box coordinates are transformed into (y, x, h, w), that is, the coordinates of the center point and the height and width of the target box. Step 1.2: Calculate the loss: The loss function formula used is: In the formula It is the true score of the sample. It's a tag for ground truth. The loss for the sample; Step 1.3: After calculating the loss, use gradient descent to optimize the parameters. The formula for calculating the stochastic gradient descent (SGD) method is as follows: This allows us to obtain the optimal parameters. The training process ends when the network's final loss is stable below 0.1 after 50 training iterations.

6. The multi-template fusion target tracking algorithm based on graph attention network according to claim 5, characterized in that, Step 2 is as follows: Step 2.1: Specify the target's location on the first frame of the video image sequence. Extract the target from the image and scale it to obtain a target image of size 127*127*3. Then, input the target image into the feature extraction network of the overall network to obtain features at layers P2, P3, P4, and P5. Use the P5 layer features as high-level features in the regression branch as the initial template for the regression branch. Use the P2 layer features as low-level features in the classification branch as the initial template for the classification branch. The initial template size is 6*6*256 pixels. ,in Given the target image as input, the function Representative feature extraction network; Step 2.2: Initialize the parameters: In the first frame of a video image sequence, the target position information given by manual calibration is called BOX. BOX has four pieces of information: the target's horizontal coordinate, vertical coordinate, width, and height. Therefore, no tracking calculation is required in the first frame. It is only necessary to set the corresponding initial center coordinates of the target and the initial width and height to the values ​​within the given BOX to complete the target initialization process.

7. The multi-template fusion target tracking algorithm based on graph attention network according to claim 6, characterized in that, Step 4, the two-stage template update threshold determination mechanism, is performed as follows: Phase 1: By calculating the Pearson correlation coefficient (PCCs) between the current tracking result and the initial template and comparing it with a pre-set threshold, it is determined whether the tracking result is the target. The formula for calculating PCCs is as follows: in This represents the i-th sample. and This is the sample mean. and The standard deviation is the sample standard deviation. If both PCCs are greater than the threshold, the second stage begins; APCE and the maximum response value F-max are used as the basis for judgment to determine whether the current tracking result is reliable; the definition formula for APCE is as follows: in and These represent the maximum and minimum values ​​on the score graph, respectively. This represents the value in the w-th row and h-th column of the score graph. If both APCE and F-max are lower than the preset threshold, the template update is abandoned and the next tracking is initiated. If both are higher than the threshold, the current tracking result is sent to the GM module for template update.

Images