A real-time single-target long-time tracking method and system

Abstract

The application provides a real-time single-target long-time tracking method, comprising the following steps: step 1, obtaining a current image frame, selecting a template in a template library to determine a tracked target; matching the image frame with the tracked target to detect whether the target exists in the image frame, if the target exists, entering step 2, if the target does not exist, re-detecting; step 2, tracking by using an FDSSST target tracking algorithm, when the output confidence is greater than or equal to a set target confidence threshold value, considering that correct tracking is achieved, entering step 3; otherwise, considering that tracking fails, entering step 1 to detect a next image frame; step 3, judging whether deviation occurs according to the feature similarity of the target in adjacent two image frames, if the deviation does not occur, updating the target template in the template library, otherwise, re-selecting the template to re-detect the target in the current image frame. The application combines a deep learning algorithm with a filter algorithm, improves the accuracy and success rate of target tracking, and also guarantees the timeliness of tracking.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing and target tracking, and particularly relates to a real-time single-target long-time tracking method and system. BACKGROUND

[0002] Target tracking, as an important task in image processing, aims to use the initial position of the tracked target to stably predict the position and size of the target in subsequent image sequences. Unlike traditional short-term target tracking tasks, long-term target tracking tasks not only have the ability to capture the tracked target in image sequences, but also need to have the ability to handle frequent disappearance and reappearance of the target. Although great progress has been made in target tracking methods in recent years, many factors such as deformation, occlusion, out-of-view, scale change, in-plane rotation, etc. can greatly affect the tracking accuracy. In addition, on terminals with limited computing power, the current target tracking algorithm cannot meet the high standards of accuracy and real-time performance required by actual scenes.

[0003] Currently, the mainstream target tracking methods mainly fall into two categories: deep learning-based target tracking algorithms and correlation filter-based target tracking algorithms. Under limited computing resources, deep learning-based target tracking algorithms generally cannot meet the real-time requirements of actual scenes. In addition, deep learning-based target tracking algorithms require a large amount of data for training, and it is difficult to collect enough target samples for model training in some scenarios. Compared with deep learning-based target tracking algorithms, correlation filter-based target tracking algorithms, although with relatively low accuracy, are faster and more suitable for applications with limited computing power. Fast discriminative scale space tracking (FDSST) is an excellent correlation filter-based target tracking algorithm that designs two consistent correlation filters: a position filter and a scale filter. The former performs target positioning, and the latter performs target size scale estimation. Researchers have proposed a series of algorithm frameworks based on the FDSST algorithm to overcome the influence of factors such as occlusion, deformation, scale change, in-plane rotation, and out-of-view on tracking accuracy.

[0004] Document [1] Jiang Rui. FDSST infrared target tracking method based on spatiotemporal context similarity [D]. Harbin University of Science and Technology, 2021. Combines the FDSST tracking algorithm with the spatiotemporal context tracking algorithm STC to improve tracking accuracy. In addition, the algorithm uses the SUSAN corner point and the hierarchical clustering improved optical flow method to reposition the lost target. The repositioning algorithm has low detection accuracy for targets with fuzzy boundaries, and the entire algorithm is time-consuming.

[0005] Document [2] Zhang R, Wang J, Yang B, et al. Long-term target tracking algorithm based on adaptive model update [J]. Command Control and Simulation, 2021, 43(2): 25-32. On the basis of FDSST algorithm, the historical peak value mean and historical APCE confidence mean are used to judge the occlusion, and the template is updated adaptively; for the lost target, the algorithm uses online CUR filter for redetection. The CUR filter in the algorithm has low detection accuracy for the large deformation of the reappearing target.

[0006] Document [3] Xie W, Zhao T. Multi-feature adaptive fusion correlation filter target tracking algorithm [J]. Signal Processing, 2021, 37(4): 603-615. In the FDSST algorithm, gradient histogram features, color features CN and gray features are fused to enhance the expression ability of the features; and a occlusion judgment strategy is proposed to judge the target occlusion phenomenon in the tracking process. In addition, the method realizes the repositioning of the lost target by increasing the target candidate region in the video frame of the target occlusion. The target repositioning method in the algorithm is limited by the number of candidate boxes, and the detection accuracy of the lost target is low. SUMMARY

[0007] In view of the problems in the prior art, a real-time single target long-time tracking method and system are provided, which can effectively solve the problems of low detection capability and poor real-time performance of the existing target tracking method.

[0008] The technical scheme adopted by the present application is as follows: a real-time single target long-time tracking method, comprising:

[0009] Step 1, acquiring a current image frame, selecting a template in a template library to determine a tracked target; matching the image frame with the tracked target to detect whether the target exists in the image frame, if the target exists, determining the target position and entering step 2, if the target does not exist, selecting the template to continue matching the next image frame;

[0010] Step 2, based on the target position, using the FDSSST target tracking algorithm to track the target, when the FDSST output confidence is greater than or equal to a set target confidence threshold, considering that the tracking is correct, continuing to track and entering step 3; otherwise, considering that the tracking fails, selecting the same template to enter step 1 for matching detection of the next image frame;

[0011] Step 3, judging whether deviation occurs in real time according to the feature similarity of the tracked target in adjacent two image frames, if no deviation occurs, updating the target template in the template library, otherwise selecting the same template to re-match the target in the current image frame and entering step 1.

[0012] Further, in step 1, the matching of the image frame and the tracked target is completed by using an attention quality perception model, and the specific process is as follows:

[0013] Step 1.1, a light-weight convolutional neural network is used to extract multi-scale features of the image, and the features are scaled to the same scale;

[0014] Step 1.2, the correlation between different channels and spatial regions is modeled by using channel attention mechanism and spatial attention mechanism in turn, the importance of each feature channel and different image regions is obtained, so as to give different weight coefficients to the features, and finally generate the feature map;

[0015] Step 1.3, the matching of the image frame and the final feature map of the tracked target in the template is completed by using the quality perception template matching algorithm, and the detection of the tracked target is completed.

[0016] Further, in step 1.2, the channel attention mechanism takes the features obtained in step 1.1 as input, uses a spatial-based global average pooling operation to compress each feature channel into a value, and then performs feature re-labeling through an activation function to strengthen important feature maps and obtain channel attention weights. Finally, the channel attention weights are multiplied with the input features to generate a feature map.

[0017] Further, in step 1.2, the spatial attention mechanism takes the feature map generated by the channel attention mechanism as input, performs a global average pooling operation based on the channel, and then performs normalization processing through an activation function to generate spatial attention weights. The spatial attention weights are multiplied with the input feature map to generate a final feature map.

[0018] Further, in step 1.3, the specific process of matching is as follows:

[0019] Step 1.3.1, the likelihood function value L(t|s) of the image block feature of the tracked target in the image frame is calculated, and the likelihood function value L(s|t) of the image block feature of the image frame in the tracked target is calculated;

[0020] Step 1.3.2, the product Q(s,t) of the likelihood function value L(t|s) and the likelihood function value L(s|t) is calculated, which is used to evaluate the matching quality;

[0021] Step 1.3.3, the best matching quality value is determined, and if the best matching quality value is greater than the set matching threshold, it means that the target is detected.

[0022] Further, in step 1.3.1, the likelihood function value

[0023] The likelihood function value

[0024] wherein s', t' represent any one image block belonging to image frame or tracked target respectively, T is the tracked target, t is the current image block of the tracked target, S is the image frame, s is the current image block of the image frame, f t and f s are the final feature maps of t and s respectively, ρ(.) is a criterion measuring the similarity between the tracked target and the current image frame, and α is a temperature parameter.

[0025] Further, the optimal matching quality value is: q(s)=max(Q(s,t)|t∈T).

[0026] Further, in step 3, the specific method for judging whether deviation occurs is:

[0027] Step 3.1, obtaining the target region in the adjacent two image frames, and scaling to the same scale at the same time;

[0028] Step 3.2, extracting the features of the target region in the two image frames by using the gradient direction histogram algorithm;

[0029] Step 3.3, calculating the feature similarity between the two by using the cosine similarity;

[0030] Step 3.4, if the feature similarity is greater than or equal to the set similarity threshold, it indicates that no deviation occurs, otherwise it indicates that deviation occurs.

[0031] The application further provides a real-time single-target long-time tracking system, comprising:

[0032] a detection module, which obtains a target template from a template library, determines a tracked target, and matches the tracked target with an image frame, continuously detects whether the target exists in the image frame, and outputs the target position when the target exists;

[0033] a tracking module, which establishes a tracking model based on the FDSST algorithm, tracks according to the target position, and continuously tracks the target in subsequent image sequences;

[0034] an online learning module, which judges the similarity of the target features in adjacent image frames in real time during the tracking process, and updates the target template in the template library in real time when the similarity is greater than a threshold value;

[0035] a template library, which is used to provide the target template.

[0036] Further, the detection module comprises:

[0037] a feature extraction module, which is realized based on a pre-trained lightweight convolutional neural network, and is used to extract the multi-scale features of the image and scale to the same scale;

[0038] The channel attention module uses the feature proposed by the input feature extraction module, compresses each feature channel into a value by using a spatial-based global average pooling operation, and then uses an activation function to complete feature recalibration to strengthen important features, obtains channel attention weights, and performs multiplication operation on the weights and the input features to generate a feature map.

[0039] The spatial attention module uses the feature map generated by the channel attention module, performs a channel-based global average pooling operation, and then performs normalization processing by using an activation function to generate spatial attention weights, and performs multiplication operation on the spatial attention weights and the input feature map to generate a final feature map.

[0040] The quality-aware template matching module generates respective final feature maps of the tracked target image and the image frame by using the feature extraction module, the channel attention module and the spatial attention module, respectively, calculates the product of the matching likelihood function values of the two, and when the maximum value of the product is greater than a set matching threshold, it is determined that the tracked target is detected.

[0041] Compared with the prior art, the beneficial effects of the above technical solutions are that the deep learning algorithm and the filter algorithm are combined to fully exert the advantages of both, not only improving the accuracy and success rate of target tracking, but also greatly guaranteeing the timeliness of tracking. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 The flow chart of the real-time single-target long-time tracking method proposed by the application.

[0043] Figure 2 The schematic diagram of the detection module in an embodiment of the application.

[0044] Figure 3 The schematic diagram of the channel attention module in an embodiment of the application.

[0045] Figure 4 The schematic diagram of the spatial attention module in an embodiment of the application.

[0046] Figure 5 The schematic diagram of the online learning process in an embodiment of the application.

[0047] Figure 6 The matching quality thermograph corresponding to different temperature coefficients in an embodiment of the application.

[0048] Figure 7 The accuracy rate under different similarity thresholds in an embodiment of the application.

[0049] Figure 8(a) is the precision comparison curve of OTB50 in an embodiment of the application.

[0050] Fig. 8(b) is a contrast curve of OTB50 success rate in an embodiment of the present application. DETAILED DESCRIPTION

[0051] Embodiments of the present application are described in detail below with reference to examples illustrated in the accompanying drawings, in which the same or similar numerals or symbols represent the same or similar modules or modules having the same or similar functions throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are for explanation of the present application only, and cannot be understood as limiting the present application. On the contrary, the embodiments of the present application include all changes, modifications and equivalents falling within the spirit and scope of the appended claims.

[0052] Embodiment 1

[0053] The prior art has the following two problems:

[0054] 1. Low detection capability after the reappearance of the disappeared target. The tracker based on correlation filtering cannot find the lost target when the target is lost and the target is not in the current region of interest. Some researchers use low-level feature descriptors to detect the reappearance of the disappeared target. Such algorithms lack high-level semantic description of the target and cannot well overcome the influence of factors such as deformation and occlusion. In addition, in order to improve the detection accuracy, some researchers use target detection algorithms based on deep learning to detect the reappearance of the disappeared target. Although such methods can effectively improve the detection capability, they are not suitable for scenarios where there is insufficient training data. In most practical application scenarios, it is very difficult to obtain sufficient target data.

[0055] 2. Poor real-time performance. The target detection algorithm based on deep learning has a relatively large model and consumes a large amount of computing resources. Such an algorithm often has poor real-time performance in implementing the reappearance detection of the disappeared target.

[0056] Based on this, as shown in the figure, the embodiment combines a lightweight deep learning model with a target tracking algorithm based on a correlation filter for application scenarios such as unmanned aerial vehicles with limited computing power, and proposes a real-time single target long-time tracking method, which can effectively solve the problems of low detection capability after the reappearance of the disappeared target and poor real-time performance in the existing target tracking method. The specific process is as follows: Figure 1 Step 1. Obtain the current image frame, select the template in the template library to determine the tracked target; match the image frame with the tracked target, detect whether the target exists in the image frame, if it exists, determine the target position and enter step 2, if it does not exist, select the template to continue matching the next image frame;

[0057]

[0058] ​Step 2, based on the target position, the FDSSST target tracking algorithm is used to track the target, and when the FDSST output confidence is greater than or equal to the set target confidence threshold, it is considered that the tracking is correct, the tracking is continued and step 3 is entered; otherwise, it is considered that the tracking fails, the same template is selected to enter step 1, and the target matching detection is performed on the next image frame;

[0059] Step 3, the deviation is judged in real time according to the feature similarity of the tracked target in the previous frame and the current frame image, if the deviation does not occur, the target template in the template library is updated, otherwise the same template is selected to re-perform the target matching detection on the current image frame, and step 1 is entered.

[0060] In this embodiment, as shown in Figure 2 the attention quality perception model specifically implements the target detection process in step 1 as follows:

[0061] Step 1.1, a lightweight convolutional neural network is used to extract multi-scale features of the image and scale them to the same scale;

[0062] Step 1.2, the channel attention mechanism and the spatial attention mechanism are used in turn to model the correlation between different channels and spatial regions, to obtain the importance of each feature channel and different image regions, so as to give different weight coefficients to the features, to strengthen important features and suppress unimportant features, and to generate the final feature map;

[0063] Step 1.3, the quality perception template matching algorithm is used to match the final feature map of the tracked target in the image frame and the template, to complete the detection of the tracked target.

[0064] In this embodiment, the lightweight convolutional neural network is implemented by using a pre-trained Mobilnet V2 network.

[0065] After extracting the features in this embodiment, the channel attention mechanism and the spatial attention mechanism are introduced to further optimize the features, specifically:

[0066] The channel attention mechanism is as shown in Figure 3 The input of the channel attention mechanism is the features extracted and scaled by the lightweight convolutional neural network, each feature channel is compressed into a value by using a spatial-based global average pooling operation, then a Sigmoid activation function is used to realize feature re-calibration, to strengthen important feature maps, to form corresponding channel attention weights, to perform multiplication operation on the obtained one-dimensional channel attention weights and the input features, and to generate the processed feature map.

[0067] The spatial attention mechanism is as shown in Figure 4As shown, the input thereof is the feature map generated by the channel attention mechanism, a channel-based global average pooling operation is performed on the input feature map, and then a spatial attention weight is generated through Sigmoid activation function normalization processing. The spatial attention weight and the input feature map are subjected to multiplication operation to generate a final feature map.

[0068] After determining the acquisition mode of the final feature map, target detection based on the tracked target and the image frame can be started. Specifically:

[0069] Suppose that the existing tracked target T and the current image frame S, t and s are the corresponding current image blocks of the two, s' and t' represent any one image block belonging to the image frame or the tracked target, and t and s are subjected to the aforementioned feature extraction, channel attention mechanism and spatial attention mechanism processing to obtain corresponding image features f t and f s , and p(.) is a standard for measuring the similarity between the tracked target and the current frame. Then, the likelihood function value of the matching image block t is:

[0070]

[0071] Wherein, a is a temperature parameter.

[0072] Correspondingly, the likelihood function value of the matching image block s is:

[0073] In this embodiment, Q(s, t) is defined as the product of the likelihood function value of s matching in T and the likelihood function value of t matching in S, which is used to evaluate the matching quality:

[0074] Q(s, t) = L(t|s)·L(s|t)

[0075] Then, the best matching quality is represented as: q(s) = max(Q(s, t)|t∈T)

[0076] When q(s) is greater than the set matching threshold, it is considered that the target is detected, and the best matching area is represented as:

[0077]

[0078] Wherein, R represents a candidate window of a fixed size, and r represents any one candidate window.

[0079] When it is determined that the target exists in the current image, the position thereof is acquired, and target tracking is started. The target tracking in the embodiment is implemented by using an FDSST target tracking algorithm, which is mainly composed of two cascaded processes of a position filter and a scale filter. When the FDSST output confidence is greater than or equal to a set target confidence threshold, it is roughly considered that the target is correctly tracked, so as to perform further online learning fine determination. Otherwise, it is considered that the target tracking fails, and a template is directly selected again for target re-detection.

[0080] As shown in Figure 5 , the embodiment proposes a process of online learning fine determination, and the specific method is as follows:

[0081] 1. A previous image frame and a current image frame are acquired, a target region is determined, and is scaled to the same scale.

[0082] 2. A gradient direction histogram algorithm is used to extract features of adjacent target regions.

[0083] 3. Cosine similarity is used to calculate the feature similarity between the two.

[0084] 4. If the feature similarity is greater than or equal to a set similarity threshold, it indicates that no deviation occurs, the latest target features are stored in a template library, the original template is updated, and tracking errors are avoided from accumulating. Otherwise, it indicates that a deviation occurs, and target detection needs to be performed again through an attention quality perception model. The tracking of the next frame is performed according to the detection result.

[0085] The embodiment proposes an attention quality perception model, which matches and identifies a tracked target in a to-be-detected image. The model extracts high-level semantic features by using a pre-trained deep learning model, strengthens the learning of important features by using an attention mechanism, and realizes the matching detection of the target by using a quality perception template matching. Then, a tracking algorithm performs target tracking, and an online learning module is established to detect possible deviations of the target, so as to continuously update the template of the tracked target and improve the detection capability of the algorithm on the reappearance of the lost target. The method combines the deep learning algorithm and the filter algorithm, fully gives play to the advantages of the two, improves the accuracy and success rate of target tracking, and greatly guarantees the timeliness of tracking.

[0086] In order to test the tracking method, OTB50 is selected as a target tracking test sequence. The data set is composed of 50 image sequences, including light changes, scale changes, object occlusions, background disturbances and rapid movements and various influencing factors. All experiments are performed in a Pytorch environment, and a RuiKaiWei rk3588 development board is used as a hardware platform.

[0087] (1) Model performance evaluation index

[0088] The experiment used the One-Time Pass Evaluation (OPE) criterion to calculate accuracy and success rate, and used these as evaluation criteria to measure algorithm performance. Specifically, accuracy (DP) is the number of frames N in which the tracking center point position error is less than a set threshold p. t With the total number of frames N sum The ratio (threshold set to p = 20 pixels) is mathematically calculated as follows:

[0089]

[0090] Success rate (OP) is the percentage of true bounding boxes B during the tracking process. gt And prediction box B t The overlap rate is greater than the set threshold T s The number of frames and the total number of frames N sum The ratio (threshold set to T) s =0.5), overlap rate S and success rate f suc The mathematical formulas are as follows:

[0091]

[0092]

[0093] In the formula I t This is an indicator function.

[0094]

[0095] (2) Model parameter selection

[0096] The parameters that need to be set in this embodiment include the temperature parameter α and the matching threshold T. q Target confidence threshold T c Similarity threshold T sim .

[0097] ① Determination of temperature coefficient α

[0098] The temperature coefficient directly affects the matching quality between the target and the image to be inspected. Figure 6 The target-matched mass thermograms are presented under different temperature coefficients. After analyzing and comparing a large number of experimental results, the final temperature coefficient value selected in this experiment is α = 250.

[0099] ② Matching threshold T q With the target confidence threshold T c .

[0100] This experiment used 200 images to test the attention quality perception algorithm. By analyzing the matching and tracking results, an empirical value T was obtained. q =0.01, T c =6.5.

[0101] ③similarity threshold T sim .

[0102] When the temperature parameter a, the matching threshold T q , the target confidence threshold T c is determined, in order to determine the value of the similarity threshold T sim , the experiment respectively sets T sim 0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9 and 1 for experiment, and the experimental results are shown in Figure 7 From Figure 7 , when T sim = 0.7, the highest accuracy is obtained, so the similarity threshold is set to T sim = 0.7.

[0103] (3) Result analysis

[0104] ①Comparison with FDSST

[0105] First, in order to verify the effectiveness of the method, it is compared with the original FDSST algorithm, and the results are shown in Figure 8. Among them, Figure 8(a) is the accuracy graph on the OTB50 dataset; Figure 8(b) is the success rate graph of the algorithm on the OTB50 dataset. From the figure, it can be seen that the accuracy and success rate of the tracking method proposed in the embodiment are greatly improved compared with the original FDSST algorithm, and the accuracy is improved by 15.8%, and the success rate is improved by 25.3%. This is because the attention quality perception model and online learning introduced in the embodiment can effectively overcome the influence of factors such as deformation, occlusion, field of view, scale change, in-plane rotation, etc.

[0106] ②Comparison with other methods

[0107] In order to comprehensively evaluate the tracking ability of the method proposed in the application for long-time target, comparative experiments are carried out with other several literature methods, and the experimental results are shown in Table 1. The best result is bold. Among them, literature [1] is the FDSST infrared target tracking method of spatiotemporal context similarity proposed by Jiang Rui; literature [2] is the long-time target tracking algorithm of adaptive model updating proposed by Zhang Rui et al.; literature [3] is the multi-feature adaptive fusion correlation filter target tracking algorithm proposed by Xie Weixin et al.

[0108] Table 1 Comparison of experimental results

[0109]

[0110]

[0111] From the experimental results of Table 1, it can be seen that compared with FDSST, the accuracy of literature [1] is increased by 4%, the success rate is increased by about 10%, but the tracking speed is greatly reduced. This is because the algorithm fuses FDSST and STC for template tracking and uses the SUSAN corner and the improved optical flow method of hierarchical clustering to reposition the target, so that the whole algorithm is very large and the operation is time-consuming. The accuracy of literature [2] is the lowest, which is 83.4%. The main reason is that the CUR filter is used to process the occlusion or tracking failure scene, and its reliability is low. Literature [3] uses multi-feature fusion technology in FDSST, which increases the target tracking accuracy by about 8%; and uses the fixed candidate frame to realize the lost target repositioning, which makes the positioning success rate low, only increased by 7.7%. The long-time target tracking algorithm proposed in the application achieves the best accuracy and success rate. This is because: the application can well overcome the influence of target occlusion, deformation, in-plane rotation and other factors by extracting high-level semantic features of the target through the attention quality perception algorithm; by introducing an online learning module, the template can be updated continuously to prevent tracking error accumulation; the NPU module on RK3588 is used to accelerate the operation of the attention quality perception model, so that the real-time performance of the application is better than other algorithms.

[0112] Embodiment 2

[0113] On the basis of embodiment 1, a real-time single target long-time tracking system is proposed, comprising:

[0114] A detection module acquires a target template from a template library, determines a tracked target, and matches with an image frame to continuously detect whether the target exists in the image frame, and outputs the target position when the target exists;

[0115] A tracking module establishes a tracking model based on the FDSST algorithm, tracks according to the target position, and continuously tracks the target in subsequent image sequences;

[0116] An online learning module judges the similarity of target features in adjacent image frames in real time during tracking, and updates the target template in the template library in real time when the similarity is greater than a threshold value;

[0117] A template library is used to provide a target template.

[0118] In this embodiment, the detection module comprises:

[0119] A feature extraction module is realized based on a pre-trained lightweight convolutional neural network, which is used to extract multi-scale features of an image and scale to the same scale;

[0120] The channel attention module uses the feature extracted by the input feature extraction module to compress each feature channel into a value by using a spatial-based global average pooling operation, and then uses an activation function to complete feature recalibration to strengthen important features, obtain channel attention weights, and perform multiplication operation between the weights and the input features to generate a feature map;

[0121] The spatial attention module uses the feature map generated by the channel attention module to perform a channel-based global average pooling operation, and then performs an activation function normalization process to generate spatial attention weights, and performs multiplication operation between the spatial attention weights and the input feature map to generate a final feature map.

[0122] The quality-aware template matching module generates respective final feature maps of the tracked target image and the image frame by the feature extraction module, the channel attention module and the spatial attention module, respectively, calculates the product of the matching likelihood function values of the two, and when the maximum value of the product is greater than a set matching threshold, it is determined that the tracked target is detected.

[0123] It should be noted that in the description of the embodiments of the present application, unless otherwise explicitly specified and limited, the terms "set", "connected" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be directly connected, or indirectly connected through an intermediate medium. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances; the drawings in the embodiments are used to clearly and completely describe the technical solutions in the embodiments of the present application, and obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various configurations.

[0124] Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and cannot be understood as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A real-time single-target long-time tracking method, characterized in that, Comprising: Step 1, obtaining a current image frame, selecting a template in a template library to determine a tracked target; Matching the image frame with the tracked target, detecting whether the target exists in the image frame, if it exists, determining the target position and entering step 2, if it does not exist, selecting a template to continue matching the next image frame; Step 2, based on the target position, using the FDSST target tracking algorithm to track the target, when the FDSST output confidence is greater than or equal to the set target confidence threshold, it is considered to be correct tracking, continuous tracking and entering step 3; Otherwise, it is considered that the tracking fails, the same template is selected to enter step 1, and the target matching detection is performed on the next frame image; Step 3, judging whether deviation occurs according to the feature similarity of the tracked target in the adjacent two frames of images in real time, if no deviation occurs, updating the target template in the template library, otherwise selecting the same template to re-perform the target matching detection on the current image frame, entering step 1; In step 1, the attention quality perception model is used to complete the matching of the image frame and the tracked target, and the specific process is as follows: Step 1.1, a lightweight convolutional neural network is used to extract multi-scale features of the image, and the features are scaled to the same scale; Step 1.2, the channel attention mechanism and the spatial attention mechanism are used in turn to model the correlation between different channels and spatial regions, obtain the importance of each feature channel and different image regions, and thus give different weight coefficients to the features, and generate the final feature map; Step 1.3, the quality perception template matching algorithm is used to match the final feature map of the tracked target in the image frame and the template, and the detection of the tracked target is completed; In step 1.3, the specific process of matching is as follows: Step 1.3.1, calculating a likelihood function value of matching the image block features of the tracked target in the image frame and calculating a likelihood function value of matching the image block features of the image frame in the tracked target ; Step 1.3.2, calculating the likelihood function value the product of the likelihood function value for evaluating the matching quality;​ Step 1.3.3, the best matching quality value is determined, if the best matching quality value is greater than the set matching threshold, it indicates that the target is detected; In step 1.3.1, the likelihood function value , Likelihood function value , wherein, , respectively denote an arbitrary image patch belonging to an image frame or a tracked target, T is a tracked target, t is a current image patch of a tracked target, S is an image frame, s is a current image patch of an image frame, f t and f s respectively t and s the final feature maps of The specific method for judging whether deviation occurs in step 3 is as follows: (.) is a criterion measuring the similarity of a tracked target and a current image frame, α is a temperature parameter.

2. The real-time single-target long-time tracking method according to claim 1, characterized in that, Step 3.1, obtaining the target region in the adjacent two frames of images, and scaling to the same scale at the same time; 3. The real-time single-target long-time tracking method according to claim 2, characterized in that, Step 3.2, using the gradient direction histogram algorithm to extract the features of the target region in the two frames of images; 4. The real-time single-target long-time tracking method according to claim 1, wherein, the best match quality value is: .

5. The real-time single-target long-time tracking method according to claim 1, wherein, Step 3.3, using the cosine similarity to calculate the feature similarity between the two; Step 3.4, if the feature similarity is greater than or equal to the set similarity threshold, it indicates that no deviation occurs, otherwise it indicates that deviation occurs. Comprising: ​ ​ 6. A real-time single-target long-time tracking system, characterized in that, ​ The detection module obtains a target template from a template library, determines a tracked target, and matches the tracked target with an image frame, continuously detects whether the target exists in the image frame, and outputs a target position when the target exists; The tracking module establishes a tracking model based on an FDSST algorithm, tracks the target according to the target position, and continuously tracks the target in subsequent image sequences; The online learning module judges the similarity of target features in adjacent image frames in real time during the tracking process, and updates the target template in the template library in real time when the similarity is greater than a threshold value; The template library is used to provide the target template; In the detection module, the matching of the image frame and the tracked target is completed by using an attention quality perception model, and the specific process is as follows: Step 1.1, a lightweight convolutional neural network is used to extract multi-scale features of the image and scale them to the same scale; Step 1.2, the channel attention mechanism and the spatial attention mechanism are used in turn to model the correlation between different channels and spatial regions, obtain the importance of each feature channel and different image regions, and thus give different weight coefficients to the features to generate the final feature map; Step 1.3, the quality perception template matching algorithm is used to match the final feature maps of the tracked target in the image frame and the template, and the detection of the tracked target is completed; In step 1.3, the specific process of matching is as follows: Step 1.3.1, calculating a likelihood function value of matching the image block features of the tracked target in the image frame and calculating a likelihood function value of matching the image block features of the image frame in the tracked target ; Step 1.3.2, calculating the likelihood function value the product of the likelihood function value for evaluating the matching quality;​ Step 1.3.3, the best matching quality value is determined, and if the best matching quality value is greater than a set matching threshold value, it indicates that the target is detected; In step 1.3.1, the likelihood function value , Likelihood function value , wherein, , respectively denote an arbitrary image patch belonging to an image frame or a tracked target, T is a tracked target, t is a current image patch of a tracked target, S is an image frame, s is a current image patch of an image frame, f t and f s respectively t and s final feature maps of The detection module comprises: (.) is a criterion measuring the similarity of a tracked target and a current image frame, α is a temperature parameter.

7. The real-time single-target long-time tracking system according to claim 6, wherein, The feature extraction module is realized based on a pre-trained lightweight convolutional neural network, and is used to extract multi-scale features of the image and scale them to the same scale; The channel attention module inputs the features output by the feature extraction module, compresses each feature channel to a value by using a spatial-based global average pooling operation, and then uses an activation function to complete feature re-labeling and strengthen important features to obtain channel attention weights, and performs multiplication operation on the weights and the input features to generate a feature map; The spatial attention module inputs the feature map generated by the channel attention module, performs a channel-based global average pooling operation, and then generates spatial attention weights through activation function normalization processing, and performs multiplication operation on the spatial attention weights and the input feature map to generate a final feature map; The quality perception template matching module generates respective final feature maps of the tracked target image and the image frame through the feature extraction module, the channel attention module and the spatial attention module, calculates the product of the matching likelihood function values of the two, and when the maximum value of the product is greater than a set matching threshold value, it is determined that the tracked target is detected. ​

Images