A feature fusion video target tracking method and system

Abstract

The application provides a video target tracking method and system based on feature fusion, and proposes a method combining a target detection algorithm and a correlation filter, a target detection frame of a target is recognized through the target detection algorithm, a target tracking edge frame is obtained through a local and global multi-feature fusion correlation filter, and the target detection frame and the target tracking edge frame are matched through a Hungarian matching algorithm, so that the problem of low accuracy and success rate of a target detection and tracking algorithm of a single correlation filter tracking algorithm or a neural network target detection algorithm is solved, target tracking stability in a complex scene such as a power scene is realized, and the real-time performance, accuracy and success rate of target detection and tracking are improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of target detection and tracking, and particularly relates to a feature fusion video target tracking method and system. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute the prior art.

[0003] Video target tracking refers to automatic identification of a target in subsequent frames after the target is specified in the first frame of a video sequence, continuous positioning of the target through a bounding box, and predictive scale change. The main challenge of video target tracking is complex changes of the target and the background in the target tracking process, including target disappearance, target occlusion, target deformation, background switching, target movement, etc., which leads to inconsistency of target tracking features such as appearance, color, shape, etc. in the time dimension in the video sequence, resulting in incorrect tracking of the tracker.

[0004] In recent years, correlation filtering and twin networks have become two major algorithm frameworks in the field of video target tracking. The correlation filtering algorithm framework has good recognition accuracy and extremely high running speed, and the extraction of reinforced features and the construction of high-robustness filters become the key improvement ideas. The famous CSK and KCF algorithms proposed by Henriques et al. The CSK algorithm is an extension of dense sampling and kernelized correlation filtering, and the KCF algorithm further increases multi-channel gradient features to improve the running speed of the algorithm. Danelljan et al. proposed multi-channel color features and used color space to improve the algorithm to obtain feature effects. Due to the superiority of deep convolution features, Danelljan et al. introduced single-layer convolution deep features of VGG network in DeepSRDCF algorithm, which greatly improved the target accuracy. Li et al. proposed SAMF algorithm, which performed multi-scale sampling on the detection region and used the filter to detect the target at each scale, and took the maximum response value as the estimation result.

[0005] To reduce the algorithm complexity and improve the detection speed, some scholars proposed one-stage detection algorithms such as SSD and YOLO. Jo et al. based on YOLO-v2 solved the problem of low real-time precision in multi-target detection. Kong et al. proposed FoveaBox model, which solved the problem of insufficient self-adaptation of multi-scale anchorbox in the detection network.

[0006] In the real-time monitoring of power operation, there are often complicated types of equipment within the construction range, high operation risk, and the need for real-time summary of the operation personnel, as well as high robustness to factors such as occlusion, scale transformation, rapid movement, and background transformation. Therefore, a target tracking method is proposed to meet the real-time tracking requirements of field operation personnel and construction vehicles in the power field under complex operation environment, and to improve the accuracy of detection and tracking, which is a problem that needs to be solved at present. SUMMARY

[0007] To overcome the shortcomings of the prior art, the present application provides a feature fusion video target tracking method and system, which combines target detection algorithms and correlation filtering to improve the stability of target tracking in complex scenes such as power scenes, and improve the real-time performance, accuracy and success rate of target detection and tracking.

[0008] To achieve the above purpose, the first aspect of the present application provides a feature fusion video target tracking method, comprising:

[0009] detecting the target in the obtained current video frame using a target detection algorithm to obtain a target detection frame;

[0010] extracting a first maximum response map and a second maximum response map from the obtained previous video frame through a local correlation filter and a global correlation filter, respectively;

[0011] obtaining a target tracking edge frame of the current video frame according to the first maximum response map and the second maximum response map;

[0012] matching the overlap degree of the target detection frame and the target tracking edge frame using a Hungarian matching algorithm to obtain the final target frame of the target.

[0013] The second aspect of the present application provides a feature fusion video target tracking system, comprising:

[0014] a target detection module configured to detect the target in the obtained current video frame using a target detection algorithm to obtain a target detection frame;

[0015] a correlation filtering module configured to extract a first maximum response map and a second maximum response map from the obtained previous video frame through a local correlation filter and a global correlation filter, respectively;

[0016] a target tracking module configured to obtain a target tracking edge frame of the current video frame according to the first maximum response map and the second maximum response map;

[0017] A matching module is configured to match the overlapping degrees of the target detection frame and the target tracking edge frame by using a Hungarian matching algorithm to obtain the final target frame of the target.

[0018] A third aspect of the present application provides a computer device, comprising: a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the computer device is running, the processor and the memory communicate through the bus, and the machine readable instructions are executed by the processor to perform a feature fusion video target tracking method.

[0019] A fourth aspect of the present application provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by the processor to perform a feature fusion video target tracking method.

[0020] The above one or more technical solutions have the following beneficial effects:

[0021] The present application creatively proposes a feature fusion video target tracking method and system, proposes a method combining a target detection algorithm and a correlation filter, identifies a target detection frame of a target through the target detection algorithm, obtains a target tracking edge frame through a local and global multi-feature fusion correlation filter, and matches the target detection frame and the target tracking edge frame by using a Hungarian matching algorithm, thereby solving the problem of low accuracy and success rate of a target detection and tracking algorithm of a single correlation filter tracking algorithm or a neural network target detection algorithm, realizing target tracking stability in a complex scene such as a power scene, and improving the real-time performance, accuracy and success rate of target detection and tracking.

[0022] The advantages of the additional aspects of the present application will be partially given in the following description, partially will become obvious from the following description, or will be known by the practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0023] The drawings accompanying the specification of the present application form a part thereof, serve to provide further understanding of the present application, and together with the description of the exemplary embodiments of the present application and their description serve to explain the present application, and do not constitute an improper limitation of the present application.

[0024] Figure 1 YOLO-v8 network structure in embodiment one of the present application;

[0025] Figure 2 CA module structure diagram in embodiment one of the present application;

[0026] Figure 3 Basic flow of correlation filtering in embodiment one of the present application;

[0027] Figure 4A flow chart of a multi-feature fusion related filtering tracking algorithm in the embodiment one of the present application;

[0028] Figure 5 A target detection edge frame and a target tracking edge frame in the embodiment one of the present application;

[0029] Figure 6 A detection and tracking fusion flow in the embodiment one of the present application;

[0030] Figure 7 An OPE precision map in the embodiment one of the present application;

[0031] Figure 8 An OPE success rate map in the embodiment one of the present application;

[0032] Figure 9 A jumping video frame sequence tracking comparison chart in the embodiment one of the present application;

[0033] Figure 10 A woman frequency frame sequence tracking comparison chart in the embodiment one of the present application. DETAILED DESCRIPTION

[0034] It should be noted that the following detailed description is exemplary in nature and is intended to provide further description of the present application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0035] It should be noted that the terms used herein are only for the purpose of describing the specific embodiments, and are not intended to limit the exemplary embodiments according to the present application.

[0036] In the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0037] Embodiment one

[0038] The present embodiment discloses a feature fusion video target tracking method, comprising:

[0039] Detecting the target in the obtained current video frame by using a target detection algorithm to obtain a target detection frame;

[0040] Extracting a first maximum response map and a second maximum response map from the obtained previous video frame by using a local correlation filter and a global correlation filter respectively;

[0041] According to the first maximum response map and the second maximum response map, a target tracking edge frame of the current video frame is obtained;

[0042] The overlap between the target detection box and the target tracking edge box is matched using the Hungarian matching algorithm to obtain the final target box.

[0043] The overall approach of the feature fusion video target tracking method proposed in this implementation is as follows: First, target features are extracted based on the YOLO algorithm. By adding a CA module, the neural network's ability to extract upstream feature information is effectively enhanced, solving the problem of information loss for small targets caused by target scale switching. Second, target tracking is performed using a correlation filtering algorithm. By combining the overlap and correlation between the target detection bounding box and the tracking bounding box, the Hungarian matching algorithm is used to optimally match the target detection and tracking results. The tracking results are used to correct the detection results, shortening the detection time and improving the network's detection capability.

[0044] In this embodiment, the YOLO series algorithms adopt the FPN+PAN framework, which is a lightweight detection network with a simple simulation structure and faster detection speed. The main idea is to segment the image into multiple grid blocks to be detected, predict the object category contained in each grid block by traversing it, and use a non-maximum suppression strategy to eliminate overlapping bounding boxes.

[0045] The YOLO-v8 algorithm is a modification of version v5, introducing the DPM network module to perform dense prediction through high-dimensional feature maps, retaining more feature information and improving detection accuracy. It also adds SGSC technology to dynamically adjust the relationship between different feature maps, thereby better acquiring multi-scale features of the detected target.

[0046] like Figure 1 The diagram shows the overall network structure of the existing YOLO-v8. The input side uses the Mosaic high-order data augmentation strategy and adds an adaptive image adjustment strategy. The backbone network consists of three modules: CBS, C2f, and SPPF. The C2f module has a richer gradient flow and can enhance the feature fusion effect of the convolutional network. The SPPF module achieves the fusion of local and global enhancement through spatial pooling. The head part is replaced with a decoupled head structure to separate classification and detection. In the loss function, DFLLOss+CIOU Loss is used as the regression loss. For the matching strategy, the Task-AlignedAssigner matching method is adopted.

[0047] like Figure 2As shown, the present embodiment is improved based on the YOLO-v8 algorithm, by adding a coordinate attention module CA at the end of the backbone network of the YOLO-v8 algorithm, enhancing shallow and fine-grained feature extraction, and improving detection accuracy. Compared with the traditional attention mechanism, the SE attention mechanism only pays attention to the direct dependence relationship of the channel and ignores the spatial features, and the CBAM attention mechanism has a high dependence on computing resources due to the large size of the convolution kernel and the large number of parameters. The CA attention mechanism can obtain channel information and spatial position information at the same time, can associate long-distance features in the horizontal and vertical directions, and realize the accurate pointing of the model to the target object of interest.

[0048] The basic process of correlation filtering is as shown in Figure 3 The original correlation filter is obtained using the initialization sample, and each subsequent frame is detected by extracting the image block from the target position region of the previous frame. The Fourier transform is used instead of the convolution operation to improve the calculation efficiency, and the position of the maximum response value is obtained as the target position. The target appearance of the current frame is extracted, and the correlation filter is updated according to the feedback of the model update strategy.

[0049] The and filter target tracking algorithm follows the basic process of correlation filtering, and adopts the ridge regression method, i.e., the least squares method with regularization, to predict the target position. The target is taken as the center, and the method of circular shift is used to select the m x n rectangular region of pixel blocks around the target. i (i∈{0,…m-1}×{0,…n-1}) as training samples to train the classifier The regularized least squares RLS classifier model is adopted, and the kernel function is introduced

[0050]

[0051] where, represents the mapping from the original input space to the hilbert feature space, y i is the known regression value corresponding to x i , and λ is the regularization parameter for controlling overfitting. At this time, the classifier weight W is solved.

[0052]

[0053] The optimization objective is converted to solving α={α1,α2,…} T , and the kernel function can be obtained as:

[0054] α=(K+λI) -1 y (3)

[0055] where I is the identity matrix; K is the kernel matrix:

[0056]

[0057] By using the circulant matrix and the discrete Fourier transform (DFT), the optimal solution of the kernelized regularized least squares is obtained as

[0058]

[0059] where F denotes the discrete Fourier transform, k xx k(x, x) is the output of the kernel k, and the Gaussian kernel is used as the kernel function, and the calculation formula is

[0060]

[0061] In the next target frame, the position of the moving target is predicted by using the trained classifier, that is, the position of the maximum classifier response value is found in all test samples in the search region, and the candidate window image block Z can obtain the output response of the classifier as follows:

[0062]

[0063] The maximum response position is the predicted position of the target.

[0064] wherein and respectively represent the target appearance template and the classifier parameters learned, and the linear interpolation method is generally used to update the parameters as follows:

[0065]

[0066]

[0067] wherein n is the sequence number of the current frame, and γ is the learning rate.

[0068] Multi-feature fusion can enrich the appearance model and improve the stability of the algorithm in a complex scene, and therefore a feature fusion method is proposed in the embodiment, in which the local feature and the global feature are reasonably weighted and fused at the decision level, so as to improve the tracking performance.

[0069] The local feature selects the HOG feature, which has a good description of the image contour and shape, and the global feature selects the color feature, which has good stability for target shape deformation and motion blur, and the two features are independent and complementary to each other, so that the target can be effectively represented and stable feature description is provided for tracking.

[0070] Supposing that the current frame is I and the related filter function is f, the target response function f(t hog ) corresponding to the HOG feature and the target response function f(t color ) corresponding to the color feature can be obtained, and the final target maximum response value max(f i (t)) can be obtained after weighting.​

[0071] max(f i (t))=λ” hog max(f i (t hog ))+λ” color max(f i (t color )) (10)

[0072] where λ” hog is the set weight of HOG feature, set according to demand, λ” color is the set weight of color feature, set according to demand; the position of the maximum response value is the position of the target. When the target position is determined by the maximum response value, the next frame target is framed by a target tracking edge box with a size of twice the size of the target size of the previous frame, and then the scale adjustment of the target box is performed.

[0073] As Figure 4 shown, in the embodiment, a one-dimensional scale-dependent filter is trained using HOG features for the scale change problem in the tracking process. A candidate region of different scales is collected with the center position of the current detected center as the center point, and the scale selection method is as follows:

[0074]

[0075] where P and R are the width and height of the previous frame respectively, a = 1.02 is the scale factor, and S = 33 is the number of scales. Therefore, 33 samples of different scales can be obtained for each frame. a n P and a n R are exponential functions with a scale coefficient as the base number greater than 1, so the scale growth of the 33 samples collected is nonlinear. The larger the scale level compared to the current scale, the larger the step; the smaller the scale level compared to the current scale, the smaller the step, that is, coarse detection is performed on larger scales, and fine detection is performed on smaller scales.

[0076] Specifically, first, a series of image blocks x sl of different scales are collected around the detected target, 31-dimensional HOG features in each image block of different scales are extracted, d represents the dimension of the HOG feature, the corresponding Gaussian function label is y sl , γ S is the learning rate, and F is the discrete Fourier transform. The RLS classifier is trained using these image blocks, and is solved in a linear space to obtain a one-dimensional scale-dependent filter. Scale detection is completed by finding the maximum output response of the filter to obtain a scale-dependent template B num , and the calculation and update formula of B den is as follows:

[0077]

[0078]

[0079] Final solution output response value f s (x si ):

[0080]

[0081] Wherein, λ is a non-zero minimum value, i represents an image block; the scale of the sampling block with the maximum value of max(f s (x sl )) is obtained as the scale of the target in the new frame.

[0082] The multi-feature fusion correlation filter tracking algorithm process is as follows: first, the initial frame of the target is given by manual calibration, and the local and global correlation filters of the HOG feature and color feature are trained through the first frame. The first maximum response map and the second maximum response map are obtained through the local and global correlation filters respectively, the weighted average of the target center positions determined based on the first maximum response map and the target center positions determined based on the second maximum response map is obtained, and the final target center position is determined. Then, the scale filter is used to realize scale updating, and the model of the correlation filter is updated.

[0083] The YOLO-v8 target detection algorithm and the multi-feature fusion correlation filter tracking algorithm output target detection edge frames and target tracking edge frames respectively, and the Hungarian algorithm is used to match the IOU overlapping areas of the detection edge frames and the tracking edge frames. After screening in three ways, the target with the minimum cost matrix is replaced as the target being tracked.

[0084] The greater the correlation between the target detection edge frame and the tracking target edge frame, the better the current detection effect.

[0085]

[0086] Wherein, S ABCD is the area of the target detection frame, S EFGH is the area of the target tracking edge frame, and S EHDP is the overlapping area of the target detection frame and the target tracking edge frame.

[0087] The Hungarian matching algorithm establishes the matching relationship between the target detection edge frame and the target tracking edge frame according to the prior relationship of IOU:

[0088]

[0089] i is the number of matching pairs of target detection edge boxes and target tracking edge boxes, d i is the target detection edge box in the ith matching pair, t i is the target tracking edge box in the ith matching pair.

[0090] Formula (15) should satisfy IOU≤threash at the same time.

[0091] The embodiment uses YOLO-v8 to improve the target detection algorithm to realize target detection, and uses the kernel filtering algorithm based on multi-feature fusion to realize target tracking. The target detection edge box is taken as the reference, and the target tracking edge box is taken as the correction, so that the target detection result is more accurate. The working process of the detection model fused with the tracking algorithm is as shown in Figure 6

[0092] The embodiment uses the following experiment to verify the effectiveness of the method: 9 video sequences are selected from the standard test videos in the video tracking benchmark video library (Visual Tracker Benchmark) for testing. These video sequences cover common fast motion, rotation, occlusion, background interference and other problems in the tracking process. All experiments use the standard initial box size and position provided by the benchmark library to maintain the same initial conditions as all comparison algorithms. The comparison tracking algorithms include: compressive tracking algorithm (Compressive Tracking, CT); online real-time tracking algorithm (Real-Time Tracking via On-line Boosting, OAB); long target tracking algorithm (Tracking-Learning-Detection, TLD); locally orderless tracking (Locally Orderless Tracking, LOT).

[0093] The above 9 groups of video sequences are respectively tested in terms of accuracy, success rate and speed, and the results are shown in the following table.

[0094] As can be seen from Table 1, the mean center error of the algorithm of the embodiment is only 6.9 pixels, so the accuracy is better than that of other algorithms. The algorithm of the embodiment does not fail to track in each sequence, while other algorithms lose the target in different sequence frames. It shows that the accuracy performance of the algorithm of the embodiment is stable and can realize target tracking in multiple scenarios.

[0095] Table 1: Target center error (pixels):

[0096] sequence CT OAB TLD LOT OURS fish 10.7 87.0 6.5 33.6 5.8 david2 76.7 33.8 5.0 4.1 3.4 girl 18.9 3.7 9.8 22.8 6.9 boy 9.0 2.9 4.5 66 4.1 jumping 47.7 45.3 5.9 5.6 3.0 singer1 15.5 12.9 8.0 141 4.9 sylvester 8.6 14.8 7.3 11.3 6.9 woman 114 31.4 139 117 9.6 mhyang 13.3 7.4 9.5 113 1.5 Average 52.8 25.1 41.1 54.0 6.0

[0097] ​Table 2 gives the overlap ratios of the target boxes in each group of video sequences. It can be seen that in the singer1 sequence with strong illumination changes, the overlap ratio of the algorithm of the embodiment is 83%, which shows that the illumination invariant feature effectively suppresses the influence of illumination changes. In the woman sequence with occlusion, the overlap ratio of 64% exceeds that of other algorithms, and the algorithm of the embodiment avoids the model drift problem caused by occlusion, and achieves good tracking effect. The average overlap ratio of the algorithm of the embodiment is 75%, and stable tracking of all sequences is basically achieved.

[0098] Table 2 Target box overlap ratio:

[0099] sequence CT OAB TLD LOT OURS fish 0.72 0.04 0.81 0.23 0.8 david2 0.50 0.32 0.69 0.6 0.78 girl 0.31 0.72 0.57 0.42 0.62 boy 0.59 0.79 0.66 0.53 0.84 jumping 0.04 0.07 0.66 0.58 0.79 Singer1 0.54 0.52 0.8 0.21 0.83 sylvester 0.67 0.56 0.67 0.57 0.73 woman 0.13 0.48 0.13 0.09 0.64 mhyang 0.60 0.75 0.63 0.22 0.83 Average 0.37 0.49 0.58 0.40 0.75

[0100] The average value of the tracking speed of each algorithm in all video sequences is given in Table 3. The algorithm of the embodiment ranks third among the compared algorithms. The CT algorithm uses a sparse matrix representation and thus is fast, but this algorithm is slightly poor in robustness due to simple feature representation. The speed of the algorithm of the embodiment reaches 56 frames per second, which is lower than that of the above two algorithms, but can guarantee real-time requirements, and is higher than that of most generative algorithms of the same type.

[0101] Table 3 Tracking speed

[0102] sequence CT OAB TLD LOT OURS Average frame rate 87 13 23 35 56

[0103] According to the data obtained from the experiments of all video sequences, the overall robustness of the algorithm is evaluated. The OPE evaluation method is used, the algorithm is initialized according to the accurate position of the initial frame, then the algorithm is executed in a test sequence, and finally the average accuracy and success rate results are obtained.

[0104] As shown in Figure 7 , the accuracy graph is drawn according to the center error value. It can be seen from the graph that the number of frames with error within a given threshold accounts for a percentage of the total number of frames. And the algorithm is ranked according to the result of the threshold of 20 pixels. It can be seen that the accuracy of the algorithm of the embodiment is the highest.

[0105] As shown in Figure 8 , the success rate graph is drawn according to the overlap ratio, which represents the percentage of the total number of frames with an overlap ratio greater than a given threshold. Using a success rate at a certain threshold (such as to=0.5) may not be fair or representative for evaluating the tracker. Therefore, the area under the curve (AUC) of each success rate graph is used as an alternative for ranking the tracking algorithm. It can be seen that the success rate of the algorithm of the embodiment is also the highest.

[0106] In the qualitative evaluation experiment, in order to be able to intuitively and clearly see the tracking effect of different algorithms, the embodiment gives the result graph of the target frame obtained by tracking of five algorithms in different video frames, wherein the tracking results of different algorithms are shown by rectangular frames of different colors, as shown in Figure 9

[0107] Figure 9 The jumping video contains fast motion and motion blur, and the tracked face range in the video is small, and fewer feature points can be extracted. In addition, the continuous fast motion and the image blur phenomenon bring great difficulty to tracking, so most trackers will fail to track. As can be seen from the five frames given in the figure, after the target moves up and down several times, OAB and CT fail to track, and TLD and LOT trackers partially deviate and have scale errors, but the algorithm in this paper can still basically frame the target.

[0108] Figure 10 In the entire image sequence 552 frames of the woman, the target appears multiple times with different degrees of local occlusion. As can be seen in the 141 frame screenshot, after the first occlusion, only the algorithm in this paper can stably frame the target, and the OAB algorithm deviates, while the remaining TLD, LOT, and CT trackers have failed to track. After the second occlusion of the target, the target frame of the OAB tracker also completely deviates. As can be seen in the 349 and 547 frames, the target occlusion is eliminated, and the OAB re-finds the target, while the target frames of the other three algorithms are still lost. In the 564 frame, the target has a scale change and becomes blurred, at which time the OAB algorithm deviates again, and the algorithm in this paper adjusts the target frame according to the scale and still accurately tracks the target. Therefore, it can be seen that when the target appears partial occlusion and motion blur, the algorithm in this paper can still find the target center through clustering according to the remaining partial feature points, and timely adjust the scale size to accurately track the target.

[0109] ​The video track tracking method provided in the embodiment can enhance features by adding the attention mechanism of the CA module at the end of the backbone network of the YOLO-V8 algorithm, improve the accurate detection capability of the model on the target, use local features and global features to construct a correlation filter to improve the tracking stability, and finally use the Hungarian matching algorithm to optimally match the correlation and overlap of the target tracking edge frame and the target detection edge frame, correct the target detection result according to the tracking result, and output the final target frame. Compared with the correlation filter tracking algorithm alone, the tracking algorithm fused with the deep learning model has been effectively improved in accuracy and success rate, especially for small target tracking, and the tracking effect is obviously improved. In the application field of the power industry, a large number of substation monitoring images are used for model training, and then combined with the real-time correlation filter algorithm, the real-time tracking of the on-site workers and construction vehicles in the complex operation environment can be effectively realized.

[0110] Embodiment two

[0111] The purpose of the embodiment is to provide a feature fusion video target tracking system, which comprises:

[0112] The target detection module is configured to detect the target in the obtained current video frame by using a target detection algorithm to obtain a target detection frame.

[0113] The correlation filter module is configured to extract a first maximum response map and a second maximum response map from the obtained previous video frame by using a local correlation filter and a global correlation filter respectively.

[0114] The target tracking module is configured to obtain a target tracking edge frame of the current video frame according to the first maximum response map and the second maximum response map.

[0115] The matching module is configured to match the overlap of the target detection frame and the target tracking edge frame by using the Hungarian matching algorithm to obtain the final target frame of the target.

[0116] Embodiment three

[0117] The purpose of the embodiment is to provide a computing device, which comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to realize the steps of the above method.

[0118] Embodiment four

[0119] The purpose of the embodiment is to provide a computer readable storage medium.

[0120] A computer readable storage medium, which stores a computer program, wherein the program is executed by a processor to perform the steps of the above method.

[0121] The steps involved in the apparatuses of embodiments two, three and four above correspond to the method of embodiment one, and the detailed description can be found in the relevant description of embodiment one. The term "computer readable storage medium" should be understood to include a single medium or multiple media, one or more sets of instructions; and should also be understood to include any medium that is capable of storing, encoding or carrying a set of instructions for execution by a processor and that causes the processor to perform any one of the methods of the present application.

[0122] Those skilled in the art should understand that the modules or steps of the present application described above can be implemented by a general computer device, or alternatively, they can be implemented by program codes executable by a computing device, so that they can be stored in a storage device and executed by a computing device, or they can be respectively manufactured into individual integrated circuit modules, or a plurality of modules or steps among them can be manufactured into a single integrated circuit module. The present application is not limited to any specific combination of hardware and software.

[0123] Although the specific embodiments of the present application have been described above with reference to the drawings, the description is not a limitation on the scope of protection of the present application, and those skilled in the art should understand that various modifications or variations made on the basis of the technical solutions of the present application without creative labor are still within the scope of protection of the present application.

Claims

1. A video target tracking method based on feature fusion, characterized in that, include: The target detection algorithm is used to detect targets in the current video frame and obtain target detection boxes; The first maximum response map and the second maximum response map are extracted from the previous video frame by using local correlation filters and global correlation filters, respectively. Based on the first maximum response map and the second maximum response map, the target tracking edge box of the current video frame is obtained; The overlap between the target detection box and the target tracking edge box is matched using the Hungarian matching algorithm to obtain the final target box of the target; Specifically, color features and histogram of directional gradients (HDRs) features are extracted from the previous video frame using local correlation filters and global correlation filters, respectively. After Fourier transform, a first maximum response map and a second maximum response map are obtained. Based on the target center position determined by the first maximum response map and the target center position determined by the second maximum response map, an adaptive weighted algorithm is used to obtain the final target center position. The target tracking bounding box is determined based on the final target center position. The local feature selection is the histogram of directional gradients (HDR) feature, and the global feature selection is the color feature. In the target tracking process, a scale filter is used to update the scale of the determined target center position, specifically as follows: Using the currently detected target center location as the center point, candidate regions of different scales are collected based on the width and height of the previous video frame; Extract oriented gradient histogram features for candidate regions at each scale; A regularized least squares classifier is trained using all extracted directional gradient histogram features to obtain scale-related templates; The output response value is solved using the relevant scale template, and the scale of the largest output response value is the scale of the target in the next video frame; Among them, scale-related templates , The calculation and update formulas are as follows: ； In the formula, i represents an image patch. For learning rate, For Discrete Fourier Transform, This represents the Gaussian function label corresponding to image patch i. This represents the dimension of the directional gradient histogram features.

2. The video target tracking method based on feature fusion as described in claim 1, characterized in that, The target detection algorithm uses the YOLO-v8 network as the main network and adds a coordinate attention mechanism at the end of the YOLO-v8 network to enhance the features of the output of the YOLO-v8 network.

3. The video target tracking method based on feature fusion as described in claim 1, characterized in that, The Intersection over Union (IOU) value between the target detection bounding box and the target tracking edge box is calculated. Based on the prior relationship of the calculated IOU value, the Hungarian matching algorithm establishes a matching relationship between the target detection edge box and the target tracking edge box.

4. A video target tracking system with feature fusion, characterized in that, include: The target detection module is configured to: use a target detection algorithm to detect targets in the current video frame and obtain target detection boxes; The correlation filtering module is configured to extract the first maximum response map and the second maximum response map from the acquired previous video frame using a local correlation filter and a global correlation filter, respectively. The target tracking module is configured to obtain the target tracking edge box of the current video frame based on the first maximum response map and the second maximum response map; Specifically, color features and histogram of directional gradients (HDRs) features are extracted from the previous video frame using local correlation filters and global correlation filters, respectively. After Fourier transform, a first maximum response map and a second maximum response map are obtained. Based on the target center position determined by the first maximum response map and the target center position determined by the second maximum response map, an adaptive weighted algorithm is used to obtain the final target center position. The target tracking bounding box is determined based on the final target center position. The local feature selection is the histogram of directional gradients (HDR) feature, and the global feature selection is the color feature. In the target tracking process, a scale filter is used to update the scale of the determined target center position, specifically as follows: Using the currently detected target center location as the center point, candidate regions of different scales are collected based on the width and height of the previous video frame; Extract oriented gradient histogram features for candidate regions at each scale; A regularized least squares classifier is trained using all extracted directional gradient histogram features to obtain scale-related templates; The output response value is solved using the relevant scale template, and the scale of the largest output response value is the scale of the target in the next video frame; Among them, scale-related templates , The calculation and update formulas are as follows: ； In the formula, i represents an image patch. For learning rate, For Discrete Fourier Transform, This represents the Gaussian function label corresponding to image patch i. The dimension representing the features of the histogram of oriented gradients; The matching module is configured to use the Hungarian matching algorithm to match the overlap between the target detection box and the target tracking edge box to obtain the final target box of the target.

5. A video target tracking system based on feature fusion as described in claim 4, characterized in that, In the matching module, the IOU value between the target detection box and the target tracking edge box is calculated. The Hungarian matching algorithm establishes a matching relationship between the target detection edge box and the target tracking edge box based on the prior relationship of the calculated IOU value.

6. A computer device, characterized in that, include: The system includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform a video target tracking method based on feature fusion as described in any one of claims 1 to 3.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs a video target tracking method based on feature fusion as described in any one of claims 1 to 3.

Images