Target tracking method based on re-detection mechanism and multi-feature fusion

Abstract

The application discloses a target tracking method based on a re-detection mechanism and multi-feature fusion, a reasonable response graph quality evaluation function DIS is designed from the robustness and accuracy of a tracking result, a fusion coefficient of responses of a gradient histogram feature and responses of a contour feature is planned with the response graph quality evaluation function DIS as a target, and the robustness of the algorithm is improved by using the complementarity of different features. Meanwhile, in order to facilitate the calculation of a re-identification module, a reliability detection module is designed, filters meeting the reliability condition in a tracking process are stored into a reliability template library, and a difference between response graph quality obtained by detecting current image features by using high-quality templates and normally updated templates is used as a basis for judging whether occlusion occurs, so that the target is effectively detected to be occluded, and it is not easy to confuse occlusion with image motion blur and the like. After the occlusion disappears, the tracker can track the target again.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a target tracking method based on a re-detection mechanism and multi-feature fusion. BACKGROUND

[0002] With the development of artificial intelligence, computer video tracking technology has been applied in many fields, such as intelligent transportation, logistics and warehousing, and life services. However, due to the complexity of the actual application scene, there are problems such as light change, scale change, obstacle occlusion, and rapid motion. How to accurately and quickly track the target in the presence of the above factors is an important research topic for those skilled in the art.

[0003] At present, the discriminative algorithm commonly used in the field of computer video tracking has the advantages of fast tracking speed and high tracking accuracy. At the same time, by means of correlation filtering algorithm, it uses cyclic matrix and frequency domain calculation method, and has a relatively low operation complexity, and the device computing power demand is not high. In practical application, the target tracking algorithm with correlation filtering algorithm and cyclic matrix and frequency domain calculation solves the target tracking problem to a certain extent. However, when facing complex video scenes, especially when there are a large number of occlusions or motion blurs in the video, it is still prone to tracking failure. SUMMARY

[0004] The present application aims to provide a target tracking method based on a re-detection mechanism and multi-feature fusion, which has better robustness and higher recognition accuracy.

[0005] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0006] The target tracking method based on the re-detection mechanism and multi-feature fusion of the present application comprises the following steps:

[0007] S1, using the filter template of the last frame determining the target search area in the current frame; the filter template includes a gradient histogram filter template and a contour filter template;

[0008] S2, extracting the manual features of the target search area of the current frame, the manual features including gradient histogram features and contour features;

[0009] correlation calculation is performed on the gradient histogram features of the target search area of the current frame and the gradient histogram filter template of the last frame to obtain the gradient histogram filter response; correlation calculation is performed on the contour features of the target search area of the current frame and the contour filter template of the last frame to obtain the contour filter response;

[0010] S3, the gradient histogram filter response and the contour filter response are linearly added to obtain a correlation filter response; a peak position in the correlation filter response is a target region in the current frame prediction; a DIS index of the correlation filter response is calculated;

[0011] S4, a manual feature of the target region in the current frame is extracted, and a ridge regression target function is used to train a training filter template of the current frame ; the training filter template includes a gradient histogram training filter template and a contour training filter template;

[0012] S5, the histogram training filter template and the contour training filter template that satisfy the reliability detection are sequentially stored in a reliability template library;

[0013] S6, when the DIS index does not satisfy the unreliability detection, it is continuously judged whether a shielding condition is satisfied;

[0014] S7, if the shielding condition is satisfied, the histogram training filter template and the contour training filter template of the nearest frame saved in the reliability template library are used to expand the range search of the current frame, and S2 to S4 steps are executed again to calculate a new DIS index, denoted as ;

[0015] S8, if the target re-identification index is satisfied, the histogram training filter template and the contour training filter template used in the range expansion search in S7 are used to update the histogram filter template and the contour filter template of the current frame, to obtain a filter template of the current frame ;

[0016] S9, otherwise, the training filter template of the current frame trained by using the ridge regression target function in S4 is used to update the filter template of the current frame .

[0017] Further, the reliability detection condition is: , wherein o is a constant, is an average value of DIS of the last M frames satisfying the reliability detection condition; the unreliability detection condition is , and .

[0018] Further, the reliability template library sequentially saves the last n target histogram training filter templates and target contour training filter templates satisfying the reliability detection condition.

[0019] Further, the shielding condition is: , wherein is a constant,

[0020] ​,

[0021] wherein n is a current frame number, and k is a preset value. is closest to the current frame and meets the reliability detection condition DIS.

[0022] Further, the enlarged range is 2.5 times of the region where the target of the previous frame is located.

[0023] Further, the target re-identification index is wherein is a coefficient greater than 1.

[0024] Further, the The calculation formula is: wherein is a constant representing a learning rate.

[0025] Further, if the target range of the first frame of the video is clear, the step S4 is directly executed.

[0026] The present application has the advantages that a reasonable response map quality evaluation function DIS is designed from the robustness and accuracy of the tracking result, the fusion coefficient of the responses of the gradient histogram features and the contour features is planned as the target of maximizing the response map quality evaluation function DIS, and the robustness of the algorithm is improved by using the complementarity of different features. Meanwhile, in order to facilitate the calculation of the re-identification module, a reliability detection module is designed, the filter meeting the reliability condition in the tracking process is stored in the reliability template library, the difference between the response map quality obtained by detecting the current image features with the high-quality template and the normally updated template is used as the basis for distinguishing the occlusion, so that the target is effectively detected to be occluded, and the occlusion and image motion blur are not easily confused. After the occlusion disappears, the tracker can re-track the target. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 is the flow chart of the method of the present application.

[0028] Figures 2-4 is the process chart of the method of the present application for tracking the target of the BOX video sequence using the OTB100 data set. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present application will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0030] As Figure 1 ​As shown, the target tracking method based on re-detection mechanism and multi-feature fusion comprises the following steps:

[0031] S1, using the filtering template of the last frame determining the target search area in the current frame, the filtering template comprising a gradient histogram filtering template and a contour filtering template;

[0032] S2, extracting the manual features of the target search area in the current frame, performing cross-correlation calculation on the gradient histogram features of the target search area in the current frame and the gradient histogram filtering template of the last frame to obtain the gradient histogram filtering response; performing cross-correlation calculation on the contour features of the target search area in the current frame and the contour filtering template of the last frame to obtain the contour filtering response; the manual features comprise the gradient histogram features and the contour features; wherein the gradient histogram features are denoted as , and comprise N image channels, denoted as

[0033]

[0034] the target contour features are denoted as , and also comprise N image channels, denoted as

[0035]

[0036] the cross-correlation calculation formula is:

[0037] Formula (1)

[0038] in formula (1), g is the correlation filtering response; when is the dth image channel in the gradient histogram features of the current frame; is the gradient histogram filtering template of the dth image channel in the gradient histogram features of the last frame, g is the gradient histogram filtering response, denoted as when is the dth image channel in the contour features of the current frame; is the contour filtering template of the dth image channel in the contour features of the last frame, g is the contour filtering response, denoted as .

[0039] That is, after performing cross-correlation calculation on the N filtering templates in the gradient histogram filtering template of the last frame corresponding to the N image channels of the gradient histogram features of the target search area in the current frame respectively and then adding and summing, the gradient histogram filtering response of the current frame can be obtained .

[0040] The N filter templates in the updated contour filter template of the N image channels of the contour features of the target search region of the current frame are respectively correlated with each other and then added to obtain the contour correlation filter response of the current frame .

[0041] S3, the gradient histogram filter response and the contour filter response are linearly added to obtain a correlation filter response ;

[0042] The peak position in the correlation filter response is the region where the target of the current frame is predicted to be located.

[0043] The DIS index of the correlation filter response is calculated

[0044] The correlation filter response calculation formula is:

[0045] Formula (2)

[0046] wherein, and are fusion coefficients, the fusion coefficients are determined by maximizing the DIS index, specifically: a planning objective function is established

[0047] Formula (3)

[0048] In formula (3), is the correlation filter response, which is essentially a two-dimensional matrix, each point in the matrix represents the probability size of the position being the target center. In general, the correlation filter response reflects a response map that may have response peaks similar to a two-dimensional Gaussian shape. In formula (3), the represents the position of the peak in the response map. is the position coordinate of the highest peak among all the peaks; is the height ratio between the second peak and the highest peak, is a parameter.

[0049] At the same time and also need to meet the following constraint conditions:

[0050] Formula (4)

[0051] S4, extract the manual features of the region where the target of the current frame is located, the manual features include gradient histogram features and contour features; a training filter template of the current frame is obtained by training using a ridge regression objective function ;

[0052] The ridge regression objective function formula is:

[0053] Formula (5)

[0054] In formula (5), is the hand-crafted feature of the dth image channel, denotes Fourier transform on ; denotes a diagonal matrix composed of ; N represents N image channels. Training the filter template using the ridge regression objective function includes training the gradient histogram filter template and the contour training filter template;

[0055] The above formula (5) calculates the gradient histogram training filter template, is the gradient histogram feature of the dth image channel in the gradient histogram feature; when calculating the contour training filter template, is the contour feature of the dth image channel in the contour feature.

[0056] is the training filter template of the dth image channel; denotes the Fourier transform of ; when calculating the gradient histogram filter template, is the gradient histogram training filter template of the dth image channel in the gradient histogram feature; when calculating the contour training filter template, is the contour training filter template of the dth image channel in the contour feature.

[0057] is a two-dimensional Gaussian function; denotes the Fourier transform of ; is a regularization parameter; is a unit vector.

[0058] That is, according to formula (5), N training filter templates of N image channels of the gradient histogram feature are obtained to form the gradient histogram training filter template; N training filter templates of N image channels of the contour feature are obtained to form the contour training filter template. The gradient histogram training filter template and the contour training filter template jointly form the training filter template, denoted as .

[0059] S5, the histogram training filter template and the contour training filter template that meet the reliability detection of the DIS index are sequentially stored in the reliability template library;

[0060] The reliability detection condition is:

[0061] Equation (6)

[0062] wherein o is a constant, is the average value of the last M DISs satisfying the reliability detection condition; the DIS satisfying the reliability detection condition, i.e., the DIS indicator of the correlation filtering response satisfying the reliability detection Equation (6), is denoted as When the tracker is just started, due to the limited number of detection frames, there are not enough , the preset is used. When the number of detection frames reaches a certain number, the M values can be guaranteed, and then the detection is performed according to Equation (6).

[0063] The last n gradient histogram training filtering templates and contour training filtering templates satisfying the reliability detection condition are sequentially saved in the reliability template library, and these templates are used in subsequent steps when the detection of the tracking target is blocked, and the target re-detection module is started.

[0064] S6, when the DIS indicator satisfies the unreliable detection, it is continuously judged whether the blocking condition is satisfied;

[0065] The unreliable detection condition is , and .

[0066] The blocking condition is:

[0067] Equation (7)

[0068] wherein is a constant,

[0069] Equation (8)

[0070] wherein n is the current frame number, and k is a preset value; ; is the histogram training filtering template and the contour training filtering template of the last frame saved in the reliability template library.

[0071] S7, if the blocking condition is satisfied, the histogram training filtering template and the contour training filtering template of the last frame saved in the reliability template library are used to expand the range search of the current frame, and S2 to S4 steps are executed again to calculate a new DIS indicator, denoted as ; the expanded range is 2.5 times the area of the target in the last frame.

[0072] That is, the target area of the current frame is identified by the filtering template of the last frame; the manual features of the target area of the current frame are extracted, and the training filtering template ; the DIS index of the correlation filtering response is calculated, and if the DIS index meets the unreliability detection and meets the occlusion condition, the training filtering template of the latest frame saved in the reliability template library is used to identify the target region of the current frame again; the manual features of the target region of the current frame are extracted, and a ridge regression target function is used to retrain the training filtering template of the current frame; the correlation filtering response is obtained again through cross-correlation calculation; the DIS index of the correlation filtering response is calculated, and the DIS index of the correlation filtering response of this time is marked as Then, S8 is executed.

[0073] S8, if The target re-identification index is met, the histogram training filtering template and the contour training filtering template used in the range expansion search in S7 are used to update the histogram filtering template and the contour filtering template of the current frame, and the filtering template of the current frame is obtained ;

[0074] The target re-identification index is:

[0075] Formula (9)

[0076] Wherein is the DIS index of the correlation filtering response calculated after S2 to S4 are executed again in the range expansion search of the current frame using the histogram training filtering template and the contour training filtering template of the latest frame saved in the reliability template library in S7; is the DIS index of the correlation filtering response calculated using the filtering template of the last frame to identify the target region of the current frame; the manual features of the target region of the current frame are extracted, and a ridge regression target function is used to train the training filtering template of the current frame ; the DIS index of the correlation filtering response obtained through cross-correlation calculation.

[0077] S9, otherwise, the training filtering template of the current frame trained by using the ridge regression target function in S4 is used to update the filtering template of the current frame .

[0078] The formula is:

[0079] Formula (10)

[0080] In the first execution, since the target range of the first frame of the video has been determined, and there is no last frame, S1 to S3 can be skipped, and S4 can be directly executed.

[0081] The inventors selected representative video sequences (Boxes) from the OTB100 dataset to illustrate the effectiveness of the method. Box sequences primarily exhibit several occlusion issues, such as... Figure 2 As shown, the item marked in the red box in frame 455 is partially obscured by the vernier caliper. Figure 3 As shown, in frame 474, the item marked in red is completely obscured by the vernier caliper. The occlusion detection mechanism of this invention detects that the item is occluded, and then the algorithm enters the occlusion phase, initiating the target re-identification mechanism. The histogram training filter template and contour training filter template of the most recent frame stored in the reliability template library are used to expand the search range for the current frame. Steps S2 to S4 are executed again to calculate a new DIS index, denoted as... ;like Figure 4 As shown, the item marked in red was detected again in frame 495. The target re-identification criteria are met, and the occlusion process is bypassed. This demonstrates that the algorithm mechanism in this application can effectively handle the occlusion problem.

Claims

1. A target tracking method based on a re-detection mechanism and multi-feature fusion, characterized in that: The method comprises the following steps: S1, using a filter template of a previous frame determining a target search region in a current frame; the filter template comprises a gradient histogram filter template and a contour filter template; S2, extracting a manual feature of a target search area of a current frame, the manual feature comprising a gradient histogram feature and a contour feature; S3, linearly adding the gradient histogram filter response and the contour filter response to obtain a correlation filter response; A peak position in the correlation filter response is a region where a target of the current frame is predicted to be located; S4, calculating a DIS index of the correlation filter response; The training filter template comprises a gradient histogram training filter template and a contour training filter template; S4, extracting the manual features of the region where the target in the current frame is located, and training a filter template of the current frame by using a ridge regression target function ; S5, storing the histogram training filter template and the contour training filter template that satisfy the reliability detection to a reliability template library in sequence; S6, when the DIS index does not satisfy the reliability detection, continuing to judge whether a blocking condition is satisfied; The reliability template library sequentially saves the most recent n target histogram training filter templates and target contour training filter templates that satisfy the reliability detection condition. S7, if the occlusion condition is satisfied, the histogram of the most recent frame saved in the reliability template library is used to train the filter template and the contour training filter template to expand the range of search for the current frame, and steps S2 to S4 are executed again to calculate a new DIS index, denoted as ; S8, if If the target re-identification index is met, the histogram training filter template and the contour training filter template in the enlarged range search in step S7 are used to update the histogram filter template and the contour filter template of the current frame, to obtain the filter template of the current frame ; S9, otherwise use the training filter template of the current frame obtained by training the ridge regression objective function in step S4 updating the filter template of the current frame . 2.The target tracking method based on re-detection mechanism and multi-feature fusion according to claim 1, characterized in that: The reliability detection condition is: wherein o is a constant, is an average value of the last M DISs satisfying the reliability detection condition; the unreliability detection condition is , and . 3.The target tracking method based on re-detection mechanism and multi-feature fusion according to claim 1, characterized in that: The enlarged range is 2.5 times of a region where a target of a previous frame is located. 4.The target tracking method based on re-detection mechanism and multi-feature fusion according to claim 1, characterized in that: The blocking condition is: wherein is a constant, ， Wherein, n is the current frame serial number, k is a preset value; ; is the closest to the current frame, and satisfies the reliability detection condition DIS.

5. The target tracking method based on re-detection mechanism and multi-feature fusion according to claim 1, characterized in that: When a target range of a first frame of a video is clear, S4 is directly executed. 6.The target tracking method based on re-detection mechanism and multi-feature fusion according to claim 1, characterized in that: The target re-identification index is wherein is a coefficient greater than 1.

7. The target tracking method based on re-detection mechanism and multi-feature fusion according to claim 1, characterized in that: The The calculation formula is: wherein is a constant representing the learning rate. 8.The target tracking method based on re-detection mechanism and multi-feature fusion according to claim 1, characterized in that: ​

Images