Distortion suppression and low-rank constraint-based long-time target tracking method

Abstract

The invention discloses a long-time target tracking method based on distortion suppression and low-rank constraint, and solves the problems that in the prior art, when a target appears along with adverse factors, the appearance of the target changes sharply in a short time due to introduction of excessive background information, and model drifting and even tracking failure easily occur. According to the long-time target tracking method based on distortion suppression and low-rank constraint provided by the invention, in the process of constructing a filter (model), the target and background information around the target are fused, the boundary effect is effectively relieved, meanwhile, the search area is enlarged, and the discrimination capability in a rapid movement, shielding or background disordered scene is improved.

Description

Technical Field

[0001] The invention relates to a target tracking method, and in particular to a long-term target tracking method based on distortion suppression and low-rank constraints. Background Art

[0002] The target tracking method mainly uses intelligent detection algorithms and manual precise labeling methods to determine the center position and range of the target in the initial image sequence, and then continuously, stably, and in real time track the target in the subsequent image sequence, and predict its trajectory and state. The current target tracking framework focuses on two categories: Discrimination Correlation Filters (DCFs) and Siamese Networks (SNs).

[0003] Although the tracking paradigm based on the twin network shows powerful performance, its complex and time-consuming convolution operations and high dependence on the GPU platform limit its practical application. In contrast, correlation filtering achieves a high degree of coordination in the time domain and frequency domain by cleverly combining the fast Fourier transform and the cyclic structure of the tracking paradigm, thereby significantly improving the target tracking efficiency. After continuous exploration, many innovative target tracking methods have been spawned, which not only enriches the theoretical system of target tracking, but also provides strong technical support for practical applications. For example: The Chinese invention patent with publication number CN115760906A discloses an "anti-occlusion single target tracking with self-attention mechanism in UAV target tracking scenario", which adopts HOG and position features, and cleverly uses the attention mechanism to achieve feature enhancement, and then obtains the optimal match by maximizing the total matching score, thereby realizing target position estimation.

[0004] The Chinese invention patent with publication number CN114926497A discloses "A single-target anti-occlusion tracking method and device based on ECO", which optimizes the model calculation amount through parameters such as detection, search area and threshold, and perceives the current tracking reliability through the peak sidelobe ratio. When the feedback is unreliable, it uses a global search mode and threshold adjustment to reposition the tracking.

[0005] The Chinese invention patent with publication number CN116468752A discloses "A single target tracking method for drones". It is oriented to drone scenarios and combines correlation filtering and Kalman filtering technology. In the target prediction stage, a response dynamic perception correlation filter is trained to handle the smoothness between responses. Then, in the subsequent sequence, Kalman filtering is used to search for the location of the area of ​​interest, effectively handling the problem of target disappearance caused by rapid motion.

[0006] The above-mentioned literatures all alleviate the impact of boundary effects by introducing an expanded search area. What is more difficult is that when the target appears with similar objects, or when the target is partially / completely occluded, or when other unfavorable factors arise, the introduction of too much background information causes the target appearance to change dramatically in a short period of time, which makes it very easy for model drift or even tracking failure to occur. Summary of the invention

[0007] In order to solve the technical problem in the prior art that when the target is accompanied by unfavorable factors, the introduction of too much background information causes the target appearance to change dramatically in a short period of time, which is very likely to cause model drift or even tracking failure, the present invention provides a long-term target tracking method based on distortion suppression and low-rank constraints.

[0008] The inventive concept of the present invention is to significantly improve the tracking performance through two core strategies: First, the present invention cleverly integrates distortion suppression and context-aware low-rank theory, aiming to explicitly integrate the target and its background information and enhance the discriminative power of the filter (model); imposes distortion suppression constraints in the filter (model) learning stage to adapt to the impermanent changes in target appearance; and emphasizes temporal consistency by guiding the filter (model) to be distributed on a low-dimensional discriminative manifold to improve tracking stability and accuracy.

[0009] Second, the present invention integrates the historical average peak energy APCE , accurately evaluate the reliability of the tracking results, and dynamically coordinate the filter (model) and the detector tracking model. When the tracking state is stable, the filter (model) constructed in the first stage is used for tracking; once the tracking is unstable or abnormal, the target recapture method is activated. First, the background subtraction strategy is used to generate a coarse candidate area, and then fine screening is performed based on the scale and position prior information of the target. Finally, the state and response confidence of the fine-grained candidate area are optimized to achieve accurate capture and recovery of the target, effectively avoiding the problem of model degradation.

[0010] In order to achieve the above purpose and complete the above invention concept, the present invention adopts the following technical solutions: A long-term target tracking method based on distortion suppression and low-rank constraint, which is special in that it includes the following steps: Step 1: Select multiple groups of continuous multi-frame images in the image sequence whose tracking status is continuously unreliable, and the previous frame image of each group of continuous multi-frame images The state and properties of are known. For each group of continuous multi-frame images, according to its previous frame image Target state Extract the current frame image Target features and its context block features ; The current frame image is the image closest to the previous frame in the corresponding continuous multi-frame image images; in, , is a positive integer, is the current frame time, Indicates the previous frame image The center position of the target, Indicates the previous frame image The width and height of the target; Step 2: Based on target features And the context block features , construct the current frame image Filter ; Step 3: Use the sliding average method to update the filter at a preset frame interval , and obtain the weighted average filter ; in, represents the learning rate; Step 4: Based on the previous frame image Target state , in the current frame image Crop the region of interest around the center point of the target , and extract the region of interest Features of interest ; Step 5: Features of interest With weighted average filter Perform convolution operation to get the current frame image Response graph of :

[0011] in, represents the inverse discrete Fourier transform, and Respectively and The discrete Fourier transform of Step 6: Get the recapture flag ; Step 7: According to the recapture flag Get the current frame image The target state; If the recapture flag is , get the current frame image through the recapture method Target state ; If the recapture flag is , according to the response diagram The peak value of the current frame image is obtained Target state ; in, Indicates the current frame image The center position of the target, Indicates the current frame image The width and height of the target; Step 8: For each frame image in each group of continuous multi-frame images in the image sequence whose tracking status is continuously unreliable, execute steps 1 to 7 in sequence according to the frame order to obtain the target state of each frame image in all continuous multi-frame images, and combine it with the target state of each frame image with a reliable tracking status to complete long-term target tracking.

[0012] Furthermore, step 2 specifically includes: 2.1. Target features extracted by step 1 and context block features , explicitly fuse the target and its surrounding background information to construct the first objective function:

[0013] in, and Represents target features and context block features The circulant matrix, , are constants, is an ideal two-dimensional Gaussian response; 2.2. Introducing distortion suppression constraints into the first objective function , and the second objective function is obtained:

[0014] in, represents a circular shift operation, p , q Respectively represent the previous frame image and the current frame image of x The peak displacement difference of the shaft and y Axis peak displacement difference, Indicates the previous frame image The response graph of is the distortion suppression penalty parameter; 2.3. Applying low-rank constraints to the second objective function , to emphasize the temporal consistency, the third objective function is obtained:

[0015] in, The history filter is stored in , , is a negative integer; Represents the matrix rank operation; is the variable in the low-rank constraint; 2.4. Use the sufficient condition of low-rank constraint to replace the low-rank constraint of the third objective function , to reconstruct the third objective function and obtain the fourth objective function:

[0016] in, is a constant and ; 2.5. Introducing equality constraints into the fourth objective function , we get the augmented Lagrangian function:

[0017] in, is the penalty parameter, is the Lagrange multiplier, is an auxiliary variable; 2.6. Use the alternating direction multiplier method to decompose the augmented Lagrangian function and obtain the fifth objective function:

[0018] Among them, the penalty parameter According to the rules renew, represents the penalty parameter The maximum value of represents the iteration step length, Indicates Iterations; 2.7. Iteratively optimize the fifth objective function and update the filter in sequence , auxiliary variables , Lagrange multipliers , penalty parameter , until the maximum preset number of iterations is reached, and the current frame image is obtained The optimal filter .

[0019] Furthermore, step 6 specifically includes: 6.1. Get the previous frame image Historical average peak energy The weighted mean of , and obtain the number of consecutive multi-frame images for which the tracking status is continuously unreliable ; 6.2. Calculate the current frame image Response graph of The average peak energy :

[0020] in, and Respectively represent the response graph The width and height of and Separate response diagram The maximum and minimum response values ​​in ; Representation response graph Middle i Line j Column response value; 6.3. Average peak energy With weighted mean Make comparisons; if , then the number of frames Add 1; if , then the number of frames Set to 0; in, represents a constant; 6.4. Determine the frame number Whether the conditions are met , T represents the minimum value that activates the background difference mechanism; if , then the recapture flag is 1; if , then the recapture flag is 0.

[0021] Furthermore, in step 7, the current frame image is acquired by recapture method Target state , specifically including the following steps: 7.1. Construct a mixed Gaussian background model and pass Update the mixed Gaussian background model to obtain the background model ; 7.2. Extracting background model and the current frame image ORB features; 7.3. Building background model With the current frame image Movement Model ; 7.4. Solving the motion model using ORB features ; 7.5. Based on the motion model , the background model With the current frame image Align to get the aligned background model ; 7.6. Align the background model and the current frame image Perform the difference to obtain a coarse-grained binary difference map ; 7.7. Coarse-grained binary difference image Perform morphological filtering to obtain a fine-grained binary difference image ; 7.8. From fine-grained binary difference images Extracting coarse-grained candidate regions from ,in, represents the first n The center position of the candidate region, represents the first n The width and height of the candidate regions, Indicates the total number of candidate regions contained in the coarse-grained candidate region set; 7.9. Combine with the previous frame image The scale and position prior information of the target in the image is obtained from the coarse-grained candidate region set. Filter out fine-grained candidate regions ,in, represents the first j The state of a fine-grained candidate region, represents the first j The center position of the fine-grained candidate region, represents the first j The width and speed of fine-grained candidate regions, A Indicates the total number of candidate regions contained in the fine-grained candidate region set; 7.10. Computing fine-grained candidate region sets The state confidence of each candidate region in :

[0022] in, is a constant; Compute fine-grained candidate region sets The response confidence of each fine-grained proposal region in :

[0023] in, and Respectively and The discrete Fourier transform of 7.11. Fusion State Confidence and response confidence , obtain the comprehensive confidence set of fine-grained candidate regions : in, , is a constant; 7.12. Obtaining the comprehensive confidence set The maximum confidence value , through the maximum confidence value Index to the best candidate region ; in, represents the center position of the optimal candidate region, Indicates the width and height of the optimal candidate region; 7.13 Based on the optimal candidate region , get the current frame image Target center position , the target scale remains unchanged, then , and the target state is .

[0024] Furthermore, in step 1: The target features and context block features Both include HOG, CN, Gray and deep convolution features.

[0025] Furthermore, step 7.1 is specifically as follows: The mixed Gaussian background model is constructed by using the neighborhood pixel relationship and regional modeling method. Update the mixed Gaussian background model to obtain the background model .

[0026] Furthermore, step 7.4 is specifically as follows: The motion model is solved by ORB features using the RSNSNC method. .

[0027] Furthermore, in step 3: The preset frame interval is 2 frames.

[0028] Furthermore, in step 2.7: The maximum preset number of iterations is 5 times.

[0029] Beneficial effects of the present invention: 1. The present invention provides a long-term target tracking method based on distortion suppression and low-rank constraint. In the process of constructing the filter (model), the target itself and its surrounding background information are integrated, which effectively alleviates the boundary effect, increases the search area, and improves the identification ability in fast-moving, occluded or cluttered background scenes.

[0030] 2. The present invention provides a long-term target tracking method based on distortion suppression and low-rank constraints. By imposing a distortion suppression constraint on the change rate of the response graph, it can effectively suppress the erratic changes in target appearance caused by adverse factors such as partial / complete occlusion and illumination changes when accompanied by background noise. At the same time, the introduction of low-rank constraints ensures the stability of the filter (model) and effectively avoids the occurrence of filter (model) drift.

[0031] 3. The present invention provides a long-term target tracking method based on distortion suppression and low-rank constraints, and designs a high-confidence target recapture method based on dynamic target detection. The current frame image is differentiated from the background model, and the differential result is subjected to morphological filtering. Then, multiple constraints are used to finely screen the candidate areas, and those candidate areas with large differences are effectively eliminated. Finally, the regional state analysis and response map calculation are combined to successfully achieve high-confidence recapture of the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a flow chart of an embodiment of a long-term target tracking method based on distortion suppression and low-rank constraint of the present invention; Figure 2 Schematic diagram of the recapture method in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] The embodiment of the present invention provides a long-term target tracking method based on distortion suppression and low-rank constraint, such as Figure 1 As shown, the following steps are included: Step 1: Select multiple groups of continuous multi-frame images in the image sequence whose tracking status is continuously unreliable through intelligent detection or manual precise screening. The previous frame image of each group of continuous multi-frame images The state and properties of are known. For each group of continuous multi-frame images, according to its previous frame image Target state Extract the current frame image Target features and its context block features ; Current frame image is the image closest to the previous frame in the corresponding continuous multi-frame image images; in, , is a positive integer, t is the current frame time, Indicates the previous frame image The center position of the target, Indicates the previous frame image The width and height of the target in and context block features Both include HOG, CN, Gray and deep convolution features.

[0035] Step 2: Based on target features And the context block features , construct the current frame image Filter ; 2.1. Target features extracted by step 1 and context block features , explicitly fuse the target and its surrounding background information to construct the first objective function:

[0036] in, and Represents target features and context block features The circulant matrix, , are constants, is an ideal two-dimensional Gaussian response; 2.2. Considering that there will be impermanent interference in actual tracking scenarios, multiple unfavorable factors overlap, such as the target being partially / completely blocked by other objects and non-rigid deformation, which causes the response graph in the detection stage to vigorous jitter, and the fusion of excessive error information causes the response graph to be distorted, and finally the filter Drift or even tracking failure. For this reason, this embodiment constructs a filter It not only integrates the contextual information of the target and its surroundings, but also suppresses potential distortion to obtain a more stable filter. Therefore, this embodiment introduces a distortion suppression constraint into the first objective function: , and the second objective function is obtained:

[0037] in, represents a circular shift operation, p , q Respectively represent the previous frame image and the current frame image of x The peak displacement difference of the shaft and y Axis peak displacement difference, Indicates the previous frame image The response graph of is the distortion suppression penalty parameter; 2.3. To further enhance the filter The discrimination power of this embodiment is to construct a filter When applying low-rank constraints on the cross-frame sequence, we emphasize the temporal smoothness, so that the learned filter On the low-dimensional identification manifold, improve the adaptability under complex conditions such as deformation and partial occlusion, specifically: Apply a low-rank constraint to the second objective function , to emphasize the temporal consistency, the third objective function is obtained:

[0038] in, The history filter is stored in , , is a negative integer; Represents the matrix rank operation; and is the variable in the low-rank constraint; 2.4. Consider the matrix rank operation is an NP-hard problem. Fortunately, such problems can be replaced by sufficient conditions of low-rank constraints to optimize the third objective function. Therefore, this embodiment uses sufficient conditions of low-rank constraints to replace the low-rank constraints of the third objective function. , to reconstruct the third objective function and obtain the fourth objective function:

[0039] in, is a constant and ; 2.5. In order to improve the filter Adaptability to complex environments, in the filter During optimization, this embodiment introduces an equality constraint into the fourth objective function: , we get the augmented Lagrangian function:

[0040] in, is the penalty parameter, is the Lagrange multiplier, is an auxiliary variable; 2.6. Use the alternating direction multiplier method to decompose the augmented Lagrangian function and obtain the fifth objective function:

[0041] Among them, the penalty parameter According to the rules renew, represents the penalty parameter The maximum value of represents the iteration step length, Indicates Iterations; 2.7. Iteratively optimize the fifth objective function: Subproblems in the fifth objective function optimization: Sub-questions The objective function can be expressed as:

[0042] Due to the sub-problem is a strictly quadratic convex optimization problem, so the subproblem is Differentiate and get the filter :

[0043] Considering the sub-problem The optimal solution involves the inversion of the matrix, and the heavy computational effort makes it difficult to meet the requirements of real-time tracking. However, by using the diagonal properties of the circulant matrix in the frequency domain, the filter can be further simplified. The matrix in and :

[0044] The above matrix and The simplified result is brought into the filter of the previous step In the equation, we get:

[0045] in Indicates signal The discrete Fourier transform of ,symbol Represents the dot product operation of the corresponding elements of the vector. It is worth noting that except FFT and IFFT Operation, sub-question In the optimal solution, all operations are bit-wise operations, and matrix multiplication and inversion operations are not involved, so it is consistent with actual real-time tracking.

[0046] Sub-questions Optimization: About sub-problems The objective function can be expressed as:

[0047] Directly on the objective function The derivative is:

[0048] Lagrange multipliers and penalty parameters It can be updated according to the following rules:

[0049] Finally, update the filter , auxiliary variables , Lagrange multipliers , penalty parameter , until the maximum preset number of iterations is reached, and the current frame image is obtained The optimal filter In this embodiment, the maximum preset number of iterations is 5 times.

[0050] Step 3: Use the sliding average method to update the filter at a preset frame interval , and obtain the weighted average filter ; in, represents the learning rate; In this embodiment, the preset frame interval is 2 frames.

[0051] Step 4: Based on the previous frame image Target state , in the current frame image Crop the region of interest around the center point of the target , and extract the region of interest Features of interest ; Step 5: Features of interest With weighted average filter Perform convolution operation to get the current frame image Response graph of :

[0052] in, represents the inverse discrete Fourier transform, and Respectively and The discrete Fourier transform of Step 6: Get the recapture flag ; 6.1. Get the previous frame image The historical average peak energy The weighted mean of , and obtain the number of consecutive multi-frame images for which the tracking status is continuously unreliable ; 6.2. Calculate the current frame image Response graph of The average peak energy :

[0053] in, and Respectively represent the response graph The width and height of and Separate response diagram The maximum and minimum response values ​​in ; Representation response graph Middle i Line j Column response value; 6.3. Average peak energy With weighted mean Make comparisons; if , then the number of frames Add 1; if , then the number of frames Set to 0; in, represents a constant; 6.4. Determine the frame number Whether the conditions are met , Trepresents the minimum value that activates the background difference mechanism; if , then the recapture flag is 1; if , then the recapture flag is 0.

[0054] Step 7: According to the recapture flag Get the current frame image The target state; If the recapture flag is , get the current frame image through the recapture method Target state ; Actual tracking applications may generally track minute-level image sequences. The target may be subject to more severe interference factors, and the filter (model) may be contaminated, resulting in accumulated positioning errors, which can easily lead to tracking failures. Therefore, it is particularly important for the tracker to configure a re-detection module to handle long-term occlusions and frequent out-of-field and re-entry situations. Currently, Kalman filtering or target detection strategies are usually used to address the re-detection problem. Although such methods are effective, they are based on global search methods and can easily lead to disastrous consequences. Therefore, if Figure 2 As shown, in this embodiment, the candidate area is first determined based on the moving target detection method, and then the shape and position constraints are injected using the pre-built background model, and finally the target is recaptured based on the response map and the spatiotemporal constraints; the details are as follows: 7.1. If the recapture flag is , a mixed Gaussian background model is constructed through neighborhood pixel relationship and regional modeling method, and Update it to get the background model ; 7.2. Extracting background model and the current frame image ORB features; 7.3. Building background model With the current frame image Movement Model ; Considering that the actual tracking task often faces the situation where the target and the background move at the same time, as well as the mobility of the target and the acquisition platform. This embodiment intends to assume that the motion relationship between the two image sequences is an affine transformation. Based on the above analysis, because after the affine transformation, the properties and spatial relative relationship of the straight lines in the image will not change, and the affine transformation matrix can be regarded as a combination of translation, scaling, rotation, reflection and shearing. It can be formulated in two-dimensional space as:

[0055] in is the product of rotation, shearing and scaling, the matrix Represents a translation transformation.

[0056] 7.4. Using RSNSNC method to solve motion model through ORB features ; Considering the grayscale difference between the previous and next two frames of the target and the complexity of the motion pattern in the actual tracking scene, a feature-based method is used to estimate the platform motion relationship; the details are as follows: 7.4.1. Get the current frame image separately and background model ; 7.4.2. Extract the current frame image separately and background model ORB feature point set , And construct a motion vector field, where K represents the number of maximum feature point pairs extracted.

[0057] 7.4.3. Screening the motion vector field and calculating the motion model based on the random sampling consistency algorithm ; 7.5. Based on motion model , the background model With the current frame image Align to get the aligned background model ; 7.6. Align the background model and the current frame image Perform the difference to obtain a coarse-grained binary difference map ; 7.7. Coarse-grained binary difference image Perform morphological filtering to obtain a fine-grained binary difference image ; 7.8. From fine-grained binary difference images Extracting coarse-grained candidate regions from ,in, represents the first n The center position of the candidate region, represents the first n The width and height of the candidate regions, Indicates the total number of candidate regions contained in the coarse-grained candidate region set; 7.9. Combine with the previous frame image The scale and position prior information of the target in the image is obtained from the coarse-grained candidate region set. Filter out fine-grained candidate regions ,in, represents the first j The state of a fine-grained candidate region, represents the first j The center position of the fine-grained candidate region, represents the first j The width and speed of fine-grained candidate regions, A Indicates the total number of candidate regions contained in the fine-grained candidate region set; 7.9.1. Based on the previous frame image The scale and position prior information of the target in the image is used to obtain a coarse-grained candidate region set. The center position flag set and range flags set ;in:

[0058]

[0059] and All are known parameters; 7.9.2. Based on the center position flag and range flag generated in 7.9.1, obtain the comprehensive flag set according to the following formula :

[0060] 7.9.3. Comprehensive flag bit set generated according to 7.9.2 , remove the coarse-grained candidate regions and focus on the previous frame image The candidate regions with large differences in target range, center position and state are selected to select a fine-grained candidate region set. ; 7.10. Computing fine-grained candidate region sets The state confidence of each candidate region in :

[0061] in, is a constant; Compute fine-grained candidate region sets The response confidence of each fine-grained proposal region in :

[0062] in, and Respectively and The discrete Fourier transform of 7.11. Fusion State Confidence and response confidence , obtain the comprehensive confidence set of fine-grained candidate regions :

[0063] in, , is a constant; 7.12. Obtaining the comprehensive confidence set The maximum confidence value , through the maximum confidence value Index to the best candidate region ; in, represents the center position of the optimal candidate region, Indicates the width and height of the optimal candidate region; 7.13 Based on the optimal candidate region , get the current frame image Target center position , the target scale remains unchanged, then , and the target state is .

[0064] If the recapture flag is , according to the response diagram The peak value of the current frame image is obtained Target state ; in, Indicates the current frame image The center position of the target, Indicates the current frame image The width and height of the target; Step 8: For each frame image in each group of continuous multi-frame images in the image sequence whose tracking status is continuously unreliable, execute steps 1 to 7 in sequence according to the frame order to obtain the target state of each frame image in all continuous multi-frame images, and combine it with the target state of each frame image with a reliable tracking status to complete long-term target tracking.

[0065] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A long-term target tracking method based on distortion suppression and low-rank constraint, characterized in that: The following steps are involved: Step 1: Select multiple groups of continuous multi-frame images in the image sequence whose tracking status is continuously unreliable, and the previous frame image of each group of continuous multi-frame images The state and properties of are known. For each group of continuous multi-frame images, according to its previous frame image Target state Extract the current frame image Target features and its context block features ; The current frame image is the image closest to the previous frame in the corresponding continuous multi-frame image images; in, , is a positive integer, is the current frame time, Indicates the previous frame image The center position of the target, Indicates the previous frame image The width and height of the target; Step 2: Based on target features And the context block features , construct the current frame image Filter ; Step 3: Use the sliding average method to update the filter at a preset frame interval , and obtain the weighted average filter ; in, represents the learning rate; Step 4: Based on the previous frame image Target state , in the current frame image Crop the region of interest around the center point of the target , and extract the region of interest Features of interest ; Step 5: Features of interest With weighted average filter Perform convolution operation to get the current frame image Response graph of : ； in, represents the inverse discrete Fourier transform, and Respectively and The discrete Fourier transform of ; the symbol ⊙ represents the dot product operation of the corresponding elements of the vector; Step 6: Get the recapture flag ; Step 7: According to the recapture flag Get the current frame image The target state; If the recapture flag is , get the current frame image through the recapture method Target state ; If the recapture flag is , according to the response diagram The peak value of the current frame image is obtained Target state ; in, Indicates the current frame image The center position of the target, Indicates the current frame image The width and height of the target; Step 8: For each frame image in each group of continuous multi-frame images in the image sequence whose tracking status is continuously unreliable, execute steps 1 to 7 in sequence according to the frame order to obtain the target state of each frame image in all continuous multi-frame images, and combine it with the target state of each frame image with a reliable tracking status to complete long-term target tracking.

2. The long-term target tracking method based on distortion suppression and low-rank constraint according to claim 1, characterized in that: Step 2 specifically includes: 2.

1. Target features extracted by step 1 and context block features , explicitly fuse the target and its surrounding background information to construct the first objective function: ； in, and Represents target features and context block features The circulant matrix, , are constants, is an ideal two-dimensional Gaussian response; 2.

2. Introducing distortion suppression constraints into the first objective function , and the second objective function is obtained: ； in, represents a circular shift operation, p , q Respectively represent the previous frame image and the current frame image of x The peak displacement difference of the shaft and y Axis peak displacement difference, Indicates the previous frame image The response graph of is the distortion suppression penalty parameter; 2.

3. Applying low-rank constraints to the second objective function , to emphasize the temporal consistency, the third objective function is obtained: ； in, The history filter is stored in , , is a negative integer; Represents the matrix rank operation; is the variable in the low-rank constraint; 2.

4. Use the sufficient condition of low-rank constraint to replace the low-rank constraint of the third objective function , to reconstruct the third objective function and obtain the fourth objective function: ； in, is a constant and ; 2.

5. Introducing equality constraints into the fourth objective function , we get the augmented Lagrangian function: ； in, is the penalty parameter, is the Lagrange multiplier, is an auxiliary variable; 2.

6. Use the alternating direction multiplier method to decompose the augmented Lagrangian function and obtain the fifth objective function: ； Among them, the penalty parameter According to the rules renew, represents the penalty parameter The maximum value of represents the iteration step length, Indicates Iterations; 2.

7. Iteratively optimize the fifth objective function and update the filter in sequence , auxiliary variables , Lagrange multipliers , penalty parameter , until the maximum preset number of iterations is reached, and the current frame image is obtained The optimal filter .

3. The long-term target tracking method based on distortion suppression and low-rank constraint according to claim 2, characterized in that: Step 6 specifically includes: 6.

1. Get the previous frame image Historical average peak energy The weighted mean of , and obtain the number of consecutive multi-frame images for which the tracking status is continuously unreliable ; 6.

2. Calculate the current frame image Response graph of The average peak energy : ； in, and Respectively represent the response graph The width and height of and Separate response plot The maximum and minimum response values ​​in ; Representation response graph Middle i Line j Column response value; 6.

3. Average peak energy With weighted mean Make comparisons; if , then the number of frames Add 1; if , then the number of frames Set to 0; in, represents a constant; 6.

4. Determine the number of frames Whether the conditions are met , T represents the minimum value that activates the background difference mechanism; if , then the recapture flag is 1; if , then the recapture flag is 0.

4. The long-term target tracking method based on distortion suppression and low-rank constraint according to claim 3 is characterized in that: In step 7, the current frame image is obtained by recapture method Target state , specifically including the following steps: 7.

1. Construct a mixed Gaussian background model and pass Update the mixed Gaussian background model to obtain the background model ; 7.

2. Extracting background model and the current frame image ORB features; 7.

3. Building background model With the current frame image Movement Model ; 7.

4. Solving the motion model using ORB features ; 7.

5. Based on the motion model , the background model With the current frame image Align to get the aligned background model ; 7.

6. Align the background model and the current frame image Perform the difference to obtain a coarse-grained binary difference map ; 7.

7. Coarse-grained binary difference image Perform morphological filtering to obtain a fine-grained binary difference image ; 7.

8. From fine-grained binary difference images Extracting coarse-grained candidate regions from ,in, represents the first n The center position of the candidate region, represents the first n The width and height of the candidate regions, Indicates the total number of candidate regions contained in the coarse-grained candidate region set; 7.

9. Combine with the previous frame image The scale and position prior information of the target in the image is obtained from the coarse-grained candidate region set. Filter out fine-grained candidate regions ,in, represents the first j The state of a fine-grained candidate region, represents the first j The center position of the fine-grained candidate region, represents the first j The width and speed of fine-grained candidate regions, A Indicates the total number of candidate regions contained in the fine-grained candidate region set; 7.

10. Computing fine-grained candidate region sets The state confidence of each candidate region in : ； in, is a constant; Compute fine-grained candidate region sets The response confidence of each fine-grained proposal region in : ； in, and Respectively and The discrete Fourier transform of 7.

11. Fusion State Confidence and response confidence , obtain the comprehensive confidence set of fine-grained candidate regions : ； in, , is a constant; 7.

12. Obtaining the comprehensive confidence set The maximum confidence value , through the maximum confidence value Index to the best candidate region ; in, represents the center position of the optimal candidate region, Indicates the width and height of the optimal candidate region; 7.13 Based on the optimal candidate region , get the current frame image Target center position , the target scale remains unchanged, then , and the target state is .

5. The long-term target tracking method based on distortion suppression and low-rank constraint according to claim 4, characterized in that: In step 1: The target features and context block features Both include HOG, CN, Gray and deep convolution features.

6. The long-term target tracking method based on distortion suppression and low-rank constraint according to claim 5, characterized in that: Step 7.1 is as follows: The mixed Gaussian background model is constructed by using the neighborhood pixel relationship and regional modeling method. Update the mixed Gaussian background model to obtain the background model .

7. The long-term target tracking method based on distortion suppression and low-rank constraint according to claim 6, characterized in that: Step 7.4 is as follows: The motion model is solved by ORB features using the RSNSNC method. .

8. The long-term target tracking method based on distortion suppression and low-rank constraint according to claim 7, characterized in that: In step 3: The preset frame interval is 2 frames.

9. The long-term target tracking method based on distortion suppression and low-rank constraint according to claim 8, characterized in that: In step 2.7: The maximum preset number of iterations is 5 times.

Images