Anti-occlusion target tracking method based on multivariate information fusion

Abstract

The invention provides an anti-shielding target tracking method based on multivariate information fusion. The method comprises the following steps: fusing multivariate information such as tracking confidence information, target template information and target space motion information; when shielding occurs, updating of a target position and a filter template is stopped in time, and a recent effective target template is replaced for target re-detection, so that the situation that a current target template containing shielding object information cannot be matched with a target again is avoided; the spatial position of the target is predicted in real time according to the information of the angle of the servo turntable and the target image deviation value, and the target is still in an effective search range when appearing again, so that the probability of successful re-tracking of the target is greatly improved; according to the method, the occurrence of target shielding can be accurately recognized, and the problem that shielding recognition is inaccurate and not timely due to a fixed threshold value is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of target tracking, and in particular to an anti-occlusion target tracking method based on multi-information fusion. Background Art

[0002] The optoelectronic tracking system is a stable platform equipped with optoelectronic detectors such as visible light cameras and infrared thermal imagers. Through the azimuth and pitch motion of the servo turntable, it can realize the search, reconnaissance and tracking of targets on the ground, sea and air. In reality, targets such as drones and vehicles often appear in complex background environments and are often blocked by buildings, towers, branches, etc., making it difficult for traditional tracking algorithms to achieve continuous and effective tracking and detection. When the moving target is temporarily blocked, the template update of the existing tracking algorithm has little impact and the target can be tracked again. However, when the target is gradually blocked, the target template will be gradually updated to the characteristics of the blocking object, resulting in a failure to match the target when it appears again and tracking failure. Summary of the invention

[0003] The technical problem to be solved by the present invention is how to continuously and effectively track the target and make an accurate judgment when the target is gradually blocked. In view of this, the present invention provides an anti-blocking target tracking method based on multi-information fusion.

[0004] The technical solution adopted by the present invention is an anti-occlusion target tracking method based on multi-information fusion, comprising:

[0005] Step S1, selecting a target through the host computer software, and sending the pixel coordinates and size of the target in the image to the tracking program of the image processing unit;

[0006] Step S2, based on the pixel coordinates and size of the target in the image, if it is the initial frame, the tracking program extracts the target area image, takes the target center point as the center, calculates the Gaussian response map G0 of the first frame image, and the Fourier transform F0 of the target image, calculates the initial frame filter template H0 and its decomposition items A0 and B0 through G0 and F0, and the filter H0 will be used as the matching template for the next frame image;

[0007] Step S3, for non-initial frames, extract the target image of the current frame from the position updated in the previous frame, and store the target history filter template once every n frames, and when occlusion occurs, it is used to replace the target template that has failed due to occlusion;

[0008] Step S4, calculate the correlation response between the current frame target image and the target filter template H trained in the previous frame, use the peak sidelobe ratio PSR as the tracking confidence index, and update the tracking confidence index in real time, by accumulating and calculating the average tracking confidence level PSR within the recent preset time range dyn , if the frame PSR drops below PSRdyn The preset ratio of PSR drops to a fixed threshold value PSR thr If the value is less than 0, it is considered that occlusion occurs, and steps S9 to S11 are performed; otherwise, steps S5 to S8 are performed;

[0009] Step S5, according to the result G of the correlation calculation between the target image and the filter template, after inverse Fourier transform, the maximum response point is obtained, the target position is updated with the point, and the target deviation is sent to the servo controller to calculate the target spatial angle position in real time;

[0010] Step S6, updating the target area image according to the latest target position;

[0011] Step S7, using the updated target image to obtain its discrete Fourier transform F, and training the latest target filter template parameter A with the relevant response result G new , B new ;

[0012] Step S8, the previous frame target filter template parameter A n-1 , B n-1 And the frame template parameter A new , B new Assign learning rate and obtain the latest target filter template parameter A through linear weighting n , B n , and use this to calculate the latest filter template H n ;

[0013] Step S9, when occlusion occurs, stop updating the target position and updating the target filter template parameters to avoid updating the target filter template to an occluder;

[0014] Step S10, enabling the target history filter template as the template for target matching in the next frame, and not updating the target filter template when the confidence level is not increased to above a preset threshold;

[0015] Step S11, after the tracking program detects that the target is occluded, it sends a memory tracking instruction to the servo turntable. The servo controller predicts the target speed in real time by tracking the historical information of the spatial position of the target, and continues to move forward at this predicted speed when receiving the memory tracking instruction, so that when the target appears again, it is still in the effective search and detection area.

[0016] Another aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the anti-occlusion target tracking method based on multi-information fusion as described in any one of the above items.

[0017] Compared with the prior art, the present invention has at least the following advantages:

[0018] The present invention can accurately identify the occurrence of target occlusion by fusing multiple information such as tracking confidence information, target template information, target spatial motion information, etc., and overcomes the problem of inaccurate and untimely occlusion identification caused by a fixed threshold. After the occlusion occurs, the update of the target position and the filter template is stopped in time, and the recent effective target template is replaced to re-detect the target, avoiding the problem that the current target template containing the occlusion information cannot be re-matched with the target. At the same time, the target spatial position is predicted in real time through the servo turntable angle and the target image deviation information. When the target reappears, it will still be within the effective search range, which greatly improves the probability of successful target re-tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of a flow chart of an anti-occlusion target tracking method based on multi-information fusion according to an embodiment of the present invention;

[0020] FIG2 is a diagram of an anti-occlusion target tracking test according to an embodiment of the present invention;

[0021] Figure 3 FIG. 1 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined purpose, the present invention is described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0023] Unless otherwise defined, all terms (including technical terms and scientific terms) used in this article have the same meaning as commonly understood by ordinary technicians in the field to which this application belongs. It should also be understood that terms (such as terms defined in commonly used dictionaries) should be interpreted as having the same meaning as their meaning in the context of the relevant technology, and will not be interpreted in an idealized or overly formal sense unless explicitly defined in this article.

[0024] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0025] The embodiment of the present invention is a method for tracking an anti-occlusion target based on multi-information fusion, which can be referred to Figure 1 ,include:

[0026] Step S1, selecting a target through the host computer software, and sending the pixel coordinates and size of the target in the image to the tracking program of the image processing unit;

[0027] Step S2, based on the pixel coordinates and size of the target in the image, if it is the initial frame, the tracking program extracts the target area image, takes the target center point as the center, calculates the Gaussian response map G0 of the first frame image, and the Fourier transform F0 of the target image, calculates the initial frame filter template H0 and its decomposition items A0 and B0 through G0 and F0, and the filter H0 will be used as the matching template for the next frame image;

[0028] Step S3, for non-initial frames, extract the target image of the current frame from the position updated in the previous frame, and store the target history filter template once every n frames, and when occlusion occurs, it is used to replace the target template that has failed due to occlusion;

[0029] Step S4, calculate the correlation response between the current frame target image and the target filter template H trained in the previous frame, use the peak sidelobe ratio PSR as the tracking confidence index, and update the tracking confidence index in real time, by accumulating and calculating the average tracking confidence level PSR within the recent preset time range dyn , if the frame PSR drops below PSR dyn The preset ratio of PSR drops to a fixed threshold value PSR thr If the value is less than 0, it is considered that occlusion occurs, and steps S9 to S11 are performed; otherwise, steps S5 to S8 are performed;

[0030] Step S5, according to the result G of the correlation calculation between the target image and the filter template, after inverse Fourier transform, the maximum response point is obtained, the target position is updated with the point, and the target deviation is sent to the servo controller to calculate the target spatial angle position in real time;

[0031] Step S6, updating the target area image according to the latest target position;

[0032] Step S7, using the updated target image to obtain its discrete Fourier transform F, and training the latest target filter template parameter A with the relevant response result G new , B new ;

[0033] Step S8, the previous frame target filter template parameter A n-1 , B n-1 And the frame template parameter A new , B new Assign learning rate and obtain the latest target filter template parameter A through linear weighting n , B n , and use this to calculate the latest filter template H n ;

[0034] Step S9, when occlusion occurs, stop updating the target position and updating the target filter template parameters to avoid updating the target filter template to an occluder;

[0035] Step S10, enabling the target history filter template as the template for target matching in the next frame, and not updating the target filter template when the confidence level is not increased to above a preset threshold;

[0036] Step S11, after the tracking program detects that the target is occluded, it sends a memory tracking instruction to the servo turntable. The servo controller predicts the target speed in real time by tracking the historical information of the spatial position of the target, and continues to move forward at this predicted speed when receiving the memory tracking instruction, so that when the target appears again, it is still in the effective search and detection area.

[0037] Can refer to again Figure 1 , a detailed implementation process based on this embodiment will be provided below.

[0038] Step 1: The host computer software obtains the pixel coordinates (x, y) and size (length h and width w) of the target in the image in the initial frame and sends them to the tracking program, as shown in Figure 2(a). The target is the box selection area.

[0039] Step 2: For the initial frame image, extract the target area image image_tar according to the target position and size, initialize the target relative position (x0, y0), train the initial frame target template filter H0 through fast Fourier transform FFT, and use it as the filter template for the next frame image;

[0040]

[0041] is the complex conjugate of H0, G0 and F0 are the Fourier transform of the correlation response result and the target image respectively;

[0042] Step 3: For non-initial frames, the position (x n-1 ,y n-1 ) Extract the current frame target image image_sub and store the target history filter template H dyn The template data is refreshed every 20 frames. When occlusion occurs, the template gradually learns to be an occluder due to the target gradient. In the tenth step, H dyn Replace target templates that fail due to occlusion;

[0043] Step 4: The target template filter H trained by the previous frame n-1 And the current frame target image image_sub, perform correlation calculation to obtain the current frame tracking confidence PSR n :

[0044]

[0045] Among them, gmax is the peak value of the correlation calculation result, μ and σ represent the mean and standard deviation respectively.

[0046] At the same time, the tracking confidence is updated in real time, and the average tracking confidence level PSR in the last k frames is calculated by the window filtering method. dyn , calculated as follows:

[0047]

[0048] If the frame PSR n Dropping more than PSR dyn 90% or PSR n Down to a fixed threshold PSR thr =8 (PSR is recommended when tracking weak targets thr If the value is below 0, it is considered that occlusion occurs and the ninth to twelfth steps are performed; otherwise, the fifth to eighth steps are performed;

[0049] Step 5: According to the target template filter H n-1 And the Fourier transform F of the target image image_sub in the frame n-1 Calculate the relevant response, obtain the target deviation (Δx, Δy), and update the target position to (x n-1 +Δx,y n-1 +Δy), and sends the target deviation to the servo control unit to calculate the target spatial angle position in real time;

[0050] Step 6: According to the latest position of the target (x n-1 +Δx,y n-1 +Δy), update the target image image_sub_new;

[0051] Step 7: Use the updated target image image_sub_new to perform discrete Fourier transform DFT and calculate the latest target filter template parameter A new and B new ;

[0052] Step 8: Use the linear weighting method to calculate the target filter template parameter A of the previous frame n-1 and B n-1 And when the frame parameter A new and B new Calculate the latest target filter template H n ;

[0053] A n =A n-1 ×(1-λ)+A new ×λ

[0054] B n =B n-1×(1-λ)+B new ×λ

[0055] Among them, λ is the learning rate parameter, which is 0.1 in this example;

[0056] Step 9: When the tracking program determines that the occlusion condition is met, such as the target is blocked by the bridge in Figure 2(b), there is no tracking target in the target frame, and the target filter template H is stopped. n-1 Match and update the target position, and stop using the original target position image image_sub_new (with occluders) of the current frame to update the target filter template;

[0057] Step 10: Use the target recent filter template H stored in step 3 dyn Replace the previous frame template H n-1 , as the target image template for re-detection, until the target reappears and the tracking confidence increases above the threshold;

[0058] Step 11: During the process of the system stably tracking the target, the servo turntable periodically collects its own angle (θ x ,θ y ) and the deviation angle (△θ x , △θ y ), and obtain the position of the target in the turntable coordinate system (θ x +△ θx ,θ y +△ θy ), predict the target speed in real time, and continue to move forward at this predicted speed when receiving the memory tracking instruction, instead of driving at the target deviation, until the target is recaptured as shown in Figure 2(c);

[0059] Compared with the prior art, the embodiments of the present invention can specifically achieve:

[0060] 1) Compare the tracking confidence calculation result of the current frame with the fixed threshold and the historical dynamic level to determine whether occlusion occurs;

[0061] In the normal tracking process, the tracking confidence within k frames is stored in real time and averaged to evaluate the tracking confidence level under the current target size, clarity, etc. When the frame calculation result drops below a fixed threshold, or the difference between it and the current historical level exceeds a certain ratio, it is judged that occlusion occurs.

[0062] 2) Fusion of multiple information such as target historical template information and target spatial motion information to re-detect the target when the occlusion ends. First, after the target occlusion condition is met, the target template and position update is stopped, and the target template destroyed by the occlusion is discarded and replaced with the target filter template n frames ago; second, during the normal tracking process, the deviation of the target from the image center is converted into angle information in real time according to the sensor focal length, and the angular velocity of the target relative to the turntable is predicted in real time in combination with the current turntable angle information. When occlusion occurs, the servo turntable enters the memory tracking mode according to the predicted speed, so that the target still appears in the effective search and detection area after the occlusion ends.

[0063] In summary, the present invention can accurately identify the occurrence of target occlusion by fusing multiple information such as tracking confidence information, target template information, target spatial motion information, etc., and overcomes the problem of inaccurate and untimely occlusion identification caused by fixed thresholds. After occlusion occurs, the update of the target position and filter template is stopped in time, and the recent valid target template is replaced for target re-detection, avoiding the problem that the current target template containing occlusion information cannot be re-matched with the target. At the same time, the target spatial position is predicted in real time through the servo turntable angle and target image deviation information. When the target reappears, it will still be within the effective search range, greatly improving the probability of successful target re-tracking.

[0064] A second embodiment of the present invention is an electronic device, such as Figure 3 As shown, it can be understood as a physical device, including a processor and a memory storing instructions executable by the processor. When the instructions are executed by the processor, the following operations are performed:

[0065] Step S1, selecting a target through the host computer software, and sending the pixel coordinates and size of the target in the image to the tracking program of the image processing unit;

[0066] Step S2, based on the pixel coordinates and size of the target in the image, if it is the initial frame, the tracking program extracts the target area image, takes the target center point as the center, calculates the Gaussian response map G0 of the first frame image, and the Fourier transform F0 of the target image, calculates the initial frame filter template H0 and its decomposition items A0 and B0 through G0 and F0, and the filter H0 will be used as the matching template for the next frame image;

[0067] Step S3, for non-initial frames, extract the target image of the current frame from the position updated in the previous frame, and store the target history filter template once every n frames, and when occlusion occurs, it is used to replace the target template that has failed due to occlusion;

[0068] Step S4, calculate the correlation response between the current frame target image and the target filter template H trained in the previous frame, use the peak sidelobe ratio PSR as the tracking confidence index, and update the tracking confidence index in real time, by accumulating and calculating the average tracking confidence level PSR within the recent preset time range dyn , if the frame PSR drops below PSR dyn The preset ratio of PSR drops to a fixed threshold value PSR thr If the value is less than 0, it is considered that occlusion occurs, and steps S9 to S11 are performed; otherwise, steps S5 to S8 are performed;

[0069] Step S5, according to the result G of the correlation calculation between the target image and the filter template, after inverse Fourier transform, the maximum response point is obtained, the target position is updated with the point, and the target deviation is sent to the servo controller to calculate the target spatial angle position in real time;

[0070] Step S6, updating the target area image according to the latest target position;

[0071] Step S7, using the updated target image to obtain its discrete Fourier transform F, and training the latest target filter template parameter A with the relevant response result G new , B new ;

[0072] Step S8, the previous frame target filter template parameter A n-1 , B n-1 And the frame template parameter A new , B new Assign learning rate and obtain the latest target filter template parameter A through linear weighting n , B n , and use this to calculate the latest filter template H n ;

[0073] Step S9, when occlusion occurs, stop updating the target position and updating the target filter template parameters to avoid updating the target filter template to an occluder;

[0074] Step S10, enabling the target history filter template as the template for target matching in the next frame, and not updating the target filter template when the confidence level is not increased to above a preset threshold;

[0075] Step S11, after the tracking program detects that the target is occluded, it sends a memory tracking instruction to the servo turntable. The servo controller predicts the target speed in real time by tracking the historical information of the spatial position of the target, and continues to move forward at this predicted speed when receiving the memory tracking instruction, so that when the target appears again, it is still in the effective search and detection area.

[0076] Through the description of the specific implementation methods, a deeper and more specific understanding of the technical means and effects adopted by the present invention to achieve the predetermined purpose should be obtained. However, the accompanying drawings are only for reference and illustration purposes and are not intended to limit the present invention.

Claims

1. A method for tracking an anti-occlusion target based on multivariate information fusion, characterized in that: include: Step S1, selecting a target through the host computer software, and sending the pixel coordinates and size of the target in the image to the tracking program of the image processing unit; Step S2, based on the pixel coordinates and size of the target in the image, if it is the initial frame, the tracking program extracts the target area image, takes the target center point as the center, calculates the Gaussian response map G0 of the first frame image, and the Fourier transform F0 of the target image, calculates the initial frame filter template H0 and its decomposition items A0 and B0 through G0 and F0, and the filter H0 will be used as the matching template for the next frame image; Step S3, for non-initial frames, extract the target image of the current frame from the position updated in the previous frame, and store the target history filter template once every n frames, and when occlusion occurs, it is used to replace the target template that has failed due to occlusion; Step S4, calculate the correlation response between the current frame target image and the target filter template H trained in the previous frame, use the peak sidelobe ratio PSR as the tracking confidence index, and update the tracking confidence index in real time, by accumulating and calculating the average tracking confidence level PSR within the recent preset time range dyn , if the frame PSR drops below PSR dyn The preset ratio of PSR drops to a fixed threshold value PSR thr If the value is less than 0, it is considered that occlusion occurs, and steps S9 to S11 are performed; otherwise, steps S5 to S8 are performed; Step S5, according to the result G of the correlation calculation between the target image and the filter template, after inverse Fourier transform, the maximum response point is obtained, the target position is updated with the point, and the target deviation is sent to the servo controller to calculate the target spatial angle position in real time; Step S6, updating the target area image according to the latest target position; Step S7, using the updated target image to obtain its discrete Fourier transform F, and training the latest target filter template parameter A with the relevant response result G new , B new ; Step S8, the previous frame target filter template parameter A n-1 , B n-1 And the frame template parameter A new , B new Assign learning rate and obtain the latest target filter template parameter A through linear weighting n , B n , and use this to calculate the latest filter template H n ; Step S9, when occlusion occurs, stop updating the target position and updating the target filter template parameters to avoid updating the target filter template to an occluder; Step S10, enabling the target history filter template as the template for target matching in the next frame, and not updating the target filter template when the confidence level is not increased to above a preset threshold; Step S11, after the tracking program detects that the target is occluded, it sends a memory tracking instruction to the servo turntable. The servo controller predicts the target speed in real time by tracking the historical information of the spatial position of the target, and continues to move forward at this predicted speed when receiving the memory tracking instruction, so that when the target appears again, it is still in the effective search and detection area.

2. An electronic device, characterized in that: The electronic device comprises: a memory, a processor and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the anti-occlusion target tracking method based on multi-information fusion as claimed in claim 1 are implemented.

Images