VideoSAR shadow tracking method, system, device and storage medium

Abstract

The application discloses a VideoSAR shadow tracking method, system and device and a storage medium. The method comprises the following steps: acquiring an input picture and an input tracking box; performing first prediction on the input picture based on the input tracking box by using a correlation filter to obtain an estimated tracking box; performing second prediction based on the motion state of a interacting multiple model filter to obtain fused prior observation and determine a prior probability function; performing double-parameter constant false alarm detection of a dynamic threshold in the estimated tracking box based on the prior probability function to obtain a detection result; adaptively updating the correlation filter and the interacting multiple model filter based on the estimated tracking box or the fused prior observation according to the detection result; and obtaining an output tracking box of a target frame VideoSAR picture based on the result of the adaptive update. The embodiment of the application can efficiently and accurately complete the VideoSAR target tracking task and can be widely applied to the technical field of shadow tracking.

Description

Technical Field

[0001] This invention relates to the field of shadow tracking technology, and in particular to a VideoSAR shadow tracking method, system, device, and storage medium. Background Technology

[0002] VideoSAR technology was first proposed by Sandia National Laboratories in the United States in 2003. Compared to optical imaging, SAR imaging technology has the advantage of all-weather, all-time operation, providing high-resolution imaging results even in fog, dust, and clouds. However, due to the long accumulation time, SAR imaging often only generates static single images, limiting its effectiveness. As a new radar imaging method, VideoSAR overcomes the low frame rate problem of traditional SAR imaging, enabling dynamic and real-time monitoring of target areas. Furthermore, VideoSAR operates in frequency bands above millimeter waves, adapting to the trend of intelligent and miniaturized development in modern battlefields by integrating it into small flight platforms such as UAVs. In summary, VideoSAR imaging technology, operating at higher radar bands, overcomes the low frame rate problem of traditional high-resolution synthetic aperture radar imaging, aligns with the development trend of intelligent modern battlefields and the requirements of radar systematization and end-to-end processes, and has a profound impact on precision guidance, rapid location, and threat target identification.

[0003] Target shadows are a crucial entry point in current VideoSAR target tracking and detection research. Target shadows arise in the region after the target's energy shift, reflecting its position and approximate shape. They possess advantages such as independence from the target's radar cross-section and the absence of blind velocities and minimum detection speeds. However, existing VideoSAR shadow tracking methods still have some problems. For tracking methods that treat shadows as point targets, complex background clutter and noise can generate false alarms, affecting the processing difficulty and accuracy of detection and tracking. For shadow tracking methods that use appearance feature tracking, strong scattering clutter sidelobes can obscure the shadow, the shadow's own energy and the defocusing energy of other targets can interfere with the shadow, and rapid movement can cause shadow blurring. All of these problems lead to the degradation and contamination of the appearance template in the tracking algorithm. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a VideoSAR shadow tracking method, system, device, and storage medium, which can efficiently and accurately achieve VideoSAR shadow tracking.

[0005] On one hand, embodiments of the present invention provide a VideoSAR shadow tracking method, including:

[0006] Acquire the input image and the input tracking box; where the input image represents the target frame VideoSAR image in the VideoSAR video frame sequence, and the input tracking box represents the output tracking box obtained based on the VideoSAR image of the previous frame of the target frame;

[0007] Using a correlation filter, a first prediction is made on the input image based on the input tracking box to obtain the estimated tracking box; wherein, the correlation filter includes a two-dimensional filter for displacement estimation and a one-dimensional filter for scale estimation;

[0008] The motion state is used for a second prediction based on the cross-multi-model filter to obtain fused prior observations and determine the prior probability function;

[0009] Based on the prior probability function, a two-parameter constant false alarm rate (CFAR) detection with dynamic threshold is performed in the estimated tracking box to obtain the detection results; the detection results include whether the target was detected in the estimated tracking box and whether the target was not detected in the estimated tracking box.

[0010] Based on the detection results, the correlation filter and the cross-model filter are adaptively updated based on the estimated tracking box or the fusion of prior observations;

[0011] Based on the results of adaptive updates, the output tracking box of the target frame VideoSAR image is obtained.

[0012] Optionally, the method further includes:

[0013] The two-dimensional filter is optimized by minimizing the loss function;

[0014] The expression for minimizing the loss function is:

[0015]

[0016] In the formula, This represents minimizing the loss function. A hierarchical index representing a feature. The total number of layers representing the features. Indicates the relevant operation. This represents the features extracted from the target. Represents the Gaussian function. Indicates the parameters affected by regularization. This represents a two-dimensional filter.

[0017] Optionally, using a correlation filter, a first prediction is made on the input image based on the input tracking box to obtain an estimated tracking box, including:

[0018] The target features of the input image are extracted using a correlation filter; the target features include histogram of gradient orientation features and pixel intensity features.

[0019] The Discrete Fourier Transform (DFT) of the two-dimensional filter is determined based on the target features; the expression for the DFT of the two-dimensional filter is:

[0020]

[0021] In the formula, Represents the discrete Fourier transform of a two-dimensional filter. and A hierarchical index representing a feature. The total number of layers representing the features. This represents the discrete Fourier transform of a Gaussian function. Discrete Fourier transform representing features, Indicates the regularization effect parameter, superscript Indicates complex conjugation;

[0022] The estimated tracking box is obtained by extracting the feature map of the input tracking box and determining the result score by combining it with the discrete Fourier transform of the two-dimensional filter; the expression for the result score is:

[0023]

[0024] In the formula, Indicates the result score. This represents the inverse discrete Fourier transform. A hierarchical index representing a feature. The total number of layers representing the features. The molecule representing the discrete Fourier transform of a two-dimensional filter. The superscript represents the denominator of the discrete Fourier transform of a two-dimensional filter. Indicates complex conjugation. This indicates the parameters affected by regularization.

[0025] Optionally, a second prediction is made based on the motion state of the cross-multi-model filter to obtain fused prior observations and determine the prior probability function, including:

[0026] By interacting with the states of various motion model filters through inter-multi-model filters, the fusion state, fusion covariance, and motion state probability of the Kalman filters under different motion states are determined.

[0027] Based on the fusion state, fusion covariance, and motion state probability, fusion prior observations are obtained, and the prior probability function is determined.

[0028] Optionally, based on the fusion state, fusion covariance, and motion state probability, fusion prior observations are obtained, and the prior probability function is determined, including:

[0029] Based on the fusion state, fusion covariance, and motion state probability, the fusion prior target state is obtained;

[0030] Based on the fusion of prior target states, combined with the observation matrix, fusion of prior observations are obtained;

[0031] Based on the fusion of prior observations, the prior probability function is determined by combining the Gaussian density function and the observation innovation covariance; whereby the observation innovation covariance is obtained through the motion model with the highest motion state probability among various motion models; the expression of the prior probability function is:

[0032]

[0033] In the formula, Denotes the prior probability function. A reference variable representing a function. Represents the Gaussian density function. This indicates the fusion of prior observations. This represents the covariance of observed new information.

[0034] Optionally, based on the detection results, the correlation filter and the cross-model filter are adaptively updated according to the estimated tracking box or the fused prior observations, including:

[0035] When a target is detected in the estimated tracking box, the cross-multi-model filter is updated using the estimated tracking box, and the relevant filter is updated using the fusion result of the motion state of the cross-multi-model filter;

[0036] When no target is detected in the estimated tracking box, the inter-model filter is updated using fused prior observations, and the estimated tracking box of the relevant filter is repositioned but not updated.

[0037] Optionally, the method further includes:

[0038] By combining a prior probability function with a search strategy, lost targets are re-tracked, and process noise parallel to the road is added to the tracking box with a fixed gain; the expression for the process noise is:

[0039]

[0040] In the formula, Indicates process noise. Indicates fixed gain. This indicates the number of frames in which the two-parameter constant false alarm rate (CFAR) detection system continuously failed to detect the target. This represents the standard deviation of the two-parameter constant false alarm rate (CFAR) detection.

[0041] On the other hand, embodiments of the present invention provide a VideoSAR shadow tracking system, comprising:

[0042] The first module is used to acquire input images and input tracking boxes; wherein, the input image represents the target frame VideoSAR image in the VideoSAR video frame sequence, and the input tracking box represents the output tracking box obtained based on the VideoSAR image of the previous frame of the target frame;

[0043] The second module is used to make a first prediction on the input image based on the input tracking box using a correlation filter to obtain an estimated tracking box; wherein the correlation filter includes a two-dimensional filter for displacement estimation and a one-dimensional filter for scale estimation.

[0044] The third module is used to make a second prediction based on the motion state of the cross-multi-model filter, obtain the fused prior observations, and determine the prior probability function.

[0045] The fourth module is used to perform two-parameter constant false alarm rate (CFAR) detection with dynamic thresholds in the estimated tracking box based on a prior probability function, and to obtain the detection results. The detection results include whether the target was detected in the estimated tracking box and whether the target was not detected in the estimated tracking box.

[0046] The fifth module is used to adaptively update the correlation filter and the cross-model filter based on the detection results, either by estimating the tracking box or by fusing prior observations.

[0047] The sixth module is used to obtain the output tracking box of the target frame VideoSAR image based on the results of adaptive updates.

[0048] On the other hand, embodiments of the present invention provide a VideoSAR shadow tracking device, including a processor and a memory;

[0049] Memory is used to store programs;

[0050] The processor executes the program as described above.

[0051] On the other hand, embodiments of the present invention provide a computer-readable storage medium storing a program that is executed by a processor to implement the method described above.

[0052] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned method.

[0053] This invention first acquires an input image and an input tracking box. The input image represents the target frame VideoSAR image in a VideoSAR video frame sequence, and the input tracking box represents the output tracking box obtained based on the VideoSAR image of the previous frame of the target frame. Using a correlation filter, a first prediction is performed on the input image based on the input tracking box to obtain an estimated tracking box. The correlation filter includes a two-dimensional filter for displacement estimation and a one-dimensional filter for scale estimation. A second prediction is performed based on the motion state of the inter-multi-model filter to obtain fused prior observations and determine the prior probability function. Based on the prior probability function, a two-parameter constant false alarm rate (CFAR) detection with a dynamic threshold is performed on the estimated tracking box to obtain a detection result. The detection result includes whether a target is detected in the estimated tracking box and whether no target is detected in the estimated tracking box. Based on the detection result, the correlation filter and the inter-multi-model filter are adaptively updated based on the estimated tracking box or the fused prior observations. Based on the adaptive update result, the output tracking box of the target frame VideoSAR image is obtained. This invention implements a tracking framework by combining correlation filters and cross-multiplexed model filters (CMMs). The CMMs supplement tracking with motion information, addressing the issues of appearance template degradation and contamination. Furthermore, dual-parameter constant false alarm rate (CFAR) detection based on dynamic thresholds reduces the impact of false alarm interference. This invention overcomes the shortcomings of existing similar technologies, efficiently and accurately completing VideoSAR target tracking tasks. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 A flowchart illustrating a VideoSAR shadow tracking method provided in an embodiment of the present invention;

[0056] Figure 2 A schematic diagram of the main framework of a VideoSAR shadow tracking method provided in an embodiment of the present invention;

[0057] Figure 3 A schematic diagram illustrating a shadow tracking comparison experiment example provided in an embodiment of the present invention;

[0058] Figure 4 This is a schematic diagram illustrating the performance of a shadow tracking comparison experiment provided in an embodiment of the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0060] like Figure 1 The flowchart of the method of the present invention is as follows: Figure 2 This is a schematic diagram of the main framework of the method of the present invention. In one aspect, an embodiment of the present invention provides a VideoSAR shadow tracking method, comprising:

[0061] S100: Obtain the input image and input tracking box;

[0062] It should be noted that the input image represents the target frame VideoSAR image in the VideoSAR video frame sequence, and the input tracking box represents the output tracking box obtained based on the VideoSAR image of the previous frame of the target frame.

[0063] S200. Using a correlation filter, perform a first prediction on the input image based on the input tracking box to obtain the estimated tracking box;

[0064] It should be noted that the correlation filter includes a two-dimensional filter for displacement estimation and a one-dimensional filter for scale estimation.

[0065] In some embodiments, the method further includes: optimizing the two-dimensional filter by minimizing a loss function; wherein the expression for minimizing the loss function is:

[0066]

[0067] In the formula, This represents minimizing the loss function. A hierarchical index representing a feature. The total number of layers representing the features. Indicates the relevant operation. This represents the features extracted from the target. Represents the Gaussian function. Indicates the parameters affected by regularization. This represents a two-dimensional filter.

[0068] In some embodiments, a correlation filter is used to perform a first prediction on the input image based on the input tracking box to obtain an estimated tracking box, including: extracting target features of the input image using the correlation filter; wherein the target features include gradient orientation histogram features and pixel intensity features; determining the discrete Fourier transform of the two-dimensional filter based on the target features; wherein the expression for the discrete Fourier transform of the two-dimensional filter is:

[0069]

[0070] In the formula, Represents the discrete Fourier transform of a two-dimensional filter. and A hierarchical index representing a feature. The total number of layers representing the features. This represents the discrete Fourier transform of a Gaussian function. Discrete Fourier transform representing features, Indicates the regularization effect parameter, superscript The complex conjugate is represented by ; the estimated tracking box is obtained by extracting the feature map of the input tracking box and determining the result score by combining it with the discrete Fourier transform of the two-dimensional filter; where the expression for the result score is:

[0071]

[0072] In the formula, Indicates the result score. This represents the inverse discrete Fourier transform. A hierarchical index representing a feature. The total number of layers representing the features. The molecule representing the discrete Fourier transform of a two-dimensional filter. The superscript represents the denominator of the discrete Fourier transform of a two-dimensional filter. Indicates complex conjugation. This indicates the parameters affected by regularization.

[0073] S300. Based on the motion state of the cross-multi-model filter, a second prediction is made to obtain the fused prior observations and determine the prior probability function.

[0074] It should be noted that some embodiments include the step of: determining the fusion state, fusion covariance, and motion state probability of the Kalman filters for different motion states by interacting the states of various different motion model filters through an interleaved multi-model filter; wherein, the expression for interacting the states of various different motion model filters is:

[0075]

[0076]

[0077]

[0078]

[0079]

[0080] In the formula, the subscript and The indexes representing Kalman filters in different motion states. Indicates the number of Kalman filters. Represents discrete time. Indicates the state of the Kalman filter. This indicates the fusion state of the Kalman filter. The covariance represents the state variable estimation error of the Kalman filter. Fusion covariance of state variable estimation errors in Kalman filters. Represents the probability of a motion state. Indicates the state of motion To the state of motion The fusion probability, Indicates the state of motion To the state of motion The state transition probability;

[0081] Then, based on the fusion state, fusion covariance, and motion state probability, fusion prior observations are obtained, and a prior probability function is determined. In some embodiments, this step includes: obtaining the fusion prior target state based on the fusion state, fusion covariance, and motion state probability; obtaining the fusion prior observations based on the fusion prior target state and the observation matrix; and determining the prior probability function based on the fusion prior observations, the Gaussian density function, and the observation innovation covariance. The observation innovation covariance is obtained from the motion model with the highest motion state probability among various motion models. The expression for the prior probability function is:

[0082]

[0083] In the formula, Denotes the prior probability function. A reference variable representing a function. Represents the Gaussian density function. This indicates the fusion of prior observations. This represents the covariance of observed new information.

[0084] S400. Based on the prior probability function, perform two-parameter constant false alarm rate detection with dynamic threshold in the estimated tracking box to obtain the detection result;

[0085] It should be noted that the detection results include both cases where the target was detected in the estimated tracking frame and cases where the target was not detected in the estimated tracking frame.

[0086] S500. Based on the detection results, adaptively update the correlation filter and the cross-model filter based on the estimated tracking box or the fusion of prior observations.

[0087] It should be noted that in some embodiments, the steps include: when a target is detected in the estimated tracking frame, updating the inter-multi-model filter using the estimated tracking frame, and updating the related filter using the fusion result of the motion state of the inter-multi-model filter; or, when no target is detected in the estimated tracking frame, updating the inter-multi-model filter using fused prior observations, and changing the position of the estimated tracking frame of the related filter without updating it.

[0088] In some embodiments, the method further includes: re-tracking the lost target by combining a prior probability function with a search strategy, and adding process noise parallel to the road to the tracking box with a fixed gain; wherein the expression for the process noise is:

[0089]

[0090] In the formula, Indicates process noise. Indicates fixed gain. This indicates the number of frames in which the two-parameter constant false alarm rate (CFAR) detection system continuously failed to detect the target. This represents the standard deviation of the two-parameter constant false alarm rate (CFAR) detection.

[0091] S600: Based on the results of adaptive updates, obtain the output tracking box of the target frame VideoSAR image.

[0092] Specifically, to facilitate understanding of the subsequent steps, a detailed explanation of the relevant radar signal model will be provided first:

[0093] Specifically, in some specific embodiments, the present invention provides a tracking framework for robust target tracking in VideoSAR. This framework mainly includes a correlation filter, an inter-parameter multi-model filter, and a two-parameter constant false alarm rate (2P-CFAR) algorithm based on dynamic thresholds. (Refer to...) Figure 2 The correlation filter locates the target shadow and extracts the appearance features of the shadow within the tracking box, training and optimizing the appearance template. Based on the prediction of the Interactive Multi-Model Filter (IMF), the dynamic threshold 2P-CFAR algorithm detects shadow targets in VideoSAR video frames. The IMF continuously tracks the target trajectory based on the motion model, and simultaneously determines whether to use the appearance feature tracking results of the correlation filter as observations based on the detection results of 2P-CFAR, correcting and iterating the target's motion state. Whether the appearance template of the correlation filter is updated and the final update position depend on the detection results of 2P-CFAR and the final prediction results of the IMF.

[0094] like Figure 2 As shown, the VideoSAR video frame sequence is input frame by frame, and the correlation filter (DCF) is applied based on the previous frame (the first frame). The tracking box obtained in frame 1, and the tracking box obtained in the current frame (frame 2). Features extracted from the VideoSAR image (frame 1) are used to predict the target state, resulting in an estimated tracking bounding box. Simultaneously, the Interactive Multi-Model (IMM) predicts based on the motion model, yielding a prior probability function. This prior probability function is then input into a dynamically thresholded 2P-CFAR, and detection is performed within the estimated tracking bounding box of the DCF. The IMM and DCF are updated based on the detection results, ultimately yielding the final result. The final tracking box of the target in the frame.

[0095] To describe in detail the functionality of the tracking framework in this embodiment of the invention, the various modules of this embodiment are described in detail below. Specific implementation details and processes will be described in detail according to the modules, specifically correlation filters, cross-multiple models, and 2P-CFAR with dynamic thresholding. It should be noted that the following is an explanation of the invention and should not be considered as a limitation thereof.

[0096] 1) Correlation filter

[0097] In the proposed shadow tracking framework, the DSST tracking algorithm, a correlation filter tracking algorithm, is introduced as a correlation filter (DCF) to track the appearance information of the target shadow. This method utilizes two independent filters to track the target: a two-dimensional filter for displacement estimation and a one-dimensional filter for scale estimation. The algorithm extracts the target's gradient orientation histogram features and pixel intensity features, and minimizes the loss function... To obtain the optimal filter .

[0098] (1)

[0099] In the formula, Representing the Layer features, This represents the total number of feature layers. The i-th layer features represent the extracted target, while For the first Filters for layer features. It is a Gaussian function. Indicates the relevant operation. The impact of the regularization term is determined by minimizing the loss function. The optimal filter is obtained by solving the problem. ,

[0100] (2)

[0101] In the formula, For the corresponding number Filters for layer features Discrete Fourier Transform, or The target number is respectively Layers and Discrete Fourier transform of layer features for The complex conjugate, This is the complex conjugate of the discrete Fourier transform of the Gaussian function. To obtain the solution more efficiently, the DSST algorithm decomposes equation (2) into the corresponding feature layers. molecules and the denominator The two are updated separately to obtain a robust approximation.

[0102] (3a)

[0103] (3b)

[0104] In the formula, This is the learning rate.

[0105] By extracting the feature map of the tracking box position in the previous frame The new target state can be obtained by maximizing the relevance score of the outcome.

[0106] (4)

[0107] In the formula, Represents the inverse discrete Fourier transform. To extract feature maps The Discrete Fourier transform of layer features The same applies to determining the effect of the regularization term.

[0108] The DSST algorithm trains a two-dimensional displacement filter using equations (3a) and (3b), and locates the target using equation (4). The located target position is also the estimated position (estimated tracking box) of the correlation filter (DCF), which will be used as the observation. The impact on the update of the interactive multi-model is explained in detail in Section 2). The one-dimensional scaling filter is trained and the target scale is determined through the same steps.

[0109] However, DSST is an algorithm designed for short-term tracking of optical targets. Directly applying it to VideoSAR tracking will result in poor sample quality for the filter due to the scarcity and variability of target shadows, thus contaminating the appearance template. Therefore, the tracking framework proposed in this invention introduces a 2P-CFAR detection algorithm, which can determine whether the target shadow is in a motion-blurred state or affected by strong scattering clutter, rendering it undetectable. Based on the detection results, the tracking framework can determine when to update the relevant filters.

[0110] 2) Interactive Multi-Model Tracking Framework

[0111] Interactive Multimodel Filter (IMF) is a tracking algorithm that incorporates several Kalman filters with different motion states, enabling accurate tracking of highly maneuverable targets. In the tracking framework proposed in this invention, the IMF includes three motion states: stationary, uniform velocity, and uniform acceleration. The target's state is set as follows: The measurement was set to In the formula It is discrete time. The target's state includes its position, velocity, and acceleration, while the measurement only includes position.

[0112] The first step in an inter-mode multi-model filter is to interact with the states of the various motion model filters.

[0113] (5)

[0114] (6)

[0115] (7)

[0116] In the formula, the subscript or Kalman filters representing different motion states, such as For the first The state of the Kalman filter in each motion state. It is the first The covariance of the state variable estimation error of a Kalman filter for each motion state It is the number of filters. and It represents the fusion state and covariance of each filter. It is the probability of motion state fusion from motion state to motion state. It is the probability of motion state, and It is the state transition probability from one state of motion to another.

[0117] The second step of the Interactive Multi-Model Filter (IMF) is prediction. The Kalman filter for each motion state predicts the target's state, resulting in a fused prior target state. and a fusion of prior observations .

[0118] (8)

[0119] (9)

[0120] (10)

[0121] (11)

[0122] (12)

[0123] In the formula, It is the state transition matrix of the motion state. It is the observation matrix. It is a state of motion The covariance matrix of the observed new information, and Respectively, motion states The covariance matrix of process noise and observation noise.

[0124] While the inter-model filter predicts the target's location, the correlation filter also predicts the target's location and scale, resulting in an estimated tracking box. Subsequently, based on the predicted location and observation information covariance, a dynamic threshold 2P-CFAR is used to detect the target. The detection process will be described in detail in Section 3).

[0125] The third step is updating. If a target is detected within the tracking box estimated by the correlation filter, the inter-multi-model filter will update the estimated location (estimated tracking box) of the correlation filter. As an observation, the filter is updated. Otherwise, the inter-model filter will select its own prediction. As an observation.

[0126] (13)

[0127] (14)

[0128] (15)

[0129] (16)

[0130] In the formula, In motion Kalman gain.

[0131] The final step is to fuse the motion states of all Kalman filters.

[0132] (17)

[0133] (18)

[0134] (19)

[0135] (20)

[0136] In the formula, Let be the likelihood function corresponding to the motion state. The final result in discrete time is... Subsequently, the correlation filter moves the estimated position for the next discrete time step to the final position, but whether the correlation filter is updated depends on the previous 2P-CFAR detection results.

[0137] Furthermore, the technical solution of this invention incorporates prior road information to constrain the target's trajectory during the interactive multi-model tracking process. For vehicles traveling on the road, the process noise parallel to the road direction must be less than the road width. This means that the original ideal Gaussian function has been truncated to a truncated Gaussian function with the following variance:

[0138] (twenty one)

[0139] In the formula, Let V be the variance of the original noise. The cumulative distribution function is... This is the Gaussian error function.

[0140] Meanwhile, within a certain error range, the target's velocity direction needs to point towards the road. Therefore, a spurious observation is added to the update process to constrain the target velocity.

[0141] (twenty two)

[0142] pseudo-observation in the formula , The angle between the road and the x-axis. It is the velocity observation matrix. This represents velocity-direction noise. By updating the inter-model filter using spurious observations, the tracking framework can correct the target's velocity.

[0143] 3) 2P-CFAR with dynamic threshold

[0144] The two-parameter constant false alarm rate (2P-CFAR) detection method is an adaptive threshold detection algorithm capable of detecting targets in complex backgrounds. By setting a target window, a protection window, and a background window, and sliding the window across the target image, the algorithm can detect the target using the following formula:

[0145] (twenty three)

[0146] In the formula and These are the mean values ​​of the target window and the background window, respectively. The standard deviation of the background window, the threshold The size of the window is determined by the false alarm rate. The size of the window is determined by the target scale estimated by the correlation filter during the tracking process.

[0147] In the traditional 2P-CFAR detection algorithm, the threshold The threshold is set to be the same globally. As a result, when performing a search and detection across the entire target image, the detection results will inevitably be affected by false alarms.

[0148] To overcome the above difficulties, a prior probability function is needed. It is introduced into the detection process. Based on historical motion information, It is a normalized two-dimensional Gaussian function, the center of which is the prediction location of the inter-model. Its covariance matrix is ​​the observation information covariance of the highest probability motion model.

[0149] (twenty four)

[0150] (25)

[0151] In the formula, This represents the Gaussian density function. The prior probability function can be obtained after the prediction step of the inter-model multi-interaction method, reconstructing the globally identical threshold of 2P-CFAR, and performing object detection on the entire image. The detection criterion is improved to...

[0152] (26)

[0153] By adding a prior probability function, the globally identical threshold is reconstructed into a dynamic threshold to suppress false alarms that may cause interference. Therefore, the dynamic threshold 2P-CFAR detection algorithm can accurately reveal the current appearance of the target shadow through the detection results.

[0154] Furthermore, the prior probability function can be combined with the search strategy to re-track the lost target. Relying solely on the prediction of the interactive multi-model filter results in a decrease in the reliability of predicting the target location as the number of lost frames increases. For this reason, the tracking framework uses a fixed gain. Increase process noise parallel to the road ,

[0155] (27)

[0156] In the formula This represents the number of frames in which 2P-CFAR has consistently failed to detect the target.

[0157] The increase in process noise parallel to the road affects the covariance matrix of the prior probability function through equations (8) and (9), causing it to increase in the road direction. This reduces the detection threshold along the road and increases the probability of re-searching for and locating the target. At the same time, the search area of ​​the correlation filter is also expanded until the target is re-detected.

[0158] 4) Experimental verification

[0159] Based on VideoSAR data provided by Sandia Labs, after preprocessing the images including filtering, grayscale inversion, and transformation, we conducted comparative experiments to test the performance of the proposed tracking framework. The experimental results are shown in the figure above. Taking two tracked targets, 2 and 9, as an example... Figure 3 The image shows the tracking segments. In frame 13 of target 2 and frame 27 of target 9, the target shadows disappear due to background clutter. Other trackers at this point may drift to nearby targets or have their templates contaminated by background clutter, causing tracking failure. The framework proposed in this embodiment of the invention will stop updating and continue moving based on historical motion information until it recaptures the target. In frame 52 of target 2 and frame 19 of target 9, based on the full use of motion information and dynamic thresholds, the framework proposed in this embodiment of the invention demonstrates good anti-false alarm capability. Other trackers may drift to background shadows and similar targets.

[0160] like Figure 4 As shown, in quantitative testing, the proposed tracking framework also demonstrated performance far exceeding that of other comparative trackers. In terms of success rate derived from the cross-union ratio and accuracy derived from the center distance difference, the proposed tracking framework achieved an accuracy of 86.9% and a success rate of 63.0%, respectively. Compared to other trackers, it outperforms SiamFC by 24.7%, ECO by 26.3%, and KCF by 32.8% in accuracy, and surpasses SiamFC by 7.1%, ECO by 17.1%, and KCF by 23.3% in success rate, proving that the tracking framework can effectively improve the robustness and accuracy of shadow tracking.

[0161] In summary, this invention effectively determines the appearance feature state of the tracked target by combining the 2P-CFAR detection algorithm and the correlation filter, thereby solving the template degradation problem in appearance tracking methods. Simultaneously, based on the motion information from the Interactive Multi-Model Tracking (IMM) method, a dynamic threshold replaces the globally fixed threshold used in most existing algorithms, reducing the impact of false alarms. Finally, road prior information, neglected by existing algorithms, is introduced, improving the robustness and accuracy of tracking. Specifically, by combining the 2P-CFAR detection algorithm with the correlation filter, the target position is effectively detected, influencing the updates of the correlation filter and the IMM, avoiding appearance template degradation caused by clutter and noise. Furthermore, the dynamic threshold, based on the motion information from the IMM filter, is used to reconstruct the 2P-CFAR detection threshold, reducing the impact of false alarms caused by complex background clutter and similar targets. At the same time, road prior information and search strategies are used in the IMM tracking process, limiting process noise and target speed, thus improving the robustness and accuracy of tracking.

[0162] On the other hand, embodiments of the present invention provide a VideoSAR shadow tracking system, comprising: a first module for acquiring an input image and an input tracking box; wherein the input image represents a target frame VideoSAR image in a VideoSAR video frame sequence, and the input tracking box represents an output tracking box obtained based on the VideoSAR image of the previous frame of the target frame; a second module for performing a first prediction on the input image based on the input tracking box using a correlation filter to obtain an estimated tracking box; wherein the correlation filter includes a two-dimensional filter for displacement estimation and a one-dimensional filter for scale estimation; a third module for performing a second prediction based on the motion state of an inter-multi-model filter to obtain fused prior observations and determine a prior probability function; a fourth module for performing two-parameter constant false alarm rate detection with dynamic threshold in the estimated tracking box based on the prior probability function to obtain a detection result; the detection result includes a target detected in the estimated tracking box and a target not detected in the estimated tracking box; a fifth module for adaptively updating the correlation filter and the inter-multi-model filter based on the detection result and the estimated tracking box or fused prior observations; and a sixth module for obtaining the output tracking box of the target frame VideoSAR image based on the result of the adaptive update.

[0163] It should be noted that, in some embodiments, the system further includes:

[0164] The seventh module is used to optimize the two-dimensional filter by minimizing the loss function.

[0165] The eighth module is used to re-track lost targets by combining prior probability functions with search strategies, and to add process noise parallel to the road to the tracking box with a fixed gain.

[0166] The content of the method embodiments of the present invention is applicable to the system embodiments. The specific functions implemented in the system embodiments are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0167] Another aspect of the present invention provides a VideoSAR shadow tracking device, including a processor and a memory;

[0168] Memory is used to store programs;

[0169] The processor executes the program as described above.

[0170] The content of the method embodiments of the present invention is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0171] Another aspect of this invention provides a computer-readable storage medium storing a program that is executed by a processor to implement the method described above.

[0172] The content of the method embodiments of the present invention is applicable to the computer-readable storage medium embodiments. The specific functions implemented by the computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0173] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned method.

[0174] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0175] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0176] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0177] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution means, apparatus, or device (such as a computer-based device, a processor-including device, or other means that can fetch and execute instructions from, or in conjunction with, an instruction execution means, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution means, apparatus, or device.

[0178] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0179] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution device. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0180] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0181] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0182] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A VideoSAR shadow tracking method, characterized by, The method comprises: obtaining an input picture and an input tracking box; wherein the input picture represents a target frame VideoSAR picture in a VideoSAR frame sequence, and the input tracking box represents an output tracking box obtained based on a VideoSAR picture of a previous frame of the target frame; performing first prediction on the input picture based on the input tracking box by using a correlation filter to obtain an estimated tracking box; wherein the correlation filter comprises a two-dimensional filter for displacement estimation and a one-dimensional filter for scale estimation; performing second prediction based on a motion state of an interacting multiple model filter to obtain fused prior observation, and determining a prior probability function; performing double-parameter constant false alarm detection of a dynamic threshold in the estimated tracking box based on the prior probability function to obtain a detection result; the detection result comprises detection of a target in the estimated tracking box and no detection of a target in the estimated tracking box; performing adaptive update on the correlation filter and the interacting multiple model filter based on the estimated tracking box or the fused prior observation according to the detection result; obtaining an output tracking box of the target frame VideoSAR picture based on a result of the adaptive update; wherein the method further comprises: optimizing the two-dimensional filter by minimizing a loss function; wherein an expression of the loss function is: wherein, denotes minimizing a loss function, denotes a hierarchical index of a feature, denotes a total number of layers of a feature, denotes a correlation operation, denotes a feature of an extraction target, denotes a Gaussian function, denotes a regularization influence parameter, denotes a two-dimensional filter; wherein the performing second prediction based on a motion state of an interacting multiple model filter to obtain fused prior observation, and determining a prior probability function, comprises: determining a fused state and a fused covariance of Kalman filters of different motion states and a motion state probability by interacting states of each different motion model filter through the interacting multiple model filter; obtaining fused prior observation and determining a prior probability function according to the fused state, the fused covariance and the motion state probability.

2. A VideoSAR shadow tracking method according to claim 1, wherein, The performing first prediction on the input picture based on the input tracking box by using a correlation filter to obtain an estimated tracking box, comprises: extracting target features of the input picture by using the correlation filter; wherein the target features comprise gradient direction histogram features and pixel intensity features; determining a discrete Fourier transform of the two-dimensional filter based on the target features; wherein an expression of the discrete Fourier transform of the two-dimensional filter is: wherein denotes the discrete Fourier transform of a two-dimensional filter, and denotes the hierarchical index of a feature, denotes the total number of layers of a feature, denotes the discrete Fourier transform of a Gaussian function, denotes the discrete Fourier transform of a feature, denotes a regularization influence parameter, superscript denotes complex conjugate; obtaining an estimated tracking box by extracting a feature map of the input tracking box and combining a determination result score of the discrete Fourier transform of the two-dimensional filter; wherein an expression of the result score is: wherein denotes the result score, denotes the inverse discrete Fourier transform, denotes the hierarchical index of the feature, denotes the total number of layers of the feature, denotes the numerator of the discrete Fourier transform of the two-dimensional filter, denotes the denominator of the discrete Fourier transform of the two-dimensional filter, the superscript denotes the complex conjugate, denotes the regularization influence parameter.

3. The VideoSAR shadow tracking method of claim 1, wherein, The obtaining fused prior observation and determining a prior probability function according to the fused state, the fused covariance and the motion state probability, comprises: obtaining a fused prior target state according to the fused state, the fused covariance and the motion state probability; obtaining fused prior observation by combining an observation matrix according to the fused prior target state; According to the fusion prior observation, a prior probability function is determined in combination with a Gaussian density function and an observation innovation covariance; wherein the observation innovation covariance is obtained through a motion model with the highest motion state probability in each different motion model; and an expression of the prior probability function is: wherein represents a prior probability function, represents a reference variable for a function, represents a Gaussian density function, represents a fused prior observation, represents an observation innovation covariance.

4. The VideoSAR shadow tracking method of claim 1, wherein, The adaptive updating of the correlation filter and the interacting multiple model filter based on the estimated tracking box or the fusion prior observation according to the detection result comprises: When a target is detected in the estimated tracking box, the interacting multiple model filter is updated using the estimated tracking box, and the correlation filter is updated using a fusion result of motion states of the interacting multiple model filter; Or, when no target is detected in the estimated tracking box, the interacting multiple model filter is updated using the fusion prior observation, and the estimated tracking box of the correlation filter is changed in position and not updated.

5. The VideoSAR shadow tracking method of claim 1, wherein, The method further comprises: The lost target is re-tracked by combining the prior probability function with a search strategy, and parallel process noise of the tracking box is increased with a fixed gain; wherein an expression of the process noise is: In the formula, represents process noise, represents a fixed gain, represents the number of frames in which the double-parameter constant false alarm detection fails to detect a target, represents a standard deviation based on the double-parameter constant false alarm detection.

6. A VideoSAR shadow tracking system characterized by, The VideoSAR shadow tracking method of claim 1 comprises: A first module is configured to acquire an input picture and an input tracking box; wherein the input picture represents a target frame VideoSAR picture in a VideoSAR video frame sequence, and the input tracking box represents an output tracking box obtained based on a previous frame VideoSAR picture of the target frame; A second module is configured to perform first prediction on the input picture based on the input tracking box by using a correlation filter, to obtain an estimated tracking box; wherein the correlation filter comprises a two-dimensional filter for displacement estimation and a one-dimensional filter for scale estimation; A third module is configured to perform second prediction based on motion states of an interacting multiple model filter, to obtain a fusion prior observation, and to determine a prior probability function; A fourth module is configured to perform dynamic threshold double-parameter constant false alarm detection in the estimated tracking box based on the prior probability function, to obtain a detection result; the detection result comprises that a target is detected in the estimated tracking box and that no target is detected in the estimated tracking box; A fifth module is configured to adaptively update the correlation filter and the interacting multiple model filter based on the estimated tracking box or the fusion prior observation according to the detection result; A sixth module is configured to obtain an output tracking box of the target frame VideoSAR picture based on a result of the adaptive updating.

7. A VideoSAR shadow tracking device comprising a processor and a memory; The memory is configured to store a program; The processor executes the program to implement the method of any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The storage medium stores a program, and the program is executed by a processor to implement the method of any one of claims 1 to 5.

Images