Filtering smoothing-based airborne video SAR image stabilization method and device

Abstract

The application discloses a kind of based on filtering smooth airborne video SAR image stabilization method and device, the method includes: the two-dimensional interframe shift observation vector sequence of computer airborne video SAR image sequence is calculated;Based on Kalman filter model, with the two-dimensional interframe shift observation vector of each frame in two-dimensional interframe shift observation vector sequence as input, Kalman filtering is executed, obtains full-frame posteriori state estimation sequence and full-frame posteriori error covariance sequence, Kalman filter model includes state equation, observation equation and filter recursion equation, and observation equation is determined according to two-dimensional interframe shift observation vector, and observation equation is two-dimensional Singer-random walk joint state transition equation;According to full-frame posteriori error covariance sequence, RTS reverse state smoothing is carried out to full-frame posteriori state estimation sequence, and video SAR image stabilization result is obtained.The application can effectively solve the problem of interframe shift of airborne video SAR image.

Description

Technical Field

[0001] This invention relates to the field of video SAR imaging technology, and in particular to an airborne video SAR image stabilization system and method based on filtering and smoothing. Background Technology

[0002] Synthetic Aperture Radar (SAR) boasts unique advantages, operating in all weather conditions and regardless of illumination, enabling high-resolution two-dimensional imaging of complex scenes and targets. However, traditional SAR suffers from long synthetic aperture times and image frame rates significantly lower than infrared or optical images, limiting its ability to capture dynamic scene changes. Video Synthetic Aperture Radar (ViSAR), a novel SAR imaging modality, combines video display technology with SAR imaging, achieving both high frame rates and high resolution simultaneously. Video SAR uses a spotlight mode to continuously adjust the radar beam, pointing it towards the center of the scene, acquiring continuous frames of the target area and ultimately displaying them as a video. Compared to traditional SAR, ViSAR has shorter synthetic aperture times, includes multiple synthetic apertures in a single flight, and can generate multi-frame image sequences at a higher frame rate, typically no less than 5Hz. Based on these advantages, ViSAR is currently widely used in the detection and tracking of moving targets.

[0003] However, due to various factors such as atmospheric turbulence, engine vibration, and changes in flight attitude, video SAR is prone to motion errors during actual echo data acquisition. ViSAR has high requirements for inter-frame image stability. Inter-frame motion errors can cause phase shifts in multiple frames, making the position of stationary targets change between consecutive frames. This interferes with the distinction between the background and the target, affecting the accuracy of target tracking and identification.

[0004] Video SAR primarily uses image registration methods to determine the optimal matching relationship between two frames of images in the same scene and complete geometric alignment, thereby achieving inter-frame image stabilization. Currently, image matching methods are mainly divided into two categories: feature-based matching and gray-level matching. While the former has good adaptability to gray-level changes, deformations, and occlusions, it is easily affected by strong multiplicative speckle noise in SAR images, leading to image distortion. Normalized Cross Correlation (NCC) matching is a typical gray-level-based image matching algorithm. This method uses the cross-correlation coefficient as a similarity metric and searches for the optimal translation amount that maximizes the correlation coefficient to achieve translational registration of the images to be registered. It incorporates mean removal and normalization operations during the calculation process, thus exhibiting good robustness and applicability in the field of image registration.

[0005] However, in practical applications of spotlight-mode airborne video SAR, relying solely on frame-by-frame normalized cross-correlation registration is insufficient to meet the requirements for high-precision image stabilization. From an imaging mechanism perspective, residual platform motion errors introduce defocusing effects in the image domain. The local geometric mismatch caused by defocusing and imaging geometric approximation violates the assumption in the NCC registration method that the same scattering object remains approximately unchanged across different frames and only undergoes translation, causing a deviation in the position of the main cross-correlation peak relative to the true drift. From an algorithm implementation perspective, sub-pixel translation requires interpolation resampling. The interpolation kernel type and parameter settings alter the local gray-scale fitting method, thereby changing the peak position of the cross-correlation surface and introducing additional systematic bias. Furthermore, SAR speckle noise randomly perturbs the position of the main cross-correlation peak, causing significant fluctuations in the frame-by-frame estimation results near the true value. These factors mean that NCC-based drift estimation simultaneously includes random errors and slowly changing systematic errors. In the image sequence registered by this method, the stationary point target still has pixel-level reciprocating positional deviation between consecutive frames, that is, the inter-frame jitter phenomenon still exists, and the multi-frame image drift problem has not been fundamentally eliminated.

[0006] The background section described above is merely a description made by the inventor based on his understanding, and the above content should not be regarded as evidence of prior art disclosed before the filing date of this application. Summary of the Invention

[0007] This invention provides an airborne video SAR image stabilization method based on filtering and smoothing to effectively solve the inter-frame drift problem of airborne video SAR images. The method includes: Two-dimensional inter-frame drift observation vector sequence of computer-loaded video SAR image sequence; Based on the Kalman filtering model, the two-dimensional inter-frame drift observation vector of each frame in the two-dimensional inter-frame drift observation vector sequence is used as input. Kalman filtering is performed to obtain the full-frame posterior state estimation sequence and the full-frame posterior error covariance sequence. The Kalman filtering model includes a state equation, an observation equation, and a filtering recursive equation. The observation equation is determined based on the two-dimensional inter-frame drift observation vector. The observation equation is a two-dimensional Singer-random walk joint state transition equation. The posterior state estimation of each frame in the full-frame posterior state estimation sequence includes drift displacement estimation, drift velocity estimation, drift acceleration estimation, and scalar bias estimation. Based on the full-frame posterior error covariance sequence, the full-frame posterior state estimation sequence is subjected to RTS inverse state smoothing to obtain the stable image result of the video SAR image.

[0008] Another aspect of the present invention provides an airborne video SAR image stabilization device based on filtering and smoothing to effectively solve the problem of inter-frame drift in airborne video SAR images. The device includes: A drift observation extraction module is used to extract two-dimensional inter-frame drift observation vector sequences from computer-loaded video SAR image sequences. The Kalman filtering module is used to perform Kalman filtering based on the Kalman filtering model, taking the two-dimensional inter-frame drift observation vector of each frame in the two-dimensional inter-frame drift observation vector sequence as input, to obtain the full-frame posterior state estimation sequence and the full-frame posterior error covariance sequence. The Kalman filtering model includes a state equation, an observation equation, and a filtering recursive equation. The observation equation is determined based on the two-dimensional inter-frame drift observation vector and is a two-dimensional Singer-random walk joint state transition equation. The posterior state estimation of each frame in the full-frame posterior state estimation sequence includes drift displacement estimation, drift velocity estimation, drift acceleration estimation, and scalar bias estimation. The state smoothing module is used to perform RTS inverse state smoothing on the full-frame posterior state estimation sequence based on the full-frame posterior error covariance sequence, so as to obtain the stable image result of the video SAR image.

[0009] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described airborne video SAR stabilization method based on filtering and smoothing.

[0010] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described airborne video SAR stabilization method based on filtering and smoothing.

[0011] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described airborne video SAR stabilization method based on filtering and smoothing.

[0012] In this embodiment of the invention, a two-dimensional inter-frame drift observation vector sequence of a computer-borne video SAR image sequence is obtained. Based on a Kalman filtering model, the two-dimensional inter-frame drift observation vector of each frame in the two-dimensional inter-frame drift observation vector sequence is used as input to perform Kalman filtering, resulting in a full-frame posterior state estimation sequence and a full-frame posterior error covariance sequence. The Kalman filtering model includes a state equation, an observation equation, and a filtering recursive equation. The observation equation is determined based on the two-dimensional inter-frame drift observation vector and is a two-dimensional Singer-random walk joint state transition equation. The posterior state estimation of each frame in the full-frame posterior state estimation sequence includes drift displacement estimation, drift velocity estimation, drift acceleration estimation, and scalar bias estimation. Based on the full-frame posterior error covariance sequence, the full-frame posterior state estimation sequence is subjected to RTS inverse state smoothing to obtain the video SAR image stabilization result. Compared with existing technical solutions, this invention introduces Kalman filtering and RTS inverse state smoothing on the basis of the two-dimensional Singer-random walk joint state transition equation. It performs forward recursive estimation and inverse global smoothing on NCC drift observations, effectively utilizes the correlation of video SAR drift trajectory over time, improves the estimation accuracy of inter-frame drift and system bias, and realizes accurate generation of drift trajectory for long-term series. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 This is a flowchart of an airborne video SAR image stabilization method based on filtering and smoothing in an embodiment of the present invention; Figure 2 This is a flowchart of the full-image registration method based on normalized cross-correlation in an embodiment of the present invention; Figure 3 This is a detailed flowchart of the Kalman filtering process in an embodiment of the present invention; Figure 4 This is a flowchart of RTS reverse state smoothing in an embodiment of the present invention; Figure 5 This is a schematic diagram of the airborne video SAR image stabilization device based on filtering and smoothing in an embodiment of the present invention; Figure 6 This is a schematic diagram of a computer device in an embodiment of the present invention. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0015] To address the aforementioned problems, this invention proposes an airborne video SAR image stabilization method and apparatus based on filtering and smoothing. The method first uses the first frame as the reference frame and performs a global coarse search on an integer pixel grid using full-image normalized cross-correlation. Then, a sub-pixel fine search is conducted near the coarse search result to obtain sub-pixel-level drift observation sequences in the azimuth and range directions. Subsequently, in the time dimension, the true drift component is modeled as a two-dimensional Singer motion process, and the systematic bias is modeled as a random walk process. These are jointly represented within a unified state-space framework, constructing a state-space model containing displacement, velocity, acceleration, and slowly varying bias components. Compared to directly using frame-by-frame NCC registration, this approach distinguishes and models the true drift and systematic bias in the state space, elevating drift estimation from a simple "single-frame correlation peak selection" to an optimal estimate constrained by both the state equation and the observation equation. Building upon this foundation, Kalman filtering and Rauch-Tung-Striebel (RTS) fixed-interval smoothing are introduced to perform temporal filtering and smoothing reconstruction on the NCC drift observations across the entire sequence. This effectively utilizes the temporal correlation of the video SAR drift trajectory, yielding a smoother inter-frame drift estimate that closely approximates the true value. Finally, sub-pixel shifting is performed on the original image sequence based on the smoothed drift trajectory, achieving overall image stabilization of multiple ViSAR frames.

[0016] Figure 1 This is a flowchart of an airborne video SAR image stabilization method based on filtering and smoothing, as described in an embodiment of the present invention. Figure 1 As shown, the method includes: Step 101: Two-dimensional inter-frame drift observation vector sequence of computer-borne video SAR image sequence; Step 102: Based on the Kalman filtering model, using the two-dimensional inter-frame drift observation vector of each frame in the two-dimensional inter-frame drift observation vector sequence as input, Kalman filtering is performed to obtain the full-frame posterior state estimation sequence and the full-frame posterior error covariance sequence. The Kalman filtering model includes a state equation, an observation equation, and a filtering recursive equation. The observation equation is determined based on the two-dimensional inter-frame drift observation vector. The observation equation is a two-dimensional Singer-random walk joint state transition equation. The posterior state estimation of each frame in the full-frame posterior state estimation sequence includes drift displacement estimation, drift velocity estimation, drift acceleration estimation, and scalar bias estimation. Step 103: Based on the full-frame posterior error covariance sequence, perform RTS inverse state smoothing on the full-frame posterior state estimation sequence to obtain the stable image result of the video SAR image.

[0017] Depend on Figure 1 As shown in the structural diagram, in this embodiment of the invention, based on the two-dimensional Singer-random walk joint state transition equation, Kalman filtering and RTS reverse state smoothing are introduced to perform forward recursive estimation and reverse global smoothing of NCC drift observations. This effectively utilizes the correlation of video SAR drift trajectory over time, improves the estimation accuracy of inter-frame drift and system bias, and realizes the accurate generation of drift trajectories for long-term sequences.

[0018] In step 101, the computer-borne video SAR image sequence is a two-dimensional inter-frame drift observation vector sequence; In spotlight mode airborne video SAR image sequences, even if the same stationary point target in the observed scene remains physically unchanged, its imaging position in different frames will still exhibit two-dimensional inter-frame drift along the azimuth and range directions. The main reason for this is that during video SAR image acquisition, there are unavoidable estimation errors in the azimuth and range directions of the aircraft's position and velocity. This causes the actual slant range between the radar antenna phase center and the point target to deviate from the ideal slant range in the imaging model, thus introducing motion errors into the echo signal. Conventional data-driven motion compensation algorithms struggle to completely eliminate these errors; their residual portion manifests as inter-frame residual motion phase errors during multi-frame imaging. After focused imaging, this phase error is converted into a two-dimensional pixel offset of the point target in the image domain, and this offset gradually accumulates with the increase in the number of frames.

[0019] To suppress the inter-frame drift caused by residual motion errors, this embodiment of the invention first employs a full-map registration method based on normalized cross-correlation to estimate the two-dimensional translation of each frame relative to the reference frame.

[0020] In one embodiment, the two-dimensional inter-frame drift observation vector sequence of a computer-borne video SAR image sequence includes: Using the first frame of the video SAR image as the reference frame and the remaining frames as the images to be registered, a coarse search is performed on each frame of the images to be registered in sequence on a preset integer pixel grid to obtain the optimal integer pixel translation amount corresponding to each frame of the images to be registered. The fine search range is determined with the optimal integer pixel translation amount as the center. Subpixel translation of each frame of the image to be registered is achieved by interpolation resampling. Fine search is performed within the fine search range to obtain the optimal subpixel translation amount of each frame of the image to be registered. The optimal subpixel translation of each frame of the image to be registered is converted into a distance drift. Based on the drift amount of each frame of the image to be registered, the two-dimensional inter-frame drift observation vector of each frame of the image to be registered is determined, and a two-dimensional inter-frame drift observation vector sequence is formed.

[0021] In one embodiment, the first frame of the video SAR image is used as the reference frame, and the remaining frames are used as images to be registered. A coarse search is performed sequentially on each image to be registered within a preset integer pixel grid to obtain the optimal integer pixel translation amount corresponding to each image to be registered, including: Based on a preset integer pixel grid, generate a set of integer pixel translation amounts in the azimuth and range directions; For each integer pixel translation amount in the set of integer pixel translation amounts, the following steps are performed: For each frame of the image to be registered, the image to be registered is translated using the integer pixel translation operator corresponding to the integer pixel translation amount, and zero-filling is used for positions that exceed the original pixel area; the first effective overlap area between the translated image to be registered and the reference frame is determined by using a visible domain mask, and the first effective overlap pixel index set with valid values ​​is selected; based on the first effective overlap pixel index set, the normalized cross-correlation coefficient between the reference frame and each translated image to be registered is calculated; For each frame of the image to be registered, a coarse search is performed on the integer pixel grid, and the integer pixel translation that maximizes the normalized cross-correlation coefficient in the set of integer pixel translations is taken as the optimal integer pixel translation for each frame of the image to be registered.

[0022] In one embodiment, a fine search range is determined centered on the optimal integer pixel translation amount. Subpixel translation of each frame of the image to be registered is achieved through interpolation resampling. A fine search is then performed within the fine search range to obtain the optimal subpixel translation amount for each frame of the image to be registered. This includes: Centered on the optimal integer pixel translation amount, generate a set of sub-pixel translation amounts in the azimuth and range directions, and use it as the fine search range; For each subpixel translation amount in the subpixel translation amount set, the following steps are performed: each frame of the image to be registered is subpixel translated according to the subpixel translation amount by interpolation resampling to obtain the resampled image to be registered for each frame; the second effective overlap region between the reference frame and each resampled image to be registered is determined by the visible domain mask, and the second effective overlap pixel index set with effective values ​​is selected; based on the second effective overlap pixel index set, the normalized cross-correlation coefficient between the reference frame and each resampled image to be registered is calculated; For each frame of the image to be registered, the subpixel translation amount that maximizes the normalized cross-correlation coefficient in the set of subpixel translation amounts is taken as the optimal subpixel translation amount.

[0023] Figure 2 This is a flowchart of the full image registration method based on normalized cross-correlation in an embodiment of the present invention, which is a detailed introduction to the above-mentioned full image registration method based on normalized cross-correlation.

[0024] See Figure 2 Based on the airborne video SAR image sequence of tracking filtering, the first frame of the video SAR image is used as the reference frame, and its grayscale (or amplitude) image is denoted as... .in and These represent the number of pixels in the azimuth and range directions, respectively. For azimuth pixel index, For distance-oriented pixel indexing. The first... The frame of the image to be registered is represented as In the specific implementation, both images are first converted to grayscale and then normalized to improve the numerical stability of subsequent correlation calculations. For ease of description, the pixel coordinate domain of the image is denoted as... For a given integer pixel translation amount ,(in This is the azimuth translation amount. (where the distance is the translation amount, and the unit is pixels). The integer pixel translation operator applied to the image to be registered is defined as follows: (1) That is, the images to be registered Perform integer pixel translation, and extend beyond the original pixel area. The positions are zero-padding. To avoid zero-padding pixels participating in correlation calculations, the first effective overlap region between the reference frame and the translated image to be registered can be determined by a "visible domain mask". The mask region corresponding to the visible domain mask is defined as follows: (2) If and only if the reference frame Image to be registered after translation At pixel position When both values ​​have valid values, the mask value is 1. Based on this, the translation amount can be... The first set of valid overlapping pixel indices with valid values ​​is defined as: (3) in, Indicates the cardinality of the first set of valid overlapping pixel indices. This represents the number of pixels in the first effective overlapping region.

[0025] Therefore, given an integer pixel translation amount Under the conditions, the first The normalized cross-correlation coefficient (NCC) of the frame to be registered relative to the reference frame is defined as follows: (4) in, and They represent the first set of valid overlapping pixel indices respectively. The mean pixel value of the reference frame and the translated image to be registered within the first effective overlapping area. When performing a coarse search on an integer pixel grid, a coarse search range needs to be set, corresponding to the set of integer pixel translations formed by all integer pixel translations in the azimuth and range directions, represented as follows in the azimuth and range directions. and : (5) in, and They represent the first The frame to be registered is coarsely searched for its center in the azimuth and range directions. and This corresponds to the coarse search step size. and This represents the coarse search radius in the corresponding direction.

[0026] For the For the frame to be registered, the optimal integer pixel translation (also known as the optimal integer pixel translation estimate) in the coarse search stage can be defined as the translation amount that makes the NCC coefficients reach their maximum value within the coarse search range mentioned above: (6) To obtain the optimal integer pixel translation amount in the coarse search phase Subsequently, to further improve the drift estimation accuracy, a sub-pixel level fine search can be performed within its neighborhood. Similar to the coarse search, the fine search also defines sets of sub-pixel translations in the azimuth and range directions, but the search points are refined into a real-valued grid. The th... The set of subpixel translations of a frame is represented as: (7) in, , , for the first The frame of the image to be registered has a fine search center in the azimuth and range directions. and This corresponds to the fine search step size (usually much smaller than 1 pixel). and This is the fine search radius in the corresponding direction.

[0027] When translation amount When the value is a real number, the pixel positions of the image to be registered will be mapped to non-integer coordinates between the original pixel grids after translation. Sub-pixel translation is required to construct the image through interpolation resampling, resulting in the resampled image to be registered. The translation amount will be... Next The image to be registered after frame resampling is defined as For any integer grid points in the resampled image to be registered. Its grayscale value can be expressed as the original image to be registered in continuous coordinates. The interpolated weighted sum of pixel grayscale values ​​within the integer neighborhood of the original video SAR image. The model is determined jointly by the interpolation kernel and the model itself. This embodiment of the invention uses a one-dimensional interpolation kernel with separable variables for modeling, for any given translation amount... , It can be represented as: (8) in, For a one-dimensional interpolation kernel, Integer pixels in the neighborhood of the original video SAR image The resampled image to be registered Interpolation weights at the specified points.

[0028] Next, you can Define the corresponding visible domain mask and the set of pixels in the effective overlapping region, and construct the NCC coefficients for the fine search stage based on this. Similar to the coarse search phase, the fine search phase determines the translation amount that maximizes the NCC coefficient within the set of sub-pixel translation amounts: (9) After completing the coarse search and subpixel fine search, the first... The optimal subpixel translation estimate of a frame relative to a reference frame is denoted as ,in and Let represent the optimal sub-pixel translation amounts (in pixels) in the azimuth and range directions, respectively. Suppose the resolutions of the video SAR image in the azimuth and range directions are respectively... and (Unit: meters / pixel), then the pixel translation can be further converted into the drift in the sense of physical distance: (10) In state-space modeling, as long as the physical dimensions of the relevant variables in the state equation and the observation equation remain consistent, a unified state-space form can be used to describe the system in either the pixel domain or the distance domain. In this embodiment of the invention, the subsequent model will... and As the first The frame drift observations. Based on this, the two-dimensional inter-frame drift observation vector of the frame can be defined as: (11) It should be pointed out that, Only under ideal conditions does it perfectly match the true drift. For actual spotlight airborne ViSAR imaging, factors such as defocusing effects and local geometric mismatches caused by imaging geometric approximations, sub-pixel-level deviations due to interpolation resampling and gray quantization, and the perturbation of the main peak position of cross-correlation by SAR speckle noise can all cause the translation estimate based on normalized cross-correlation to deviate from the true drift. In view of the above sources of error, It can be represented as an observation consisting of the superposition of the true drift, the slowly changing systematic bias over time, and the zero-mean random observation error. The corresponding statistical decomposition will be given in the subsequent observation equations.

[0029] In one embodiment, the method further includes: Construct a discrete one-dimensional Singer motion model, wherein the state vector of the discrete one-dimensional Singer motion model includes drift displacement, drift velocity, and drift acceleration; Construct a discrete one-dimensional random walk deviation model. The state vector of the discrete one-dimensional random walk deviation model includes scalar deviation. A one-dimensional joint state transition equation is constructed based on the discrete one-dimensional Singer motion model and the discrete one-dimensional random walk deviation model. Based on the one-dimensional joint state transition equations of the azimuth and range directions, a two-dimensional Singer-random walk joint state transition equation is constructed.

[0030] To further filter and smooth the inter-frame drift obtained from correlation registration, this embodiment of the invention models the drift trajectory as a two-dimensional random process: its backbone is characterized by the Singer acceleration model, and the observation end is superimposed with a random walk bias term that changes slowly over time.

[0031] The process of constructing a one-dimensional Singer motion model includes: taking a single drift component in the azimuth (or range) direction as an example, and denoting the one-dimensional motion state in continuous time as... ,in , , These represent drift displacement, drift velocity, and drift acceleration, respectively. The Singer model assumes that the acceleration follows a first-order Markov process. (12) in, To accelerate the exponential decay rate of autoregression, This indicates the acceleration-related time. Zero-mean Gaussian white noise is used to drive the random acceleration process. For the Dirac function, It represents the mathematical expectation. Acceleration-driven noise The power spectral density. To ensure the steady-state acceleration variance is... , .

[0032] Based on the relationship between displacement, velocity, and acceleration , The equations for the continuous-time one-dimensional Singer motion model can be obtained as follows: (13) in For the system matrix, This represents the noise gain matrix.

[0033] Suppose that the inter-frame interval of video SAR during multi-frame imaging is . (each time elapsed) (Obtain the position coordinates once), then the first... The imaging time corresponding to a frame can be expressed as: Then ordered , , In the interval By equivalent discretization of equation (13), the following expression for the discrete one-dimensional Singer motion model can be obtained: (14) (15) in, This is the discrete-time state transition matrix. This represents discrete-time process noise. Conforms to Gaussian white noise characteristics. Let it be its corresponding covariance matrix.

[0034] The process of constructing a discrete one-dimensional random walk bias model includes the following: In ViSAR image sequences, although the imaging scene is static, the two-dimensional inter-frame drift observation vector based on gray-level / amplitude correlation is usually not equivalent to the "true image drift amount," but rather contains multiple systematic error components. Specifically, typical influencing factors include: local geometric mismatch caused by defocusing effect and imaging geometric approximation, sub-pixel level bias caused by image interpolation resampling and gray-level quantization, and the shift in the main peak position of the cross-correlation function caused by SAR speckle noise. In summary, due to the combined effect of the above factors, the drift amount obtained by the full-image registration method can be statistically decomposed into: the true drift amount, the slowly changing systematic bias over time, and the zero-mean high-frequency observation noise.

[0035] To explicitly characterize such slowly changing system deviations in a state-space model, this embodiment of the invention introduces a scalar deviation state in addition to the "true drift" state. It is modeled as a continuous-time random walk process, which can be expressed as: (16) in, For zero-mean Gaussian white noise, the corresponding power spectral density is .

[0036] At discrete time Sampling is performed on the above, and equation (16) is applied in the interval. Integrating the equations, we obtain the discrete-time random walk equations, forming a discrete one-dimensional random walk bias model, which can be expressed as: (17) The one-dimensional Singer state and the random walk deviation are combined to form a one-dimensional extended state vector. : (18) Where the subscript x indicates the azimuth direction. From equations (14)-(15) and (17), the one-dimensional joint state transition equation can be obtained, as follows: (19) (20) in, Let be the joint state transition matrix in the x-direction. For the joint process noise vector, Let its corresponding covariance matrix be... Let be the variance of the random walk deviation in the x-direction per frame increment. The one-dimensional joint state transition equation for the distance to y-direction is completely consistent with the forms of equations (18)-(20), only the parameter values ​​are different. Let the corresponding state and matrix be denoted as . , , .

[0037] To establish a unified state-space representation of the drift process of video SAR images in a two-dimensional plane, the one-dimensional state vectors in the azimuth and range directions are represented at discrete time intervals. Stacked by components. Define an 8-dimensional joint state vector in two-dimensional discrete time. and joint process noise vector They are respectively: (twenty one) Assuming that the process noises in the x and y directions are independent, the two one-dimensional joint state transition equations can be combined into a two-dimensional joint state transition equation: (twenty two) in, Represents a two-dimensional joint state transition matrix. Let be the covariance matrix of the two-dimensional joint process noise. and The specific expression is: (twenty three) Equation (22) is the two-dimensional Singer-random walk joint state transition equation used in ViSAR drift estimation in this embodiment of the invention. This model describes the displacement, velocity, acceleration and their slowly varying deviation terms in both the azimuth and range directions under a unified framework, and reflects the statistical independence of the process noise in the two directions through a block diagonal matrix, providing a strict state-space basis for subsequent Kalman filtering and RTS smoothing.

[0038] Based on the aforementioned two-dimensional Singer-random walk joint state transition equation, it is also necessary to combine the two-dimensional inter-frame drift observation vector obtained from the video SAR image sequence to construct an observation equation that matches the state vector.

[0039] Since the two-dimensional inter-frame drift observation vector can be considered as the sum of the actual drift and the slowly varying system bias, superimposed with zero-mean random observation noise, and combined with the definition of the joint state vector, the observation equation can be established: (twenty four) in, For the observation matrix, Given a two-dimensional observation noise vector, if we assume... If the noise is zero-mean Gaussian white noise, then it satisfies the following condition: (25) In the formula, and These represent the standard deviations of the observation noise in the azimuth and range directions, respectively, and are used to quantify the degree of uncertainty caused by the two-dimensional inter-frame drift observation vector. To observe the noise covariance matrix.

[0040] In the state equation With observation equation Under known conditions, the two-dimensional inter-frame drift observation vector can be modeled as a discrete-time linear Gaussian state-space model. Let... As of the end of the The posterior state estimate obtained from frame observations For only use before Prior predictions obtained from frame observations. and These are the corresponding error covariance matrices. The filter initialization can be expressed as: (26) in, The settings are usually based on empirical values ​​or coarse registration results. The uncertainty of the initial estimate is described by specifying an appropriate initial error variance for each state component, and its value has a direct impact on the distribution of the Kalman gain in the first few frames.

[0041] In obtaining the first Before frame observation, the prior state estimate is updated based on the state transition equation and the error covariance matrix. and prior error covariance : (27) Obtain the observation values Then, construct the observed predicted values. With new information : (28) in, This indicates the deviation between the actual observed value and the predicted observed value. This is used to quantify the statistical uncertainty of the deviation.

[0042] Calculate the Kalman gain according to equation (28) to complete the observation update: (29) Based on the recursion from equation (27) to equation (29), This is the posterior state estimation, also known as the filtered state estimation over the entire frame sequence. , The posterior error covariance is used for the next filtering calculation. Equations (27) to (29) are the filtering recursive formulas for the Kalman filtering model. and The drift displacement estimate relative to the reference frame is given, while and It is used to absorb the scalar bias estimation accumulated frame by frame during the correlation registration process, realize the modeling of the difference between the actual drift and the observation bias, and provide a basis for subsequent fixed interval smoothing and multi-frame image stabilization.

[0043] In step 102, based on the Kalman filtering model, the two-dimensional inter-frame drift observation vector of each frame in the two-dimensional inter-frame drift observation vector sequence is used as input to perform Kalman filtering, resulting in a full-frame posterior state estimation sequence and a full-frame posterior error covariance sequence. The Kalman filtering model includes a state equation, an observation equation, and a filtering recursive equation. The observation equation is determined based on the two-dimensional inter-frame drift observation vector and is a two-dimensional Singer-random walk joint state transition equation. The posterior state estimation of each frame in the full-frame posterior state estimation sequence includes drift displacement estimation, drift velocity estimation, drift acceleration estimation, and scalar bias estimation. Figure 3 This is a detailed flowchart of the Kalman filtering process in one embodiment of the invention. Based on the Kalman filtering model, using the two-dimensional inter-frame drift observation vector of each frame in the two-dimensional inter-frame drift observation vector sequence as input, Kalman filtering is performed to obtain the full-frame posterior state estimation sequence and the full-frame posterior error covariance sequence, including: Step 301: Initialize the posterior state estimate and error covariance matrix; Step 302: Based on the state transition equation and the error covariance matrix, calculate the prior state estimate and prior error covariance for each frame; Step 303: Calculate the observation prediction value and innovation based on the two-dimensional inter-frame drift observation vector of each frame and the prior state estimate of each frame; Step 304: Calculate the Kalman gain. Based on the Kalman gain, the prior state estimate of each frame, and the innovation, calculate the posterior state estimate of each frame. Form the posterior state estimates of all frames into a full-frame posterior state estimate sequence. Step 305: Calculate the full-frame posterior error covariance sequence based on the Kalman gain and prior error covariance.

[0044] In step 103, based on the full-frame posterior error covariance sequence, the full-frame posterior state estimation sequence is subjected to RTS inverse state smoothing to obtain the stable image result of the video SAR image.

[0045] Figure 4 This is a flowchart of the RTS inverse state smoothing process in an embodiment of the present invention. In one embodiment, based on the full-frame posterior error covariance sequence, RTS inverse state smoothing is performed on the full-frame posterior state estimation sequence to obtain the stable video SAR image result, including: Step 401: Starting from the second-to-last frame, proceed backwards to each frame up to the first frame, and obtain the RTS smoothing gain of the current frame based on the posterior error covariance of the current frame in the full frame posterior error covariance sequence. Step 402: Based on the posterior error covariance, posterior state estimate, and RTS smoothing gain of the current frame, obtain the smoothed state estimate and smoothing error covariance of the current frame. Step 403: The smooth drift estimate in the smooth state estimate of the current frame is shifted relative to the reference frame to achieve inter-frame image stabilization in the image domain and obtain the video SAR image stabilization result.

[0046] The posterior state estimation sequence was obtained through Kalman forward filtering. and its corresponding error covariance matrix ,in, For causal state estimation, only the previous state is used. Frame-2D Inter-Frame Drift Observation Vector Sequence The calculations yielded the following results. Regarding the multi-frame video SAR image stabilization problem studied in this embodiment of the invention, since the two-dimensional inter-frame drift observation vector sequence has already been obtained during the processing stage... Therefore, an RTS fixed-interval smoother can be introduced based on the posterior state estimation sequence obtained by forward filtering to perform reverse recursive correction on the posterior state estimation sequence, so that the posterior state estimation of each frame simultaneously integrates past and future observation information to obtain a smoothed state estimate. And error covariance matrix Compared to causal estimation results using only forward Kalman filtering, RTS smoothing estimation achieves the best non-causal estimation in the sense of mean square error. It not only more accurately approximates the true drift state, but also exhibits better smoothness consistency across the entire frame sequence.

[0047] In the last frame of the frame sequence Due to the posterior state estimation of this frame Having utilized all two-dimensional inter-frame drift observation vectors, its smoothing estimate is the same as the filtering estimate; therefore, the initial condition for RTS smoothing can be defined as follows: Subsequently, regarding Perform reverse recursion sequentially, and then... The RTS smoothing gain of a frame is defined as: (30) in, It is a two-dimensional joint state transition matrix. For the first Frame post-hoc error covariance For the first The frame state prediction obtained from the first Frame prior error covariance.

[0048] In a given Under the conditions, the first The smoothed state estimation of the frame and the corresponding smoothing error covariance can be expressed as: (31) After completing the backward recursion, the full-frame smooth state estimation sequence and the full-frame smooth error covariance sequence can be obtained. . State components in and Corresponding to the first Frame smooth drift estimation in azimuth and range directions. By using the first... The original video SAR image of the frame is translated relative to the reference frame. This achieves inter-frame image stabilization in the image domain and utilizes... The uncertainty of drift estimation for each frame is quantitatively assessed.

[0049] This invention also provides an airborne video SAR image stabilization device based on filtering and smoothing, as described in the following embodiments. Since the principle by which this device solves the problem is similar to that of the airborne video SAR image stabilization method based on filtering and smoothing, the implementation of this method can be found in the implementation of the airborne video SAR image stabilization method based on filtering and smoothing; repeated details will not be elaborated further.

[0050] Figure 5 This is a schematic diagram of the airborne video SAR image stabilization device based on filtering and smoothing in an embodiment of the present invention, including: The drift observation extraction module 501 is used for the two-dimensional inter-frame drift observation vector sequence of computer-borne video SAR image sequences; Kalman filtering module 502 is used to perform Kalman filtering based on a Kalman filtering model, taking the two-dimensional inter-frame drift observation vector of each frame in the two-dimensional inter-frame drift observation vector sequence as input, to obtain a full-frame posterior state estimation sequence and a full-frame posterior error covariance sequence. The Kalman filtering model includes a state equation, an observation equation, and a filtering recursive equation. The observation equation is determined based on the two-dimensional inter-frame drift observation vector and is a two-dimensional Singer-random walk joint state transition equation. The posterior state estimation of each frame in the full-frame posterior state estimation sequence includes drift displacement estimation, drift velocity estimation, drift acceleration estimation, and scalar bias estimation. The state smoothing module 503 is used to perform RTS inverse state smoothing on the full-frame posterior state estimation sequence based on the full-frame posterior error covariance sequence to obtain the stable image result of the video SAR image.

[0051] In this embodiment, the drift observation extraction module is used for: Using the first frame of the video SAR image as the reference frame and the remaining frames as the images to be registered, a coarse search is performed on each frame of the images to be registered in sequence on a preset integer pixel grid to obtain the optimal integer pixel translation amount corresponding to each frame of the images to be registered. The fine search range is determined with the optimal integer pixel translation amount as the center. Subpixel translation of each frame of the image to be registered is achieved by interpolation resampling. Fine search is performed within the fine search range to obtain the optimal subpixel translation amount of each frame of the image to be registered. The optimal subpixel translation of each frame of the image to be registered is converted into a distance drift. Based on the drift amount of each frame of the image to be registered, the two-dimensional inter-frame drift observation vector of each frame of the image to be registered is determined, and a two-dimensional inter-frame drift observation vector sequence is formed.

[0052] In this embodiment, the drift observation extraction module is used for: Based on a preset integer pixel grid, generate a set of integer pixel translation amounts in the azimuth and range directions; For each integer pixel translation amount in the set of integer pixel translation amounts, the following steps are performed: For each frame of the image to be registered, the image to be registered is translated using the integer pixel translation operator corresponding to the integer pixel translation amount, and zero-filling is used for positions that exceed the original pixel area; the first effective overlap area between the translated image to be registered and the reference frame is determined by using a visible domain mask, and the first effective overlap pixel index set with valid values ​​is selected; based on the first effective overlap pixel index set, the normalized cross-correlation coefficient between the reference frame and each translated image to be registered is calculated; For each frame of the image to be registered, a coarse search is performed on the integer pixel grid, and the integer pixel translation that maximizes the normalized cross-correlation coefficient in the set of integer pixel translations is taken as the optimal integer pixel translation for each frame of the image to be registered.

[0053] In this embodiment, the drift observation extraction module is used for: Centered on the optimal integer pixel translation amount, generate a set of sub-pixel translation amounts in the azimuth and range directions, and use it as the fine search range; For each subpixel translation amount in the subpixel translation amount set, the following steps are performed: each frame of the image to be registered is subpixel translated according to the subpixel translation amount by interpolation resampling to obtain the resampled image to be registered for each frame; the second effective overlap region between the reference frame and each resampled image to be registered is determined by the visible domain mask, and the second effective overlap pixel index set with effective values ​​is selected; based on the second effective overlap pixel index set, the normalized cross-correlation coefficient between the reference frame and each resampled image to be registered is calculated; For each frame of the image to be registered, the subpixel translation amount that maximizes the normalized cross-correlation coefficient in the set of subpixel translation amounts is taken as the optimal subpixel translation amount.

[0054] In this embodiment, the device further includes a model building module 504, used for: Construct a discrete one-dimensional Singer motion model, wherein the state vector of the discrete one-dimensional Singer motion model includes drift displacement, drift velocity, and drift acceleration; Construct a discrete one-dimensional random walk deviation model. The state vector of the discrete one-dimensional random walk deviation model includes scalar deviation. A one-dimensional joint state transition equation is constructed based on the discrete one-dimensional Singer motion model and the discrete one-dimensional random walk deviation model. Based on the one-dimensional joint state transition equations of the azimuth and range directions, a two-dimensional Singer-random walk joint state transition equation is constructed.

[0055] In this embodiment, the Kalman filtering module is used for: Initialize the posterior state estimate and the error covariance matrix; Based on the state transition equation and the error covariance matrix, the prior state estimate and prior error covariance of each frame are calculated. Based on the two-dimensional inter-frame drift observation vector of each frame and the prior state estimate of each frame, the observation prediction value and the innovation are calculated. Calculate the Kalman gain, and based on the Kalman gain, the prior state estimate and the innovation for each frame, calculate the posterior state estimate for each frame, and form a full-frame posterior state estimate sequence from the posterior state estimates of all frames. Calculate the full-frame posterior error covariance sequence based on Kalman gain and prior error covariance.

[0056] In this embodiment, the state smoothing module is used for: Starting from the second-to-last frame, proceed backwards to each frame up to the first frame. Based on the posterior error covariance of the current frame in the full-frame posterior error covariance sequence, obtain the RTS smoothing gain of the current frame. Based on the posterior error covariance, posterior state estimate, and RTS smoothing gain of the current frame, the smoothed state estimate and smoothing error covariance of the current frame are obtained. The smooth drift estimate in the smooth state estimate of the current frame is shifted relative to the reference frame to achieve inter-frame image stabilization in the image domain, resulting in a video SAR image stabilization result.

[0057] This invention also provides a computer device. Figure 6 This is a schematic diagram of a computer device in an embodiment of the present invention. The computer device 600 includes a memory 610, a processor 620, and a computer program 630 stored in the memory 610 and executable on the processor 620. When the processor 620 executes the computer program 630, it implements the above-mentioned airborne video SAR image stabilization method based on filtering and smoothing.

[0058] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described airborne video SAR stabilization method based on filtering and smoothing.

[0059] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described airborne video SAR stabilization method based on filtering and smoothing.

[0060] The beneficial effects achieved by the method and apparatus proposed in the embodiments of the present invention are as follows: First, this invention overcomes the problem that frame-by-frame registration methods based on NCC are susceptible to both systematic biases and random noise, leading to image sequence jitter in the registration results and making it difficult to reflect the true trajectory. While maintaining the robustness of NCC registration, this embodiment treats the output two-dimensional sub-pixel drift sequence as a temporal observation and performs unified modeling and processing within a state-space framework, thus elevating drift estimation from "independent frame-by-frame solution" to "joint estimation utilizing temporal correlation."

[0061] Specifically, firstly, using the first frame as the reference frame, a two-dimensional inter-frame drift observation sequence in the azimuth and range directions is constructed by employing full-image normalized cross-correlation combined with a visible domain mask and a coarse-to-fine two-level search strategy. Subsequently, the true drift component is modeled as a two-dimensional Singer acceleration process, and the system drift error introduced by factors such as imaging geometric approximation, interpolation resampling, and speckle noise is explicitly modeled as a random walk bias, which, together with the displacement, velocity, and acceleration components, forms a two-dimensional Singer-random walk joint state transition equation. The correlation-registered two-dimensional inter-frame drift observation sequence is interpreted as a superposition of "true drift + slowly varying bias + random observation noise," enabling subsequent filtering processes to statistically distinguish and suppress these three factors, thereby alleviating the problem of difficult-to-eliminate inter-frame jitter in traditional NCC methods at its source.

[0062] Second, it significantly improves the estimation accuracy of inter-frame drift and system bias, achieving accurate generation of drift trajectories for long-term sequences. Based on the two-dimensional Singer-random walk joint state transition equation, this embodiment introduces Kalman filtering and RTS fixed-interval smoothing to perform forward recursive estimation and reverse global smoothing of NCC drift observations, effectively utilizing the time-varying correlation of video SAR drift trajectories.

[0063] Specifically, a linear observation equation is constructed that relates only to displacement and deviation components, and an observation noise covariance is set to characterize the statistical properties of random errors in NCC observations. Based on this, a Kalman filtering module predicts and updates the state of each frame, enabling a joint estimate of the true drift and gradually varying deviation under the minimum mean square error. Furthermore, through RTS inverse state smoothing (based on a fixed RTS interval), the posterior state estimate over the entire time period is recursively corrected, allowing the posterior state estimate of each frame to simultaneously incorporate past and future observation information. Compared to schemes that rely solely on frame-by-frame NCC registration or use only forward filtering, this invention provides smoother, lower-noise, and more accurate two-dimensional drift and deviation estimates across the entire sequence, and simultaneously outputs the error covariance, providing a reliable uncertainty measure for subsequent applications.

[0064] Third, while ensuring imaging resolution, overall image stabilization of multi-frame sequences of video SAR is achieved, significantly improving scene display stability and subsequent moving target detection and tracking performance. This embodiment of the invention directly feeds back the smoothed drift estimation results to the image domain, performing sub-pixel translation compensation on the original ViSAR image to achieve accurate correction of residual motion errors between frames.

[0065] Specifically, after completing Singer-Random Walk state estimation and RTS smoothing, this embodiment of the invention performs subpixel translation and interpolation resampling frame by frame on the original video SAR image. Since the drift trajectory used has sufficiently suppressed random fluctuations and slowly accumulating system biases in NCC registration, the pixel positions of stationary scatterers remain essentially unchanged between consecutive frames in the compensated image sequence, significantly reducing inter-frame drift and jitter. This avoids residual drift that is difficult to eliminate with traditional frame-by-frame registration alone, and provides a stable and unified image reference coordinate system for subsequent processing such as target detection, trajectory estimation, and change detection based on multi-frame accumulation, thereby improving the overall application effect of the video SAR system in complex scene surveillance and moving target tracking tasks.

[0066] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0067] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0068] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0069] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0070] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An airborne video SAR image stabilization method based on filtering and smoothing, characterized in that, include: Two-dimensional inter-frame drift observation vector sequence of computer-loaded video SAR image sequence; Based on the Kalman filtering model, the two-dimensional inter-frame drift observation vector of each frame in the two-dimensional inter-frame drift observation vector sequence is used as input. Kalman filtering is performed to obtain the full-frame posterior state estimation sequence and the full-frame posterior error covariance sequence. The Kalman filtering model includes a state equation, an observation equation, and a filtering recursive equation. The observation equation is determined based on the two-dimensional inter-frame drift observation vector. The observation equation is a two-dimensional Singer-random walk joint state transition equation. The posterior state estimation of each frame in the full-frame posterior state estimation sequence includes drift displacement estimation, drift velocity estimation, drift acceleration estimation, and scalar bias estimation. Based on the full-frame posterior error covariance sequence, the full-frame posterior state estimation sequence is subjected to RTS inverse state smoothing to obtain the stable image result of the video SAR image.

2. The method as described in claim 1, characterized in that, The two-dimensional inter-frame drift observation vector sequence of computer-loaded video SAR image sequences includes: Using the first frame of the video SAR image as the reference frame and the remaining frames as the images to be registered, a coarse search is performed on each frame of the images to be registered in sequence on a preset integer pixel grid to obtain the optimal integer pixel translation amount corresponding to each frame of the images to be registered. The fine search range is determined with the optimal integer pixel translation amount as the center. Subpixel translation of each frame of the image to be registered is achieved by interpolation resampling. Fine search is performed within the fine search range to obtain the optimal subpixel translation amount of each frame of the image to be registered. The optimal subpixel translation of each frame of the image to be registered is converted into a distance drift. Based on the drift amount of each frame of the image to be registered, the two-dimensional inter-frame drift observation vector of each frame of the image to be registered is determined, and a two-dimensional inter-frame drift observation vector sequence is formed.

3. The method as described in claim 2, characterized in that, Using the first frame of the video SAR image as the reference frame and the remaining frames as images to be registered, a coarse search is performed on each frame of the image to be registered sequentially within a preset integer pixel grid to obtain the optimal integer pixel translation amount corresponding to each frame of the image to be registered, including: Based on a preset integer pixel grid, generate a set of integer pixel translation amounts in the azimuth and range directions; For each integer pixel translation amount in the set of integer pixel translation amounts, the following steps are performed: For each frame of the image to be registered, the image to be registered is translated using the integer pixel translation operator corresponding to the integer pixel translation amount, and zero-filling is used for positions that exceed the original pixel area; the first effective overlap area between the translated image to be registered and the reference frame is determined by using a visible domain mask, and the first effective overlap pixel index set with valid values ​​is selected; based on the first effective overlap pixel index set, the normalized cross-correlation coefficient between the reference frame and each translated image to be registered is calculated; For each frame of the image to be registered, a coarse search is performed on the integer pixel grid, and the integer pixel translation that maximizes the normalized cross-correlation coefficient in the set of integer pixel translations is taken as the optimal integer pixel translation for each frame of the image to be registered.

4. The method as described in claim 2, characterized in that, The fine search range is determined centered on the optimal integer pixel translation amount. Subpixel translation of each frame of the image to be registered is achieved through interpolation resampling. A fine search is then performed within the fine search range to obtain the optimal subpixel translation amount for each frame of the image to be registered, including: Centered on the optimal integer pixel translation amount, generate a set of sub-pixel translation amounts in the azimuth and range directions, and use it as the fine search range; For each subpixel translation amount in the subpixel translation amount set, the following steps are performed: each frame of the image to be registered is subpixel translated according to the subpixel translation amount by interpolation resampling to obtain the resampled image to be registered for each frame; the second effective overlap region between the reference frame and each resampled image to be registered is determined by the visible domain mask, and the second effective overlap pixel index set with effective values ​​is selected; based on the second effective overlap pixel index set, the normalized cross-correlation coefficient between the reference frame and each resampled image to be registered is calculated; For each frame of the image to be registered, the subpixel translation amount that maximizes the normalized cross-correlation coefficient in the set of subpixel translation amounts is taken as the optimal subpixel translation amount.

5. The method as described in claim 1, characterized in that, Also includes: Construct a discrete one-dimensional Singer motion model, wherein the state vector of the discrete one-dimensional Singer motion model includes drift displacement, drift velocity, and drift acceleration; Construct a discrete one-dimensional random walk deviation model. The state vector of the discrete one-dimensional random walk deviation model includes scalar deviation. A one-dimensional joint state transition equation is constructed based on the discrete one-dimensional Singer motion model and the discrete one-dimensional random walk deviation model. Based on the one-dimensional joint state transition equations of the azimuth and range directions, a two-dimensional Singer-random walk joint state transition equation is constructed.

6. The method as described in claim 1, characterized in that, Based on the full-frame posterior error covariance sequence, RTS inverse state smoothing is performed on the full-frame posterior state estimation sequence to obtain the stabilized video SAR image results, including: Starting from the second-to-last frame, proceed backwards to each frame up to the first frame. Based on the posterior error covariance of the current frame in the full-frame posterior error covariance sequence, obtain the RTS smoothing gain of the current frame. Based on the posterior error covariance, posterior state estimate, and RTS smoothing gain of the current frame, the smoothed state estimate and smoothing error covariance of the current frame are obtained. The smooth drift estimate in the smooth state estimate of the current frame is shifted relative to the reference frame to achieve inter-frame image stabilization in the image domain, resulting in a video SAR image stabilization result.

7. An airborne video SAR image stabilization device based on filtering and smoothing, characterized in that, include: A drift observation extraction module is used to extract two-dimensional inter-frame drift observation vector sequences from computer-loaded video SAR image sequences. The Kalman filtering module is used to perform Kalman filtering based on the Kalman filtering model, taking the two-dimensional inter-frame drift observation vector of each frame in the two-dimensional inter-frame drift observation vector sequence as input, to obtain the full-frame posterior state estimation sequence and the full-frame posterior error covariance sequence. The Kalman filtering model includes a state equation, an observation equation, and a filtering recursive equation. The observation equation is determined based on the two-dimensional inter-frame drift observation vector and is a two-dimensional Singer-random walk joint state transition equation. The posterior state estimation of each frame in the full-frame posterior state estimation sequence includes drift displacement estimation, drift velocity estimation, drift acceleration estimation, and scalar bias estimation. The state smoothing module is used to perform RTS inverse state smoothing on the full-frame posterior state estimation sequence based on the full-frame posterior error covariance sequence, so as to obtain the stable image result of the video SAR image.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 6.

Images