A Real-time Video Stabilization Method Based on Adaptive Kalman Filter

Abstract

The present invention discloses a real-time video stabilization method based on adaptive Kalman filtering. The steps include: 1) obtaining the original video frame sequence captured by the camera in real time; 2) performing feature point detection on the original video frame sequence; 3) performing feature point tracking and matching between two adjacent frames in the original video frame sequence; 4) calculating the motion parameters of the camera and the original camera path; 5) filtering the original motion trajectory by using the adaptive Kalman filtering algorithm; 6) performing motion compensation on the original video frame sequence to remove the jitter in the current frame and output a stable video frame. The present invention uses adaptive Kalman filtering to smooth the camera path, enhancing the robustness of the technology in the application scenario and effectively removing a large amount of jitter components. While ensuring that the camera motion path is smoother, this method effectively reduces the deviation generated by the Kalman filter with fixed parameters in different shooting scenarios. The processing process is simple, fast, with low system cost and good real-time performance.

Description

Technical Field

[0001] The present invention belongs to the technical field of video image processing, and particularly relates to a real-time video stabilization method based on adaptive Kalman filtering. Background Art

[0002] With the development of today's self-media, shooting short videos through intelligent devices has become the mainstream way for people to record their lives. Subsequently, the public's pursuit of the smoothness and stability of the captured videos makes video stabilization technology extremely important when users shoot videos. In addition, some mobile airborne imaging devices such as vehicle-mounted camera systems are vulnerable to external interferences (strong winds, road bumps, etc.), which causes the collected video sequences to have translations, rotations, and even random jitters in all directions, resulting in poor visual effects and even affecting the further experience of relevant users. Therefore, whether in daily life or in scientific research work, a video stabilization system that can generate high-quality and stable videos has become a rigid demand of users, and the research and optimization of stabilization methods have profound application values.

[0003] Video stabilization technology is divided into three methods: mechanical stabilization, optical stabilization, and electronic stabilization according to different principles of removing jitters. Due to the poor portability of mechanical stabilization systems and the high cost of optical stabilization systems, electronic stabilization technology has been widely applied. Electronic stabilization technology mainly includes three main steps: motion estimation, motion filtering, and motion compensation. Among them, motion estimation aims to calculate the global motion of the camera, and its calculation results will affect subsequent steps such as motion filtering and motion compensation; motion filtering is to separate the scanning motion and jitter components of the camera, eliminate or reduce the random jitters in the video frame sequence, and obtain the corresponding compensation components; motion compensation is to correct and improve the video frame sequence according to the compensation components.

[0004] In the invention patent with the authorization announcement number of CN106550174B and the invention name of "A Real-time Video Stabilization Method Based on Homography Matrix", the Harris corner detection method is used to extract feature points in the corresponding video frames, and the optical flow vector is used to track the feature points. According to the matched feature points, the trajectory of the camera motion is obtained in the form of calculating the homography matrix. Then, the fixed-parameter Kalman filtering method is used to eliminate video jitters on the camera motion trajectory and then perform corresponding motion compensation to achieve stable image output. This system needs to calculate the 3D distortion parameters after obtaining the homography transformation to obtain the 2D motion components and then perform filtering processing. The calculation process is complex, and the use of the fixed-parameter Kalman filtering method makes the algorithm's application scenario less robust and unable to achieve the self-adaptability of the filtering performance.

[0005] In the invention patent with the authorization announcement number CN108805908B and the invention name of "A Real-time Video Stabilization Method Based on Temporal Grid Flow Superposition", after the feature point extraction and matching algorithm is processed, the corresponding video frame image is subjected to grid processing. After obtaining the motion vectors at the grid vertices according to the motion vectors of the feature points and performing smoothing processing, motion compensation is performed, and finally stable image output is achieved. The patent method has a large amount of calculation, complex algorithm processing, and poor real-time performance. Summary of the Invention

[0006] The purpose of the present invention is to propose a real-time video stabilization method based on adaptive Kalman filtering for the problems existing in the prior art. The method includes the following steps:

[0007] Step 1: Obtain the original video frame sequence captured by the camera in real time;

[0008] Step 2: Perform Shi-Tomasi feature point detection and description on the original video frame sequence to obtain the feature point information in each frame of the original video frame sequence;

[0009] Step 3: According to the feature point information of two adjacent frames in the original video frame sequence, use the Kanade-Lucas-Tomasi (KLT) algorithm for feature point tracking and matching;

[0010] Step 4: Calculate the motion parameters of the camera and the original camera path. Calculate the affine transformation matrix from the position information of the feature points obtained by matching, and obtain four motion parameters of horizontal displacement, vertical displacement, scaling ratio, and rotation angle between adjacent frames from the affine transformation matrix parameters;

[0011] Step 5: Use traditional Kalman filtering to filter the two motion parameters of rotation angle and scaling ratio with fixed parameters, and use adaptive Kalman filtering to perform parameter self-adjusting filtering on horizontal displacement and vertical displacement, so as to obtain an accurate and smooth camera motion trajectory, that is, a smooth path;

[0012] Step 6: Perform motion compensation on the original video frame sequence according to the relationship between the original camera path and the smooth path, and then obtain a stable video.

[0013] In the above technical solution, further, in step S2, when performing Shi-Tomasi feature point detection and description on the original video frame sequence, Shi-Tomasi feature point detection is an improvement of Harris feature point detection and is a local feature detection method. The main idea is to calculate the change in grayscale within the window by changing the moving window. If there is a large grayscale change in any direction of sliding, it can be considered that there are feature points within the window.

[0014] In step S3, the KLT algorithm is used to track feature points for the feature point information between two adjacent frames. Under the premise that the brightness remains unchanged and the gray values of the image pixels are continuous and consistent, the sum of squared differences (SSD) of the gray values between adjacent frames of the window to be tracked is used as the similarity criterion for feature matching.

[0015] Further, in step S4, the Random Sample Consensus (RANSAC) algorithm is used to calculate the affine transformation matrix.

[0016] The beneficial effects of the present invention are as follows: (1) The present invention uses the Shi-Tomasi algorithm to detect feature points, detects the points with large gradient changes in the image, and then tracks the feature points between adjacent frames, thereby estimating the affine transformation. The motion parameters obtained from the affine transformation between frames are accumulated to accurately calculate the camera path;

[0017] (2) Aiming at the problem of poor adaptability of the traditional Kalman filter, the adaptive Kalman filter introduces a jitter parameter to describe the degree of deviation of the filter and then distinguishes application scenarios, and updates the measurement noise covariance in real time according to the jitter parameter, realizing a Kalman filter with self-adjusting parameters. It not only enhances the robustness in different shooting scenarios. Specifically, in the camera pursuit scenario, the adaptive Kalman filter enhances the following characteristics of the filter; in the camera jitter scenario, it improves the following characteristics of the filter, and can effectively remove a large number of jitter components and retain the true motion of the camera;

[0018] (3) The present invention uses the difference between the original camera path and the smoothed path to obtain the motion compensation for each adjacent frame, and finally performs motion compensation on each frame, so as to achieve a stable video. The system has low cost and good real-time performance, which is conducive to popularization and application. This method not only ensures that the camera motion path is smoother but also effectively reduces the deviation generated by the Kalman filter with fixed parameters in different shooting scenarios. The processing process is simple and fast. Description of the Drawings

[0019] Figure 1 It is a flowchart of the real-time video stabilization method based on adaptive Kalman filtering according to an embodiment of the present invention.

[0020] Figure 2 It is a graph of the adaptive Kalman filter parameter adjustment function in an embodiment of the present invention.

[0021] Figure 3 It is a comparison graph of the adaptive Kalman filtering algorithm and the traditional Kalman filtering algorithm in the pursuit scenario in an embodiment of the present invention.

[0022] Figure 4This is a comparison graph of the jitter scenarios between the adaptive Kalman filter algorithm and the traditional Kalman filter algorithm in the embodiments of the present invention. Detailed implementation manners

[0023] To facilitate the understanding and implementation of the present invention by those of ordinary skill in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0024] As Figure 1 , a real-time video stabilization method based on an adaptive Kalman filter provided by the present invention is implemented according to the following steps:

[0025] Step 1: Obtain the original video frame sequence captured in real time by a camera (such as a motion camera, a camera phone, or an airborne imaging device, etc.);

[0026] Step 2: Perform Shi-Tomasi feature point detection and description on the original video frame sequence to obtain the feature point information in each frame of the original video frame sequence;

[0027] Feature points, also known as corner points, refer to the corner positions or the intersections of straight lines where the pixel changes rapidly. Shi-Tomasi feature point detection is an improvement of Harris feature point detection and is a local feature detection method. The main idea is to calculate the change in grayscale within the window by changing the moving window. If there is a large change in grayscale for any direction of sliding, it can be considered that there are feature points within the window.

[0028] Assume that I(u, v) represents the grayscale value of the grayscale image, (, v) represents the pixel coordinates of a point in the moving window, and the specific calculation formula for the grayscale value change E(Δx, Δy) obtained by moving the window (Δx, Δy) is:

[0029]

[0030] Among them, represents the window function, and σ represents the scale parameter.

[0031] To obtain the feature point position, it is only necessary to find the maximum value of E(Δx, Δy). Perform Taylor expansion on the grayscale value change calculation formula and discard the high-order terms. The specific calculation formula is:

[0032]

[0033] Among them, the expression of M is:

[0034]

[0035] Among them, I x , I y respectively represent the gradient values in the x and y directions. If the two eigenvalues λ of M1 , λ 2 Given a threshold λ, if a certain pixel satisfies min(λ 1 , λ 2 ) > λ, then there are Shi - Tomasi feature points; otherwise, there are no feature points.

[0036] Step 3: According to the feature point information of two adjacent frames in the original video frame sequence, use the KLT algorithm for feature point tracking and matching.

[0037] After detecting the Shi - Tomasi feature points, they need to be tracked. Considering the combination of duration and effect, the KLT tracking algorithm is used for implementation. Under the premise that the brightness remains unchanged and the gray values of image pixels are continuous and consistent, the sum of squared differences SSD of the gray differences between adjacent frames of the window to be tracked is used as the similarity criterion for feature matching.

[0038] Suppose a feature window W containing feature texture information. Let the video frame corresponding to time t be represented by I(x, y, t), and the video frame corresponding to time t + τ can be represented by I(x, y, t + τ). The specific calculation formula for the corresponding position gray value is:

[0039] I(x, y, t + τ) = I(x + Δx, y + Δy, t)

[0040] where Δx and Δy respectively represent the motion offsets of each pixel point in I(x, y, t) in the horizontal and vertical directions, and each pixel point in the video frame corresponding to time t + τ can be obtained by translating d(Δx, Δy) the corresponding pixel point in I(x, y, t).

[0041] Suppose adjacent frames I and J. To achieve feature tracking of the adjacent frame video sequence, first, it is necessary to calculate SSD (represented by ε(d(Δx, Δy))), and its specific calculation formula is:

[0042] ε(d(Δx, Δy)) = ∫∫[J(X + (d(Δx, Δy)) - I(X)] 2 ω(X)dX

[0043] where the integration region of the above formula is the feature window W, ω(X) is a weight coefficient with a value of 1, X represents the video frame pixel point coordinates, and I(X) is the gray value of the video frame corresponding to time I.

[0044] In order to find \(d(\Delta x,\Delta y)\) that minimizes \(\varepsilon(d(\Delta x,\Delta y))\), since \(d(\Delta x,\Delta y)\) is usually much smaller than \(X\), \(J(X + d(\Delta x,\Delta y))\) can be Taylor-expanded and the highest-order term can be ignored, only retaining the first two terms. To obtain the best-matched corner points, take the derivative of \(\varepsilon(d(\Delta x,\Delta y))\) with respect to \(d(\Delta x,\Delta y)\) and set it to 0, then the offset \(d(\Delta x,\Delta y)\) can be obtained, thereby obtaining the positions of the matching feature points.

[0045] Step 4: Calculate the motion parameters of the camera and the original camera path. Calculate the affine transformation matrix from the position information of the feature points obtained by matching, and obtain four motion parameters, namely the horizontal displacement, vertical displacement, scaling ratio, and rotation angle, between adjacent frames from the parameters of the affine transformation matrix.

[0046] The global motion estimation between two adjacent frames of a jittery video is mainly manifested in transformations such as translation, rotation, and scaling. The present invention uses an affine transformation as the global motion between adjacent frames and uses the Random Sample Consensus (RANSAC) algorithm to estimate the affine transformation matrix between frames. Let \(p\) 0 (x y 1) T and \(p\) 1 (x′ y′ 1) T respectively represent the matching feature points in the current frame and the reference frame. The mathematical model of the affine transformation between the feature points can be expressed as:

[0047]

[0048] is the affine transformation matrix. \(\cos\theta\) and \(\sin\theta\) represent the rotation part in the video frame, \(a\) represents the scaling ratio between frames in the video, and \(dx\) and \(dy\) respectively represent the horizontal displacement and vertical displacement. The affine transformation matrix can well reflect the motions such as translation, rotation, and scaling of the image and can satisfy the image transformation in daily shooting scenarios.

[0049] Accumulate the four motion parameters obtained according to the affine transformation to obtain the original motion trajectories of the horizontal motion, vertical motion, and rotational motion of the camera.

[0050] Step 5: Use traditional Kalman filtering to perform filtering with fixed parameters on the two motion parameters of the rotation angle and the scaling ratio to achieve the purpose of smoothing; use adaptive Kalman filtering to perform filtering with self-adjusting parameters on the horizontal displacement and the vertical displacement, so as to finally obtain an accurate and smooth camera motion trajectory (i.e., a smooth path);

[0051] Motion smoothing is mainly used to remove the jitter of the original camera path and make the path smooth. Motion smoothing can usually be accomplished by means of Gaussian filters, Kalman filters, or curve fitting. The Kalman filter has good real-time performance and only requires the predicted value of the motion parameters of the previous frame and the measured value of the motion parameters of the current frame to generate the estimated value of the current frame.

[0052] The macroscopic process of the Kalman filter is to obtain the predicted value using the optimal result of the previous moment, and at the same time use the observed value to correct the predicted value, so as to obtain the optimal result. Therefore, the Kalman filter can be understood as two processes, namely the prediction process and the update process. In the prediction process, the Kalman filter algorithm uses the target state vector of the previous moment to predict the state of the target at the next moment; in the update process, by calculating the Kalman gain, the predicted value in the previous step is corrected with the help of the system observed value.

[0053] Modeling the camera motion process, the Kalman filter formula can be specifically written in this solution system as:

[0054]

[0055] is the optimal motion state result of the (i - 1)-th frame, which corresponds to an updated covariance P i-1|i-1 to describe the error degree between this optimal motion state and the observed value at the corresponding moment; is the estimated value of the motion state of the current frame based on the optimal motion state of the previous frame which simultaneously corresponds to a predicted covariance P i|i-1 to describe the error degree between the estimated value and the measured value; Q is the system noise covariance; K i is the Kalman filter gain of the i-th frame; R is the measurement noise covariance, and the optimal motion state value of the i-th frame after prediction and update is denoted as Using the Kalman filter gain K i and the prediction error P i|i-1 the updated covariance P of the i-th frame can be obtained i|i .

[0056] In the Kalman filter formula, the calculation formula of K i is:

[0057]

[0058] It can be seen from the above formula that the Kalman filter gain is approximately equal to the ratio of the system noise covariance Q to the sum of the system noise covariance Q and the measurement noise covariance R, and is independent of the specific value. And the smaller the Q value, the faster the Kalman filter convergence speed. If Q is fixed, the smaller R is, the larger K i is, and the optimal motion state result trusts the observed value more; conversely, the larger R is, the smaller K i is, and the optimal result value trusts the predicted value more. From the perspective of filtering, if the optimal motion state result trusts the predicted value more, that is, the algorithm shows better smoothing effect. If it trusts the actual value more, it can show better tracking effect.

[0059] In the process of Kalman filtering, there are mainly two situations for the filtering results in non-ideal cases: one is that the filtering curve deviates from the original video trajectory, and the other is that some jitter components are retained. Therefore, in the present invention, the main purpose of smoothing is achieved by adopting fixed-parameter Kalman filtering for the camera rotation motion, while for the horizontal and vertical motions, the adaptive Kalman filtering method is adopted to achieve smoothing and keep the motion trend consistent with the original motion trajectory.

[0060] To implement the adaptive Kalman filtering, a jitter parameter μ is introduced in the present invention as a feedback factor for the filtering performance. Let the jitter vector represent the deviation degree between the filtering result in the x direction of the i-th frame in the video sequence (i.e., the horizontal displacement value obtained through the aforementioned Kalman filtering) and the original motion state (observed value, i.e., the horizontal displacement value obtained through the global motion estimation in the aforementioned step 4). Similarly, the jitter vector represents the deviation degree between the filtering result in the y direction of the i-th frame in the video sequence (i.e., the vertical displacement value obtained through the aforementioned Kalman filtering) and the original motion state (observed value, i.e., the vertical displacement value obtained through the global motion estimation in the aforementioned step 4). It is expressed by the formula:

[0061]

[0062] The jitter parameter μ can then be calculated as:

[0063]

[0064] In the formula, L is the length of the jitter vector window within the selected time period, represents the jitter parameter in the x direction of the i-th frame, which is defined as the average jitter vector in the x direction within this window; similarly represents the jitter parameter in the y direction of the i-th frame, which is defined as the average jitter vector in the y direction within this window. In this embodiment, L takes 5.

[0065] The jitter parameter can preferably reflect the deviation degree of the motion filtering result based on measuring the camera motion condition, so as to identify whether the current scene is a pursuit motion or a jitter motion. It can be analyzed that when there is a pursuit motion in a certain parameter direction, the jitter parameter will be very large. In order to give full play to the following characteristics of the Kalman filter, the measurement noise covariance should be very small; on the contrary, if there is only random jitter, the jitter parameter will be very small, and a larger value of the measurement noise covariance can make good use of the smoothing effect of the Kalman filter, so as to achieve the adaptive motion filtering of the Kalman filter in different scenes.

[0066] Therefore, in the present invention, it is proposed that there is a non-linear function between the jitter parameter and the measurement noise covariance, and it can be obtained from experimental observations that this functional relationship should satisfy that if the jitter parameter is in the interval of [-25, -15] ∪ [15, 25] (unit: pixel value), the measurement noise covariance should decrease rapidly, so as to achieve a good transition from the smoothing effect to the tracking effect. Therefore, the functional relationship is proposed so as to be able to determine the observation noise R in the i-th frame according to the value of the jitter parameter μ i value.

[0067] In order to obtain the values of a and k in the function, the present invention processes the original video frame sequence through steps 1 to 4 to obtain the original camera path, and modifies the value of the measurement noise covariance R manually in segments based on step 5 for the original camera path, so as to obtain the filtering value that is smooth and can well track the camera motion trend, and obtain the ideal motion path. The specific method is that if the camera starts to show a trend of pursuit motion in a certain direction in the i-th frame, the value of R should be continuously adjusted at this time to change from the smoothing effect to the tracking effect. According to the obtained original camera path and the ideal motion path, the functional relationship is solved that is, the motion prediction value obtained by Kalman filtering through the above-proposed functional relationship should be as close as possible to the manually calibrated filtering value, and then the values of a and k are solved. In this embodiment, a = 1.88 and k = 0.17, and its function image is referred to Figure 2 as shown.

[0068] The value of the measurement noise covariance R in the i-th frame is obtained according to the jitter parameter μ, i so as to realize the adaptive Kalman filtering for the horizontal and vertical motions of the camera.

[0069] Figure 3 It is a comparison diagram of the effects after processing the horizontal motion of the video by using the adaptive Kalman filtering algorithm and the fixed-parameter Kalman filtering algorithm in the pursuit scene. Figure 3The blue line marked with "original" is the original motion trajectory for testing the horizontal motion of the test image. The red line marked with "alter - parameter" is the motion trajectory smoothed by the adaptive Kalman filtering algorithm. The yellow line marked with "fixed - parameter" is the motion trajectory obtained by processing with the Kalman filtering algorithm under fixed parameters. From Figure 3 It can be seen that after being processed by the adaptive Kalman filtering algorithm of the present invention, a smooth path of the camera motion can be obtained quickly, and compared with the Kalman filtering algorithm under fixed parameters, the smooth motion trend of the camera can be tracked more accurately, and there is no phenomenon of deviating from the original motion trajectory.

[0070] Refer to Figure 4 , which is a comparison diagram of the effects after processing the vertical motion of the video with the adaptive Kalman filtering algorithm and the Kalman filtering algorithm with fixed parameters under a conventional jitter scenario. Figure 4 The blue line marked with "original" in it is the original motion trajectory for testing the vertical motion of the test image. The red line marked with "alter - parameter" is the motion trajectory smoothed by the adaptive Kalman filtering algorithm. The yellow line marked with "fixed - parameter" is the motion trajectory obtained by processing with the Kalman filtering algorithm under fixed parameters. From Figure 4 It can be known that after being processed by the adaptive Kalman filtering algorithm of the present invention, a smooth path that retains the original motion trend of the camera is obtained. Therefore, in subsequent motion compensation, the disadvantage of a large number of undefined regions generated in the video frames after being processed by the Kalman filtering algorithm with fixed parameters is effectively avoided, and at the same time, the potential risk of the disappearance of important visual information caused by being processed by the traditional Kalman filtering algorithm is effectively avoided.

[0071] Step 6: Perform motion compensation on the original video frame sequence according to the relationship between the original camera path and the smooth path, and then obtain a stable video.

[0072] After obtaining the smooth path from motion smoothing, after removing the jitter component according to the relationship between the original camera path and the smooth path, a new transformation matrix will be obtained. Use this matrix to perform inverse compensation on the current frame, and the compensated frame is output to the video writing stream, and finally a stabilized video sequence is obtained.

[0073] The calculation time per frame of the method of the present invention on a personal computer (3.00GHz Intel(R) Core(TM) i5 - 8500 CPU, 16GB DDR4 memory, 64 - bit Windows 10 operating system) is less than 20ms, and real - time video stabilization can be achieved.

Claims

1. A real-time video stabilization method based on adaptive Kalman filtering, characterized in that, the steps are as follows: S1: Obtain the original video frame sequence captured by the camera in real time; S2: Perform Shi-Tomasi feature point detection and description on the original video frame sequence to obtain the feature point information in each frame of the original video frame sequence; S3: According to the feature point information of two adjacent frames in the original video frame sequence, use the KLT algorithm for feature point tracking and matching; S4: Calculate the motion parameters of the camera and the original camera path, calculate the affine transformation matrix from the position information of the feature points obtained by matching, and obtain four motion parameters of horizontal displacement, vertical displacement, scaling ratio, and rotation angle between adjacent frames from the affine transformation matrix parameters; S5: Use traditional Kalman filtering to filter the two motion parameters of rotation angle and scaling ratio with fixed parameters, and use adaptive Kalman filtering to filter the horizontal displacement and vertical displacement with self-adjusting parameters to obtain a smooth path; S6: Perform motion compensation on the original video frame sequence according to the relationship between the original camera path and the smooth path, and then obtain a stable video; In step S5, adaptive Kalman filtering is used to perform parameter self - adjustment filtering on the horizontal displacement and vertical displacement. Specifically: First, calculate the jitter parameter μ, which is used to reflect the deviation degree of the motion filtering result, so as to identify whether the current scene is a pursuit motion or a jitter motion; then, establish the jitter parameter μ of each frame i and the measurement noise covariance R in the Kalman filtering algorithm i functional relationship: where a and k are constant terms in the function, so that the observation noise R in the i - th frame of the video sequence can be determined according to the μ value, realizing the Kalman filtering with parameter self - adaptation. i value.

2. The real-time video stabilization method based on adaptive Kalman filtering according to claim 1, characterized in that, in the step S4, the random sample consensus algorithm is used to calculate the affine transformation matrix.

3. The real-time video stabilization method based on adaptive Kalman filtering according to claim 1, characterized in that, the jitter parameter of each frame specifically includes the jitter parameter in the x direction and the jitter parameter in the y direction, and its calculation formula is: Among them, L is the window length of the jitter vector within the selected time period. represents the jitter parameter in the x direction of the i-th frame, which is defined as the average jitter vector in the x direction within this window; similarly represents the jitter parameter in the y direction of the i-th frame, which is defined as the average jitter vector in the y direction within this window. is the jitter vector in the x direction, representing the deviation degree between the filtering result in the x direction of the k-th frame in the video sequence and the original motion state obtained in step S4; is the jitter vector in the y direction, representing the deviation degree between the filtering result in the y direction of the k-th frame in the video sequence and the original motion state obtained in step S4.

4. The real-time video stabilization method based on adaptive Kalman filtering according to claim 1, characterized in that, the step S6 is specifically that, after removing the jitter component according to the relationship between the original camera path and the smooth path, a new transformation matrix will be obtained, and this matrix is used to perform reverse compensation on the current frame, and the compensated frame is output to the video writing stream, and finally a stabilized video sequence is obtained.

Images