A general video stabilization method based on multiple strategy fusion

Abstract

The application discloses a general video stabilization method based on multiple strategy fusion, and solves the video jitter problem in a complex dynamic scene. A variety of algorithms are combined to improve the adaptability and robustness of the system, and are widely applicable to different types of camera equipment and video scenes. In the motion estimation link, the nR-AKAZE feature matching algorithm is adopted, which can quickly detect feature points in the video frame and generate robust descriptors. Then, the phase correlation method is used for Fourier transform to realize fine translation estimation and ensure high-precision motion detection. In the motion compensation stage, an improved method based on parameter filtering is introduced, a Gaussian filter is used to smooth the global motion vector, thereby reducing the influence of unintentional motion and avoiding error accumulation. In addition, image correction and zero value filling technology are used when generating a stabilized video frame to ensure the coherence and smooth transition of the visual effect. The whole provides an efficient and accurate real-time video stabilization processing solution.

Description

Technical Field

[0001] The present invention belongs to the technical field of video stabilization, and in particular relates to a universal video stabilization method based on multiple strategy fusion. Background Art

[0002] Continuous technological advancements and improved living standards have led to the widespread use of electronic cameras in various fields. Video sequences not only serve as a tool for transmitting information but also contain rich, in-depth content, making them widely used in various fields, including civil, industrial, and military. However, when shooting in complex environments, camera shake can cause unstable movement or distortion in the video, which not only increases visual fatigue but also makes subsequent video analysis and interpretation more difficult. To address these issues, video stabilization technology has emerged. It effectively removes unnecessary random jitter, laying a solid foundation for further video processing. Video stabilization not only improves the viewing experience but also ensures accuracy in various application scenarios.

[0003] Video stabilization technologies are primarily categorized into three types: mechanical, optical, and electronic. Mechanical stabilization uses physical movement to suppress jitter, but its reliance on complex mechanical structures often increases system weight, hinders miniaturization, and can be easily damaged in extreme environments. Optical stabilization achieves stabilization through lens movement. However, this method also has limitations, primarily requiring high lens requirements, resulting in high costs and poor performance in low-light conditions.

[0004] In comparison, electronic image stabilization (EIS) technology directly stabilizes video through image processing algorithms, offering significant advantages such as ease of operation, compact size, and high flexibility. This has made it a major trend in current image stabilization research and development. However, current EIS technology faces challenges in practical application, requiring a balance between accuracy and efficiency. High-precision stabilization improves video quality but often requires longer processing times, potentially impacting real-time performance. High-efficiency algorithms, while enabling fast video processing, may compromise stabilization effectiveness. Therefore, finding the optimal balance between these two factors is crucial to ensure the technology delivers a high-quality visual experience while also being adaptable to a wide range of devices and environments, facilitating deployment and application. This approach will enable broader and more flexible applications of EIS. Summary of the Invention

[0005] To address the technical issues presented in the previous article, this paper provides a universal video stabilization method based on the fusion of multiple strategies. This method integrates the advantages of multiple intelligent image processing algorithms, including motion estimation based on feature matching and phase correlation, motion compensation based on parameter filtering and trajectory smoothing, and visual optimization based on image transformation and zero-value padding. This technology aims to comprehensively consider accuracy, generalizability, and real-time performance to effectively reduce the visual discomfort caused by jitter, improve the user's viewing experience, and enhance the accuracy of subsequent analysis. It also ensures that processing speeds meet real-time requirements and can flexibly adapt to different devices and application scenarios.

[0006] In order to solve the technical problem, the technical solution of the present invention is:

[0007] A general video stabilization method based on multi-strategy fusion, the method comprising:

[0008] Motion estimation: Accurately estimate the translation of video frames through feature matching and phase correlation methods;

[0009] Motion compensation: Based on the estimated translation amount, a Gaussian filter is applied to smooth the motion vector to obtain the smoothed transformation parameters and the processed current frame information;

[0010] Visual optimization: Based on the obtained transformation parameters and current frame information, image transformation and zero-value filling methods are used to obtain the stabilized video sequence.

[0011] Furthermore, the motion estimation specifically includes:

[0012] Use feature matching to preliminarily estimate the translation amount:

[0013] The nR-AKAZE algorithm is used to detect feature points in video frames and quickly generate descriptors. The BF algorithm is used to match descriptors using Hamming distance, and the best match is found by calculating the distance between feature points. The RANSAC algorithm is used to remove incorrect matching points to improve matching accuracy and stability. The displacement of the remaining valid matching points is calculated and averaged to obtain a preliminary translation component.

[0014] Phase correlation fine estimation of translation:

[0015] Preliminary motion estimation: The preliminary motion vector obtained through methods such as feature matching is used to perform a preliminary image transformation on the current frame to make it visually closer to the reference frame. Fourier transform: Fourier transforms are performed on the current frame and the reference frame after the preliminary transformation. This step converts the image from the spatial domain to the frequency domain, making the analysis of translation transformation between the images more efficient. Frequency domain processing: The Fourier transform result of the current frame is multiplied by the conjugate complex number of the Fourier transform of the reference frame. This operation utilizes the characteristics of the frequency domain and emphasizes the phase information between the images through complex operations, effectively filtering out information related to translation. Inverse Fourier transform: The inverse Fourier transform is performed on the above product to obtain a correlation image. The value of each position in the correlation image represents the degree of similarity between the current frame and the reference frame at the corresponding position. Maximum value location: The location of the maximum value in the correlation image is found. The location of the maximum value corresponds to the optimal translation between the current frame and the reference frame, that is, the refined motion vector. Motion vector superposition: The refined motion vector is superimposed with the preliminary motion vector to obtain the final motion vector, which represents the comprehensive translation of the current frame relative to the reference frame.

[0016] Furthermore, the motion compensation specifically includes:

[0017] Parametric filtering and trajectory smoothing: Smoothes unintentional motion in the final motion vector; introduces a Gaussian filter to perform real-time smoothing based on the global transformation parameters of the current frame, adjacent stabilized output frames, and unstabilized input frames; uses a Gaussian filter to perform neighborhood weighted and smoothing on the motion vectors of the current frame and consecutive frames to obtain the smoothed transformation parameters and processed current frame information, thereby achieving stable video output.

[0018] Furthermore, the visual optimization specifically includes:

[0019] Image transformation and black edge filling: Use the smoothed transformation parameters to perform image correction on the current frame to reduce the impact of unintentional motion and retain intentional motion. For the black edges caused by image transformation, a zero-value filling method is used to fill the black edges of the corrected current frame with the stable output content of the previous frame to reduce visual discontinuity. In this process, the corresponding part of the previous frame is seamlessly spliced ​​with the edge of the current frame to ensure the integrity of the image and the naturalness of the transition. Finally, all consecutive frames after stabilization are written to the video file in sequence to ensure the smoothness of the output video and improve the visual quality.

[0020] Compared with the prior art, the advantages of the present invention are:

[0021] First, this invention provides a universal video stabilization method based on the fusion of multiple strategies, designed to address video jitter in complex dynamic scenes. By integrating multiple algorithms and technical approaches, this method is not only highly adaptable and robust, but also widely applicable to different types of camera equipment and video scenarios. Whether in real-time video stream processing or post-processing video interpretation, this invention demonstrates excellent results, ensuring stable video output and possessing broad application value.

[0022] Secondly, the nR-AKAZE feature matching algorithm is used in motion estimation. This algorithm can quickly detect feature points in video frames and generate robust descriptors that remain invariant under different conditions. Then, phase correlation methods are used to perform fine translation estimation using Fourier transforms, thereby improving the accuracy and robustness of motion estimation and ensuring the ability to detect even small image displacements.

[0023] Finally, during the motion compensation phase, the present invention introduces an improved method based on parametric filtering. The estimated global motion vector is smoothed using a Gaussian filter. This method not only considers the motion of the current frame but also incorporates information from previous and subsequent frames, reducing the impact of unintended motion and preventing error accumulation. When generating stabilized video frames, the smoothed transformation parameters are used for image correction. Zero values ​​in the image are filled with the corresponding transformed portion of the previous frame, ensuring smooth transitions and visual continuity.

[0024] Based on this series of designs, the present invention constructs an efficient and accurate video stabilization processing solution, which is particularly suitable for real-time applications in dynamic scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 , an overall flow chart of a universal video stabilization method based on multiple strategy fusion of the present invention;

[0026] Figure 2 , the main flow chart of a universal video stabilization method based on multiple strategy fusion of the present invention. DETAILED DESCRIPTION

[0027] The specific implementation of the present invention is described below in conjunction with examples:

[0028] It should be noted that the structures, proportions, sizes, etc. shown in this specification are only used to match the contents disclosed in the specification for people familiar with this technology to understand and read, and are not used to limit the conditions under which the present invention can be implemented. Any structural modification, change in proportional relationship or adjustment of size should still fall within the scope of the technical content disclosed in the present invention without affecting the efficacy and purpose that can be achieved by the present invention.

[0029] Meanwhile, the terms such as "upper", "lower", "left", "right", "middle", and "one" cited in the present specification are merely for the convenience of clear description, and are not intended to limit the scope of the present application, and the change or adjustment of the relative relationship is also considered as the scope of the present application without substantial change of the technical content.

[0030] Embodiment 1:

[0031] The following is a detailed explanation of the input and output of the three steps (motion estimation, motion compensation, visual optimization) of the video stabilization scheme of the present application.

[0032] 1. Motion Estimation

[0033] Input:

[0034] Video frame sequence: A plurality of consecutive image frames from which motion information is extracted.

[0035] Feature detection algorithm: Parameters used by the nR-AKAZE algorithm (such as threshold, number of key points, etc.).

[0036] Image processing tool: Mathematical tool for Fourier transform and phase correlation calculation.

[0037] Output:

[0038] Preliminary global motion vector: A rough translation calculated in the first stage, representing the motion of the current frame relative to the reference frame.

[0039] Accurate global motion vector: A motion vector fine-tuned by the phase correlation method, providing more accurate translation information.

[0040] 2. Motion Compensation

[0041] Input:

[0042] Superposition of accurate global motion vector and preliminary global motion vector: Output from the first step, containing intended and unintended motion information of the camera.

[0043] Information of the current frame and the adjacent frame: Global motion vectors of the stabilized output frame and the un-stabilized input frame, used for smoothing processing.

[0044] Gaussian filter parameters: Mathematical parameters for neighborhood weighting.

[0045] Output:

[0046] Smoothed transformation parameters: Global motion vectors after Gaussian filtering, removing unintended motion effects, providing stable motion information.

[0047] 3. Visual Optimization

[0048] enter:

[0049] Smoothed transformation parameters: The smoothed result from the second step, used for image rectification.

[0050] Current frame and adjacent frames: the stabilized output frame and the unstabilized input frame images.

[0051] Output:

[0052] Stable output video frames: After transformation and black edge filling, the final video frames are output with more stable and coherent visual effects.

[0053] Final video file: Combine the processed frames into a video file in sequence, ensuring that the parameters are consistent with the original video.

[0054] In summary, motion estimation outputs motion features derived from video frames, which are used to obtain a preliminary and accurate global motion vector. The motion compensation stage uses these superimposed motion vectors for smoothing, aiming to eliminate unintended motion and stabilize the current frame. Visual optimization ultimately outputs a stable, coherent image sequence and generates a playable, stabilized video file. This step-by-step process effectively improves the quality and stability of video stabilization, ultimately resulting in a smoother and more comfortable viewing experience.

[0055] Example 2:

[0056] This embodiment 2 is a further explanation of embodiment 1, which mainly includes three steps: motion estimation, motion compensation, and visual optimization. The overall framework and specific process of the technical solution are as follows: Figure 1 、 2 shown.

[0057] 1. Motion Estimation Based on Feature Matching and Phase Correlation

[0058] In video stabilization, it's often necessary to focus solely on image translation while ignoring rotation, scaling, and other distortions. This is because, in most cases, jitter between video frames manifests primarily as translation. First, a rotation-ignoring accelerated KAZE (nR-AKAZE) feature detection algorithm is used to rapidly extract key features from video frames and provide a preliminary estimate of the translation. This process minimizes redundant computational operations, resulting in high efficiency and suitability for real-time processing. Next, a more refined translation estimate is performed using phase correlation. Phase correlation leverages the properties of the Fourier transform to analyze phase information in the frequency domain, enabling more accurate identification of subtle translations between images. This strategy, which first uses feature matching to obtain a rough estimate and then uses phase correlation for finer adjustments, leverages the advantages of both approaches. This effectively improves the accuracy and robustness of image stabilization without sacrificing efficiency, resulting in more stable and clearer image output in dynamic scenes.

[0059] 1.1. Feature matching based on nR-AKAZE

[0060] The feature matching process based on nR-AKAZE mainly includes the steps of feature point detection, descriptor generation, feature matching, error elimination and displacement estimation.

[0061] First, the nR-AKAZE (non-Rotation AKAZE) algorithm is used to detect feature points in the input image and quickly generate descriptors for each feature point while ignoring rotation operations. These descriptors capture local image information and are highly robust under various image conditions, providing a solid foundation for subsequent matching.

[0062] Next, the Brute Force Matcher (BF) algorithm is used to match the descriptors extracted from the previous and next frames of the video. BF calculates the distance between descriptors to find the best matching pair. To improve matching accuracy, the RANSAC (Random Sample Consensus) algorithm is then applied to eliminate false matches. This process, through random sampling and model fitting, effectively identifies and removes matching points that do not conform to the overall transformation relationship, thereby enhancing the reliability of the matching results.

[0063] Finally, for the remaining valid matching points, the displacement between them is calculated. The specific method is to extract the displacement of each pair of matching points in the previous and next frames of the video and average all the displacements to obtain the preliminary translation component.

[0064] Phase Correlation Method

[0065] The translation component obtained by feature matching is applied to the video frame for a preliminary transformation, making the states of the previous and next frames visually closer. Next, a phase correlation method is used to calculate the secondary translation of the video frame after the preliminary transformation.

[0066] The phase correlation method leverages the properties of the Fourier transform to determine the displacement component between two images by calculating the frequency domain product of the two images. Specifically, the two images are first Fourier transformed, and the product of the Fourier domain of one image and the conjugate complex number of the other image is calculated. The result is then inverse Fourier transformed to obtain a new image in which the maximum value corresponds to the translation between the two images. This method, through frequency domain analysis, can more accurately identify small displacements and is highly robust to noise.

[0067] The obtained secondary translation is superimposed on the preliminary translation to obtain a more accurate overall translation component, that is, the global motion vector of the current frame relative to the reference frame.

[0068] 2. Motion compensation based on parameter filtering and trajectory smoothing

[0069] The global motion vector obtained by motion estimation contains both intentional motion components (such as normal camera scanning motion) and unintentional motion components (such as random camera jitter). To eliminate unintentional motion components in videos, it is usually necessary to smooth and compensate the estimated global motion vector. However, traditional methods often smooth the accumulated global motion parameters, which easily introduces cumulative errors and ultimately causes video sequence drift.

[0070] To address this issue, the present invention employs an improved real-time motion smoothing method. This method not only smooths unstable input frame sequences but also incorporates information from stabilized output frames, effectively eliminating high-frequency jitter. Specifically, a Gaussian filter is used to smooth the global transformation parameters calculated in real time between the current frame and designated adjacent stabilized output frames and stabilized input frames, forming a dynamic motion compensation mechanism.

[0071] During this process, a Gaussian filter acts as a motion filter, performing neighborhood weighting and iteration on the motion vectors of the current frame and several consecutive frames before and after it, ultimately obtaining smoothed transformation parameters. This motion compensation method based on parameter filtering and trajectory smoothing avoids the error accumulation problem caused by simple cumulative estimation in traditional methods. By achieving smoother and more stable video output without increasing computational overhead, the quality of motion compensation is significantly improved.

[0072] 3. Visual optimization based on image transformation and zero-value filling

[0073] The current frame is transformed using the smoothed transformation parameters, and the image is corrected by repositioning each pixel in the image. In this way, the unintentional motion component can be reduced or eliminated while preserving the intentional motion component of the video frame, achieving the effect of video stabilization. However, image transformation can cause zero values at the edges of the image, i.e. black edges, affecting the visual effect.

[0074] To solve this problem, the previous frame after transformation (stabilized output) can be used to fill in the black edges generated by the transformation of the current frame, making the image more complete and natural. Finally, all consecutive stabilized frames are written in order to the video file, and the parameters of the output video are ensured to be consistent with the original video, thereby generating a playable stabilized output video.

[0075] Embodiment 3:

[0076] The present application effectively stabilizes the RGB and infrared video sequences in offline and real-time scenarios. The motion estimation method used in the present application, which combines nR-AKAZE feature matching and phase correlation, has significantly improved stability and real-time performance compared to traditional feature matching algorithms such as SIFT (Scale-Invariant Feature Transform). Specifically, the present application achieves a higher frame rate (Frames Per Second, FPS) during processing. Tests have shown that in a 16vCPU Intel(R) Xeon(R) Platinum 8481C environment, the processing speed for a video sequence with a size of 640*480 pixels is above 30FPS, and the video processing is smooth, making it suitable for real-time applications in dynamic environments.

[0077] After stabilization, the Inter-Frame Transform Fidelity (ITF) of the video is significantly improved compared to the original video. This means that the image quality remains more stable between different frames, with clearer details. This improvement has been verified in multiple static / motion test scenarios, demonstrating the significant advantage of the present application in video quality, especially in irregular fast movement and severe shaking scenarios, where it can effectively reduce shaking and blur.

[0078] Compared with other stabilization methods, the present application achieves a better balance between precision and efficiency. For example, traditional stabilization methods may sacrifice processing speed while improving stability, while the present application ensures efficient processing without compromising image quality through a multi-strategy fusion design. Through comparative analysis, it is proved that the stabilization technology of the present application can provide high-quality stabilization effect while maintaining real-time processing efficiency.

[0079] This method is particularly well-suited for video stabilization applications with a wide range of scene requirements on mobile devices and in low-computing resource scenarios, enabling users to obtain smoother and clearer video data on a variety of devices. The broad application potential of this technology will significantly improve areas such as video surveillance, drone aerial photography, and mobile device photography.

[0080] The specific implementation of the present invention includes three aspects: motion estimation, motion compensation, and visual optimization.

[0081] 1. Motion Estimation Based on Feature Matching and Phase Correlation

[0082] In the technical solution of the present invention, the purpose of motion estimation is to identify and correct the inter-frame translation caused by camera shake. This method combines feature matching and phase correlation techniques to improve accuracy and robustness.

[0083] Feature matching based on nR-AKAZE

[0084] The original video frame is converted into a single-channel grayscale image, and a nR-AKAZE algorithm is used to detect feature points P = {p1, p2, ..., p n}, each feature point p i With location and scale information. Generate a descriptor D for each feature point i , capturing local image features. The nR-AKAZE algorithm simplifies the operations used in the original AKAZE to handle significant image rotation. That is, the calculation of the main direction of the feature points in the AKAZE algorithm is canceled, the reference direction angles of the feature points are set to 0°, and the rotation operation of the feature descriptor is canceled.

[0085] The BF matching algorithm is used to calculate the distance between the descriptors of the feature points of the two frames I1 and I2. Finding the best matching pair Where d can be calculated using the Hamming distance:

[0086]

[0087] Where m is the length of the descriptor.

[0088] Apply the RANSAC algorithm to remove false matches. Set a model (for example, a translation or affine transformation model) and estimate the model through random sampling, identifying matching points that do not conform to the model.

[0089] For the remaining valid matching points Calculate the displacement between them:

[0090]

[0091] Where N is the number of valid matching points, and are the x and y coordinates of the feature points in frame I1, and are the x and y coordinates of frame I2 and its corresponding valid feature points, respectively.

[0092] Phase correlation method

[0093] Apply the preliminary translation (Δx1, Δy1) obtained by the nR-AKAZE algorithm to the video frame I1 to obtain the transformed frame I1′:

[0094] I1′(x,y)=I1(x+Δx1,y+Δy1)

[0095] Perform Fourier transform on the transformed frame I1′ and the reference frame I2:

[0096]

[0097] Compute the product of the complex conjugates of F1′ and F2:

[0098]

[0099] Perform inverse Fourier transform on the product result to obtain the related image:

[0100]

[0101] Find the maximum position (x0, y0) in the correlation image R(x,y), which corresponds to the fine translation (Δx2, Δy2) between the two images. Add the fine translation to the preliminary translation to get the final motion vector:

[0102] Δx=Δx1+Δx2

[0103] Δy=Δy1+Δy2

[0104] 2. Motion compensation based on parameter filtering and trajectory smoothing

[0105] The goal of motion compensation is to improve video quality by removing unintended motion components caused by camera motion. This method uses a Gaussian filter to smooth the global motion vector in the neighborhood of the current frame. This method combines information from unstable input frames and stable output frames, rather than just the original input frame, to achieve dynamic motion compensation.

[0106] Assume T i j represents the coordinate transformation from frame i to frame j, and the adjacent frames are represented as:

[0107] NF c ={n:ck≤n≤c+k}

[0108] Then, the smooth motion transformation can be expressed as:

[0109]

[0110] Among them, T USF (j) is the transformation parameter from the current frame to the specific stable output frame j, and T UOF (j) are the transformation parameters between the current frame and the unstable input frame j, and:

[0111]

[0112] This is a standard deviation of Gaussian kernel, * represents the convolution operation. This process is done by N blur iterations to achieve the best smoothing effect. Finally, the smoothed motion frame can be transformed from the original frame by:

[0113] I c ′=S c I c

[0114] 3. Visual optimization based on image transformation and zero-value filling

[0115] During video stabilization, when the smoothed transformation parameters are used to correct the current frame, "black edges" may appear due to zero pixel values ​​at the edges of the image due to the spatial position transformation. To improve the visual effect while maintaining the same video size, these zero-value areas can be filled with the corresponding parts of the transformed (stabilized) previous frame, resulting in a more natural transition. The pixel value at position (x, y) of the tth frame after zero-value filling is expressed as:

[0116]

[0117] Among them, I t ′(x,y) represents the pixel value at position (x,y) in the tth frame after image transformation.

[0118] Finally, the parameters and encoding format of the output video are set to be consistent with the input video, and all consecutive stabilized frames are written to the video file in sequence to generate the stabilized output video.

[0119] The preferred embodiments of the present invention are described in detail above, but the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in this field without departing from the scope of the present invention.

[0120] Many other changes and modifications can be made to the application without departing from the spirit and scope of the application. It is to be understood that the application is not limited to particular embodiments described, and is intended in its broadest aspect to cover all such changes and modifications thereof. The scope of the application is defined by the appended claims.

Claims

1. A general video stabilization method based on multi-strategy fusion, characterized in that: The method comprises: Motion estimation: Accurately estimate the translation of video frames through feature matching and phase correlation methods; Motion compensation: Based on the estimated translation amount, a Gaussian filter is applied to smooth the motion vector to obtain the smoothed transformation parameters and the processed current frame information; Visual optimization: Based on the obtained transformation parameters and the current frame information, the stabilized video sequence is obtained by using image transformation and zero-value filling methods; The motion estimation specifically includes: Use feature matching to preliminarily estimate the translation amount: The nR-AKAZE algorithm is used to detect feature points in video frames and quickly generate descriptors. The BF algorithm is used to match descriptors using Hamming distance, and the best match is found by calculating the distance between feature points. The RANSAC algorithm is used to remove incorrect matching points to improve matching accuracy and stability. The displacement of the remaining valid matching points is calculated and averaged to obtain a preliminary translation component. Phase correlation fine estimation of translation: Preliminary motion estimation: The preliminary motion vector obtained through feature matching is used to perform a preliminary image transformation on the current frame to make it visually closer to the reference frame. Fourier transform: Fourier transforms are performed on the current frame and the reference frame after the preliminary transformation. This step converts the image from the spatial domain to the frequency domain, making the analysis of translation transformation between images more efficient. Frequency domain processing: The Fourier transform result of the current frame is multiplied by the conjugate complex number of the Fourier transform of the reference frame. This operation utilizes the characteristics of the frequency domain to emphasize the phase information between the images through complex operations, effectively filtering out information related to translation. Inverse Fourier transform: The above product result is inverse Fourier transformed to obtain a correlation image. In the correlation image, the value of each position represents the degree of similarity between the current frame and the reference frame at the corresponding position. Maximum value location: The location of the maximum value in the correlation image is found. The location of the maximum value corresponds to the optimal translation between the current frame and the reference frame, that is, the refined motion vector. Motion vector superposition: The refined motion vector obtained is superimposed with the preliminary motion vector to obtain the final motion vector, which represents the comprehensive translation of the current frame relative to the reference frame. The motion compensation specifically includes: Parametric filtering and trajectory smoothing: Smoothes unintended motion in the final motion vector. A Gaussian filter is introduced to perform real-time smoothing based on the global transformation parameters of the current frame, adjacent stabilized output frames, and unstabilized input frames. The Gaussian filter performs neighborhood weighted summing on the motion vectors of the current frame and successive frames, yielding smoothed transformation parameters and processed current frame information, thereby achieving stable video output. The visual optimization specifically includes: Image transformation and black edge filling: Use the smoothed transformation parameters to perform image correction on the current frame to reduce the impact of unintentional motion and retain intentional motion. For the black edges caused by image transformation, a zero-value filling method is used to fill the black edges of the corrected current frame with the stable output content of the previous frame to reduce visual discontinuity. In this process, the corresponding part of the previous frame is seamlessly spliced ​​with the edge of the current frame to ensure the integrity of the image and the naturalness of the transition. Finally, all consecutive frames after stabilization are written to the video file in sequence to ensure the smoothness of the output video and improve the visual quality.

Images