Unmanned Vehicle Wide Field-of-View Imaging Video Stabilization System and Method
The unmanned vehicle wide field-of-view imaging video stabilization system utilizes multi-aperture cameras and deep learning networks to solve the problem of image shaking during unmanned vehicle movement, improve video stability and environmental perception capabilities, and ensure driving safety and stability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- FUZHOU UNIV
- Filing Date
- 2024-09-18
- Publication Date
- 2026-05-26
AI Technical Summary
During the movement of an autonomous vehicle, factors such as uneven ground, poor road conditions, and vehicle vibration cause the images captured by the camera to shake, affecting image stability and reducing the autonomous vehicle's environmental perception capabilities and path planning accuracy.
An unmanned vehicle large field-of-view imaging video stabilization system is adopted, which uses a multi-aperture camera, acquisition card, computing card, display, video acquisition module, real-time image stitching module and large field-of-view video stabilization module, and is trained with a deep learning network to improve the stability of large field-of-view video.
It improves the stability of wide-field-of-view video, reduces the loss of video image information, and enhances the visual perception capability and path planning accuracy of autonomous vehicles.
Smart Images
Figure CN119211703B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to a video stabilization system and method for large field-of-view imaging of unmanned vehicles. Background Technology
[0002] During the movement of autonomous vehicles, factors such as uneven ground, poor road conditions, and vehicle vibrations can cause image jitter in the camera-captured images, resulting in unstable images. This image instability makes it difficult for autonomous vehicles to accurately identify and understand their surroundings, thus reducing their ability to perceive roads and obstacles. Secondly, image jitter also affects the autonomous vehicle's path planning, leading to unstable or even deviated routes, increasing driving risks and uncertainties. Therefore, solving the problem of image jitter during autonomous vehicle movement and improving image stability and quality is crucial for enhancing the vehicle's visual perception capabilities and the accuracy of path planning. Employing effective video stabilization algorithms can effectively suppress image jitter, ensuring that the vehicle obtains clear and stable visual information during operation, thereby improving its autonomous navigation and obstacle avoidance capabilities and ensuring driving safety and stability.
[0003] Autonomous vehicles (RVs) typically require extensive environmental information to perform navigation and obstacle avoidance functions, making a wide field of view crucial. This allows the vehicle to perceive its surroundings more comprehensively, improving its ability to identify road conditions and obstacles. To meet this need, using multiple cameras to expand the field of view has become a common choice for RVs. Combining multiple cameras to create a wide field of view image can cover a broader area, providing the vehicle with more environmental information. By analyzing wide field-of-view video images, RVs can detect road conditions, recognize traffic signs, and identify obstacles in real time, contributing to accurate navigation and effective obstacle avoidance. However, if this wide field-of-view video experiences severe jitter, affecting video quality, it will significantly impact the realization of the RV's functions. Summary of the Invention
[0004] In view of this, the purpose of this invention is to provide a large field-of-view imaging video stabilization system and method for unmanned vehicles. By combining a deep learning network and training it with a large field-of-view video dataset, this invention improves the stability of large field-of-view videos and reduces information loss in video images compared to traditional small field-of-view video stabilization methods.
[0005] To achieve the above objectives, the present invention adopts the following technical solution: an unmanned vehicle large field-of-view imaging video stabilization system, comprising an unmanned vehicle, a multi-aperture camera, a capture card, a computing card, a display, a video capture module, a real-time image stitching module, and a large field-of-view video stabilization module;
[0006] The unmanned vehicle is equipped with a computing card, a data acquisition card, and a multi-aperture camera. The unmanned vehicle will generate vibrations during its movement, which is the source of vibration in the wide field-of-view video of the unmanned vehicle.
[0007] The video acquisition module includes a capture card, a multi-aperture camera, a multi-threaded parallel acquisition module, and a frame synchronization control module. The multi-aperture camera is used for multi-aperture video. The capture card is connected to the multi-aperture camera and is used to deploy multi-aperture video acquisition algorithms. The multi-threaded parallel acquisition module is used to control the operation of the multi-aperture camera. The frame synchronization control module is used to control the multi-aperture camera to achieve synchronous acquisition.
[0008] The real-time image stitching module is connected to the video acquisition module and is used to stitch the received multi-aperture images into a complete large field-of-view image;
[0009] The wide field-of-view video stabilization module is connected to the real-time image stitching module. The wide field-of-view video stabilization module includes a feature point detection module, a key point motion estimation module, a grid motion propagation module, and a motion smoothing module, which are used to stun the stitched wide field-of-view video and output a stable wide field-of-view video.
[0010] The computing card is used to deploy real-time stitching algorithms and wide-field-of-view video stabilization algorithms, and is responsible for performing the corresponding computing tasks; the display is used to present the stabilized wide-field-of-view video.
[0011] In a preferred embodiment, a multi-aperture camera is fixed at a certain angle on the unmanned vehicle to expand the field of view. A data acquisition card is connected to the multi-aperture camera and deploys a video acquisition algorithm to control the camera to perform multi-aperture video acquisition. A real-time image stitching module and a large field-of-view video stabilization algorithm module are deployed on a computing card and are responsible for stabilizing the received large field-of-view video. A display is connected to the data acquisition card to display the stabilized large field-of-view video.
[0012] In a preferred embodiment, the video acquisition module includes a multi-aperture camera module, a multi-threaded parallel acquisition module, an acquisition card, and a frame synchronization control module; the acquisition card is connected to the multi-aperture camera module, the multi-threaded parallel acquisition module, and the frame synchronization control module respectively; the multi-threaded parallel acquisition module controls the transmission of multi-aperture images in a multi-threaded parallel manner; the frame synchronization control module sends signals to the multi-aperture camera using a continuous soft-trigger method, while controlling parameters such as frame rate; the multi-threaded parallel acquisition module and the frame synchronization control module are used together to obtain synchronized multi-aperture images.
[0013] In a preferred embodiment, the real-time image stitching includes an image distortion correction module, an image registration module, and an image transformation module. The image distortion correction module uses calibrated camera intrinsic parameters to distort the multi-aperture images separately, and simultaneously records the image distortion correction matrix. The image registration module is connected to the image distortion correction module. First, the image registration module receives a multi-aperture image with sufficient features, detects the feature information of the input image, records the camera's intrinsic and extrinsic parameters using a multi-view geometric calibration method, and calculates the transformation relationship parameters of the stitched image. The image transformation module is connected to the image registration module, uses the recorded image distortion correction parameters and image transformation parameters to perform distortion transformation on the multi-aperture images, and fuses the transformed multi-aperture images into a single image to obtain a large field-of-view image.
[0014] In a preferred embodiment, the wide field-of-view video stabilization module includes a feature point detection module, a key point motion estimation module, a grid motion propagation module, and a motion smoothing module. The feature point detection module detects feature points in the input wide field-of-view image. The key point motion estimation module, connected to the feature point detection module, filters the detected feature points and obtains key point motion vectors by matching feature points in the current frame with those in past frames. The grid motion propagation module, connected to the key point motion estimation module, first divides the wide field-of-view video frame into an M×N grid, then propagates the key point motion data into grid vertex motion data with certain weights, the weights being related to the distance between the key points and grid vertices. Finally, multiple one-dimensional convolutions are used to extract motion features, fuse information, and generate optimized grid motion trajectories. The motion smoothing module, connected to the wide field-of-view video stabilization module, smooths the calculated grid vertex motion, performs wide field-of-view image transformation based on the offset grid motion vectors, and outputs a stable wide field-of-view video.
[0015] The aforementioned grid motion propagation module and motion smoothing module first use the system's video acquisition module and real-time image stitching module to create a large field-of-view video dataset. The dataset is an unstable large field-of-view video dataset collected when the unmanned vehicle shakes during its movement. The dataset is then input into the grid motion propagation module and motion smoothing module for training, thereby improving the robustness of the anti-shake algorithm in smoothing complex shaking in a large field of view.
[0016] This invention also provides a method for stabilizing video images of unmanned vehicles with a wide field of view, which employs the aforementioned video stabilization system for unmanned vehicles with a wide field of view, and includes the following steps:
[0017] Step S1: Construct a large field-of-view video dataset based on the autonomous vehicle, including image data from different scenes;
[0018] Step S2: Train the grid motion propagation module and motion smoothing module using the constructed dataset to obtain a grid motion propagation module and motion smoothing module that are robust to image stabilization of large field of view for unmanned vehicles;
[0019] Step S3: Acquire a calibration image for the real-time image stitching module. This image must contain enough feature points to ensure the accuracy of the transformation parameters between images, and calculate the image transformation parameters using this image.
[0020] Step S4: Drive the driverless car. The driverless car may vibrate due to uneven road surfaces or its own characteristics.
[0021] Step S5: The video acquisition module uses a multi-aperture camera mounted on the unmanned vehicle to capture multi-aperture images of shaking motions, which are affected by the shaking posture information of the unmanned vehicle.
[0022] Step S6: Input the jittery multi-aperture image into the real-time image stitching module, and transform the multi-aperture image using pre-calculated image transformation parameters to obtain a large field-of-view image;
[0023] Step S7: The wide field-of-view video stabilization module receives the shaky wide field-of-view image, performs feature point detection, key point motion estimation, grid motion propagation and motion smoothing on the shaky image, and finally outputs a stable wide field-of-view video.
[0024] Compared with the prior art, the present invention has the following beneficial effects: The present invention performs shake smoothing processing on the large field-of-view video collected by the unmanned vehicle, which improves the stability of the large field-of-view video and reduces the loss of information in the video image compared with the traditional small field-of-view video anti-shake method. Attached Figure Description
[0025] Figure 1 This is a schematic diagram of a preferred embodiment of the unmanned vehicle large field-of-view imaging video stabilization method and system of the present invention;
[0026] Figure 2 This is a schematic diagram of the multi-aperture camera assembly structure according to a preferred embodiment of the present invention;
[0027] Figure 3 This is a schematic diagram of the multi-aperture video acquisition algorithm of a preferred embodiment of the present invention;
[0028] Figure 4 This is a schematic diagram of the real-time image stitching algorithm of a preferred embodiment of the present invention. Detailed Implementation
[0029] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0030] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.
[0031] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations according to this application; as used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise; furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.
[0032] See Figure 1-4 A large field-of-view imaging video stabilization system for unmanned vehicles is described below: Based on the large field-of-view imaging video stabilization method and system for unmanned vehicles, the system is characterized by including an unmanned vehicle, a multi-aperture camera, a data acquisition card, a computing card, a display, a video acquisition module, a real-time image stitching module, and a large field-of-view video stabilization module.
[0033] The unmanned vehicle, equipped with a computing card, a data acquisition card, and a multi-aperture camera, will generate vibrations during its movement, which are the source of vibrations in the wide field-of-view video of the unmanned vehicle.
[0034] The video acquisition module includes a capture card, a multi-aperture camera, a multi-threaded parallel acquisition module, and a frame synchronization control module. The multi-aperture camera is used for multi-aperture video; the capture card connects to the multi-aperture camera and is used to deploy the multi-aperture video acquisition algorithm; the multi-threaded parallel acquisition module controls the operation of the multi-aperture camera; and the frame synchronization control module controls the multi-aperture camera to achieve synchronous acquisition, providing the necessary conditions for stitching.
[0035] The real-time image stitching module, connected to the video acquisition module, is used to stitch the received multi-aperture images into a complete large field-of-view image;
[0036] The wide field-of-view video stabilization module is connected to the real-time image stitching module. It includes a feature point detection module, a key point motion estimation module, a grid motion propagation module, and a motion smoothing module, which are used to process the stitched wide field-of-view video for stabilization and output a stable wide field-of-view video.
[0037] The computing card is used to deploy real-time stitching algorithms and wide-field-of-view video stabilization algorithms, and is responsible for performing the corresponding computing tasks; the display is used to present the stabilized wide-field-of-view video.
[0038] The video acquisition module includes a multi-aperture camera module, a multi-threaded parallel acquisition module, an acquisition card, and a frame synchronization control module. The multi-aperture camera is an industrial USB camera with a resolution of 3920 × 2080 and a maximum adjustable frame rate of 60 FPS; the acquisition card is a Jeston Xavier NX; and the video acquisition algorithm is programmed using QT5.12.12.
[0039] The acquisition card is connected to the multi-aperture camera module, the multi-threaded parallel acquisition module, and the frame synchronization control module. The multi-threaded parallel acquisition module controls the transmission of multi-aperture images in a multi-threaded parallel manner. The frame synchronization control module sends signals to the multi-aperture camera using a continuous soft-trigger method, while controlling parameters such as the frame rate. The multi-threaded parallel acquisition module and the frame synchronization control module are used together to obtain synchronized multi-aperture images, providing the necessary conditions for stitching.
[0040] Multi-aperture camera according to Figure 1 The data acquisition card is fixed in place and mounted on the unmanned vehicle, with a wired connection between the acquisition card and the multi-aperture camera. A schematic diagram of the multi-aperture video acquisition algorithm is shown below. Figure 3 As shown, the specific steps include the following:
[0041] Step SA1: Enumerate and configure camera devices: Scan all accessible camera devices and access them one by one. Set acquisition parameters for each camera, including exposure time, gain, soft trigger mode, etc., to ensure the quality and consistency of acquired images;
[0042] Step SA2: Independent Thread Triggered Acquisition: Create an independent thread for each camera, and continuously execute soft trigger operations within the thread. After each trigger, send a command to the camera to acquire a single frame image, ensuring that the acquisition process is efficient and unaffected by other cameras;
[0043] Step SA3: Parallel Callback Image Acquisition: Register callback functions in parallel within an independent thread for each camera. Each time a camera completes image acquisition, the callback function is triggered, thus acquiring one frame of image data.
[0044] Step SA4: Main Thread Image Display: The acquired image is sent to the main thread to interact with the interface controls and achieve real-time output and display of the image;
[0045] Step SA5: Calculate and evaluate performance: Record the time of each image update, calculate and analyze performance parameters such as frame rate to evaluate the real-time performance and efficiency of the system;
[0046] Step SA6: Multi-threaded image saving: Start three independent threads specifically for saving multi-aperture images, supporting offline image processing and subsequent analysis.
[0047] The aforementioned unmanned vehicle large field-of-view imaging video stabilization method is characterized by real-time image stitching comprising an image distortion correction module, an image registration module, and an image transformation module. The image distortion correction module uses calibrated camera intrinsic parameters to distort multi-aperture images separately, while simultaneously recording the image distortion correction matrix. The image registration module is connected to the image distortion correction module. This module first receives a multi-aperture image with sufficient features, detects the feature information of the input image, records the camera's intrinsic and extrinsic parameters using multi-view geometric calibration, and calculates the transformation parameters for the stitched image. The image transformation module is connected to the image registration module and uses the recorded image distortion correction parameters and image transformation parameters to perform distortion transformation on the multi-aperture images. The transformed multi-aperture images are then fused into a single image to obtain a large field-of-view image. A schematic diagram of the real-time image stitching process is shown below. Figure 4 As shown, the specific steps include:
[0048] Step SB1: Use Zhang's calibration method to accurately estimate the camera intrinsic parameters, and calculate the distortion correction matrix based on these parameters to ensure that image distortion is effectively corrected;
[0049] Step SB2: Use the video acquisition module to acquire a multi-aperture image containing enough feature points to provide rich geometric information for subsequent processing.
[0050] Step SB3: Calculate the camera's extrinsic parameters using a multi-view geometric estimation method, and then derive the projection parameters required in the panoramic image generation process.
[0051] Step SB4: Create a stitching weight map to seamlessly fuse the multi-aperture image into the large field-of-view image, and record the weight parameters of each pixel to optimize the fusion effect.
[0052] Step SB5: Pass the image stitching parameters, weight parameters, and the size of the large field-of-view image to the stitching thread to ensure smooth and accurate real-time processing.
[0053] Step SB6: Perform frame transformation on the multi-aperture image according to the aforementioned parameters to finally generate a high-quality large field-of-view image and achieve real-time stitching.
[0054] The aforementioned unmanned vehicle wide-field-of-view imaging video stabilization method is characterized by the wide-field-of-view video stabilization module comprising a feature point detection module, a key point motion estimation module, a grid motion propagation module, and a motion smoothing module. The feature point detection module detects feature points in the input wide-field-of-view image; the key point motion estimation module, connected to the feature point detection module, filters the detected feature points and obtains key point motion vectors by matching feature points from the current frame with those from past frames; the grid motion propagation module, connected to the key point motion estimation module, first divides the wide-field-of-view video frame into an M×N grid, then propagates the key point motion data into grid vertex motion data according to a certain weight, where the weight is related to the distance between the key point and the grid vertex; the motion smoothing module, connected to the wide-field-of-view video stabilization module, smooths the calculated grid vertex motion, performs wide-field-of-view image transformation based on the offset grid motion vectors, and outputs a stable wide-field-of-view video.
[0055] The aforementioned grid motion propagation module and motion smoothing module first use the system's video acquisition module and real-time image stitching module to create a large field-of-view video dataset. This dataset consists of large field-of-view videos captured during the autonomous vehicle's movement when shaking occurs. The dataset is then input into the grid motion propagation module and motion smoothing module respectively for training, improving the robustness of the anti-shake algorithm in smoothing complex shaking within a large field of view.
[0056] The unmanned vehicle wide-field-of-view imaging video stabilization method based on the above-mentioned unmanned vehicle wide-field-of-view imaging video stabilization system includes the following steps:
[0057] Step SC1: Construct a large field-of-view video dataset based on the autonomous vehicle, including image data from different scenes;
[0058] Step SC2: Train the grid motion propagation module and motion smoothing module using the constructed dataset to obtain a grid motion propagation module and motion smoothing module that are robust to image stabilization for large field-of-view images of unmanned vehicles;
[0059] Step SC3: Acquire a calibration image for the real-time image stitching module. This image must contain enough feature points to ensure the accuracy of the transformation parameters between images, and calculate the image transformation parameters using this image.
[0060] Step SC4: Drive the driverless car. The driverless car may vibrate due to uneven road surfaces or its own characteristics.
[0061] Step SC5: The video acquisition module uses a multi-aperture camera mounted on the unmanned vehicle to capture jittery multi-aperture images, which are affected by the shaking posture information of the unmanned vehicle.
[0062] Step SC6: Input the jittery multi-aperture image into the real-time image stitching module, and transform the multi-aperture image using pre-calculated image transformation parameters to obtain a large field-of-view image;
[0063] Step SC7: The wide field-of-view video stabilization module receives the shaky wide field-of-view image, performs feature point detection, key point motion estimation, grid motion propagation and motion smoothing on the shaky image, and finally outputs a stable wide field-of-view video.
Claims
1. A wide field-of-view imaging video stabilization system for unmanned vehicles, characterized in that... It includes unmanned vehicles, multi-aperture cameras, acquisition cards, computing cards, displays, video acquisition modules, real-time image stitching modules, and large field-of-view video stabilization modules; The unmanned vehicle is equipped with a computing card, a data acquisition card, and a multi-aperture camera. The unmanned vehicle will generate vibrations during its movement, which is the source of vibration in the wide field-of-view video of the unmanned vehicle. The video acquisition module includes a capture card, a multi-aperture camera, a multi-threaded parallel acquisition module, and a frame synchronization control module. The multi-aperture camera is used to acquire multi-aperture video. The capture card is connected to the multi-aperture camera and is used to deploy the multi-aperture video acquisition algorithm. The multi-threaded parallel acquisition module is used to control the operation of the multi-aperture camera. The frame synchronization control module is used to control the multi-aperture camera to achieve synchronous acquisition. The real-time image stitching module is connected to the video acquisition module and is used to stitch the received multi-aperture images into a complete large field-of-view image; The wide field-of-view video stabilization module is connected to the real-time image stitching module. The wide field-of-view video stabilization module includes a feature point detection module, a key point motion estimation module, a grid motion propagation module, and a motion smoothing module, which are used to stun the stitched wide field-of-view video and output a stable wide field-of-view video. The computing card is used to deploy real-time stitching algorithms and wide-field-of-view video stabilization algorithms, and is responsible for performing the corresponding computing tasks; the display is used to present the stabilized wide-field-of-view video. The large field-of-view video stabilization module includes a feature point detection module, a key point motion estimation module, a grid motion propagation module, and a motion smoothing module; The feature point detection module detects feature points in the input large field-of-view image; The keypoint motion estimation module is connected to the feature point detection module. It is used to filter the detected feature points and obtain the keypoint motion vector by matching the feature points of the current frame with those of past frames. The grid motion propagation module is connected to the keypoint motion estimation module. It first divides the large field-of-view video frame into an M×N grid, then propagates the keypoint motion data into grid vertex motion data according to weights. These weights are related to the distance between the keypoint and the grid vertex. Finally, it uses multiple one-dimensional convolutions to extract motion features, fuse information, and generate an optimized grid motion trajectory. The motion smoothing module is connected to the large field-of-view video stabilization module. It is used to smooth the calculated grid vertex motion, perform large field-of-view image transformation based on the canceled grid motion vectors, and output a large field-of-view stable video. The aforementioned grid motion propagation module and motion smoothing module first use the system's video acquisition module and real-time image stitching module to create a large field-of-view video dataset. The dataset is an unstable large field-of-view video dataset collected when the unmanned vehicle shakes during its movement. The dataset is then input into the grid motion propagation module and motion smoothing module for training, thereby improving the robustness of the anti-shake algorithm in smoothing complex shaking in a large field of view.
2. The unmanned vehicle large field-of-view imaging video stabilization system according to claim 1, characterized in that: A multi-aperture camera is fixed at a certain angle on the unmanned vehicle to expand the field of view. The acquisition card is connected to the multi-aperture camera and deploys video acquisition algorithms to control the camera to perform multi-aperture video acquisition. The real-time image stitching module and the large field of view video stabilization algorithm module are deployed on the computing card and are responsible for stabilizing the received large field of view video. The monitor connects to the capture card to display stabilized, wide-field-of-view video.
3. The unmanned vehicle large field-of-view imaging video stabilization system according to claim 1, characterized in that, The video acquisition module includes a multi-aperture camera module, a multi-threaded parallel acquisition module, an acquisition card, and a frame synchronization control module. The acquisition card is connected to the multi-aperture camera module, the multi-threaded parallel acquisition module, and the frame synchronization control module. The multi-threaded parallel acquisition module controls the transmission of multi-aperture images in a multi-threaded parallel manner. The frame synchronization control module sends signals to the multi-aperture camera using a continuous soft-trigger method while controlling the frame rate parameters. The multi-threaded parallel acquisition module and the frame synchronization control module are used together to obtain synchronized multi-aperture images.
4. The unmanned vehicle large field-of-view imaging video stabilization system according to claim 1, characterized in that... The real-time image stitching includes an image distortion correction module, an image registration module, and an image transformation module. The image distortion correction module uses calibrated camera intrinsic parameters to distort the multi-aperture images separately, and simultaneously records the image distortion correction matrix. The image registration module is connected to the image distortion correction module. First, the image registration module receives a multi-aperture image with sufficient features, detects the feature information of the input image, records the camera's intrinsic and extrinsic parameters using a multi-view geometric calibration method, and calculates the transformation relationship parameters of the stitched image. The image transformation module is connected to the image registration module, uses the recorded image distortion correction parameters and image transformation parameters to perform distortion transformation on the multi-aperture images, and fuses the transformed multi-aperture images into a single image to obtain a large field-of-view image.
5. A method for stabilizing video in wide-field-of-view imaging for unmanned vehicles, characterized in that... The unmanned vehicle large field-of-view imaging video stabilization system according to any one of claims 1-4 includes the following steps: Step S1: Construct a large field-of-view video dataset based on the autonomous vehicle, including image data from different scenes; Step S2: Train the grid motion propagation module and motion smoothing module using the constructed dataset to obtain a grid motion propagation module and motion smoothing module that are robust to image stabilization of large field of view for unmanned vehicles; Step S3: Acquire a calibration image for the real-time image stitching module. This image must contain enough feature points to ensure the accuracy of the transformation parameters between images, and calculate the image transformation parameters using this image. Step S4: Drive the driverless car. The driverless car may vibrate due to uneven road surfaces or its own characteristics. Step S5: The video acquisition module uses a multi-aperture camera mounted on the unmanned vehicle to capture multi-aperture images of shaking motions, which are affected by the shaking posture information of the unmanned vehicle. Step S6: Input the jittery multi-aperture image into the real-time image stitching module, and transform the multi-aperture image using pre-calculated image transformation parameters to obtain a large field-of-view image; Step S7: The wide field-of-view video stabilization module receives the shaky wide field-of-view image, performs feature point detection, key point motion estimation, grid motion propagation and motion smoothing on the shaky image, and finally outputs a stable wide field-of-view video.