Real-time panoramic image stitching method of aerial videos shot by unmanned plane

Abstract

The invention discloses a real-time panoramic image stitching method of aerial videos shot by an unmanned plane. The method comprises the steps of: utilizing a video acquisition card to acquire images which are transmitted to a base station in real time by an unmanned plane through microwave channels, carrying out key frame selection on an image sequence, and carrying out image enhancement on key frames; in the image splicing process, firstly carrying out characteristic detection and interframe matching on image frames by adopting an SURF (speeded up robust features) detection method with good robustness; then reducing the series-multiplication accumulative errors of images in a frame-to-mosaic image transformation mode, determining images which are not adjacent in time sequence but adjacent in space on a flight path according to the GPS (global positioning system) position information of the unmanned plane, optimizing the frame-to-mosaic transformation relation, determining image overlapping areas, thereby realizing image fusion and the panoramic image construction and realizing real-time effect of carrying out flying and stitching simultaneously; and in image transformation, based on adjacent frame information in a vision field and adjacent frame information in airspace, optimizing image transformation to obtain the accurate panoramic images. The stitching method has good real-time performance, is fast and accurate and meets the requirements of application occasions in multiple fields.

Description

technical field [0001] The invention relates to the splicing of panoramas of video images, in particular to a method for splicing real-time panoramas of unmanned aerial vehicle video by combining modern video scene splicing technology with a flight control system of an unmanned aerial vehicle. Background technique [0002] In recent years, with the development of technologies such as automatic control and wireless transmission, UAVs have developed rapidly in both military and civilian applications. Video shooting by drones equipped with cameras has played a huge role in military reconnaissance, advertising photography, and disaster judgment. However, because there is always a difference between the shooting range and resolution of the camera, the video of the entire flight process often contains all the information of the entire scene, while the information of a single frame image is very limited, thus the video image stitching technology was born. Video image splicing tech...

AI Technical Summary

Problems solved by technology

The main problems are: 1. These stitching methods are all stitching the acquired image sets, which cannot achieve the effect of stitching while the drone is flying. 2. These softwares are only suitable for scene restoration with a small number of image sets. Unable to process image sequences with very long time series. 3. Using SIFT for feature detection and full image frame bundle adjustment strategy, the time efficiency is low

From the above splicing process, we can see that each calculation we get is the motion relationship of adjacent images, and the corresponding relationship between the current frame and the reference frame is obtained through matrix multiplication, and the transformation T between adjacent images i，i+1 It is estimated by using a nonlinear optimization algorithm on the feature point set, which shows that the obtained transformation can only be the best motion parameter in the sense of least squares, and there must be a certain error

Images