Unmanned aerial vehicle inclined video and vector map real-time projection matching method

Abstract

The invention discloses an unmanned aerial vehicle tilt video and vector map real-time projection sleeving method, which comprises an unmanned aerial vehicle body, and is characterized in that a video data acquisition module and a video preprocessing module for preprocessing the tilt video acquired by the video data acquisition module are arranged in the unmanned aerial vehicle body; a map data processing module is further arranged in the unmanned aerial vehicle body, an improved ORB algorithm is adopted in the feature extraction and matching module, and a posture correction and projection transformation module used in cooperation with the video data acquisition module is installed in the unmanned aerial vehicle body. And a real-time display module in the unmanned aerial vehicle body displays the sleeved video and vector map on terminal equipment in real time through a signal module arranged in the unmanned aerial vehicle body. The problems that the real-time performance is poor, the nesting precision is reduced, and long-time high-precision nesting cannot be achieved during working in an existing unmanned aerial vehicle tilt video and vector map fusion technology at present are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles (UAVs), and in particular to a method for real-time projection overlay of UAV tilted video and vector maps. Background Art

[0002] A drone is an unmanned aerial vehicle controlled by a radio remote control device or its own program control device. It is widely used in geographic surveying and mapping, security monitoring, emergency rescue, and other fields. Vector maps are maps that accurately represent geographic features using geometric elements such as points, lines, and surfaces by recording coordinates. They have the advantages of small data volume, high accuracy, and arbitrary scalability. Drone oblique photography refers to the process of photographing a target area from multiple angles to obtain oblique images with rich texture information, which can truly reflect the actual situation of the ground. The fusion technology of drone oblique video and vector maps aims to accurately match the real-time video information collected by drones with vector maps, providing users with more intuitive and accurate geographic information services.

[0003] The current existing technology for fusing drone oblique videos with vector maps has poor real-time performance and is unable to meet the needs of real-time drone video processing. Moreover, in complex scenarios, such as lighting changes and occlusions, the accuracy of feature point extraction and matching will drop significantly, resulting in reduced alignment accuracy. During operation, GPS signals are easily affected by occlusion and interference, resulting in inaccurate positioning. Inertial navigation equipment has cumulative errors, which will gradually increase over time, causing the accuracy of video and map alignment to continue to decrease, making it impossible to achieve long-term high-precision alignment.

[0004] To solve the above problems, this application proposes a method for real-time projection superposition of drone oblique video and vector map. Summary of the Invention

[0005] In response to the problems in the related technology, the present invention provides a method for real-time projection matching of drone oblique video and vector map, which can realize fast, high-precision real-time projection matching of drone oblique video and vector map, providing users with more accurate and real-time geographic information display, and meeting the needs of geographic surveying and mapping, security monitoring, emergency rescue and other fields for high-precision real-time geographic information.

[0006] To this end, the specific technical solutions adopted in the present invention are as follows: A method for real-time projection overlay of unmanned aerial vehicle (UAV) oblique video and vector map includes an unmanned aerial vehicle (UAV) body, wherein the UAV body is provided with a video data acquisition module and a video preprocessing module for preprocessing the oblique video acquired by the video data acquisition module. The UAV body is also provided with a map data processing module. During overlay calculation, the vector map coordinate system in the map data processing module and the positioning coordinates of the video are unified into a coordinate system matching the video image. The UAV body is provided with a feature extraction and matching module for simultaneously extracting feature points in the video frame and the vector map, and the feature extraction and matching module adopts an improved ORB algorithm. The UAV body is installed with a posture correction and projection transformation module used in conjunction with the video data acquisition module. The real-time display module in the UAV body displays the overlaid video and vector map in real time on a terminal device through a signal module built into the UAV body.

[0007] As a further solution of the present invention, the video data acquisition module includes a high-definition camera, a GPS module and an inertial navigation unit (IMU) carried by the drone body. The high-definition camera is used to collect oblique videos of the target area, the GPS module obtains the drone's geographic location information (latitude and longitude, altitude) in real time, and the inertial navigation unit (IMU) obtains the drone's attitude information (pitch angle, roll angle, yaw angle) in real time.

[0008] As a further solution of the present invention, the video preprocessing module preprocesses the collected oblique video, including operations such as denoising and contrast enhancement, to improve the quality of the video image and provide a better basis for subsequent feature extraction and matching.

[0009] As a further solution of the present invention, the map data processing module resamples point / line / surface vector data: the point data is directly converted into a set of coordinates; the line data is discretized into an ordered point chain; the surface data is converted into a closed point ring; and a topological relationship database (such as a road intersection and turning point feature library) is established to support fast matching.

[0010] As a further solution of the present invention, the ORB (Oriented FAST and Rotated BRIEF) algorithm is optimized based on the FAST (Features from Accelerated Segment Test) corner detection algorithm and the BRIEF (Binary Robust Independent Elementary Features) algorithm descriptor, and has the characteristics of fast calculation speed and good real-time performance.

[0011] As a further solution of the present invention, the attitude correction and projection transformation module, when used in combination with the drone body position and attitude information obtained by the GPS module and the inertial navigation unit (IMU), performs attitude correction on the video frame to eliminate image deformation caused by the change of the drone's attitude. Then, based on the matched feature point pairs, the video frame is projected onto the vector map using the perspective transformation principle to achieve the superposition of the video and the map.

[0012] As a further solution of the present invention, the real-time display module displays the superimposed video and vector map on the terminal device in real time, so that the user can intuitively view the real-time situation of the target area and the corresponding relationship with the geographic information.

[0013] The beneficial effects of the present invention are: The present invention adopts an improved ORB algorithm for feature extraction and matching. Compared with traditional algorithms such as SIFT, the computational complexity is greatly reduced. It can complete the extraction and matching of a large number of feature points in a short time, meet the needs of real-time video processing of drones, and realize real-time projection matching of oblique videos and vector maps, thereby improving real-time performance.

[0014] The present invention improves the matching accuracy of feature points by adopting a feature point screening mechanism based on an improved ORB algorithm. At the same time, it performs attitude correction on video frames by combining the position and attitude information obtained by the GPS module and the inertial navigation device (IMU), effectively eliminating the impact of drone attitude changes and positioning errors, thereby greatly improving the accuracy of video and map overlay. Therefore, in complex scenarios, the overlay accuracy of the present invention is improved by more than 30% compared with existing feature point matching-based technologies, and by more than 40% compared with technologies based on GPS and inertial navigation.

[0015] By preprocessing the video and adopting an improved feature extraction algorithm, the present invention can maintain a high feature point extraction and matching accuracy in complex environments such as lighting changes and occlusions, thereby ensuring the stability and reliability of the fitting and further enhancing the robustness of the fitting system.

[0016] The present invention can provide more accurate real-time data for geographic surveying and mapping, assist security monitoring in more clearly locating targets, quickly provide accurate geographic information in emergency rescue, and effectively improve work efficiency and decision-making accuracy in related fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 The present invention is a technical flowchart of a method for real-time projection overlay of oblique video from a drone and a vector map according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] To further illustrate each embodiment, the present invention provides drawings, which are part of the disclosure of the present invention. They are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. By referring to these contents, ordinary technicians in this field should be able to understand other possible implementation methods and advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0020] According to an embodiment of the present invention, a method for real-time projection overlay of oblique video of a drone and a vector map is provided.

[0021] Please refer to the instruction manual Figure 1According to an embodiment of the present invention, a method for real-time projection alignment of a drone oblique video and a vector map includes a drone body, wherein the drone body is provided with a video data acquisition module, and a video preprocessing module for preprocessing the oblique video acquired by the video data acquisition module. The drone body is also provided with a map data processing module. During the alignment calculation, the vector map coordinate system in the map data processing module and the positioning coordinates of the video are unified into a coordinate system that matches the video image. The map data processing module resamples point / line / surface vector data: wherein the point is a direct coordinate set; the line is discretized into an ordered point chain; and the surface is converted into a closed point. loop; establish a topological relationship database (such as a road intersection and inflection point feature library) to support fast matching; the unmanned aerial vehicle is provided with a feature extraction and matching module for simultaneously extracting feature points in the video frame and the vector map, and the feature extraction and matching module adopts an improved ORB algorithm; the unmanned aerial vehicle is installed with a posture correction and projection transformation module used in conjunction with the video data acquisition module; the real-time display module in the unmanned aerial vehicle displays the superimposed video and vector map in real time on the terminal device through the signal module built into the unmanned aerial vehicle, so that the user can intuitively view the real-time situation of the target area and the corresponding relationship with the geographic information. By adopting the improved ORB algorithm for feature extraction and matching, the computational complexity is greatly reduced compared with traditional algorithms such as SIFT, and a large number of feature points can be extracted and matched in a short time, meeting the needs of real-time video processing of drones, and realizing real-time projection matching of inclined videos and vector maps, thereby improving real-time performance. In addition, by adopting the feature point screening mechanism of the improved ORB algorithm, the matching accuracy of feature points is improved; at the same time, the position and attitude information obtained by the GPS module and the inertial navigation device (IMU) are combined to perform attitude correction on the video frames, effectively eliminating the influence caused by the drone attitude change and positioning error, thereby greatly improving the matching accuracy of the video and map. Therefore, in complex scenes, the matching accuracy of the present invention is improved by more than 30% compared with the existing technology based on feature point matching, and more than 40% compared with the technology based on GPS and inertial navigation.

[0022] Among them, the video data acquisition module includes a high-definition camera, a GPS module and an inertial navigation unit (IMU) carried by the drone body. The high-definition camera is used to collect tilted videos of the target area, the GPS module obtains the drone's geographic location information (latitude, longitude, altitude) in real time, and the inertial navigation unit (IMU) obtains the drone's attitude information (pitch angle, roll angle, yaw angle) in real time.

[0023] The ORB (Oriented FAST and Rotated BRIEF) algorithm is an optimization of the FAST (Features from Accelerated Segment Test) corner detection algorithm and the BRIEF (Binary Robust Independent Elementary Features) algorithm descriptor, offering fast computational speed and excellent real-time performance. The specific implementation involves grayscale processing of video frames and vector maps, then detecting feature points using the FAST algorithm. The BRIEF algorithm then generates feature descriptors. Finally, the Hamming distance is used to calculate the similarity between feature descriptors to select highly matching feature point pairs.

[0024] The attitude correction and projection transformation module combines the drone's position and attitude information obtained by the GPS module and inertial navigation unit (IMU) to perform attitude correction on the video frames, eliminating image distortion caused by the drone's attitude changes. Based on the matched feature point pairs, the video frames are then projected onto the vector map using the principle of perspective transformation, achieving video-map alignment. Specifically, the module calculates the geographic coverage (Xmin, Xmax, Ymin, Ymax) of each frame based on the camera's center position (Xs, Ys, Zs) and attitude angle. The video frames are then rotated, translated, and scaled according to the drone's attitude angle to align with the perspective of the vector map. Finally, the coordinates in the video frames are mapped to the vector map's coordinate system using a perspective transformation matrix.

[0025] In one embodiment, please refer to the appendix of the specification. Figure 1 As a further aspect of the present invention, the video preprocessing module performs preprocessing on the captured oblique video, including operations such as denoising and contrast enhancement, improving the quality of the video image and providing a better foundation for subsequent feature extraction and matching. By preprocessing the video and employing an improved feature extraction algorithm, the present invention maintains high feature point extraction and matching accuracy even in complex environments such as lighting changes and occlusions, ensuring the stability and reliability of the overlay, thereby enhancing the robustness of the overlay system.

[0026] In one embodiment, please refer to the appendix of the specification. Figure 1 As a further aspect of the present invention, the feature extraction and matching module can also utilize a convolutional neural network (CNN) to extract features from video frames and vector maps, using a trained model to learn matching relationships between features. Deep learning models can automatically learn image features, offering greater adaptability in complex scenarios and further improving the accuracy of feature extraction and matching.

[0027] In another embodiment, as a further solution to the present invention, the drone can also be equipped with a laser radar device to obtain 3D point cloud data of the target area. By fusing the 3D point cloud data with a vector map and then overlaying it with video information, the laser radar data provides precise 3D spatial information, helping to improve the accuracy and reliability of the overlay, making it particularly suitable for areas with complex terrain.

[0028] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for real-time projection superposition of oblique video of a UAV and a vector map, comprising a UAV body, characterized by: The unmanned aerial vehicle body is provided with a video data acquisition module and a video preprocessing module for preprocessing the oblique video acquired by the video data acquisition module. The unmanned aerial vehicle body is also provided with a map data processing module. During the fitting calculation, the vector map coordinate system in the map data processing module and the positioning coordinates of the video are unified into a coordinate system that matches the video image. The unmanned aerial vehicle body is provided with a feature extraction and matching module for simultaneously extracting feature points in the video frame and the vector map, and the feature extraction and matching module adopts an improved ORB algorithm. The unmanned aerial vehicle body is installed with a posture correction and projection transformation module used in conjunction with the video data acquisition module. The real-time display module in the unmanned aerial vehicle body displays the fitted video and vector map in real time on the terminal device through the signal module built into the unmanned aerial vehicle body.

2. The method for real-time projection superposition of drone oblique video and vector map according to claim 1 is characterized by: The video data acquisition module includes a high-definition camera, a GPS module and an inertial navigation unit (IMU) carried by the drone. The high-definition camera is used to collect tilted videos of the target area. The GPS module obtains the drone's geographic location information (latitude, longitude, and altitude) in real time. The inertial navigation unit (IMU) obtains the drone's attitude information (pitch angle, roll angle, and yaw angle) in real time.

3. The method for real-time projection superposition of drone oblique video and vector map according to claim 1 is characterized by: The video preprocessing module performs preprocessing on the collected tilted video, including operations such as denoising and contrast enhancement, to improve the quality of the video image and provide a better basis for subsequent feature extraction and matching.

4. The method for real-time projection superposition of drone oblique video and vector map according to claim 1, characterized in that: The map data processing module resamples point / line / surface vector data: points are directly converted into coordinate sets; lines are discretized into ordered point chains; surfaces are converted into closed point rings; and a topological relationship database (such as a road intersection and inflection point feature library) is established to support fast matching.

5. The method for real-time projection superposition of drone oblique video and vector map according to claim 1 is characterized by: The ORB (Oriented FAST and Rotated BRIEF) algorithm is optimized based on the FAST (Features from Accelerated Segment Test) corner detection algorithm and the BRIEF (Binary Robust Independent Elementary Features) algorithm descriptor, and has the characteristics of fast calculation speed and good real-time performance.

6. The method for real-time projection superposition of drone oblique video and vector map according to claim 2, characterized in that: The attitude correction and projection transformation module combines the drone body position and attitude information obtained by the GPS module and the inertial navigation unit (IMU) to perform attitude correction on the video frame, eliminating image deformation caused by the change of the drone's attitude. Then, based on the matched feature point pairs, the video frame is projected onto the vector map using the perspective transformation principle to achieve the superposition of the video and the map.

7. The method for real-time projection superposition of drone oblique video and vector map according to claim 1, characterized in that: The real-time display module displays the superimposed video and vector map on the terminal device in real time, so that the user can intuitively view the real-time situation of the target area and the corresponding relationship with the geographic information.

Images