Unmanned aerial vehicle high-precision space positioning and virtual-real mapping three-dimensional reconstruction method

Abstract

The invention relates to a high-precision spatial positioning and virtual-real mapping three-dimensional reconstruction method for an unmanned aerial vehicle. Comprising the following steps: deploying and calibrating a ground base station, deploying and debugging unmanned aerial vehicle equipment, synchronously acquiring live-action data, establishing a spatial association data set, preprocessing a live-action photo, transforming spatial coordinates, matching photo characteristics and carrying out geometric constraint, constructing an initial three-dimensional point cloud model, optimizing the three-dimensional point cloud model and generating a high-precision three-dimensional reconstruction model. Millimeter-level positioning of the unmanned aerial vehicle is achieved through the ground base station and the optimization algorithm, then the three-dimensional reconstruction model with millimeter-level precision is obtained, applicability is high, and the method can be widely applied to multiple fields.

Description

Technical Field

[0001] This invention relates to the field of UAV surveying and 3D reconstruction technology, and in particular to a method for high-precision spatial positioning and virtual-real mapping 3D reconstruction of UAVs. Background Technology

[0002] With the rapid development of UAV technology, UAVs have been widely used in the field of real-scene scanning 3D reconstruction. They offer advantages such as flexible operation, wide coverage, and high efficiency, and have been widely applied in various fields including architectural surveying, cultural relic protection, topographic surveying, and urban planning. However, current UAV real-scene scanning 3D reconstruction technology generally suffers from insufficient accuracy in the reconstructed models, making it difficult to meet the needs of high-precision calculation and analysis, and precision engineering construction.

[0003] Research has revealed that one of the core reasons for the insufficient accuracy of the reconstructed model lies in the limitation of UAV positioning accuracy. Most existing UAVs rely on GNSS satellite positioning systems for positioning, with accuracy typically at the centimeter level. Even with differential positioning technology, it is difficult to break through this centimeter-level limitation. In the 3D reconstruction process, the UAV's positioning accuracy directly determines the spatial accuracy of the collected real-world data, thus affecting the accuracy of the final reconstructed model. Therefore, to achieve high-precision 3D reconstruction, it is essential to overcome the bottleneck of UAV positioning accuracy.

[0004] To improve the positioning accuracy of UAVs, various solutions have been proposed in existing technologies, such as multi-satellite system fusion positioning and optimized positioning algorithms. However, these solutions offer limited improvement in positioning accuracy and are insufficient to meet millimeter-level positioning requirements. Furthermore, some solutions enhance reconstruction accuracy by increasing the precision of the equipment carried by the UAV, but neglect the fundamental impact of the UAV's own positioning accuracy on the reconstruction results, leading to unsatisfactory improvements in the accuracy of the reconstructed model.

[0005] Based on the above problems, there is an urgent need for a method that can effectively improve the positioning accuracy of UAVs and thus achieve millimeter-level 3D reconstruction, so as to meet the needs of various fields for high-precision 3D reconstruction models. Summary of the Invention

[0006] This invention aims to solve the problem of insufficient accuracy of the reconstructed model in UAV 3D reconstruction technology, and provides a UAV high-precision spatial positioning and virtual-real mapping 3D reconstruction method.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: a method for high-precision spatial positioning and virtual-real mapping 3D reconstruction of unmanned aerial vehicles, comprising the following steps:

[0008] S1. Deployment and calibration of ground base stations: Deploy at least three fixed ground base stations around the area to be measured, complete the positioning calibration and signal debugging of the ground base stations, and enable the ground base stations to have the ability to receive, process and feed back coordinate information.

[0009] S2. Deployment and debugging of UAV equipment: Control the UAV equipped with image acquisition module, lidar module and laser positioning and transmission module to fly to the airspace above the test area, complete the initialization and debugging of each module of the UAV, and ensure that the image acquisition module, lidar module and laser positioning and transmission module work synchronously;

[0010] S3. Real-world data synchronous acquisition: Start the drone to synchronously acquire the following real-world data:

[0011] The image acquisition module continuously captures real-scene photos of the measured area and simultaneously records the lens resolution and wide-angle parameters corresponding to each photo.

[0012] The lidar module synchronously collects distance information between the drone and various sampling points in the real scene;

[0013] The laser positioning transmitter module synchronously transmits laser positioning signals to the ground base station. After receiving the laser positioning signals, the ground base station calculates the real-time accurate coordinates of the UAV and feeds them back to the UAV.

[0014] S4. Establishment of Spatial Association Dataset: The UAV receives real-time accurate coordinates fed back by the ground base station, and combines the distance information collected by the lidar module and the parameters recorded by the image acquisition module to establish a UAV-real scene spatial association dataset for each sampling time. The spatial association dataset shall at least include the sampling time, UAV coordinates, distance between the UAV and the real scene sampling point, real scene photos and photo parameters.

[0015] S5. Real-scene photo preprocessing: Perform preprocessing on the real-scene photos acquired in step S3, including noise reduction, distortion correction and feature point extraction.

[0016] S6. Spatial coordinate transformation: Based on the spatial association dataset established in step S4, the pixel coordinates corresponding to each real-world photo are converted into world coordinates through a spatial coordinate transformation algorithm.

[0017] S7. Photo Feature Matching and Geometric Constraints: Feature matching algorithms are used to match feature points in multiple pre-processed real-world photos. Combined with distance information collected by LiDAR, multi-view geometric constraints are applied to eliminate mismatched feature points.

[0018] S8. Initial 3D point cloud model construction: Based on the matched feature points and their corresponding world coordinate information, an initial 3D point cloud model of the measured area is constructed using a dense point cloud generation algorithm.

[0019] S9. Optimization of 3D Point Cloud Model and Generation of High-Precision 3D Reconstruction Model: The initial 3D point cloud model is optimized by including point cloud denoising, simplification and smoothing, and finally a high-precision 3D reconstruction model is obtained.

[0020] Specifically, in step S1, the positioning calibration of the ground base station adopts the static differential positioning method. Specifically, the ground base station receives GNSS satellite signals, combines them with preset reference station coordinate data to perform differential calculations, and completes the accurate calibration of its own coordinates. The coordinate error of the calibrated ground base station is controlled within ±0.1 mm.

[0021] Specifically, in step S2, the synchronous operation of the image acquisition module, the lidar module, and the laser positioning and emission module is achieved through a high-precision time synchronizer, with the synchronization error controlled within ±1μs; and the ranging accuracy of the lidar module is ±0.5 mm, with a measurement range of 0.1-500 m.

[0022] Specifically, in step S3, the propagation time of the laser positioning signal is t, the speed of laser propagation in air is c, and the distance d between the ground base station and the UAV is calculated using the following formula:

[0023] ;

[0024] Based on its own precise coordinates and the calculated distance d, the ground base station determines the real-time precise coordinates of the UAV using a triangulation algorithm. The coordinate calculation process of the triangulation algorithm is as follows: Let the coordinates of the ground base station be (x0, y0, z0) and the coordinates of the UAV be (x, y, z), then the following formula is satisfied:

[0025] ;

[0026] By deploying at least three ground base stations, a system of equations is established and solved to obtain the real-time accurate coordinates (x, y, z) of the UAV.

[0027] Specifically, in step S5, the noise reduction in photo preprocessing uses a Gaussian filtering algorithm. The convolution kernel function of the Gaussian filter is shown in the following formula:

[0028] ;

[0029] Where σ is the standard deviation of the Gaussian filter, x y These are the relative coordinates of the pixels within the convolution kernel;

[0030] Distortion correction is performed based on the camera intrinsic parameter matrix, M, as shown in the following formula:

[0031] ;

[0032] Among them, f x f y c represents the focal length of the camera in the x and y directions, respectively. x c y The coordinates of the camera's principal point;

[0033] Feature point extraction uses the SIFT algorithm, which includes the following steps:

[0034] Construct a Gaussian difference pyramid to detect extreme points;

[0035] Precisely locate extreme points and eliminate unstable extreme points;

[0036] Assign a direction vector to each extreme point to generate a feature descriptor with rotation invariance;

[0037] Euclidean distance was used as a similarity metric to match feature descriptors of different photos.

[0038] Specifically, in step S6, the spatial coordinate transformation algorithm uses perspective projection transformation. The transformation formula for converting pixel coordinates (u, v) to world coordinates (X, Y, Z) is shown in the following formula:

[0039] ;

[0040] Where [RT] is the extrinsic parameter matrix of the camera, R is the rotation matrix, and T is the translation vector.

[0041] Specifically, in step S7, feature matching uses the feature descriptors of the SIFT algorithm for matching, and epipolar constraints are applied based on the distance information collected by the lidar module to eliminate mismatched feature points.

[0042] Specifically, in step S8, the dense point cloud generation algorithm adopts a multi-view stereo matching algorithm based on patches. By matching pixels in the photo point by point and combining the distance information between the UAV and the real-world sampling points, dense three-dimensional point cloud data is generated.

[0043] Specifically, in step S9, point cloud denoising uses a statistical filtering algorithm to calculate the average distance of points within the k-neighborhood of each point and remove outliers whose distance is greater than the product of the average distance and the standard deviation; point cloud simplification uses a voxel grid downsampling algorithm to divide the point cloud space into regular voxel grids and retain a representative point in each grid to achieve uniform simplification of point cloud data; point cloud smoothing uses the moving least squares method to perform weighted fitting on the points within the neighborhood of each point to obtain the smoothed point cloud coordinates.

[0044] The beneficial effects of this invention are:

[0045] 1. High positioning accuracy: This invention achieves millimeter-level positioning for UAVs by deploying fixed ground base stations around the measured area and combining laser positioning technology with triangulation algorithms. The positioning accuracy can reach ±0.5 mm, which is a qualitative leap compared to the centimeter-level accuracy of existing GNSS positioning technology, providing a precise positioning foundation for high-precision 3D reconstruction.

[0046] 2. Good data synchronization: A high-precision time synchronizer is used to realize the synchronous operation of image acquisition, lidar ranging and laser positioning signal transmission. The synchronization error is controlled within ±1μs, which ensures the time consistency of photo data, distance data and UAV coordinate data and avoids reconstruction errors caused by data asynchrony.

[0047] 3. High reconstruction accuracy: Through optimized photo preprocessing algorithms, spatial coordinate transformation algorithms, feature matching algorithms, and point cloud optimization algorithms, combined with accurate UAV positioning data and LiDAR ranging data, millimeter-level accuracy 3D reconstruction is finally achieved. The accuracy of the reconstructed model can reach ±1mm, which can meet the needs of high-precision calculation and analysis, precision engineering construction and other scenarios.

[0048] 4. Strong applicability: The method of this invention is not limited by the terrain, environment and other factors of the area being measured, and can be widely applied to multiple fields such as architectural surveying, cultural relic protection, topographic surveying, and urban planning, and has strong versatility and applicability.

[0049] 5. Simple operation: The deployment of ground base stations and debugging of UAVs in this invention are simple and convenient. The data collection and processing can be automated without complicated manual intervention, which effectively improves the efficiency of operation. Attached Figure Description

[0050] Figure 1 This is a flowchart of the method of the present invention;

[0051] Figure 2 This is a schematic diagram of the ground base station deployment according to the present invention;

[0052] The following will describe in detail, with reference to the accompanying drawings, embodiments of the present invention. Detailed Implementation

[0053] The present invention will be further described below with reference to embodiments:

[0054] like Figures 1-2 As shown, a method for high-precision spatial positioning and virtual-real mapping 3D reconstruction of unmanned aerial vehicles (UAVs) includes the following steps:

[0055] S1. Deployment and calibration of ground base stations: Deploy at least three fixed ground base stations around the area to be measured, complete the positioning calibration and signal debugging of the ground base stations, and enable the ground base stations to have the ability to receive, process and feed back coordinate information.

[0056] Specifically, fixed ground base stations are deployed around the area to be measured. The deployment locations of the ground base stations must be able to receive the laser positioning signals emitted by the UAV, and the number of base stations deployed must be no less than three to achieve accurate positioning of the UAV. The ground base stations are fixed with high-strength brackets to ensure their stability during operation and to avoid positioning errors caused by base station movement.

[0057] The positioning calibration of the ground base station adopts the static differential positioning method. Specifically, the ground base station receives GNSS satellite signals and continuously collects satellite observation data for a period of time (no less than 30 minutes). It then performs differential calculations based on preset reference station coordinate data to eliminate systematic errors such as satellite orbit errors, ionospheric errors, and tropospheric errors, thus completing the precise calibration of its own coordinates. The calibrated ground base station coordinate error is controlled within ±0.1mm, providing accurate reference coordinates for UAV positioning.

[0058] After completing the positioning calibration, the signal receiving module, data processing module and communication module of the ground base station are debugged to ensure that the ground base station can quickly receive the laser positioning signal emitted by the UAV, accurately process the signal data and calculate the distance of the UAV, and at the same time, can feed back the calculation results to the UAV in a timely manner.

[0059] S2. Deployment and debugging of UAV equipment: Control the UAV equipped with image acquisition module, lidar module and laser positioning and transmission module to fly to the airspace above the test area, complete the initialization and debugging of each module of the UAV, and ensure that the image acquisition module, lidar module and laser positioning and transmission module work synchronously;

[0060] The UAV is equipped with an image acquisition module, a high-precision lidar module, a laser positioning and transmitting module, a high-precision time synchronizer, and a communication module; the ground base station includes a laser signal receiving module, a data processing module, a GNSS receiving module, and a communication module. The UAV and the ground base station exchange data via laser signals and wireless communication links, enabling precise positioning and data transmission for the UAV.

[0061] Specifically, a drone platform with stable flight performance was selected, and an image acquisition module, a lidar module, and a laser positioning and emission module were mounted on the drone. The image acquisition module uses a high-resolution industrial camera with adjustable parameters, capable of recording parameters such as lens resolution and wide-angle during shooting. The lidar module uses a lidar device with a ranging accuracy of ±0.5mm and a measurement range of 0.1-500 m, used to accurately acquire distance information between the drone and various sampling points in the real scene. The laser positioning and emission module uses a high-frequency, high-stability laser emitter capable of transmitting continuous laser positioning signals to the ground base station.

[0062] A high-precision time synchronizer is installed on the UAV and connected to the image acquisition module, the high-precision lidar module, and the laser positioning and transmission module, respectively, to achieve synchronous operation of the three modules. The synchronization error is controlled within ±1μs, ensuring that photo shooting, lidar ranging, and laser positioning signal transmission occur at the same time, providing a time reference for subsequent data association and coordinate calculation.

[0063] After the equipment installation is completed, the UAV is initialized and debugged, including the flight control system, the working status of each module, and the communication link, to ensure that the UAV can fly normally, each module can work stably, and the communication link between the UAV and the ground base station is unobstructed.

[0064] S3. Real-world data synchronous acquisition: Start the drone to synchronously acquire the following real-world data:

[0065] The image acquisition module continuously captures real-scene photos of the measured area and simultaneously records the lens resolution and wide-angle parameters corresponding to each photo.

[0066] The lidar module synchronously collects distance information between the drone and various sampling points in the real scene;

[0067] The laser positioning transmitter module synchronously transmits laser positioning signals to the ground base station. After receiving the laser positioning signals, the ground base station calculates the real-time accurate coordinates of the UAV and feeds them back to the UAV.

[0068] The propagation time of the laser positioning signal is t, the speed of laser propagation in air is c, and the distance d between the ground base station and the drone is calculated using the following formula:

[0069] ;

[0070] Based on its own precise coordinates and the calculated distance d, the ground base station determines the real-time precise coordinates of the UAV using a triangulation algorithm. The coordinate calculation process of the triangulation algorithm is as follows: Let the coordinates of the ground base station be (x0, y0, z0) and the coordinates of the UAV be (x, y, z), then the following formula is satisfied:

[0071] ;

[0072] By deploying at least three ground base stations, a system of equations is established and solved to obtain the real-time accurate coordinates (x, y, z) of the UAV.

[0073] Specifically, the drone is controlled to fly over the area to be tested and fly according to the preset flight path and shooting plan. The data acquisition program is started, and the image acquisition module continuously takes real-scene photos of the area to be tested at a preset shooting frequency (1-10 frames / second), while automatically recording parameters such as lens resolution, wide angle, and shooting time for each photo; the high-precision lidar module is started simultaneously to scan the area to be tested, collect distance information between the drone and each sampling point in the real scene, and record the acquisition time corresponding to each distance data; the laser positioning transmission module simultaneously transmits laser positioning signals to the ground base station. The frequency of the laser positioning signal is 100 MHz. After receiving the laser positioning signal, the ground base station records the signal reception time and calculates the distance between the drone and each ground base station based on the propagation time of the laser signal.

[0074] Based on its own precise coordinates and the calculated distances to the UAV, the ground base station determines the UAV's real-time precise coordinates using a triangulation algorithm. Let the coordinates of the ground base stations be (x1, y1, z1), (x2, y2, z2), and (x3, y3, z3), and the distances between the UAV and the three ground base stations be d1, d2, and d3, respectively. Let the UAV's coordinates be (x, y, z). Then, based on the triangulation principle, the following system of equations is established:

[0075] ;

[0076] ;

[0077] ;

[0078] By solving the above system of equations, the real-time accurate coordinates (x, y, z) of the UAV are obtained, and this coordinate information is fed back to the UAV through the communication link, with the feedback delay controlled within 10 ms.

[0079] S4. Establishment of Spatial Association Dataset: The UAV receives real-time accurate coordinates fed back by the ground base station, and combines the distance information collected by the lidar module and the parameters recorded by the image acquisition module to establish a UAV-real scene spatial association dataset for each sampling time. The spatial association dataset shall at least include the sampling time, UAV coordinates, distance between the UAV and the real scene sampling point, real scene photos and photo parameters.

[0080] Specifically, after receiving real-time precise coordinates from the ground base station, the UAV combines this data with distance information collected by the high-precision LiDAR module and photos and parameters recorded by the image acquisition module. The data is then correlated according to the shooting timestamp to establish a UAV-real-world spatial correlation dataset for each sampling moment. This dataset is stored in tabular form, with each record containing the sampling time, UAV coordinates (x, y, z), distances between the UAV and each sampling point in the real-world scene, real-world photo filename, lens resolution, and wide-angle information. The timestamp correlation ensures that each photo, distance, and UAV coordinate data corresponds to the same sampling time, providing a precise data foundation for subsequent coordinate transformations and 3D reconstruction.

[0081] S5. Real-scene photo preprocessing: Perform preprocessing on the real-scene photos acquired in step S3, including noise reduction, distortion correction and feature point extraction.

[0082] Image Denoising: Gaussian filtering is used to denoise the image. Gaussian filtering effectively suppresses Gaussian noise while preserving edge details relatively well. The core of Gaussian filtering is constructing a Gaussian convolution kernel, which is then convolved with the image's pixel matrix to obtain the denoised image. The Gaussian filtering convolution kernel function is shown in the following formula:

[0083] ;

[0084] Where σ is the standard deviation of the Gaussian filter, which is adjusted according to the noise level of the photo, and usually ranges from 0.5 to 2. In this embodiment, σ is set to 1, and a 3×3 Gaussian convolution kernel is constructed to perform convolution operations on the photo to complete the noise reduction process; x y These are the relative coordinates of the pixels within the convolution kernel;

[0085] Distortion Correction: Due to the optical characteristics of camera lenses, captured images will exhibit radial and tangential distortion, requiring distortion correction. Distortion correction is based on the camera intrinsic parameter matrix. First, the camera intrinsic parameter matrix M is obtained through camera calibration, as shown in the following formula:

[0086] ;

[0087] Among them, f x f y c represents the focal length of the camera in the x and y directions, respectively. x c y The coordinates of the camera's principal point are given. Camera calibration employs the Zhang Zhengyou calibration method, which calculates the camera's intrinsic parameter matrix and distortion coefficients by capturing multiple chessboard images.

[0088] Based on the obtained distortion coefficients and intrinsic parameter matrix, the following formula is used to correct the distortion of the pixels in the photo:

[0089] ;

[0090] ;

[0091] Where (x, y) are the pixel coordinates before correction, (x... corrected , y corrected ) represents the corrected pixel coordinates, k1, k2, and k3 are the radial distortion coefficients, p1 and p2 are the tangential distortion coefficients, and r 2 = x 2 + y 2 .

[0092] Feature point extraction: The SIFT algorithm is used to extract feature points from the corrected image. SIFT is scale-invariant and rotation-invariant, enabling it to accurately extract stable feature points from images of different scales and angles. The specific steps are as follows:

[0093] Constructing a Gaussian difference pyramid: Convolve the photo with Gaussian functions of different standard deviations to obtain Gaussian images of different scales. Then subtract Gaussian images of adjacent scales to obtain Gaussian difference images, thus forming a Gaussian difference pyramid.

[0094] Extreme point detection: In the difference of Gaussian pyramid, each pixel is compared with its eight adjacent pixels at the upper and lower scales and its eight adjacent pixels at the same scale. If the pixel is a local extreme point, it is used as a candidate feature point.

[0095] Precise localization of extreme points: Quadratic function fitting is used to precisely locate candidate feature points, eliminating unstable extreme points with low contrast and edge response, and obtaining stable feature points.

[0096] Feature point orientation assignment: Calculate the gradient magnitude and direction of pixels in the neighborhood of each feature point, statistically analyze the gradient orientation histogram, and take the direction corresponding to the peak value in the histogram as the main orientation of the feature point to achieve rotation invariance of the feature point.

[0097] S6. Spatial Coordinate Transformation: Based on the spatial association dataset established in step S4, the pixel coordinates corresponding to each real-scene photo are converted into world coordinates using a spatial coordinate transformation algorithm. The spatial coordinate transformation algorithm uses perspective projection transformation, and the transformation formula for converting pixel coordinates (u, v) into world coordinates (X, Y, Z) is shown in the following formula:

[0098] ;

[0099] Where [RT] is the camera's extrinsic parameter matrix, R is the rotation matrix, and T is the translation vector; [u; v; 1] are the pixel homogeneous coordinates, [X; Y; Z; 1] are the world homogeneous coordinates, M is the camera's intrinsic parameter matrix, R is a 3×3 rotation matrix describing the rotation relationship between the camera coordinate system and the world coordinate system, and T is a 3×1 translation vector describing the translation relationship between the origin of the camera coordinate system and the origin of the world coordinate system.

[0100] In this invention, the world coordinate system is established based on the calibration coordinates of the ground base station, and the camera coordinate system is established with the optical center of the lens of the image acquisition module carried by the UAV as the origin. Based on the UAV coordinates (i.e., the world coordinates of the camera optical center) and the photo parameters in the spatial association dataset established in step S4, the extrinsic parameter matrix [RT] can be determined. Substituting the pixel coordinates into the above formula converts each pixel in the photo into its corresponding world coordinates.

[0101] S7. Photo Feature Matching and Geometric Constraints: Feature matching algorithms are used to match feature points in multiple pre-processed real-world photos. Combined with distance information collected by LiDAR, multi-view geometric constraints are applied to eliminate mismatched feature points.

[0102] Feature point matching is performed on multiple preprocessed real-world photos using the SIFT algorithm. First, a 128-dimensional feature descriptor is generated for each extracted feature point, obtained by statistically analyzing pixel gradient information within the feature point's neighborhood. Then, Euclidean distance is used as a similarity metric to calculate the Euclidean distance between feature descriptors in different photos. If the Euclidean distance between two feature descriptors is less than a preset threshold, the two feature points are considered a match.

[0103] To eliminate mismatched feature points, multi-view geometric constraints are applied using distance information acquired by high-precision LiDAR. Based on the UAV coordinates and LiDAR ranging data in the spatial association dataset, the camera pose and positional relationship at the time of taking different photos can be calculated, establishing an epipolar constraint model. The epipolar constraint model, based on the epipolar geometry principle of binocular vision, can limit the positional range of matching feature points in another photo. If a matching feature point does not satisfy the epipolar constraint, it is identified as a mismatched feature point and eliminated. Through epipolar constraints, the accuracy of feature point matching can be improved to over 95%.

[0104] S8. Initial 3D Point Cloud Model Construction: Based on the matched feature points and their corresponding world coordinates, an initial 3D point cloud model of the measured area is constructed using a dense point cloud generation algorithm. The dense point cloud generation algorithm employs a patch-based multi-view stereo matching algorithm, which generates dense 3D point cloud data by matching pixels one by one in the photograph and combining the distance information between the UAV and the real-world sampling points. The specific steps are as follows:

[0105] S81. Based on the matched feature points, the RANSAC algorithm is used to estimate the fundamental matrix, and then the essential matrix is ​​solved. The relative pose of the camera is obtained through the decomposition of the essential matrix.

[0106] S82. Using one of the photos as a reference photo, and based on the relative attitude of the camera and the distance information between the drone and the real scene, perform projection transformation on the other photos to generate a depth map corresponding to the reference photo; each pixel value in the depth map represents the distance from the real point corresponding to that pixel to the optical center of the camera.

[0107] S83. Perform fusion processing on the generated depth maps to eliminate redundancy and conflicts between different depth maps and obtain a unified depth map;

[0108] S84. Based on the pixel coordinates of the unified depth map and reference photos, and combined with the spatial coordinate transformation results, calculate the world coordinates of the real point corresponding to each pixel, and generate dense 3D point cloud data.

[0109] S85. The generated 3D point cloud data is stitched together according to the spatial location of the measured area to obtain the initial 3D point cloud model of the measured area.

[0110] S9. 3D Point Cloud Model Optimization and High-Precision 3D Reconstruction Model Generation: The initial 3D point cloud model undergoes optimization processing, including point cloud denoising, simplification, and smoothing, ultimately yielding a high-precision 3D reconstruction model. Point cloud denoising employs a statistical filtering algorithm, calculating the average distance between k-neighbors of each point and removing outliers whose distances exceed the product of the average distance and the standard deviation. Point cloud smoothing utilizes the moving least squares method, performing a weighted fitting of the neighborhood points of each point to obtain the smoothed point cloud coordinates.

[0111] Specifically, point cloud denoising involves using a statistical filtering algorithm to denoise the initial 3D point cloud. First, the average distance between points in the k-neighborhood of each point is calculated. The value of k is adjusted according to the point cloud density, typically between 20 and 50. Then, the standard deviation of the average distances of all points is calculated. Finally, a threshold is set (usually the average distance plus 2-3 times the standard deviation). If the average distance of a point's k-neighborhood exceeds this threshold, it is identified as a noise point and removed. Statistical filtering effectively eliminates isolated noise points while preserving the overall structure of the point cloud.

[0112] Point Cloud Simplification: To improve the efficiency of subsequent model processing, the denoised point cloud needs to be simplified. A voxel grid downsampling algorithm is used to divide the point cloud space into voxel grids of a certain size. Each voxel grid retains a representative point (usually the centroid of all points within the voxel grid), thereby reducing the amount of point cloud data while maintaining the overall shape and features of the point cloud. The size of the voxel grid is adjusted according to the point cloud density and reconstruction accuracy requirements. In this embodiment, the voxel grid size is set to 0.1 mm.

[0113] Point cloud smoothing: The simplified point cloud is smoothed using the moving least squares method to eliminate surface undulations and unevenness. The moving least squares method obtains a smooth surface by weighted fitting of the neighborhood points of each point. The new coordinates of each point are then calculated based on this surface, achieving point cloud smoothing. A Gaussian function is used as the weighting function, and the neighborhood radius is adjusted according to the point cloud density to ensure that enough points in the neighborhood of each point participate in the fitting process.

[0114] High-precision 3D reconstruction model generation: Surface reconstruction is performed on the optimized point cloud data, and a 3D surface model of the measured area is constructed using the Poisson reconstruction algorithm. The Poisson reconstruction algorithm can generate a continuous and smooth 3D surface model based on the normal vector information of the point cloud, while preserving the detailed features of the measured area. The generated 3D surface model is combined with texture mapping technology, mapping the texture of the preprocessed real-world photograph onto the 3D surface model to obtain a high-precision 3D reconstruction model with realistic texture.

[0115] Example 1

[0116] This embodiment uses a super high-rise office building as the object of measurement and employs the method of the present invention to perform high-precision three-dimensional reconstruction. The specific steps are as follows:

[0117] S1. Ground Base Station Deployment and Calibration: Three ground base stations will be evenly deployed around the high-rise office building, located to the east, south, and west of the building, at a distance of 80-150m. This ensures that the ground base stations can receive the laser positioning signals emitted by the drone, while avoiding signal obstruction from the high-rise buildings. The ground base stations are fixed with high-strength aluminum alloy brackets, and the bottom of the brackets is connected to the ground with expansion bolts to ensure the stability of the base stations.

[0118] The static differential positioning method was used to calibrate the ground base stations. Each base station received GPS and BeiDou satellite signals and continuously collected 30 minutes of satellite observation data. This data was then combined with local reference station coordinate data to perform differential calculations, thus achieving precise calibration of its own coordinates. The calibrated ground base station coordinate errors were as follows: Base Station 1 (x1, y1, z1) error ±0.08 mm, Base Station 2 (x2, y2, z2) error ±0.09 mm, and Base Station 3 (x3, y3, z3) error ±0.07 mm.

[0119] The signal receiving module, data processing module, and communication module of the ground base station were debugged. The test results showed that the response time of the ground base station to receive the laser positioning signal was 0.5 ms, the data processing time was 2 ms, and the delay in feeding back coordinate information to the UAV was 5 ms, which met the design requirements.

[0120] S2. UAV Equipment Deployment and Debugging: A quadcopter UAV platform with stable flight control capabilities is selected as the flight carrier. A full-frame industrial camera with a resolution of no less than 7900×5300 pixels is mounted on it as the image acquisition module; a 16-line LiDAR with a ranging accuracy better than ±0.5mm and a measurement range covering 0.1-500m is mounted as the LiDAR module; and a high-frequency laser emitter with a transmission frequency of 100 MHz is mounted as the laser positioning emission module.

[0121] A high-precision time synchronizer (model: TS-8000) was installed on the UAV and connected to the image acquisition module, lidar module and laser positioning and emission module respectively. Synchronization tests were conducted, and the test results showed that the synchronization error of the three modules was 0.8μs, which meets the design requirements.

[0122] Debug the drone's flight control system, the working status of each module, and the communication link to ensure that the drone can fly stably along the preset path, that each module can work normally, and that the communication link between the drone and the ground base station is unobstructed.

[0123] S3. Real-world Data Acquisition: The drone is controlled to fly over the super high-rise office building. Based on the building's height (149.8 m), the flight altitude is set to 180 m, the flight speed to 3 m / s, and the shooting frequency to 8 frames / second (increased shooting frequency due to the rich details of the building's facade). The data acquisition program is started. The image acquisition module continuously captures real-world photos of the office building, simultaneously recording parameters such as resolution (7952×5304), wide angle (24 mm), and shooting time for each photo. The lidar module simultaneously scans the office building, focusing on collecting distance information for key parts such as the building's facade, doors, windows, and curtain wall. The laser positioning transmission module simultaneously transmits laser positioning signals to three ground base stations.

[0124] After receiving the laser positioning signal, the ground base station calculates the distance between the UAV and each base station based on the propagation time of the laser signal. For example, at a certain sampling moment, the time it takes for the laser signal to travel from the UAV to base station 1 is t1 = 3.33 × 10⁻⁶. -7 The propagation time to base station 2 is t2 = 3.67 × 10. -7 The propagation time to base station 3 is t3 = 3.17 × 10⁻⁶. -7 s, the speed of laser propagation in air c = 3×10 8 m / s, according to formula (1), the distance between the UAV and base station 1 is d1 = 50 m, the distance between the UAV and base station 2 is d2 = 55 m, and the distance between the UAV and base station 3 is d3 = 47.5 m.

[0125] Based on the precise coordinates of the ground base station and the calculated distance, the coordinates of the UAV at the sampling time are obtained by solving the system of equations shown in formula (6) as (x, y, z) = (123567.891 m, 456789.123 m, 50.000 m), and the coordinate information is fed back to the UAV.

[0126] S4. Establishment of Spatial Association Dataset: After receiving coordinate information from the ground base station, the UAV associates the photo data, LiDAR ranging data, and UAV coordinate data according to the shooting timestamp to establish a spatial association dataset. Partial datasets are shown in the table below:

[0127] Table 1. Related data of the spatial association dataset

[0128]

[0129] S5. Real-scene photo preprocessing:

[0130] S51: Photo Denoising: A Gaussian filtering algorithm with σ = 1 is used to construct a 3×3 Gaussian convolution kernel, which is then used to perform convolution operations on the acquired photos to remove Gaussian noise from the photos.

[0131] S52: Distortion Correction: The camera is calibrated using Zhang Zhengyou's calibration method to obtain the camera intrinsic parameter matrix M.

[0132] M = {[5210.123, 0, 3980.567], [0, 5212.345, 2650.789], [0, 0, 1]};

[0133] Simultaneously, the radial distortion coefficients k1 = -0.0012, k1 = 0.0003, k1 = -0.0001, and the tangential distortion coefficients p1 = 0.0002, p2 = -0.0001 are obtained. According to the formula:

[0134] ;

[0135] ;

[0136] Distortion correction is performed on the photograph to eliminate radial and tangential distortion.

[0137] S53: Feature point extraction: The SIFT algorithm is used to extract feature points from the corrected photos. The number of feature points extracted from each photo is about 2,000-3,000, and the feature points have good stability and discriminability.

[0138] S6. Spatial Coordinate Transformation: Based on the camera intrinsic parameter matrix M and extrinsic parameter matrix [RT] (determined by the UAV coordinates and photo parameters), the pixel coordinates in the photo are converted to world coordinates using the corresponding formula. For example, the coordinates of a pixel in the photo IMG_0001.jpg are (u = 2000, v = 1500). Substituting these coordinates into the corresponding formula, its corresponding world coordinates are calculated to be (X = 123560.123 m, Y = 456782.345 m, Z = 10.567 m).

[0139] S7. Photo Feature Matching and Geometric Constraints: The SIFT algorithm is used to match feature descriptors from different photos, with an Euclidean distance threshold of 0.6, initially obtaining matched feature point pairs. Then, an epipolar constraint model is established based on distance information collected by LiDAR to eliminate mismatched feature points. For example, in the matching of IMG_0001.jpg and IMG_0002.jpg, 1200 pairs of feature points were initially matched. After epipolar constraints, 200 pairs of mismatched feature points were eliminated, ultimately resulting in 1000 correctly matched feature points.

[0140] S8. Initial 3D Point Cloud Model Construction: Based on the matched feature points, the RANSAC algorithm is used to estimate the fundamental matrix, solve for the essential matrix, and decompose it to obtain the camera's relative pose. Using IMG_0001.jpg as a reference photo, its corresponding depth map is generated. After fusion processing of the depth maps, the world coordinates corresponding to each pixel are calculated to generate dense 3D point cloud data. The point cloud data corresponding to all photos are stitched together to obtain the initial 3D point cloud model of the ancient building, with a point cloud density of 1000 points / mm. 2 .

[0141] S9. Optimization of 3D point cloud model and generation of high-precision 3D reconstruction model:

[0142] S91: Point cloud denoising: A statistical filtering algorithm is used, with k = 30. The average distance of each point's 30-neighborhood is calculated, and the threshold is set to the average distance + 2 times the standard deviation, eliminating approximately 5% of noisy points.

[0143] S92: Point Cloud Simplification: A voxel grid downsampling algorithm is used, with the voxel grid size set to 0.1 mm, to simplify the denoised point cloud, reducing the point cloud data volume by about 40% while preserving the detailed features of the ancient building.

[0144] S93: Point cloud smoothing: Using the moving least squares method, with a neighborhood radius of 0.5 mm, the simplified point cloud is smoothed to eliminate the undulations on the point cloud surface.

[0145] S94: High-precision 3D reconstruction model generation: The Poisson reconstruction algorithm is used to reconstruct the surface of the optimized point cloud to generate a 3D surface model of the ancient building. Then, the texture of the pre-processed real-scene photo is mapped onto the 3D surface model to obtain a high-precision 3D reconstruction model of the ancient building with real texture.

[0146] The accuracy of the generated 3D reconstruction model was tested. A total station was used to conduct on-site measurements of key feature points of the facade of the super high-rise office building (such as curtain wall splicing seams, door and window corners, etc.). The measurement results were compared with the coordinates of the corresponding feature points in the 3D reconstruction model. The results showed that the accuracy of the reconstruction model was ±1.2 mm, which meets the requirements of high-precision 3D models for the maintenance, renovation and digital management of the facade of the super high-rise office building.

[0147] This invention achieves millimeter-level positioning of UAVs through ground base stations and optimization algorithms, thereby obtaining a three-dimensional reconstruction model with millimeter-level accuracy. It has strong applicability and can be widely applied in multiple fields.

[0148] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0149] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection, an electrical connection, or a connection that allows communication between them; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0150] The present invention has been described above by way of example. Obviously, the specific implementation of the present invention is not limited to the above-described manner. Any improvements made by adopting the inventive concept and technical solution of the present invention, or direct application to other occasions without modification, are all within the protection scope of the present invention.

Claims

1. A method for high-precision spatial positioning and virtual-real mapping 3D reconstruction of unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: S1. Deployment and calibration of ground base stations: Deploy at least three fixed ground base stations around the area to be measured, complete the positioning calibration and signal debugging of the ground base stations, and enable the ground base stations to have the ability to receive, process and feed back coordinate information. S2. Deployment and debugging of UAV equipment: Control the UAV equipped with image acquisition module, lidar module and laser positioning and transmission module to fly to the airspace above the test area, complete the initialization and debugging of each module of the UAV, and ensure that the image acquisition module, lidar module and laser positioning and transmission module work synchronously; S3. Real-world data synchronous acquisition: Start the drone to synchronously acquire the following real-world data: The image acquisition module continuously captures real-scene photos of the measured area and simultaneously records the lens resolution and wide-angle parameters corresponding to each photo. The lidar module synchronously collects distance information between the drone and various sampling points in the real scene; The laser positioning transmitter module synchronously transmits laser positioning signals to the ground base station. After receiving the laser positioning signals, the ground base station calculates the real-time accurate coordinates of the UAV and feeds them back to the UAV. S4. Establishment of Spatial Association Dataset: The UAV receives real-time accurate coordinates fed back by the ground base station, and combines the distance information collected by the lidar module and the parameters recorded by the image acquisition module to establish a UAV-real scene spatial association dataset for each sampling time. The spatial association dataset shall at least include the sampling time, UAV coordinates, distance between the UAV and the real scene sampling point, real scene photos and photo parameters. S5. Real-scene photo preprocessing: Perform preprocessing on the real-scene photos acquired in step S3, including noise reduction, distortion correction and feature point extraction. S6. Spatial coordinate transformation: Based on the spatial association dataset established in step S4, the pixel coordinates corresponding to each real-scene photo are converted into world coordinates through a spatial coordinate transformation algorithm. S7. Photo Feature Matching and Geometric Constraints: Feature matching algorithms are used to match feature points in multiple pre-processed real-world photos. Combined with distance information collected by LiDAR, multi-view geometric constraints are applied to eliminate mismatched feature points. S8. Initial 3D point cloud model construction: Based on the matched feature points and their corresponding world coordinate information, an initial 3D point cloud model of the measured area is constructed using a dense point cloud generation algorithm. S9. Optimization of 3D Point Cloud Model and Generation of High-Precision 3D Reconstruction Model: The initial 3D point cloud model is optimized by including point cloud denoising, simplification and smoothing, and finally a high-precision 3D reconstruction model is obtained.

2. The method for high-precision spatial positioning and virtual-real mapping 3D reconstruction of an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, In step S1, the positioning calibration of the ground base station adopts the static differential positioning method. Specifically, the ground base station receives GNSS satellite signals, performs differential calculations in combination with preset reference station coordinate data, and completes the accurate calibration of its own coordinates. The coordinate error of the calibrated ground base station is controlled within ±0.1 mm.

3. The method for high-precision spatial positioning and virtual-real mapping 3D reconstruction of an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, In step S2, the synchronous operation of the image acquisition module, the lidar module, and the laser positioning and emission module is achieved through a high-precision time synchronizer, with the synchronization error controlled within ±1μs; and the ranging accuracy of the lidar module is ±0.5 mm, with a measurement range of 0.1-500 m.

4. The method for high-precision spatial positioning and virtual-real mapping 3D reconstruction of a UAV according to claim 1, characterized in that, In step S3, the propagation time of the laser positioning signal is t, the speed of laser propagation in air is c, and the distance d between the ground base station and the UAV is calculated using the following formula: ； Based on its own precise coordinates and the calculated distance d, the ground base station determines the real-time precise coordinates of the UAV using a triangulation algorithm. The coordinate calculation process of the triangulation algorithm is as follows: Let the coordinates of the ground base station be (x0, y0, z0) and the coordinates of the UAV be (x, y, z), then the following formula is satisfied: ； By deploying at least three ground base stations, a system of equations is established and solved to obtain the real-time accurate coordinates (x, y, z) of the UAV.

5. The method for high-precision spatial positioning and virtual-real mapping 3D reconstruction of an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, In step S5, the noise reduction in photo preprocessing uses a Gaussian filtering algorithm. The convolution kernel function of the Gaussian filter is shown in the following formula: ； Where σ is the standard deviation of the Gaussian filter, x y These are the relative coordinates of the pixels within the convolution kernel; Distortion correction is performed based on the camera intrinsic parameter matrix, M, as shown in the following formula: ； Among them, f x f y c represents the focal length of the camera in the x and y directions, respectively. x c y The coordinates of the camera's principal point; Feature point extraction uses the SIFT algorithm, which includes the following steps: Construct a Gaussian difference pyramid to detect extreme points; Precisely locate extreme points and eliminate unstable extreme points; Assign a direction vector to each extreme point to generate a feature descriptor with rotation invariance; Euclidean distance was used as a similarity metric to match feature descriptors of different photos.

6. The method for high-precision spatial positioning and virtual-real mapping 3D reconstruction of an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, In step S6, the spatial coordinate transformation algorithm uses perspective projection transformation. The transformation formula for converting pixel coordinates (u, v) to world coordinates (X, Y, Z) is shown below: ； Where [RT] is the extrinsic parameter matrix of the camera, R is the rotation matrix, and T is the translation vector.

7. The method for high-precision spatial positioning and virtual-real mapping 3D reconstruction of an unmanned aerial vehicle (UAV) according to claim 5, characterized in that, In step S7, feature matching is performed using the feature descriptors of the SIFT algorithm, and epipolar constraints are applied based on the distance information collected by the lidar module to eliminate mismatched feature points.

8. The method for high-precision spatial positioning and virtual-real mapping 3D reconstruction of a UAV according to claim 1, characterized in that, In step S8, the dense point cloud generation algorithm adopts a multi-view stereo matching algorithm based on patches. By matching pixels in the photo point by point and combining the distance information between the UAV and the real-world sampling points, dense three-dimensional point cloud data is generated.

9. The method for high-precision spatial positioning and virtual-real mapping 3D reconstruction of an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, In step S9, point cloud denoising uses a statistical filtering algorithm to calculate the average distance of points in the k-neighborhood of each point and remove outliers whose distance is greater than the product of the average distance and the standard deviation. Point cloud simplification uses a voxel grid downsampling algorithm to divide the point cloud space into regular voxel grids and retain a representative point in each grid to achieve uniform simplification of point cloud data. Point cloud smoothing uses the moving least squares method to perform weighted fitting on the points in the neighborhood of each point to obtain the smoothed point cloud coordinates.

Images