Scene reconstruction and automatic correction methods using drones

Abstract

This invention provides a method for scene reconstruction and automatic correction using unmanned aerial vehicles (UAVs), belonging to the field of scene reconstruction and correction technology. It involves comprehensive data acquisition across different sub-regions, synchronously recording flight trajectory information for each round, and aligning sensor data with timestamps. A dynamic error compensation model is constructed, combining the dynamic parameter set of each round of random flight by the UAV to correct motion blur, environmental interference, and trajectory deviation errors in the acquired data. The corrected multi-round data is integrated to establish a spatial mapping between sub-region textures and point clouds, generating an initial 3D model. A closed-loop correction mechanism is constructed based on the UAV's GNSS absolute coordinates and high-precision ground control points to calculate the deviation of the initial 3D model. If the deviation exceeds a threshold, the random flight coverage density, flight rounds, and error compensation coefficients are adjusted in reverse until the deviation meets the standard. This improves the accuracy of scene reconstruction and correction.

Description

Technical Field

[0001] This invention relates to the field of scene reconstruction and correction technology, and in particular to a method for scene reconstruction and automatic correction using unmanned aerial vehicles (UAVs). Background Technology

[0002] With the development of UAV and sensor technologies, UAV scene reconstruction has been widely applied in various fields due to its high flexibility and wide coverage. However, existing technologies still have many shortcomings in practical applications: Traditional UAV scene reconstruction relies heavily on preset flight paths, which, while ensuring the regularity of data collection, is prone to omissions in sub-region coverage when facing complex terrains (such as mountains and building clusters), and is difficult to flexibly adapt to scene changes; during UAV data collection, changes in flight attitude (such as pitch and roll), environmental interference (such as changes in lighting and reduced atmospheric visibility), and trajectory deviations can easily lead to motion blur and uneven lighting in binocular vision images, and coordinate misalignment in lidar point clouds. Existing error compensation methods are mostly designed for single error sources and do not consider the coupling effect of multiple error sources, resulting in limited correction accuracy. After the initial 3D model is generated, existing technologies rely heavily on manual verification of deviations and have not established an automated closed-loop correction mechanism based on absolute coordinates (such as GNSS) and ground control points, making it difficult to adjust flight parameters and error compensation coefficients in real time, resulting in unstable model accuracy. All these factors contribute to reduced reconstruction accuracy and efficiency.

[0003] Therefore, this invention proposes a method for scene reconstruction and automatic correction using unmanned aerial vehicles (UAVs). Summary of the Invention

[0004] This invention provides a method for scene reconstruction and automatic correction using unmanned aerial vehicles (UAVs) to solve the aforementioned technical problems.

[0005] This invention provides a method for scene reconstruction and automatic correction using unmanned aerial vehicles (UAVs), comprising:

[0006] Step 1: Based on the multi-sensor system consisting of binocular vision sensor and lidar on the drone, the drone performs multiple omnidirectional random flights to cover different sub-regions of the target scene to collect full-domain data, synchronously record the flight trajectory information of each round, and perform timestamp alignment processing on the sensor data.

[0007] Step 2: Construct a dynamic error compensation model, and combine it with the dynamic parameter set of each round of random flight of the UAV to correct motion blur, environmental interference and trajectory deviation errors in the collected data;

[0008] Step 3: Integrate the corrected multi-round data, establish a spatial mapping between sub-region textures and point clouds, and generate an initial 3D model;

[0009] Step 4: Based on the GNSS absolute coordinates of the UAV and the high-precision ground control points, a closed-loop correction mechanism is constructed to calculate the deviation of the initial three-dimensional model. If the deviation exceeds the threshold, the random flight coverage density, flight rounds and error compensation coefficient are adjusted in reverse until the deviation meets the standard.

[0010] Preferably, step 2 includes:

[0011] Obtain the dynamic parameter set for each round of random flight of the UAV, wherein the dynamic parameter set includes: three-dimensional attitude angles, time rate of change, and instantaneous altitude. Flight trajectory randomness coefficient and environmental parameters;

[0012] The ratio of the standard deviation of the distance from the center of the sub-region to each sampling point in a single-wheel flight trajectory to the characteristic length of the sub-region is considered as the randomness coefficient of the flight trajectory. Extract the unit direction vector of trajectory deviation ;

[0013] Constructing an error compensation function based on motion blur ,in, These are pitch angle, roll angle, and yaw angle, respectively. These are the time rates of change of pitch angle and roll angle, respectively, in rad / s. Standard flight altitude; These are the height attenuation factor, visibility impact index, and trajectory randomness coupling index, respectively. Atmospheric visibility; Baseline visibility; These are the attitude-fuzzy transformation coefficients, in units of: ;

[0014] Using binocular vision images Driven adaptive Gaussian kernel deblurring, kernel size varies Increases and expands linearly;

[0015] Constructing an error compensation function based on environmental disturbance Where L is the light intensity; Standard light intensity; Light sensitivity coefficient; It is the coupling index for trajectory randomness; As the reference lighting;

[0016] Perform normalization correction on image pixel values: ,in, These are the pixel grayscale values ​​of the original image; These are the pixel grayscale values ​​after illumination correction; For all The maximum value.

[0017] Preferably, step 2 further includes:

[0018] Construct the spatial coordinate correction matrix:

[0019] ;

[0020] ;

[0021] ;

[0022] in, for The amount; This is the trajectory deviation coefficient; These are the translation vectors in the x, y, and z directions, respectively; This represents the average laser scanning distance between the drone and the target scene sub-region;

[0023] based on The rotation matrix R is calculated, and then processed to obtain the final matrix. ; It is a translation matrix;

[0024] Perform on lidar point cloud data The coordinate transformation corrects the point cloud misalignment caused by trajectory offset, where P is the original lidar point cloud data; This is the corrected lidar point cloud data;

[0025] Based on closed-loop feedback from multiple rounds of flight data, the objective is to minimize the average deviation between the corrected data and the ground control points, and to iteratively optimize... Continue until the average deviation is ≤3cm.

[0026] Preferred options also include:

[0027] Retrieve several historical flight trajectories of the UAV flying along a preset route and at a preset speed, and determine the trajectory position and historical speed of each historical flight trajectory based on the same historical flight time point, thereby obtaining the trajectory distance deviation set of each historical flight trajectory at the corresponding historical flight time point. and flight speed deviation set ,in, Let be the trajectory distance deviation vector for the corresponding historical flight time point under the j-th historical flight trajectory. The modulus length; Let $j$ be the flight speed deviation vector at the corresponding historical flight time point under the $j$-th historical flight trajectory. The modulus length; The total number of historical flight paths;

[0028] The trajectory distance deviation set and the flight speed deviation set are respectively fitted to obtain the distance fitting deviation. and velocity fitting deviation And based on the first minimum value z1 in the trajectory distance deviation set and the second minimum value z2 in the flight speed deviation set, the trajectory deviation coefficient k07 is obtained:

[0029] ,in, It is the exponential coefficient.

[0030] Preferably, step 3 includes:

[0031] The data integrates multi-sensor data from multiple UAVs after dynamic error compensation correction, including texture images of each sub-region collected by binocular vision sensors and point cloud data of the corresponding sub-regions collected by lidar. Each sub-region data is associated with the UAV's GNSS absolute coordinates and flight trajectory information.

[0032] A multimodal feature collaborative registration algorithm is adopted to extract visual feature points of texture images and geometric features of point clouds. Constraints are established by combining the GNSS absolute coordinates and flight trajectory information to map texture pixels to point cloud spatial coordinates one by one. The mapping error is verified based on ground high-precision control points.

[0033] Based on the texture-point cloud spatial mapping relationship of all sub-regions, an initial 3D model is generated using a layered progressive fusion strategy.

[0034] Preferably, an initial 3D model is generated using a layered progressive fusion strategy, including:

[0035] Triangulation is performed on the point cloud of each sub-region, and the texture image is projected onto the triangular mesh based on the spatial mapping relationship to form a local 3D model of the sub-region.

[0036] Simultaneously extract the normal and curvature features of the point cloud and the SIFT features of the texture image from the local 3D model of adjacent sub-regions;

[0037] Based on the cross-modal feature fusion network, geometric and visual features are mapped to the same high-dimensional space, feature similarity is calculated and matching constraints are established;

[0038] Based on GNSS absolute coordinates and flight trajectory information, rigid body transformation registration is performed on point clouds of adjacent sub-regions to initially eliminate large-scale positional deviations.

[0039] Pixel-level deformation correction is performed on the texture image based on the matching constraints, while Poisson fusion is performed on the overlapping point cloud to seamlessly connect the topology of the texture and the point cloud.

[0040] The texture consistency of the model after merging all sub-regions is adjusted and the point cloud density is equalized to generate the initial 3D model.

[0041] Preferably, step 4 includes:

[0042] The absolute coordinates of each sub-region during the initial 3D model construction process are obtained by relying on the GNSS module carried by the UAV, and at the same time, no less than 3 high-precision ground control points are set up in the target scene.

[0043] The model coordinates corresponding to the high-precision control points on the ground in the initial 3D model are compared with the actual coordinates of the high-precision control points on the ground to calculate the global deviation value of the model. At the same time, the local deviation of the model at the mapping of texture and point cloud space in each sub-region is calculated point by point to obtain the local deviation distribution.

[0044] If the global deviation value is greater than the first preset threshold or there is a sub-region where the local deviation is greater than the second preset threshold, the coverage density of the UAV's random flight, the randomness coefficient of the flight trajectory, and the compensation coefficient of the dynamic error compensation model are adjusted in reverse. The data acquisition, error compensation, initial 3D model construction, and deviation calculation steps are iteratively executed until the global deviation value is not greater than the first preset threshold and all local deviations are not greater than the second preset threshold, thus completing the closed-loop correction of the initial 3D model.

[0045] Preferably, the model after fusing all sub-regions is subjected to texture consistency adjustment and point cloud density equalization to generate an initial 3D model, including:

[0046] The texture consistency energy term is obtained by calculating the sum of squared grayscale differences and the sum of squared color space distances at the texture stitching points of adjacent sub-regions. ,in, The pixel grayscale values ​​of the k1th and k2th adjacent points in the connecting area; Let k1 and k2 be the color vectors of the adjacent points in the connecting region. This refers to the color weighting coefficient;

[0047] Average density of point cloud in target scene Based on this, the point cloud density of each sub-region is calculated and... Sum of squared deviations Where M2 is the total number of sub-regions; Let be the point cloud density of the o-th sub-region;

[0048] Based on high-precision ground control points, calculate the sum of squared Euclidean distances between the control point coordinates and the actual coordinates. Where M3 is the number of control points; Let u1 be the coordinates of the control point. The actual coordinates of the u1th control point;

[0049] based on as well as Generate an initial 3D model.

[0050] Compared with the prior art, the beneficial effects of this application are as follows:

[0051] By taking into account both the flexibility and completeness of data collection through multiple omnidirectional random flights, ensuring the spatiotemporal consistency of multi-source data through timestamp alignment, and combining dynamic error compensation and closed-loop correction mechanisms, the target scene can be reconstructed efficiently and with high precision.

[0052] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0053] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0054] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0055] Figure 1 This is a flowchart of a scene reconstruction and automatic correction method using a drone in an embodiment of the present invention. Detailed Implementation

[0056] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0057] This invention provides a method for scene reconstruction and automatic correction using unmanned aerial vehicles (UAVs), such as... Figure 1 As shown, it includes:

[0058] Step 1: Based on the multi-sensor system consisting of binocular vision sensor and lidar on the drone, the drone performs multiple omnidirectional random flights to cover different sub-regions of the target scene to collect full-domain data, synchronously record the flight trajectory information of each round, and perform timestamp alignment processing on the sensor data.

[0059] Step 2: Construct a dynamic error compensation model, and combine it with the dynamic parameter set of each round of random flight of the UAV to correct motion blur, environmental interference and trajectory deviation errors in the collected data;

[0060] Step 3: Integrate the corrected multi-round data, establish a spatial mapping between sub-region textures and point clouds, and generate an initial 3D model;

[0061] Step 4: Based on the GNSS absolute coordinates of the UAV and the high-precision ground control points, a closed-loop correction mechanism is constructed to calculate the deviation of the initial three-dimensional model. If the deviation exceeds the threshold, the random flight coverage density, flight rounds and error compensation coefficient are adjusted in reverse until the deviation meets the standard.

[0062] In this embodiment, the binocular vision sensor is a ZED2i stereo camera used to acquire scene texture images; the lidar is a Velodyne VLP-16 used to acquire 3D point cloud data of the scene, and both are fixed by a drone mounting bracket.

[0063] In this embodiment, "multiple" refers to ≥3 flight rounds, and "all-round" refers to different directions of the scene covered by each flight round, such as the first round covering the eastern half of the area, the second round covering the western half of the area, and the third round covering the edge sub-area. "Random flight" means that the flight path does not need to be preset and is dynamically adjusted by the UAV flight control system according to the real-time sub-area coverage. The overlap of the flight trajectory in each round is controlled at 20%-30% to avoid data redundancy or omission.

[0064] In this embodiment, full-domain data acquisition involves dividing the target scene into several sub-regions, with each flight covering 4-5 sub-regions, ensuring that all sub-regions are covered by at least 2 flights, thus achieving full-domain data acquisition without blind spots.

[0065] In this embodiment, the flight trajectory information is recorded by the GNSS module on the UAV, which records the trajectory coordinates of each flight. The sampling frequency is 10Hz, which means 10 trajectory points are recorded per second. The trajectory information is stored in CSV format, which includes timestamp, longitude, latitude, and altitude. At the same time, the flight control system records the UAV's flight speed, attitude angle, and other information, which are stored in association with the trajectory information.

[0066] In this embodiment, a hardware synchronizer such as the TrimbleBD990 synchronization module is used to connect the binocular camera, lidar, and GNSS module to add a unified timestamp to all sensor data. After data acquisition, the timestamp of each sensor data is read through a Python script, and data with a timestamp deviation of ≤2ms are determined to be data from the same time, thereby achieving timestamp alignment and avoiding modeling deviations caused by data asynchrony.

[0067] In this embodiment, the dynamic error compensation model refers to a multi-input multi-output compensation model that integrates flight attitude, altitude, trajectory randomness, and environmental parameters to correct three types of errors: motion blur (for binocular images), environmental interference (for binocular images), and trajectory deviation (for laser point clouds).

[0068] In this embodiment, the dynamic parameter set includes three-dimensional attitude angles, attitude angle time change rate, instantaneous altitude, flight trajectory randomness coefficient, and environmental parameters.

[0069] The initial 3D model refers to the scene 3D model generated by integrating all sub-region corrected data, establishing the spatial mapping relationship between texture and point cloud, and using 3D reconstruction algorithms (such as Poisson reconstruction and Delaunay triangulation). The model includes point cloud, mesh, and image pixels.

[0070] In this embodiment, the UAV GNSS absolute coordinates refer to the absolute coordinates in the WGS84 coordinate system obtained by the GNSS module when the UAV collects data, including: longitude range, latitude range, and altitude range, which are used to provide a spatial reference for sub-regional data.

[0071] Ground high-precision control points refer to marker points with known precise coordinates set up in the target scene, using stainless steel forced-centering observation piers. For example, a pier with a height of 30cm and a top diameter of 10cm is used. Its coordinates are calibrated using a total station with an accuracy ≤5mm. The number of piers should be ≥3. For example, 4 piers are set up in a rectangular distribution for a 5000 square meter scene, spaced 50-100m apart. It should be noted that the number of piers = max(3, ceil(scene area / 1000)), where ceil is a rounding function, and the scene area is in square meters.

[0072] The closed-loop correction mechanism refers to a loop mechanism that calculates model deviation, determines whether the deviation exceeds the threshold, adjusts parameters in reverse, and re-acquires and models the data. This mechanism enables automated optimization of model accuracy without manual intervention.

[0073] Random flight coverage density refers to the length of the drone's flight path per unit area. The initial coverage density is set to a certain value. If the model bias exceeds a certain threshold... It can be upgraded to This means increasing the length of the flight trajectory per unit area and increasing the frequency of data collection.

[0074] Flight rounds refer to the number of times a drone completes a full-area coverage flight. The initial flight rounds are set to 5 rounds. If the deviation of a local sub-region exceeds the threshold, 2 more flight rounds can be added to specifically cover the sub-region with the excessive deviation.

[0075] Error compensation coefficient refers to the adjustable parameter in the dynamic error compensation model. The initial coefficient can be set by empirical values. If the model deviation exceeds the threshold, the coefficient value is optimized iteratively by the least squares method.

[0076] The beneficial effects of the above technical solution are: by taking into account both the flexibility and integrity of data collection through multiple omnidirectional random flights, by ensuring the spatiotemporal consistency of multi-source data through timestamp alignment, and by combining dynamic error compensation and closed-loop correction mechanisms, the target scene can be reconstructed efficiently and with high precision.

[0077] This invention provides a method for scene reconstruction and automatic correction using unmanned aerial vehicles (UAVs), step 2 of which includes:

[0078] Obtain the dynamic parameter set for each round of random flight of the UAV, wherein the dynamic parameter set includes: three-dimensional attitude angles, time rate of change, and instantaneous altitude. Flight trajectory randomness coefficient and environmental parameters;

[0079] The ratio of the standard deviation of the distance from the center of the sub-region to each sampling point in a single-wheel flight trajectory to the characteristic length of the sub-region is considered as the randomness coefficient of the flight trajectory. Extract the unit direction vector of trajectory deviation ;

[0080] Constructing an error compensation function based on motion blur ,in, These are pitch angle, roll angle, and yaw angle, respectively. These are the time rates of change of pitch angle and roll angle, respectively, in rad / s. Standard flight altitude; These are the height attenuation factor, visibility impact index, and trajectory randomness coupling index, respectively. Atmospheric visibility; Baseline visibility; These are the attitude-fuzzy transformation coefficients, in units of: ;

[0081] Using binocular vision images Driven adaptive Gaussian kernel deblurring, kernel size varies Increases and expands linearly;

[0082] Constructing an error compensation function based on environmental disturbance Where L is the light intensity; Standard light intensity; Light sensitivity coefficient; It is the coupling index for trajectory randomness; As the reference lighting;

[0083] Perform normalization correction on image pixel values: ,in, These are the pixel grayscale values ​​of the original image; These are the pixel grayscale values ​​after illumination correction; For all The maximum value.

[0084] In this embodiment, pitch angle refers to the angle between the drone's longitudinal axis and the horizontal plane (range -90° to 90°, with pitch being positive), roll angle refers to the angle between the drone's transverse axis and the horizontal plane (range -180° to 180°, with rightward tilt being positive), and yaw angle refers to the angle between the drone's longitudinal axis and due north (range 0° to 360°, with clockwise being positive).

[0085] The attitude angle time change rate refers to the pitch angle time change rate and the roll angle time change rate. Among them, the yaw angle change rate has a relatively small impact on motion ambiguity and is not included for the time being. The unit is rad / s.

[0086] Instantaneous altitude h refers to the real-time altitude of the drone relative to the ground, measured in meters (m).

[0087] The flight trajectory randomness coefficient is a dimensionless parameter used to quantify the degree to which the flight trajectory deviates from the center of the sub-region. The sub-region characteristic length refers to the longest side of the sub-region. For example, if the rectangular sub-region is 50m × 5m, the characteristic length is 50m. The distance standard deviation is calculated by statistically analyzing the distances from the trajectory sampling points to the center of the sub-region. For example, if the standard deviation of the distance data from 100 sampling points is 10m, then the flight trajectory randomness coefficient = 10 / 50 = 0.2.

[0088] The trajectory deviation unit direction vector is a three-dimensional unit vector based on the x, y, and z directions. It is dimensionless and used to characterize the direction of trajectory deviation. It is calculated from GNSS trajectory data: the vector between the average position of a single-wheel flight trajectory and the center of the sub-region is taken (e.g., average position (100.5m, 50.3m, 50.2m), sub-region center (100.0m, 50.0m, 50.0m), vector is (0.5, 0.3, 0.2)). This vector is normalized by dividing it by the vector magnitude. For example, if the magnitude is ≈ 0.6245, the normalized value is (0.801, 0.480, 0.320). The direction vector deviation is ≤ 0.05.

[0089] Environmental parameters include atmospheric visibility V (in meters) and light intensity L (in lux).

[0090] The motion blur error compensation function K1 is a dimensionless coefficient used to quantify the degree of motion blur and guide the deblurring process.

[0091] The value of k01 ranges from 0.015 to 0.03, and is generally taken as 0.025.

[0092] h0 is the normalization coefficient for instantaneous altitude, in meters (m). It is taken as the standard flight altitude of the UAV (e.g., 100m) and is used to normalize the instantaneous altitude h.

[0093] k02 is the height attenuation factor, which is dimensionless and ranges from 0.5 to 0.7. For example, if it is 0.6, the higher the height, the smaller the impact of motion blur on the image. Therefore, the exponent of h / h0 is negative. The larger k02 is, the more obvious the height attenuation effect.

[0094] V0 is the baseline visibility, measured in meters (m). It is taken as the standard atmospheric visibility (e.g., 1000m) and is used to normalize the atmospheric visibility V.

[0095] k03 is the visibility impact index, which is dimensionless and ranges from 0.3 to 0.5. If it is 0.4, the lower the visibility, the more obvious the motion blur. Therefore, the exponent of V / V0 is negative. The larger the k03, the more significant the impact of visibility on blur.

[0096] k04 is the trajectory randomness coupling index, which is dimensionless and ranges from 0.1 to 0.3. If it is 0.2, the higher the trajectory randomness, the more drastic the attitude change and the more obvious the motion ambiguity. Therefore, the exponent of the flight trajectory randomness coefficient is positive. The larger k04 is, the stronger the coupling effect of trajectory randomness on ambiguity.

[0097] Adaptive Gaussian kernel deblurring is a deblurring method for binocular vision images. The Gaussian kernel size expands linearly with the increase of K1: the initial kernel size is set to 3×3 (when K1=0.1), and the kernel size expansion formula is kernel size=3+2×floor(K1 / 0.1); the deblurring process is implemented by the GaussianBlur function of the OpenCV library, and the kernel standard deviation is adjusted synchronously with the kernel size (e.g., the standard deviation of a 19×19 kernel is 3.8).

[0098] The environmental interference error compensation function K2 is a dimensionless coefficient used to quantify the degree of interference of illumination changes on the image.

[0099] k05 is the light sensitivity coefficient, which is dimensionless and ranges from 0.0002 to 0.0005. For example, if it is 0.0003, the larger the light intensity deviation, the larger K2 is and the stronger the compensation.

[0100] This is the standard illumination intensity, measured in lux. The value represents the average illumination intensity of the scene, such as 5000 lux. On a cloudy day, it can be adjusted to 3000 lux.

[0101] L0 is the reference illumination, measured in lux, and its value is related to... Consistency, such as 5000 lux, is used to account for illumination deviations. Normalization;

[0102] k06 is the trajectory randomness coupling index, dimensionless, ranging from 0.1 to 0.3. A value of 0.2 indicates higher trajectory randomness, more frequent changes in UAV attitude, and more pronounced changes in illumination angle. The exponent is positive, and the larger k06 is, the stronger the coupling effect of trajectory randomness on illumination interference.

[0103] It is the attenuation term of height to light interference, which is dimensionless. The higher h is, the more gradual the change in light intensity, the smaller the attenuation term, and the weaker the compensation.

[0104] In this embodiment, The value range is 0-255, such as I=128 for a certain pixel. It is the reciprocal of the light compensation factor, used to counteract the effects of excessively strong or weak light.

[0105] In this embodiment, the parameter values ​​for different scenario types can be found in Table 1:

[0106] Table 1. Parameter values ​​for different scenario types

[0107]

[0108] The beneficial effects of the above technical solution are: defining a trajectory randomness coefficient to quantify the trajectory characteristics of random flight, constructing compensation functions K1 and K2 that couple attitude, altitude, visibility, illumination and trajectory randomness to achieve accurate correction of motion blur and environmental interference; at the same time, pixel normalization correction ensures that the image grayscale value is within a reasonable range, solving the problem that existing single error compensation methods ignore the influence of multi-parameter coupling, significantly improving the quality of binocular vision data, and laying the foundation for subsequent texture-point cloud mapping.

[0109] This invention provides a method for scene reconstruction and automatic correction using unmanned aerial vehicles (UAVs), step 2 of which further includes:

[0110] Construct the spatial coordinate correction matrix:

[0111] ;

[0112] ;

[0113] ;

[0114] in, for The amount; This is the trajectory deviation coefficient; These are the translation vectors in the x, y, and z directions, respectively; This represents the average laser scanning distance between the drone and the target scene sub-region;

[0115] based on The rotation matrix R is calculated, and then processed to obtain the final matrix. ; It is a translation matrix;

[0116] Perform on lidar point cloud data The coordinate transformation corrects the point cloud misalignment caused by trajectory offset, where P is the original lidar point cloud data; This is the corrected lidar point cloud data;

[0117] Based on closed-loop feedback from multiple rounds of flight data, the objective is to minimize the average deviation between the corrected data and the ground control points, and to iteratively optimize... Continue until the average deviation is ≤3cm.

[0118] In this embodiment, the spatial coordinate correction matrix includes a translation matrix. The rotation matrix R is used to correct the original coordinates of the lidar point cloud and eliminate point cloud misalignment caused by trajectory deviation.

[0119] Translation vector These are the coordinate corrections in the x, y, and z directions, respectively, in meters (m).

[0120] It is the unit direction vector of trajectory deviation. The quantity is dimensionless, such as =(0.801,0.480,0.320);

[0121] The rotation matrix R is a 3×3 matrix used to correct point cloud rotation deviations caused by changes in the UAV's attitude angles, based on... Calculated in ZYX order (yaw-pitch-roll), the matrix elements are trigonometric functions of attitude angles, for example:

[0122] .

[0123] In this embodiment, the translation matrix This is used to superimpose translation vectors onto the original coordinates of the point cloud, thereby achieving translation correction of the point cloud.

[0124] In this embodiment, First, perform rotation correction (multiply by R) on the original point cloud, then perform translation correction (multiply by T). Matrix multiplication follows the right-multiplication rule (i.e., when multiplying point cloud coordinates P and M, the order is...). For example, the homogeneous coordinates (x,y,z,1) of the original lidar point cloud, in units of m (e.g., a point cloud P=(100.2,50.3,20.1,1)), and the corrected homogeneous coordinates (x',y',z',1), are obtained by matrix multiplication, in units of m.

[0125] In this embodiment, the closed-loop feedback of multi-round flight data refers to iteratively optimizing the compensation coefficients k01-k07 by using the deviation between the corrected data from multiple rounds of flight (e.g., 5 rounds) and the ground control points: after each round of flight, the average deviation between the corrected data (binocular image mapping deviation, point cloud coordinate deviation) and the ground control points is calculated. If the deviation is >3cm, the coefficient values ​​are adjusted by the least squares method (e.g., k07 is adjusted from 0.01 to 0.011) until the average deviation is ≤3cm.

[0126] The compensation coefficients k01 to k07 are the core parameters of the dynamic error compensation model. Initial values ​​are set empirically, such as k01=0.025, k02=0.6, k03=0.4, k04=0.2, k05=0.0003, k06=0.2, and k07=0.01. The gradient descent algorithm is used during optimization. The objective function is to minimize the average deviation between the corrected data and the ground control points. The number of iterations is ≤20, and the convergence accuracy is [not specified]. .

[0127] The beneficial effects of the above technical solution are: by constructing a translation vector formula that couples attitude angle, altitude, trajectory randomness, and direction vector, and combining it with Euler angle rotation matrix, the trajectory deviation of LiDAR point cloud can be accurately corrected; at the same time, by optimizing the compensation coefficient through multi-round closed-loop feedback, the spatial accuracy of point cloud data is significantly improved, providing an accurate geometric basis for texture-point cloud spatial mapping.

[0128] This invention provides a method for scene reconstruction and automatic correction using unmanned aerial vehicles (UAVs), and further includes:

[0129] Retrieve several historical flight trajectories of the UAV flying along a preset route and at a preset speed, and determine the trajectory position and historical speed of each historical flight trajectory based on the same historical flight time point, thereby obtaining the trajectory distance deviation set of each historical flight trajectory at the corresponding historical flight time point. and flight speed deviation set ,in, Let be the trajectory distance deviation vector for the corresponding historical flight time point under the j-th historical flight trajectory. The modulus length; Let $j$ be the flight speed deviation vector at the corresponding historical flight time point under the $j$-th historical flight trajectory. The modulus length; The total number of historical flight paths;

[0130] The trajectory distance deviation set and the flight speed deviation set are respectively fitted to obtain the distance fitting deviation. and velocity fitting deviation And based on the first minimum value z1 in the trajectory distance deviation set and the second minimum value z2 in the flight speed deviation set, the trajectory deviation coefficient k07 is obtained:

[0131] ,in, It is the exponential coefficient.

[0132] In this embodiment, the preset route refers to the fixed flight path (such as a rectangular route or a grid route) that the UAV has completed in the target scene, which is used to obtain historical deviation data; the preset speed refers to a fixed flight speed (such as 5m / s, with fluctuations ≤0.5m / s); and "several" refers to the number of historical trajectories, which is ≥10.

[0133] The same historical flight time point refers to the same moment in all historical trajectories that corresponds to the current flight time point (e.g., data is collected at the 30th second of the current flight, and the trajectory position and velocity at the 30th second are extracted from all historical trajectories), ensuring the time consistency between historical data and current data.

[0134] The trajectory position and historical velocity refer to the absolute GNSS coordinates (longitude, latitude, and altitude) of the historical trajectory at the corresponding time point, in meters (after conversion to planar coordinates); the historical velocity refers to the flight velocity (x, y, and z components) of the UAV at the corresponding time point, in meters per second (m / s), obtained through speed data recorded by the flight control system.

[0135] In this embodiment, The calculation method for the trajectory distance deviation vector of the j-th historical trajectory at the corresponding time point is as follows: the position difference between the actual trajectory position and the preset route position at the time point. For example, if the preset route position is (100.0m, 50.0m, 50.0m) and the actual position is (100.5m, 50.3m, 50.2m), then... =(0.5,0.3,0.2). It should be noted that the calculation principle of the flight speed deviation vector and the trajectory distance deviation vector is similar, which will not be repeated here.

[0136] In this embodiment, the fitting process refers to performing curve fitting on the deviation set to obtain the variation law of the deviation with time or trajectory, and using the least squares method for fitting (such as fitting the trajectory distance deviation set as a linear function or a quadratic function), such as the fitting function being Ln=0.02t+0.01 (t is the flight time, and Ln=0.61m when t=30s).

[0137] Velocity fitting deviation sn: The output value of the fitting function at the current time point (in m / s), such as the fitting function sn=0.01t+0.005 (sn=0.305m / s when t=30s).

[0138] In this embodiment, the first minimum value z1 of the trajectory distance deviation set and the second minimum value z2 of the flight speed deviation set are used together as the historical best deviation benchmark to quantify the gap between the current fitting deviation and the historical best.

[0139] In this embodiment, It is the exponential coefficient of the distance deviation, dimensionless, with a value range of 1.0-1.5. For example, a value of 1.2... The larger the value, the more significant the effect of the distance deviation on k07; It is the exponential coefficient of the speed deviation, dimensionless, with a value range of 1.0-1.5. For example, if we take 1.1, its... The larger the value, the more significant the effect of the speed deviation on k07.

[0140] The beneficial effects of the above technical solution are as follows: by retrieving deviation data from historical preset flight routes, a deviation set is constructed and a pattern is fitted. The deviation degree of the current deviation is quantified based on the historical minimum deviation, and the trajectory deviation coefficient k07 is finally determined. This method combines historical flight experience with the current flight status, solves the problem that the existing static value of k07 is difficult to adapt to different flight scenarios, makes the determination of k07 more targeted and accurate, and improves the accuracy of trajectory deviation correction.

[0141] This invention provides a method for scene reconstruction and automatic correction using unmanned aerial vehicles (UAVs), step 3 of which includes:

[0142] The data integrates multi-sensor data from multiple UAVs after dynamic error compensation correction, including texture images of each sub-region collected by binocular vision sensors and point cloud data of the corresponding sub-regions collected by lidar. Each sub-region data is associated with the UAV's GNSS absolute coordinates and flight trajectory information.

[0143] A multimodal feature collaborative registration algorithm is adopted to extract visual feature points of texture images and geometric features of point clouds. Constraints are established by combining the GNSS absolute coordinates and flight trajectory information to map texture pixels to point cloud spatial coordinates one by one. The mapping error is verified based on ground high-precision control points.

[0144] Based on the texture-point cloud spatial mapping relationship of all sub-regions, an initial 3D model is generated using a layered progressive fusion strategy.

[0145] In this embodiment, the texture images of each sub-region acquired by the binocular vision sensor refer to the images captured by the binocular camera that contain surface texture information of the sub-region (such as building wall texture and ground vegetation texture). The texture image of each sub-region needs to cover all viewpoints of the sub-region (such as front, side, and top).

[0146] The point cloud data of the corresponding sub-region acquired by the lidar refers to the point cloud containing the three-dimensional geometric information of the sub-region (such as building outlines and ground elevation) scanned by the lidar. The point cloud data needs to remove noise points and retain effective geometric feature points.

[0147] Multimodal feature co-registration algorithm refers to an algorithm that simultaneously extracts visual features from texture images and geometric features from point clouds, and establishes a spatial mapping between the two through feature matching. Specific steps include:

[0148] Visual feature extraction: Extract SIFT feature points from texture images (using OpenCV's SIFT_create function, with ≥500 feature points per image) and generate 128-dimensional feature descriptors;

[0149] Geometric feature extraction: Extract normal features (using PCL's NormalEstimation class, search radius 0.1m) and curvature features (calculated using the distance variance of neighborhood points in the point cloud, curvature > 0.1m) from the laser point cloud. (The points are regarded as feature points), generating a 6-dimensional feature descriptor (3-dimensional normal + 3-dimensional curvature);

[0150] Feature matching: The similarity between visual features and geometric features is calculated using the FLANN matcher (Fast Library for Approximate Nearest Neighbors). Feature pairs with a matching score > 0.8 are retained to establish a preliminary mapping between texture pixels and point cloud coordinates.

[0151] In this embodiment, GNSS absolute coordinates are used as the spatial reference to constrain the feature pairs of the initial mapping: the deviation between the GNSS coordinates when the texture image is acquired and the GNSS coordinates when the point cloud is acquired is calculated. If the deviation is greater than 5cm, the feature pair is removed. At the same time, combined with the attitude angle information of the flight trajectory segment, erroneous matching pairs caused by drastic attitude changes (such as feature pairs with an attitude angle change rate greater than 0.5rad / s) are filtered out. Finally, the number of feature pairs retained is greater than or equal to 60% of the initial matching pairs.

[0152] In this embodiment, the one-to-one mapping between texture pixels and point cloud spatial coordinates means that each texture pixel (u,v) corresponds to a unique point cloud spatial coordinate (x,y,z). The mapping is calculated through a perspective projection model: based on the intrinsic parameters (focal length f, principal point coordinates (u0,v0)) and extrinsic parameters (attitude angle, GNSS coordinates) collected by the UAV, the projection relationship between pixel coordinates and world coordinates is established, and the formula is as follows:

[0153] Where K is the camera intrinsic parameter matrix, Given the extrinsic parameter matrix, this formula allows us to calculate the corresponding pixel coordinates (u,v) from known coordinates (x, y, z), achieving a one-to-one mapping.

[0154] In this embodiment, the mapping error refers to the Euclidean distance between the point cloud coordinates corresponding to the texture pixel and the actual coordinates of the ground control point. The verification method is as follows: select the corresponding pixel of the ground control point in the texture image, calculate the point cloud coordinates mapped by the pixel, and calculate the distance between the pixel and the actual coordinates of the control point, such as (100.0m, 50.0m, 50.0m). The mapping deviation within a single sub-region is ≤2cm to ensure mapping accuracy.

[0155] The texture-point cloud spatial mapping relationship of all sub-regions, such as the pixel-point cloud mapping table of each of the 20 sub-regions, is stored in JSON format (e.g., the mapping table of sub-region 1: {(1280,360):(100.2,50.3,20.1) etc.}). The mapping table contains the mapping relationship of all valid pixels in the sub-region (excluding blurred pixels at the edges), and the number of mapping table entries for each sub-region is ≥10,000.

[0156] The layered progressive fusion strategy refers to the strategy of gradually generating the initial 3D model in three levels: fusion within a molecular region, fusion between sub-regions, and global optimization. This avoids excessive computation and loss of accuracy caused by fusing all data at once. Each fusion layer has a clear accuracy target (e.g., fusion deviation within a sub-region ≤ 1cm, fusion deviation between sub-regions ≤ 3cm).

[0157] The beneficial effects of the above technical solution are: by establishing accurate spatial mapping between texture and point cloud through multimodal feature collaborative registration and GNSS-trajectory constraints; by adopting a hierarchical progressive fusion strategy, the defects of one-time fusion of large amounts of data are avoided, and the problems of large texture-point cloud mapping deviation and discontinuous sub-region fusion in existing model generation are solved, significantly improving the geometric accuracy and texture consistency of the initial 3D model.

[0158] This invention provides a method for scene reconstruction and automatic correction using unmanned aerial vehicles (UAVs), which employs a layered progressive fusion strategy to generate an initial 3D model, including:

[0159] Triangulation is performed on the point cloud of each sub-region, and the texture image is projected onto the triangular mesh based on the spatial mapping relationship to form a local 3D model of the sub-region.

[0160] Simultaneously extract the normal and curvature features of the point cloud and the SIFT features of the texture image from the local 3D model of adjacent sub-regions;

[0161] Based on the cross-modal feature fusion network, geometric and visual features are mapped to the same high-dimensional space, feature similarity is calculated and matching constraints are established;

[0162] Based on GNSS absolute coordinates and flight trajectory information, rigid body transformation registration is performed on point clouds of adjacent sub-regions to initially eliminate large-scale positional deviations.

[0163] Pixel-level deformation correction is performed on the texture image based on the matching constraints, while Poisson fusion is performed on the overlapping point cloud to seamlessly connect the topology of the texture and the point cloud.

[0164] The texture consistency of the model after merging all sub-regions is adjusted and the point cloud density is equalized to generate the initial 3D model.

[0165] In this embodiment, triangulation refers to converting the point cloud into a triangular mesh (a surface composed of triangular patches) to carry the texture image. The implementation method is to use the Delaunay triangulation algorithm (through the Delaunay3D class of PCL). The triangulation parameters include: search radius: 0.1m (to ensure that adjacent points can form triangular patches); minimum patch area: 0.001㎡ (to remove invalid patches that are too small); after triangulation, the number of triangular meshes in each sub-region is ≥10000.

[0166] In this embodiment, projection refers to mapping the color information of texture pixels onto the vertices or faces of a triangular mesh. Specifically, vertex projection involves finding the corresponding texture pixel coordinates for each vertex of the triangular mesh, obtaining the pixel color value, and assigning it to the vertex.

[0167] Patch interpolation: For non-vertex positions within a triangular patch, calculate their color values ​​(based on the colors of the three vertices) using bilinear interpolation; after projection, the texture matching error of the triangular mesh is ≤1 pixel (e.g., the RGB difference between the projected color of a vertex and the actual pixel color is ≤10).

[0168] In this embodiment, the sub-region local 3D model refers to the independent 3D model formed by triangulation and texture projection of each sub-region, which includes triangular mesh (geometric structure) and texture (color information), and the model format is PLY (PolygonFileFormat).

[0169] In this embodiment, the normal feature is the normal vector (three-dimensional vector) of a point on the point cloud surface, which is used to characterize the orientation of the point cloud surface. It is calculated by the NormalEstimation class of PCL with a search radius of 0.1m. The consistency of the normal direction is adjusted by principal component analysis (PCA).

[0170] Curvature is the degree of curvature (dimensionless) of a point on the point cloud surface, calculated by the variance of the distance between that point and its neighboring points (points within a search radius of 0.1m). Curvature > Points are considered as feature points (such as edge points and corner points) and used for registration between sub-regions.

[0171] SIFT features of texture images refer to Scale-Invariant Feature Transform features, which are used to characterize local features of texture images (such as edges and corners). They are scale and rotation invariant and are extracted using the SIFT_create function of OpenCV. The number of feature points is ≥500 per image, and the feature descriptor is 128-dimensional.

[0172] Cross-modal feature fusion networks refer to neural networks that map the geometric features (normals + curvature, 6D) of point clouds and the visual features (SIFT, 128D) of texture images to the same high-dimensional space (e.g., 256D). They employ lightweight CNN (Convolutional Neural Network) structures and include:

[0173] Geometric feature branch: 2 convolutional layers (3×3 kernel, stride 1) map 6-dimensional features to 128-dimensional features;

[0174] Visual feature branch: 2 convolutional layers (3×3 kernel, stride 1) map 128-dimensional features to 128-dimensional features;

[0175] Fusion layer: The 128-dimensional features of the two branches are combined and mapped to 256-dimensional high-dimensional features through a fully connected layer;

[0176] Network training: The network is trained using historical registration data (1000 correctly matched feature pairs) with a contrastive loss function. The feature matching accuracy after training is ≥90%.

[0177] Feature similarity and matching constraints:

[0178] Feature similarity: Calculate the Euclidean distance between geometric features and visual features in high-dimensional space. The smaller the distance, the higher the similarity. The similarity threshold is set to 0.5 (distance ≤ 0.5 is considered a match).

[0179] Matching constraints include spatial constraints (GNSS coordinate deviation of the matched feature pair ≤ 5cm) and attitude constraints (attitude angle change of the matched feature pair ≤ 5°). Matching pairs that do not meet the constraints are removed, and the number of valid matching pairs is retained ≥ 60% of the initial matching pairs.

[0180] In this embodiment, rigid body transformation registration refers to a translation and rotation transformation that preserves the shape of the point cloud. This is used to initially eliminate large-scale positional discrepancies between point clouds in adjacent sub-regions. Specifically:

[0181] Initial transformation matrix: Based on the GNSS coordinate difference between adjacent sub-regions, an initial translation matrix is ​​constructed (e.g., if the GNSS coordinate difference between sub-region 1 and sub-region 2 is (10m, 0m, 0m), the initial translation matrix... =10m);

[0182] ICP registration: The Iterative ClosestPoint algorithm (PCL's IterativeClosestPoint class) is used. The effective matching pair is used as the initial corresponding point. The transformation matrix is ​​iteratively optimized. The number of iterations is ≤50. After registration, the Euclidean distance deviation of the overlapping area of ​​the point cloud is ≤3cm.

[0183] Pixel-level deformation correction refers to adjusting the texture images of adjacent sub-regions at the pixel level to eliminate texture misalignment at the stitching point. Specifically:

[0184] Feature point alignment involves selecting matching feature points (≥20) from adjacent sub-region texture images and calculating the perspective transformation matrix of image deformation (using OpenCV's findHomography function and robust estimation using the RANSAC algorithm).

[0185] Pixel resampling is a process that performs bilinear resampling on a texture image of one of its sub-regions based on a perspective transformation matrix, adjusting the pixel positions to align the texture edges at the stitching points (e.g., reducing the misalignment of a building corner texture from 5 pixels to 1 pixel).

[0186] In this embodiment, Poisson fusion refers to solving the Poisson equation to fuse overlapping point clouds of adjacent sub-regions into a continuous surface, eliminating gaps in the point cloud stitching. Specifically:

[0187] Point cloud fusion: The PoissonReconstruction class of PCL is used, with a reconstruction depth of 8 levels and a sampling density of [missing information]. ;

[0188] Gaps Detection: After fusion, gaps are detected by the continuity of point cloud normals (a change in normal direction >10° is considered a gap). If gaps exist, the point cloud sampling density in the overlapping area is increased (to a higher level). ), and re-integrate;

[0189] After fusion, the splicing deviation of point clouds in adjacent sub-regions is ≤2cm, with no obvious gaps.

[0190] Seamless topological alignment between textures and point clouds means that the pixel positions of the texture and the geometric positions of the point cloud correspond perfectly, without any topological inconsistencies (such as texture pixels corresponding to incorrect facets of the point cloud). Specifically:

[0191] Topology verification: For the stitched model, 100 texture pixels are randomly selected to verify whether the corresponding point cloud belongs to the correct triangular facet, with a verification accuracy of ≥99%;

[0192] Adjustment and optimization: If topological errors exist (such as pixels corresponding to incorrect patches), the spatial mapping relationship is recalculated and the texture projection is corrected until the topological connection is correct.

[0193] The beneficial effects of the above technical solution are: based on the hierarchical progressive strategy of triangulation and texture projection within sub-regions, cross-modal registration and Poisson fusion between sub-regions, and global energy optimization, it achieves accurate fusion from local to global, significantly improves the geometric accuracy and texture coherence of the initial 3D model, and lays a high-quality foundation for subsequent closed-loop correction.

[0194] This invention provides a method for scene reconstruction and automatic correction using unmanned aerial vehicles (UAVs), step 4 of which includes:

[0195] The absolute coordinates of each sub-region during the initial 3D model construction process are obtained by relying on the GNSS module carried by the UAV, and at the same time, no less than 3 high-precision ground control points are set up in the target scene.

[0196] The model coordinates corresponding to the high-precision control points on the ground in the initial 3D model are compared with the actual coordinates of the high-precision control points on the ground to calculate the global deviation value of the model. At the same time, the local deviation of the model at the mapping of texture and point cloud space in each sub-region is calculated point by point to obtain the local deviation distribution.

[0197] If the global deviation value is greater than the first preset threshold or there is a sub-region where the local deviation is greater than the second preset threshold, the coverage density of the UAV's random flight, the randomness coefficient of the flight trajectory, and the compensation coefficient of the dynamic error compensation model are adjusted in reverse. The data acquisition, error compensation, initial 3D model construction, and deviation calculation steps are iteratively executed until the global deviation value is not greater than the first preset threshold and all local deviations are not greater than the second preset threshold, thus completing the closed-loop correction of the initial 3D model.

[0198] In this embodiment, the initial 3D model construction process refers to the entire process from multi-sensor data acquisition and dynamic error compensation to the generation of the initial 3D model. During this process, the GNSS module continuously records the coordinates of each sub-region at the time of data acquisition, providing a spatial reference for the model.

[0199] The absolute coordinates of each sub-region during data acquisition are the GNSS absolute coordinates of the UAV when acquiring data for each sub-region.

[0200] In this embodiment, the internal coordinates of the generated initial 3D model correspond to the actual positions of the high-precision ground control points. For example, if the actual coordinates of the ground control points are (116.3°E, 39.9°N, 50.0m), the corresponding coordinates in the model are (116.3005°E, 39.9003°N, 50.05m). The actual coordinates are the true coordinates of the control points obtained from total station calibration, such as (116.3°E, 39.9°N, 50.0m), with an accuracy of 5mm.

[0201] The global deviation value of the model is the average Euclidean distance calculated after comparing the model coordinates with the actual coordinates of all ground high-precision control points. For example, if the deviations of the four control points are 3cm, 2.5cm, 2.8cm, and 3.2cm, respectively, the global deviation value is 2.875cm.

[0202] The mapping between texture and point cloud space in each sub-region is the location where texture pixels and point cloud coordinates are mapped one-to-one within each sub-region. For example, in sub-region 1, the texture image pixel (1280, 360) corresponds to the point cloud coordinates (100.2m, 50.3m, 20.1m).

[0203] Local deviation of the model is the deviation between the model coordinates and the actual coordinates at the mapping point cloud space of the texture in the sub-region. It is calculated point by point, that is, the deviation is calculated for each mapping point. For example, the local deviation of a mapping point in sub-region 1 is 2cm, and that of another mapping point is 1.5cm.

[0204] The local deviation distribution is the statistical distribution of local deviations in all sub-regions. For example, the local deviations in sub-region 1 are mostly between 1-2 cm, while the deviations at some points in sub-region 3 are 3 cm.

[0205] The first preset threshold is the maximum allowable value of the global deviation, set to 5cm. When the global deviation exceeds this value, the parameter needs to be adjusted.

[0206] The second preset threshold is the maximum allowable value for local deviation, set to 3cm. If any sub-region has a local deviation exceeding this value, it needs to be adjusted.

[0207] In this embodiment, the flight trajectory randomness coefficient is dimensionless, quantifying the degree to which the flight trajectory deviates from the center of the sub-region. If it is initially 0.3, it can be adjusted to 0.2 to reduce trajectory randomness and improve the spatial consistency of data collection.

[0208] In this embodiment, the compensation coefficient is an adjustable parameter in the dynamic error compensation model. For example, k01 is initially 0.025 and can be adjusted to 0.03 to enhance the error compensation strength.

[0209] The beneficial effects of the above technical solution are: by constructing a closed loop with GNSS absolute coordinates and high-precision ground control points, the global and local deviations of the model are detected in real time, the flight and compensation parameters are adjusted in reverse and iteratively optimized, so as to achieve dynamic improvement of model accuracy and ensure that the global and local deviations of the final initial 3D model meet the preset thresholds, thus laying a high-precision foundation for subsequent scenario applications.

[0210] This invention provides a scene reconstruction and automatic correction method using unmanned aerial vehicles (UAVs), which performs texture consistency adjustment and point cloud density equalization on the model after fusion of all sub-regions to generate an initial 3D model, including:

[0211] The texture consistency energy term is obtained by calculating the sum of squared grayscale differences and the sum of squared color space distances at the texture stitching points of adjacent sub-regions. ,in, The pixel grayscale values ​​of the k1th and k2th adjacent points in the connecting area; Let k1 and k2 be the color vectors of the adjacent points in the connecting region. This refers to the color weighting coefficient;

[0212] Average density of point cloud in target scene Based on this, the point cloud density of each sub-region is calculated and... Sum of squared deviations Where M2 is the total number of sub-regions; Let be the point cloud density of the o-th sub-region;

[0213] Based on high-precision ground control points, calculate the sum of squared Euclidean distances between the control point coordinates and the actual coordinates. Where M3 is the number of control points; Let u1 be the coordinates of the control point. The actual coordinates of the u1th control point;

[0214] based on as well as Generate an initial 3D model.

[0215] In this embodiment, the texture consistency energy term is calculated. In this context, adjacent sub-regions refer to spatially adjacent sub-regions in the target scene, such as sub-region 1 and sub-region 2, which are adjacent to each other in the scene; the texture splicing point is the connecting area of ​​the texture images of these two sub-regions, such as the corner of the exterior wall of a building, which is covered by the textures of both sub-regions at the same time.

[0216] In this embodiment, the sum of squares of grayscale differences is the sum of squares of the grayscale value differences between adjacent pixels at the stitching point, such as the pixels at the stitching point. The grayscale value is 140. The grayscale value is 145, and the squared grayscale difference is 25. The squared distance in the color space and the squared distance of the pixel color vector in the color space (e.g., RGB) are... =(140,150,160), =(142,153,162), then the squared color distance is 17. In this embodiment, Used to balance the impact of grayscale and color on texture consistency, the value can be 0.9. Specifically, it is adjusted through testing multiple sets of texture samples to adapt to the texture differences in different scenes.

[0217] In this embodiment, based on and By minimizing the combination of these three energy terms through optimization algorithms (such as iterative least squares), an initial 3D model with natural texture stitching, uniform point cloud density, and accurate global coordinates is generated.

[0218] In this embodiment, based on and Integrate into a global energy function QR, and balance the importance of each objective through weights:

[0219] ,in, These are the weighting coefficients. They are respectively for The normalized result, and Initial settings =0.4, =0.3, =0.3, which can be adjusted according to the needs of the scenario:

[0220] If high texture quality is required (such as architectural exterior modeling), increase the size. ;

[0221] If high uniformity of point cloud density is required (e.g., in topographic mapping), increase... ;

[0222] If high global coordinate accuracy is required (such as in engineering surveying), increase... .

[0223] The beneficial effects of the above technical solution are: by constructing a multi-dimensional energy optimization model with texture consistency, point cloud density balance and global accuracy, the model achieves coordinated optimization in texture, geometric density and spatial coordinates, providing a high-precision and highly consistent three-dimensional basic model for the precise application of subsequent scenarios.

[0224] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1.A method for scene reconstruction and automatic correction using a UAV, characterized in that, Comprising: Step 1: Based on the multi-sensor composed of dual-vision sensor and laser radar carried by the unmanned aerial vehicle, through multiple omnidirectional random flight covering different sub-regions of the target scene for global data collection, the flight trajectory information of each round is recorded synchronously, and the sensor data is timestamped and aligned; Step 2: Construct a dynamic error compensation model, combine the dynamic parameter set of each round of random flight of the unmanned aerial vehicle, and correct the motion blur, environmental interference and trajectory deviation error in the collected data; Step 3: Integrate the corrected multi-round data, establish the spatial mapping of sub-region texture and point cloud, and generate an initial three-dimensional model; Step 4: Relying on the absolute coordinates of the GNSS of the unmanned aerial vehicle and the ground high-precision control points to build a closed-loop correction mechanism, calculate the deviation of the initial three-dimensional model, if the deviation exceeds the threshold, adjust the random flight coverage density, flight round and error compensation coefficient in the reverse direction, until the deviation meets the standard; Step 2, comprising: Obtain the dynamic parameter set of each round of random flight of the unmanned aerial vehicle, wherein the dynamic parameter set comprises three-dimensional attitude angle, time variation rate, instantaneous height , flight trajectory randomness coefficient and environmental parameters; The ratio of the standard deviation of the distance of each sampling point in the single-wheel flight trajectory from the center of the sub-region to the characteristic length of the sub-region is regarded as a flight trajectory randomness coefficient , and a trajectory deviation unit direction vector is extracted ; the characteristic length of the sub-region refers to the longest side of the sub-region Constructing error compensation function based on motion blur wherein, are respectively pitch angle, roll angle, yaw angle, are respectively time-varying rate of pitch angle, time-varying rate of roll angle, unit is rad / s; is standard flight height; are respectively height attenuation factor, visibility influence index, trajectory randomness coupling index; is atmospheric visibility; is reference visibility; is attitude-blur conversion coefficient, unit is: ; For binocular vision images, the adaptive Gaussian kernel deblurring is adopted The kernel size is linearly extended with the increase of the driving. The kernel size is linearly extended with the increase of the driving. Constructing error compensation function based on environmental disturbance wherein L is the light intensity; is the standard light intensity; is the light sensitivity coefficient; is the trajectory randomness coupling index; is the reference light; A normalization correction is performed on the image pixel values: wherein, is the pixel gray value of the original image; is the pixel gray value after the illumination correction; is the maximum value of all is the maximum value of all Step 2, further comprising: Construct a spatial coordinate correction matrix: ； ； ； wherein, is a component of; is a trajectory deviation coefficient; are translation vectors in x, y, z directions, respectively; is the average laser scanning distance of the UAV and the target scene sub-region; Based on The rotation matrix R is calculated and processed to obtain the final matrix ; is a translation matrix; Performing coordinate transformation on the laser radar point cloud data , correcting point cloud dislocation caused by trajectory deviation, wherein P is the original laser radar point cloud data; is the corrected laser radar point cloud data; Closed loop feedback based on multiple rounds of flight data to iteratively optimize for minimum average deviation of post data from ground control points until average deviation < 3 cm. 2.The scene reconstruction and automatic correction method using a UAV according to claim 1, wherein, Further comprising: Call the unmanned aerial vehicle according to the preset route and the preset speed flight several historical flight trajectories, and determine the trajectory position and the historical speed based on the same historical flight time point in each historical flight trajectory, obtain the trajectory distance deviation set of each historical flight trajectory at the corresponding historical flight time point And the flight speed deviation set , wherein The trajectory distance deviation vector of the jth historical flight trajectory under the corresponding historical flight time point is The module length of The flight speed deviation vector of the jth historical flight trajectory under the corresponding historical flight time point is The module length of The total number of historical flight trajectories; fitting processing is performed on the trajectory distance deviation set and the flight speed deviation set respectively to obtain distance fitting deviation and speed fitting deviation and a second minimum value z2 in the flight speed deviation set to obtain a trajectory deviation coefficient k07: wherein is an exponential coefficient. 3.The scene reconstruction and automatic correction method using a UAV according to claim 1, wherein, Step 3, comprising: Integrate the multi-sensor data of the unmanned aerial vehicle after dynamic error compensation correction, including the texture image of each sub-region collected by the dual-vision sensor and the corresponding sub-region point cloud data collected by the laser radar, and each sub-region data is associated with the GNSS absolute coordinates and flight trajectory information of the unmanned aerial vehicle; Using a multi-modal feature collaborative registration algorithm, extract the visual feature points of the texture image and the geometric features of the point cloud, combine the GNSS absolute coordinates and flight trajectory information to establish constraints, and one-to-one map the texture pixels and point cloud spatial coordinates, and the mapping error is verified based on the ground high-precision control points; Based on the texture-point cloud spatial mapping relationship of all sub-regions, an initial three-dimensional model is generated using a hierarchical progressive fusion strategy. 4.The scene reconstruction and automatic correction method using a UAV according to claim 3, wherein, Using a hierarchical progressive fusion strategy to generate an initial three-dimensional model, comprising: Triangulate the point cloud of each sub-region, project the texture image to the triangular mesh based on the spatial mapping relationship, and form a local three-dimensional model of the sub-region; Synchronously extract the normal feature and curvature feature of the point cloud and the SIFT feature of the texture image of the adjacent sub-region local three-dimensional model; Map the geometric and visual features to the same high-dimensional space based on the cross-modal feature fusion network, calculate the feature similarity and establish the matching constraint; Based on the GNSS absolute coordinates and flight trajectory information, perform rigid transformation registration on the point clouds of adjacent sub-regions to preliminarily eliminate large-scale position deviation; Based on the matching constraint, perform pixel-level deformation correction on the texture image, and simultaneously perform Poisson fusion on the overlapping point cloud to seamlessly connect the topology of the texture and the point cloud; Adjust the texture consistency and equalize the point cloud density of the fused model of all sub-regions to generate an initial three-dimensional model. 5.The scene reconstruction and automatic correction method using a UAV according to claim 1, wherein, Step 4, comprising: Relying on the GNSS module carried by the unmanned aerial vehicle to obtain the absolute coordinates of each sub-region data collection during the construction of the initial three-dimensional model, and arranging not less than 3 ground high-precision control points in the target scene; The model coordinates corresponding to the high-precision ground control point positions in the initial three-dimensional model are compared with the actual coordinates of the high-precision ground control points, and a model global deviation value is calculated. Meanwhile, the model local deviation at the spatial mapping of each sub-region texture and point cloud is calculated point by point to obtain a local deviation distribution; If the global deviation value is greater than a first preset threshold or there is a sub-region with a local deviation greater than a second preset threshold, the coverage density of the random flight of the unmanned aerial vehicle, the flight trajectory randomness coefficient, and the compensation coefficient of the dynamic error compensation model are adjusted reversely, and the steps of data acquisition, error compensation, initial three-dimensional model construction, and deviation calculation are iteratively executed until the global deviation value is not greater than the first preset threshold and all local deviations are not greater than the second preset threshold, and the closed-loop correction of the initial three-dimensional model is completed. 6.The scene reconstruction and automatic correction method using a UAV according to claim 4, wherein, The model fused after all the sub-regions is subjected to texture consistency adjustment and point cloud density equalization to generate an initial three-dimensional model, including: Calculate the sum of square of gray level difference and the sum of square of color space distance of the texture splicing of adjacent sub-regions to obtain a texture consistency performance energy term wherein, is the pixel gray level value of the k1th and k2th adjacent points in the transition area; is the color vector of the k1th and k2th adjacent points in the transition area; is the color weight coefficient; with the target scene point cloud average density as a reference, calculate the deviation square sum of each sub-region point cloud density and wherein M2 is the total number of sub-regions; is the point cloud density of the oth sub-region.​ Calculate the Euclidean distance square sum of the control point coordinate and the actual coordinate based on the ground high-precision control point wherein M3 is the number of control points; is the u1th control point coordinate; is the u1th control point actual coordinate; Based on and an initial three-dimensional model is generated.

Images