Unmanned aerial vehicle automatic lofting method based on real scene three-dimensional model and multi-source data fusion

Abstract

The invention relates to the technical field of unmanned aerial vehicle lofting measurement, and provides an unmanned aerial vehicle automatic lofting method based on a live-action three-dimensional model and multi-source data fusion, specifically, an unmanned aerial vehicle is used as a hardware platform, and real-time high-precision geodetic coordinates of the unmanned aerial vehicle are obtained through a GNSS; and converting the coordinates into a pre-constructed live-action three-dimensional model coordinate system to realize accurate positioning of the unmanned aerial vehicle in the digital twin environment. Preliminary navigation is carried out by calculating the deviation between a projection point of an unmanned aerial vehicle model and a designed lofting point, a visual positioning auxiliary correction module is introduced, and accurate marking of a ground target spot is realized by comparing a real-time downward-looking image of the unmanned aerial vehicle with the lofting point in the model and a peripheral view. The lofting high precision of + / -3 cm can still be kept in a complex environment, efficiency, environmental adaptability and operation safety are remarkably improved, the lofting device can be widely applied to the fields of engineering construction, surveying and mapping investigation, precision agriculture and the like, and traditional lofting is promoted to be transformed to be intelligent and unmanned.

Description

Technical Field

[0001] This invention relates to the field of UAV stakeout measurement technology, and more specifically, to an automatic UAV stakeout method based on the fusion of real-scene 3D models and multi-source data. Background Technology

[0002] In fields such as engineering construction, land surveying, and precision agriculture, accurately pinpointing design points on the ground is a fundamental and crucial task. Traditional surveying mainly relies on equipment such as total stations and GNSS-RTK handheld devices, requiring surveyors to repeatedly travel between various points, which has limitations such as low efficiency, high labor intensity, and difficulty in operating in steep or dangerous areas. With the development of drone technology and digital twin technology, drone surveying, with its advantages of "aerial perspective and precise positioning," is gradually becoming an alternative to traditional methods.

[0003] Current UAV (Unmanned Aerial Vehicle) stakeout technologies mainly fall into two categories: one is coordinate-driven stakeout that directly relies on GNSS-RTK, whose accuracy is limited by RTK signal quality and absolute positioning accuracy, and its performance degrades when signals are blocked or multipath effects exist; the other is vision-based relative stakeout, which relies on the recognition of preset markers, lacks absolute geographic information, and has poor versatility. In recent years, high-precision real-world 3D models generated by oblique photogrammetry have provided UAVs with rich prior environmental information. The deep integration of UAV absolute positioning and model-based visual perception enables high-precision autonomous stakeout, ensuring accuracy stability while improving adaptability to complex environments, which has significant practical implications for promoting intelligent engineering stakeout.

[0004] Internationally, Trimble's UX5 UAV, equipped with an RTK module, achieved a layout accuracy of ±5cm, but without integrating a real-world 3D model, projection deviations were prone to occur in undulating terrain areas. SenseFly proposed a "model navigation + GNSS calibration" solution, but the coordinate transformation process required offline processing, resulting in insufficient real-time performance. Domestically, Zhang Hong et al. proposed a positioning method based on the fusion of UAV IMU and GNSS, improving dynamic positioning accuracy, but it did not address the visual assistance application of real-world 3D models. Li Juan et al. used real-world 3D models for UAV path planning, but the layout process lacked a direct mapping between aerial coordinates and ground projection, leading to insufficient accuracy control.

[0005] In summary, existing UAV lofting technology has not yet achieved deep integration of real-world 3D models and multi-source positioning data, making it difficult to balance high precision, strong adaptability, and real-time performance in complex environments. There is an urgent need to propose an automatic UAV lofting method with high precision and high applicability to solve the above problems. Summary of the Invention

[0006] In view of this, this invention proposes an automatic UAV lofting method based on the fusion of real-scene 3D models and multi-source data. It designs a precise conversion and projection process from geodetic coordinates to model coordinates, realizes real-time mapping of UAVs in digital twin space, and adopts a visual feedback refinement algorithm based on matching real-time images with model rendered views to effectively compensate for model system errors and GNSS random jitter, thereby improving lofting accuracy.

[0007] To achieve the above objectives, this invention proposes an automatic UAV lofting method based on the fusion of real-scene 3D models and multi-source data, comprising:

[0008] Step 1: Construct a realistic 3D model and unify the coordinate system;

[0009] Step 2: Based on the multi-source fusion positioning of GNSS-RTK and IMU, realize the real-time conversion of UAV geodetic coordinates to the coordinates of the real scene 3D model and ground projection;

[0010] Step 3: Using a visual-assisted refinement algorithm, the UAV pose is corrected by matching features between real-time images and the model rendering view, and ground target marking is completed.

[0011] Further, step 1 includes:

[0012] Step 1.1: Use a drone equipped with an RTK module to collect oblique photography data, and control the flight altitude, overlap and exposure time;

[0013] Step 1.2: Construct a real-world 3D model with a resolution ≤0.02m through aerial triangulation, ground control point constraints, and dense point cloud generation;

[0014] Step 1.3: Unify the model coordinate system with the design coordinate system, load the lofting points and verify their consistency.

[0015] Furthermore, in step 1.1, the flight altitude is calculated using the target ground resolution, camera focal length, and pixel size;

[0016] The photographic baseline is calculated using the long side of the image, the forward overlap, the flight altitude, and the focal length.

[0017] The exposure time was calculated using the photographic baseline and flight speed.

[0018] The overlap should be controlled to be ≥80%.

[0019] Further, step 1.2 includes:

[0020] Image distortion correction and automatic matching of corresponding points are performed on oblique photogrammetry data;

[0021] Aerial triangulation was performed using bundle adjustment of the regional network, and the exterior orientation elements of the image were output.

[0022] The exterior orientation elements of the image are constrained and adjusted by importing ground control points;

[0023] Based on the adjusted results, a dense point cloud is generated, and the real-scene 3D model is constructed.

[0024] Further, step 2 includes:

[0025] Step 2.1: Use the extended Kalman filter algorithm to fuse GNSS-RTK and IMU data to output the real-time pose of the UAV;

[0026] Step 2.2: Convert the geodetic coordinates to model plane coordinates using Gauss-Kruger projection, and perform elevation transformation using the geoid undulation model;

[0027] Step 2.3: Perform ray-mesh intersection based on DSM, calculate the deviation between the UAV projection point and the lofting point, and set the flight control logic.

[0028] Furthermore, in step 2.3, the process of ray-mesh intersection based on DSM includes:

[0029] Traverse the DSM mesh surfaces of the real-world 3D model and determine the intersection points of ray R with the mesh surfaces. When the intersection point satisfies the condition of "being located inside the mesh surface and having an elevation value that deviates from the elevation of the DSM mesh by ≤ ±1cm", then the intersection point is determined as the ground projection point.

[0030] Further, step 3 includes:

[0031] Step 3.1: Acquire real-time downward-looking images from the UAV and perform distortion correction, grayscale normalization, and region cropping;

[0032] Step 3.2: Render the virtual view based on the UAV pose and the parameters of the real-world 3D model;

[0033] Step 3.3: Extract features using ORB or SuperPoint algorithms and perform matching and purification;

[0034] Step 3.4: Calculate the pixel offset and convert it into a ground correction vector;

[0035] Step 3.5: Use a dynamic weighted fusion algorithm based on matching confidence to combine the visual correction vector with the GNSS-IMU fusion positioning result, update the UAV model coordinates, and iterate until the accuracy requirements are met.

[0036] Furthermore, in step 3.5, the specific rules of the dynamic weighted fusion algorithm based on matching confidence are as follows:

[0037] When matching accuracy At that time, weight value ;

[0038] when At that time, weight value ;

[0039] when At that time, weight value ;

[0040] when hour, Visual corrections are paused, and positioning relies solely on GNSS-IMU.

[0041] Furthermore, in step 3.5, the termination condition for iterative correction is:

[0042] If the overall deviation between the updated projection point and the stakeout point If, the iteration is terminated directly, then... Then iterative correction is performed, with a maximum of 3 iterations. If after 3 iterations... Terminate the iteration; if after 3 iterations This triggers an alarm mechanism, switching to GNSS-dominated and manual fine-tuning mode.

[0043] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0044] This invention achieves real-time, accurate mapping and ground projection from geodetic coordinates to model coordinates for UAVs by integrating GNSS-RTK, IMU, and visual multi-source data. It also innovatively introduces a visual closed-loop refinement algorithm that combines virtual view rendering and real-image matching to dynamically correct model system errors and random positioning deviations. This enables UAV stakeout to maintain a high accuracy of ±3cm even in complex environments, significantly improving stakeout efficiency, environmental adaptability, and operational safety. It can be widely applied in engineering construction, surveying, precision agriculture, and other fields, driving the transformation of traditional stakeout towards intelligent and unmanned operations. Attached Figure Description

[0045] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. In the drawings:

[0046] Figure 1 This is an overall framework diagram of the UAV automatic lofting method based on the fusion of real-scene 3D models and multi-source data of the present invention;

[0047] Figure 2 This is a wireframe diagram of the crisscross oblique photography flight path planning for aerial equipment in an embodiment of the present invention;

[0048] Figure 3 is a schematic diagram of the real-scene 3D model and lofting point loading in an embodiment of the present invention, wherein (a) is a schematic diagram of image loading and aerial triangulation calculation, (b) is a schematic diagram of the triangular mesh line width model, (c) is a schematic diagram of the real-scene 3D model, and (d) is a schematic diagram of lofting point loading.

[0049] Figure 4 This is a multi-source data fusion positioning and coordinate transformation diagram in an embodiment of the present invention;

[0050] Figure 5 This is a schematic diagram of the real-time location projection of the UAV in an embodiment of the present invention;

[0051] Figure 6 This is a flowchart illustrating the visual-assisted refinement position in an embodiment of the present invention.

[0052] Figure 7 This is a schematic diagram of the matching point pair between the downward view image and the lofting point 1 image in an embodiment of the present invention;

[0053] Figure 8 This is an installation diagram of the camera and spraying device in an embodiment of the present invention. Detailed Implementation

[0054] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0055] This embodiment proposes an automatic UAV lofting method based on the fusion of real-scene 3D models and multi-source data, such as... Figure 1 As shown, it includes the following steps:

[0056] Step 1: Construction of a High-Precision Realistic 3D Model and Coordinate System

[0057] As a "digital twin" benchmark for UAV-based automated layout, the geometric accuracy and coordinate consistency of the model directly determine the reliability of the layout results. This step constructs a realistic 3D model that meets the ±2cm accuracy requirement through oblique photogrammetry data acquisition, refined modeling optimization, and coordinate benchmark unification. The specific implementation is as follows:

[0058] Step 1.1: Oblique Photogrammetry Data Acquisition

[0059] (1) Hardware configuration: such as Figure 2As shown, a multi-rotor drone equipped with an RTK module (such as the DJI M300 RTK) and a five-lens tilting camera (such as the CERNET 202S) is selected, and the core parameters of the camera are clearly defined. The core parameters of the CERNET 202S camera are: pixel size 4.5μm, downward-facing lens focal length 40mm, and image physical size 35.9mm×24mm (long side L=35.9mm). If a lightweight platform (such as the DJI Phantom 4 RTK) is selected, it must be ensured that its single lens supports five-directional flight attitude (downward, forward, backward, left, and right) data acquisition.

[0060] (2) Operation parameter design (taking CERNET 202S as an example):

[0061] a. Accuracy target: Clearly define the ground resolution (GSD) of the 3D model of the real scene as ≤0.02m (i.e. 2cm), and use this to deduce key parameters such as flight altitude and photographic baseline;

[0062] b. Flight altitude calculation: Based on formula (1), the theoretical flight altitude is 177.78m by substituting the parameters of the CERNET camera. In actual operation, the flight altitude is rounded down to 150m (lower than the theoretical value to ensure accuracy).

[0063] (1)

[0064] c. Overlap control: Forward overlap ≥ 80%, Lateral overlap ≥ 80%.

[0065] d. Calculation of photographic baseline: The photographic baseline position is calculated to be 26.925m based on formula (2).

[0066] (2)

[0067] e. Exposure time setting: After setting the UAV flight speed, calculate the time interval for the camera to collect photos according to formula (3). Generally, the speed of a multi-rotor UAV can be controlled between 8m / s and 12m / s when conducting surveying projects. If the speed is set to 10m / s, the calculated exposure time is 2.7s. In actual operation, it is required to round down to 2s (round down to ensure the overlap of the images).

[0068] (3)

[0069] Where: GSD is the ground image resolution, f is the focal length, pix is ​​the pixel size, L is the long side of the image, p is the forward overlap, B is the photographic baseline, and v is the flight speed.

[0070] Step 1.2: Real-world 3D modeling and optimization

[0071] (1) Data preprocessing: Import the image data obtained by oblique photogrammetry into modeling software (such as Agisoft Metashape Professional, DJI Terra), first perform image distortion correction (based on camera intrinsic parameter file) and automatic matching of corresponding points;

[0072] (2) Aerial triangulation: As shown in Figure 3, aerial triangulation is performed using bundle adjustment of the area network, and the exterior orientation elements (position and attitude parameters) of the image are output. The overall reprojection error after the calculation is ≤0.5 pixels.

[0073] (3) Absolute accuracy optimization: Import the coordinates of at least 3 evenly distributed ground control points (GCPs) (which must be consistent with the subsequent design coordinate system) to constrain and adjust the aerial triangulation results to ensure the absolute geographic coordinate accuracy of the model;

[0074] (4) Model generation: Based on the optimized aerial triangulation results, a dense point cloud (point cloud density ≥ 50 points / m²) is generated, and then a three-dimensional mesh model and a digital surface model (DSM) are constructed. The DSM resolution is ≤ 0.02m and the size of the triangular facet of the mesh model is ≤ 0.03m.

[0075] Step 1.3: Coordinate datum unification and layout point loading

[0076] (1) Coordinate System 1:

[0077] a. Target coordinate system: Determine the coordinate system to which the design layout points belong (e.g., CGCS2000 geodetic coordinate system, UTM38 zone plane coordinate system), and obtain the core parameters of this coordinate system (ellipsoid parameters: semi-major axis a=6378137m, first eccentricity e=0.0818191908426, projection parameters: central meridian longitude, projection zone type);

[0078] b. Solving for transformation parameters: Based on the two sets of data of "model native coordinates - design coordinate system coordinates" of ground control points, the transformation matrix is ​​solved using the Bursa model (translation ΔX, ΔY, ΔZ, rotation angle εx, εy, εz, scaling ratio k). After transformation, the plane deviation of the control points is ≤±1cm and the elevation deviation is ≤±2cm.

[0079] (2) Loading of lofting points:

[0080] a. Data format: Import the design layout point data into the modeling software in a standardized format (such as SHP format, including point ID, design coordinates (X-axis, Y-axis, H-axis), and point type);

[0081] b. Visual association: The software automatically maps the lofting points to the corresponding spatial locations in the real-world 3D model, generating a visual model with point markers;

[0082] Consistency verification: Select no fewer than 3 independent checkpoints (not involved in coordinate transformation parameter solving), extract their model coordinates (X-model, Y-model, H-model) and design coordinates, and calculate the deviation: planar comprehensive deviation. Elevation deviation Only after verification can the subsequent layout process begin.

[0083] Step 2: UAV multi-source positioning and real-time coordinate transformation

[0084] like Figure 4 As shown, this step is the core hub connecting UAV aerial positioning and ground surveying. Through a closed-loop process of "GNSS-IMU multi-source fusion positioning → real-time conversion of geodetic coordinates to model coordinates → ground projection and deviation calculation", it achieves accurate mapping of the UAV in digital twin space and physical space, providing a high-precision position reference for subsequent navigation flight. The specific implementation is as follows:

[0085] Step 2.1: UAV Multi-Source Fusion Positioning and Coordinate Transformation

[0086] (1) Acquisition of multi-source positioning data:

[0087] a. The UAV is equipped with a GNSS-RTK module and an IMU (Inertial Measurement Unit). The GNSS-RTK module outputs the absolute coordinates (L, B, H) in the geodetic coordinate system in real time, with a positioning frequency ≥10Hz and a static positioning accuracy ≤±1cm+1ppm. The IMU collects the UAV's attitude data (roll angle, pitch angle, yaw angle) and acceleration data in real time, with a sampling frequency ≥100Hz, to compensate for positioning interruptions when the GNSS signal is temporarily blocked.

[0088] b. Data fusion is performed using the extended Kalman filter (EKF) algorithm: The coordinates output by GNSS-RTK are used as the observation values, and the attitude and acceleration output by IMU are used as the state prediction values. A fusion equation is constructed to output a stable real-time pose of the UAV (including geodetic coordinates (L, B, H) and attitude parameters), ensuring the continuity of positioning in complex environments. The dynamic positioning accuracy after fusion is ≤ ±2cm.

[0089] (2) Transformation from geodetic coordinates to model plane coordinates:

[0090] a. Conversion algorithm: The Gauss-Kruger 3° zone projection forward calculation formula (formula (4)) is used to convert the geodetic coordinates (L, B) into the plane coordinates (X, Y) corresponding to the real scene 3D model. The formula is as follows:

[0091] (4)

[0092] in: This is the abscissa value of the central meridian (the starting abscissa of the projection zone); The radius of curvature of the circle is denoted as . For the semi-major axis of the ellipsoid, For the first eccentricity, This is the difference (in radians) between the real-time longitude of the UAV and the longitude of the central meridian. , This is the second eccentricity.

[0093] (3) Elevation conversion:

[0094] a. Using the normal height system, calculate the elevation anomaly values ​​using the EGM2008 elevation anomaly model. Convert the GNSS-RTK output geodetic elevation H (ellipsoidal height) into the absolute elevation corresponding to the model. The conversion formula is: Ensure that the elevation data is consistent with the DSM elevation datum of the real-world 3D model, with a conversion accuracy of ≤±2cm.

[0095] Step 2.2: Model Projection and Deviation Calculation

[0096] (1) Implementation of ground projection:

[0097] a. such as Figure 5 As shown, the coordinates of the drone model after fusion positioning and transformation are defined as spatial points. Generate a projection ray along the negative Z-axis of the model coordinate system. .

[0098] b. Using a ray-mesh intersection algorithm: Traverse the DSM mesh surfaces of the real-world 3D model, determine the intersection points of the ray R with the mesh surfaces. When an intersection point satisfies the condition that it is located inside the mesh surface and its elevation value deviates from the DSM mesh elevation by ≤ ±1cm, the intersection point is determined as the ground projection point. ,in , To map the elevation corresponding to the DSM grid, we can achieve a precise mapping of the UAV's aerial position to the ground.

[0099] (2) Deviation calculation and navigation command generation:

[0100] a. The coordinates of the design layout points are: Calculate the deviation between the projection point and the layout point, as shown in formula (5):

[0101] (5)

[0102] b. Flight control logic:

[0103] Calculate flight heading angle (Using angles, with the X-axis of the model coordinate system as 0°, increasing clockwise), calculate the flight distance. When the distance is less than 0.5 meters, the speed is reduced to 1 m / s to ensure positioning accuracy when approaching at close range.

[0104] Step 3: Visual-assisted refinement algorithm

[0105] To compensate for GNSS-RTK random jitter and systematic errors in the real-world 3D model (such as modeling accuracy deviations and coordinate transformation residual errors), and to further improve the final accuracy of the stakeout points, this step designs a closed-loop visual refinement algorithm consisting of "virtual view rendering - real image acquisition - feature matching - deviation correction - pose fusion," achieving precise marking of ground target points at the ±3cm level. Figure 6 As shown, the specific implementation is as follows:

[0106] Step 3.1: Visual Data Acquisition and Preprocessing

[0107] (1) Hardware configuration and parameter settings: The UAV is equipped with a downward-looking high-definition camera (resolution ≥ 4000×3000 pixels, frame rate ≥ 15fps). The camera intrinsic parameters are calibrated and stored in advance using the Zhang Zhengyou calibration method. The core intrinsic parameters are fixed as follows: focal length Principal point coordinates Distortion coefficient During flight, strictly control the attitude of the UAV: ​​roll angle ≤ ±2°, pitch angle ≤ ±2°, yaw angle ≤ ±1°, to avoid excessive perspective distortion of the image due to attitude tilt (see step 3.6 for camera installation).

[0108] (2) Real image preprocessing:

[0109] a. Distortion correction: Based on camera intrinsic parameters, a Brownian distortion model is used to correct distortion in the acquired real-time images, eliminating the influence of lens optical distortion;

[0110] b. Gray-level normalization: Histogram equalization is performed on the corrected image to normalize the gray-level range to [0,255], thereby enhancing the feature stability under illumination changes;

[0111] c. Region cropping: Centered on the coordinates of the UAV model, crop the effective area of ​​the image (fixed size = 512×512 pixels), focusing on the terrain features around the lofting point to reduce redundant information interference.

[0112] Step 3.2: Model Virtual View Rendering

[0113] (1) Rendering parameter alignment: Based on the real-world 3D model, the current pose and attitude angle of the UAV are aligned. ,θ,ψ) represents the virtual camera pose. The virtual camera parameters are set to be completely consistent with the real downward-looking camera (focal length 8mm, resolution 4000×3000 pixels, field of view 60°) to ensure that the perspective relationship between the virtual view and the real image matches. When rendering, the texture mapping accuracy is enabled ≥1cm / pixel to preserve small features of the ground surface (such as stone edges, vegetation textures, ground cracks, etc.).

[0114] (2) Virtual image generation: Extract lofting points from the real-world 3D model. A 5m x 5m grid model is used to render a virtual downward-facing image (with the same resolution and grayscale range as the real image). The lighting parameters of the virtual image are dynamically adjusted according to the real-time ambient light intensity (light intensity error ≤ ±10 lux) to ensure consistency with the brightness of the real image.

[0115] Step 3.3: Feature Extraction and Matching

[0116] like Figure 7 As shown, the essence of this part is: first, find the "key points" that can represent the image features in the two images (similar to landmarks in reality, such as corners or texture inflection points), then match the same landmarks in the two images, and finally eliminate the mismatched pairs to ensure accuracy.

[0117] (1) Feature extraction algorithm selection:

[0118] a. Typical scenarios (stable lighting, no occlusion): The ORB algorithm is used to extract feature points, balancing efficiency and robustness. The specific parameters for the ORB algorithm are set as follows: pyramid layer number n = 8 (scale range 1-1.2). 7 ), FAST corner threshold t=20 (pixel grayscale difference threshold), edge threshold edgeThreshold=31, maximum number of feature points maxFeatures=1000, number of feature points ≥500 / frame.

[0119] b. Complex scenes (large changes in lighting, sparse surface texture): The SuperPoint algorithm based on deep learning is adopted, the model weights are selected from the weights of the training COCO dataset, the feature point confidence threshold confThreshold=0.8, the feature point non-maximum suppression radius nmsRadius=4px, and dense and stable feature points (≥300 / frame) are generated to ensure matching redundancy.

[0120] (2) Feature matching and purification:

[0121] a. Initial matching: The FLANN algorithm is used for feature descriptor nearest neighbor matching. The matching distance threshold d≤2.0 (ORB algorithm) or d≤0.8 (SuperPoint algorithm) is set to filter the initial matching pairs;

[0122] b. Matching purification: The RANSAC algorithm is used to remove outliers. Specific parameters are: number of iterations N = 1000 (according to...). Calculate, where the confidence level Interior point probability Minimum number of samples The interior point threshold ε = 1.5 pixels, the homography matrix degrees of freedom dof = 8, and the number of interior point matching pairs retained after purification is ≥30 pairs to ensure matching accuracy. The matching accuracy rate after purification is ≥95%.

[0123] Step 3.4: Offset Calculation and Correction Vector Generation

[0124] (1) Image offset calculation: Based on the purified interior point matching pairs, a homography moment H (3×3 matrix) is constructed, and the pixel offset between the real image and the virtual image is solved by the least squares method. ,Right now ,in The coordinates of the feature points in the real image. The coordinates of the feature points corresponding to the virtual image are given, and the residual res is constrained to be ≤ 0.5 pixels when solving the problem.

[0125] (2) Ground offset conversion: Based on the camera imaging geometry and the UAV flight altitude With camera focal length , pixel offset Convert to ground horizontal offset The conversion formula is:

[0126] (6)

[0127] in Use camera pixel dimensions to ensure conversion accuracy ≤ ±1mm.

[0128] (3) Corrected vector output: The ground offset (δ) x δ y As a visual correction vector, it is output to the flight control system to adjust the UAV's attitude.

[0129] Step 3.5: Pose Fusion and Accuracy Verification

[0130] (1) Dynamic weighted pose fusion strategy: A dynamic weighted fusion algorithm based on matching confidence is adopted to combine the visual correction vector with the GNSS-IMU fusion positioning result to update the UAV model coordinates. :

[0131] (7)

[0132] Among them, weight The system is dynamically adjusted based on the matching confidence level. The specific rules are as follows:

[0133] Matching accuracy : Visual correction is the primary method to enhance the sensitivity of fine-tuning.

[0134] : A balanced integration of visual and GNSS technologies, balancing accuracy and stability;

[0135] : Reduce visual weight and minimize the impact of error correction;

[0136] : Visual corrections are paused, and GNSS-IMU positioning is relied upon exclusively to avoid ineffective corrections.

[0137] (2) Iterative correction termination condition:

[0138] a. Accuracy constraint: After a single fusion, calculate the updated projection points. With the stakeout point Overall deviation :

[0139] (8)

[0140] like The high precision requirement has been met, so the iteration is terminated directly.

[0141] b. Number constraint: If Repeat steps 3.1-3.4 for iterative correction, with a maximum of 3 iterations. If after 3 iterations... If the layout accuracy requirement is met, the iteration terminates; if after 3 iterations... This triggers an alarm mechanism and switches to a GNSS-dominated + manual fine-tuning mode.

[0142] c. Final target marking: After the iteration terminates, the UAV hovers above the stakeout point for 3 seconds to re-collect and verify the coordinate deviation, confirming... Afterwards, the ground station sends a marking command, and the high-pressure spraying device completes the ground target point spraying according to the preset pattern (cross shape).

[0143] Step 3.6: Installation and configuration of the high-definition camera and high-pressure spraying device, see details below. Figure 8

[0144] (1) Installation requirements for high-definition cameras:

[0145] a. Installation location: Fixed in the center area directly below the drone fuselage (horizontal projection deviation from the RTK positioning antenna ≤ ±5cm), using a "vibration damping bracket + bolt fastening" structure. The bracket has 3 layers of rubber vibration damping pads (5mm thick, Shore hardness 50°) built in to reduce the impact of drone propeller vibration on camera imaging.

[0146] b. Angle calibration: After installation, adjust the camera attitude using a level and laser calibrator to ensure that the angle between the camera optical axis and the vertical direction (Z-axis) of the UAV is ≤ ±0.5°, that is, the optical axis is perpendicular to the ground; after calibration, fix it with a lock nut to prevent angle deviation during operation;

[0147] c. Unobstructed field of view: There are no structural components obstructing the view below and around the camera lens, and the lens surface is equipped with an anti-fog coating protective cover to ensure that there are no blind spots or blurs in image acquisition during operation.

[0148] (2) Installation requirements for high-pressure spraying equipment:

[0149] a. Installation location: Fix it around the high-definition camera, ensuring that the center of gravity of the device is aligned with the central axis of the drone, so as not to affect flight stability;

[0150] b. Nozzle position: Eight nozzles are set around the high-definition camera, four on the main axis and four at the diagonal points. The outlet direction is parallel to the camera's optical axis (all perpendicular to the ground). The pigment pipeline of the spraying device is arranged along the outside of the high-definition camera and uses a high-pressure resistant flexible hose (pressure ≥1MPa). The pipeline interface is sealed with a sealing gasket to prevent pigment leakage under high pressure.

[0151] Step 3.7: Control parameters of the high-pressure spraying device

[0152] (1) Core control parameters:

[0153] a. Printing pressure: Set to 0.3~0.5MPa, and dynamically adjust according to the actual flight altitude of the drone: 0.4MPa when the flight altitude is 120~150m, 0.5MPa when the altitude is 150~180m, and 0.3MPa when the altitude is ≤120m, to ensure uniform pigment atomization and accurate coverage.

[0154] b. Pigment viscosity: Suitable range 20~50mPa·s (at ambient temperature of 25℃). Use fast-drying environmentally friendly pigments (drying time ≤5 minutes) to avoid rain or damp ground causing the markings to become blurred.

[0155] c. Nozzle diameter: Use 0.8~1.2mm stainless steel nozzles to ensure stable pigment spray flow (0.5~1.0mL / s), without dripping or clogging;

[0156] d. Printing time: The printing time for a single print is 0.8~1.2 seconds, which is dynamically adjusted according to the size of the print pattern to ensure complete coverage of the pattern without diffusion;

[0157] e. Spray-painted patterns: fixed as cross or circle, cross pattern with horizontal and vertical line length ≥10cm and line width ≥1cm; circle pattern with diameter ≥10cm and deviation between pattern center and layout point ≤±3cm.

[0158] (2) Triggering and stopping control logic:

[0159] a. Triggering conditions: If the overall deviation ΔS collected from three consecutive data acquisitions within 3 seconds of the drone hovering is ≤ ±3cm and the IMU attitude is stable (roll angle and pitch angle fluctuation ≤ ±0.5°), the ground station will automatically send a printing trigger command.

[0160] Stop conditions: After the printing time reaches the set value, the high-pressure air source will be automatically cut off, and the solenoid valve of the pigment pipeline will be closed to prevent excess pigment leakage; if the drone's attitude fluctuates more than ±1° during the printing process, the printing will be paused immediately, and the attitude will be re-verified before triggering again.

[0161] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. An unmanned aerial vehicle automatic lofting method based on real scene three-dimensional model and multi-source data fusion, characterized in that, Includes the following steps: Step 1: Construct a realistic 3D model and unify the coordinate system; Step 2: Based on the multi-source fusion positioning of GNSS-RTK and IMU, realize the real-time conversion of UAV geodetic coordinates to the coordinates of the real scene 3D model and ground projection; Step 3: Using a visual-assisted refinement algorithm, the UAV pose is corrected by matching features between real-time images and the model rendering view, and ground target marking is completed.

2. The method of claim 1, wherein, Step 1 includes: Step 1.1: Use a drone equipped with an RTK module to collect oblique photography data, and control the flight altitude, overlap and exposure time; Step 1.2: Construct a real-world 3D model with a resolution ≤0.02m through aerial triangulation, ground control point constraints, and dense point cloud generation; Step 1.3: Unify the model coordinate system with the design coordinate system, load the lofting points and verify their consistency.

3. The method of claim 2, wherein, In step 1.1, the flight altitude is calculated using the target ground resolution, camera focal length, and pixel size. The photographic baseline is calculated using the long side of the image, the forward overlap, the flight altitude, and the focal length. The exposure time was calculated using the photographic baseline and flight speed. The overlap should be controlled to be ≥80%.

4. The method of claim 2, wherein, Step 1.2 includes: Image distortion correction and automatic matching of corresponding points are performed on oblique photogrammetry data; Aerial triangulation was performed using bundle adjustment of the regional network, and the exterior orientation elements of the image were output. The exterior orientation elements of the image are constrained and adjusted by importing ground control points; Based on the adjusted results, a dense point cloud is generated, and the real-scene 3D model is constructed.

5. The method of claim 1, wherein, Step 2 includes: Step 2.1: Use the extended Kalman filter algorithm to fuse GNSS-RTK and IMU data to output the real-time pose of the UAV; Step 2.2: Convert the geodetic coordinates to model plane coordinates using Gauss-Kruger projection, and perform elevation transformation using the geoid undulation model; Step 2.3: Perform ray-mesh intersection based on DSM, calculate the deviation between the UAV projection point and the lofting point, and set the flight control logic.

6. The method of claim 5, wherein, In step 2.3, the process of ray-mesh intersection based on DSM includes: Traverse the DSM mesh surfaces of the real-world 3D model and determine the intersection points of ray R with the mesh surfaces. When the intersection point satisfies the condition of "located inside the mesh surface and the elevation value deviating from the DSM mesh elevation by ≤ ±1cm", then the intersection point is determined as the ground projection point.

7. The method of claim 1, wherein, Step 3 includes: Step 3.1: Acquire real-time downward-looking images from the UAV and perform distortion correction, grayscale normalization, and region cropping; Step 3.2: Render the virtual view based on the UAV pose and the parameters of the real-world 3D model; Step 3.3: Extract features using ORB or SuperPoint algorithms and perform matching and purification; Step 3.4: Calculate the pixel offset and convert it into a ground correction vector; Step 3.5: Use a dynamic weighted fusion algorithm based on matching confidence to combine the visual correction vector with the GNSS-IMU fusion positioning result, update the UAV model coordinates, and iterate until the accuracy requirements are met.

8. The method of claim 7, wherein, In step 3.5, the specific rules of the dynamic weighted fusion algorithm based on matching confidence are as follows: When the matching accuracy , the weight value ; When the weight value ; When the weight value ; when hour, Visual corrections are paused, and positioning relies solely on GNSS-IMU.

9. The method according to claim 7, characterized in that, In step 3.5, the termination condition for iterative correction is: If the overall deviation between the updated projection point and the stakeout point If, the iteration is terminated directly, then... Then iterative correction is performed, with a maximum of 3 iterations. If after 3 iterations... Terminate the iteration; if after 3 iterations This triggers an alarm mechanism, switching to GNSS-dominated and manual fine-tuning mode.

Images