A high-precision three-dimensional modeling method and system for unmanned aerial vehicles for space measurement

Abstract

This invention relates to the field of 3D modeling technology, specifically to a high-precision 3D modeling method and system for UAVs used in spatial measurement. The method includes: simultaneously acquiring laser point clouds, multi-view images, and combined navigation data; forming an initial fused dataset after spatiotemporal alignment and coordinate transformation; extracting point cloud edges and image edges and establishing a corresponding relationship; classifying edges into three categories based on the ratio of point cloud confidence to image confidence; employing mapping strategies of correcting textures based on point clouds, correcting point clouds based on images, or retaining dual references for the three categories of edges, iteratively constructing a local elastic deformation field, updating only the parameters corresponding to the category of edges with the largest residual during iteration; correcting the point cloud and image based on the converged deformation field to generate a 3D reality model with reference labels. This invention effectively improves the misalignment caused by different edge definitions between laser point clouds and images, making the geometry and texture of the 3D model more consistent at structural edges.

Description

Technical Field

[0001] This invention relates to the field of 3D modeling technology, and more specifically, to a high-precision 3D modeling method and system for UAVs used in spatial measurement. Background Technology

[0002] 3D modeling is a core technology for engineering surveying, infrastructure inspection, and digital twin construction. In practical applications, for spatial measurement needs, drones are usually equipped with lidar and tilt cameras to obtain high-precision geometric point clouds and image textures of the target through multi-sensor fusion, thereby generating a real-scene 3D model that can be used for quantitative measurement.

[0003] In the process of generating a realistic 3D model, the point cloud and image data collected by the two sensors on the drone, namely the lidar and the tilt camera, need to be fused and registered based on a unified spatial reference. Existing technologies generally unify point clouds and images into the same coordinate system by pre-calibrating sensor installation parameters and UAV navigation trajectory data, minimizing the weighted sum of reprojection error and point cloud geometric residual, and optimizing relevant parameters to achieve global and accurate alignment between the two.

[0004] However, the above scheme assumes that the spatial positioning of the laser point cloud and the image is consistent at the same physical edge, and there are only translation and rotation deviations between the two that can be eliminated by simple adjustments.

[0005] However, in practical applications, the laser spot has a certain diameter. When it happens to illuminate the edge of the structure, part of the light falls on one side of the edge and the other part falls on the other side. The sensor receives a mixed echo and cannot accurately determine the surface to which the point belongs. Therefore, a band-shaped area with a width approximately equal to the diameter of the laser spot will be formed near the edge. The point cloud in this area does not belong to the left or the right side, but is scattered between the two sides, making it impossible to determine a clear dividing line.

[0006] In contrast, the pixels on both sides of the edge of the image show obvious differences in brightness, which can accurately locate a clear dividing line.

[0007] This difference in edge definition causes a systematic misalignment in the edge region after point cloud and image fusion, which is difficult to eliminate with conventional optimization algorithms, thus affecting the consistency of spatial measurement benchmarks.

[0008] There is an urgent need for a high-precision 3D modeling method for UAVs that unifies the definition benchmarks of laser point clouds and image edges. Summary of the Invention

[0009] One objective of this invention is to provide a high-precision 3D modeling method for UAVs used in spatial measurement. This method categorizes edge regions based on the response characteristics of laser point clouds and imagery at their respective edges, employs differentiated alignment strategies, and iteratively optimizes these methods to achieve benchmark unification between the two types of data at structural edges. This solves the problems mentioned in the background art, namely: When drones are equipped with lidar and tilt cameras for high-precision 3D modeling, the laser point cloud and the image are misaligned due to different edge definition mechanisms, which ultimately causes contradictions between the geometric structure and texture expression of the edge region in the 3D model.

[0010] To achieve the above objectives, this 3D modeling method includes the following steps: S1. Collect laser point cloud and multi-view images of the measured area using drones; S2. Extract point cloud edges from laser point clouds, extract image edges from multi-view images, project each point cloud edge onto the image plane and perform nearest neighbor matching with the image edges to establish a corresponding relationship; Calculate the point cloud confidence and image confidence for each pair of edges with the same name, and classify the edges into three categories: point cloud-dominated, image-dominated, and balanced based on the ratio of the two. S3. For point cloud-dominated edges, establish a thin plate spline interpolation model based on the point cloud edges to correct the image texture coordinates. For image-dominant edges, a radial basis function network is constructed based on the image edges to correct the point cloud positions; For balanced edges, two implicit surfaces are constructed based on the point cloud and the image, respectively, and the normal distance field between the two is used as the local offset vector. Establish a local elastic deformation field and initialize it to zero; Calculate the reprojection and backprojection residuals for all edges, and update and smooth the constraints only for the deformation field parameters corresponding to the edge type with the largest residual. If there is a residual greater than or equal to the preset threshold, the residual is recalculated and the constraints are updated and smoothed until they are satisfied, at which point the local elastic deformation field and edge reference labels are output. S4. Based on the local elastic deformation field, perform geometric mapping correction on the point cloud and image, construct a triangular mesh model, and map the edge reference labels to the attribute fields of each triangular facet to generate a 3D real scene model with edge reference labels.

[0011] Unlike conventional methods that use uniform transformation parameters to fuse point clouds and images, this solution employs a classification mechanism to address the response differences between the two types of sensors at edges. Without distinguishing edge types, point cloud-dominant and image-dominant regions will mutually constrain each other under the same set of transformation parameters, leading to a misalignment between geometry and texture at the edges. Therefore, this solution first categorizes edges into three types based on confidence level: point cloud-dominant, image-dominant, and balanced. For the first two types, a unidirectional correction strategy is adopted, adjusting only the side with lower confidence to avoid residual bias caused by bidirectional compromise. For the third type, dual benchmarks are retained for subsequent selection as needed. During the iteration process, only the parameters of the type with the largest residual are updated to prevent error coupling between different edge types. Finally, the classification information is embedded into the triangular mesh model through edge benchmark labels, ensuring clear benchmark traceability for spatial measurement results. Without the above classification and differential mapping, the geometric and texture positions of edge regions will continuously exhibit systematic deviations, and the measurement reliability of the model at structural boundaries cannot be guaranteed.

[0012] Based on this, the edges are classified into three categories: point cloud-dominated, image-dominated, and balanced. Specifically, this is achieved by comparing the ratio of point cloud confidence to image confidence with a classification threshold. The classification threshold is dynamically determined by the ratio of the angle between the planes on both sides of the point cloud edge to the gray-level gradient magnitude of the image edge. The angle between the planes on both sides of the point cloud edge is calculated by fitting the normal vectors of the local planes on both sides of the edge, and the gray-level gradient magnitude of the image edge is calculated by calculating the gray-level change rate of pixels at the edge.

[0013] In another technical solution, the reprojection residual is obtained by projecting the point cloud edges onto the image plane after deformation field transformation and calculating the Euclidean distance between the projected point and the corresponding image edge point. The backprojection residual is obtained by projecting the image edges onto the point cloud space after inverse deformation field transformation and calculating the Euclidean distance between the projected point and the corresponding point cloud edge point.

[0014] This technical solution, if a fixed value is used for the classification threshold, cannot adapt to the fluctuations in data quality in different regions. For example, edges in sparse point clouds may be more reliable than the point cloud itself, even if the image confidence is low. A fixed threshold would incorrectly classify such edges as point cloud-dominated. Therefore, a dynamic threshold is used, allowing the classification criteria to automatically adjust according to the angle between the two planes of the point cloud and the rate of change of image grayscale. A larger plane angle indicates a more prominent edge, and a smaller grayscale gradient indicates a more blurred texture. The threshold changes accordingly, making the classification results more consistent with the actual data quality. The calculation of residuals clarifies the physical meaning of iterative optimization. The reprojection residual reflects the degree of deviation of the image texture from the point cloud reference, while the backprojection residual reflects the degree of deviation of the point cloud position from the image reference. Together, they constitute the basis for two-way verification. If only one-way residuals are calculated, misalignment in one direction can only be detected, while misalignment in the other direction cannot be detected. The iterative process is prone to converge to local optima rather than global consistency.

[0015] The second objective of this invention is to provide a high-precision 3D modeling system for UAVs used in space measurement, including a data acquisition and registration module, an edge extraction and classification module, a deformation field construction and optimization module, a model reconstruction and benchmark embedding module, and an adaptive space measurement module. The data acquisition and registration module is used to simultaneously acquire laser point cloud, multi-view image and integrated navigation data, and form an initial fusion dataset after spatiotemporal alignment and coordinate transformation. The edge extraction and classification module is used to extract point cloud edges and image edges from the initial fusion dataset and establish a corresponding relationship, calculate the point cloud confidence and image confidence, and classify the edges into three categories according to the ratio of the two. The deformation field construction and optimization module is used to establish a local elastic deformation field based on the classification results, and output the local elastic deformation field and edge reference labels through iterative optimization. The model reconstruction and reference embedding module is used to correct the point cloud and image according to the local elastic deformation field, reconstruct the triangular mesh model and embed edge reference labels to generate a three-dimensional real scene model with reference labels. The adaptive spatial measurement module is used to select a measurement reference based on the relationship between the measurement path and the edge direction, and output a measurement report.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention employs a thin-plate spline interpolation model to correct image texture coordinates at point cloud-dominated edges. This model ensures that the corrected image texture remains smooth and continuous near the edges, avoiding local distortion. At image-dominated edges, a radial basis function network is used to correct point cloud positions. This network uses edge-matching points as the central function, making the control of the point cloud deformation field more localized and reducing long-distance interference. At balanced edges, the offset distribution between the point cloud and the image is quantified using the normal distance field of a dual-reference implicit surface, providing a reference for the difference between two references for subsequent measurements. The above algorithm models are adapted to the physical characteristics of different edge types, improving the targeting and stability of the correction process.

[0017] 2. This invention addresses the high-precision 3D modeling requirements in UAV spatial measurement. Based on the data quality of point cloud edges and image edges, it classifies edges into three categories: point cloud-dominated, image-dominated, and balanced. This enables subsequent mapping strategies to differentiate and process these categories, improving the consistency of multi-source data fusion at structural edges.

[0018] 3. In the iterative optimization stage of 3D modeling, this invention only updates the deformation field parameters corresponding to the edge with the largest residual at each step, while keeping the other two types unchanged. This unidirectional driving method reduces the mutual interference between point cloud position correction and image texture correction, which is conducive to the stable convergence of the deformation field.

[0019] 4. This invention uses a two-way cross-validation method of reprojection residual and backprojection residual to control the accuracy of the deformation field. The reprojection residual constrains the image texture to converge with the point cloud geometry, while the backprojection residual constrains the point cloud geometry to converge with the image texture. The two work together to make the geometric position and texture representation of the 3D model at the edge of the structure more consistent, providing a unified benchmark for spatial measurement. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the overall process of the high-precision 3D modeling method for UAVs used in space measurement according to the present invention; Figure 2 This is a flowchart of step S2 of the present invention; Figure 3 This is a flowchart of step S3 of the present invention; Figure 4 This is a schematic diagram of the overall structure of the high-precision 3D modeling system for UAVs used in space measurement according to the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Currently, when performing high-precision 3D modeling on UAVs equipped with LiDAR and oblique cameras, the different edge definition mechanisms of LiDAR point clouds and images lead to fusion misalignment, ultimately causing contradictions between the geometric structure and texture representation of edge regions in the 3D model. One objective of this invention is to provide a high-precision 3D modeling method for UAVs used in spatial measurement. See [link to invention]. Figure 1 As shown, it includes the following steps: S1. Collect laser point cloud and multi-view images of the measured area using drones; S2. Extract point cloud edges from laser point clouds, extract image edges from multi-view images, project each point cloud edge onto the image plane and perform nearest neighbor matching with the image edges to establish a corresponding relationship; Calculate the point cloud confidence and image confidence for each pair of edges with the same name, and classify the edges into three categories: point cloud-dominated, image-dominated, and balanced based on the ratio of the two. S3. For point cloud-dominated edges, establish a thin plate spline interpolation model based on the point cloud edges to correct the image texture coordinates. For image-dominant edges, a radial basis function network is constructed based on the image edges to correct the point cloud positions; For balanced edges, two implicit surfaces are constructed based on the point cloud and the image, respectively, and the normal distance field between the two is used as the local offset vector. Establish a local elastic deformation field and initialize it to zero; Calculate the reprojection and backprojection residuals for all edges, and update and smooth the constraints only for the deformation field parameters corresponding to the edge type with the largest residual. If there is a residual greater than or equal to the preset threshold, the residual is recalculated and the constraints are updated and smoothed until they are satisfied, at which point the local elastic deformation field and edge reference labels are output. S4. Based on the local elastic deformation field, perform geometric mapping correction on the point cloud and image, construct a triangular mesh model, and map the edge reference labels to the attribute fields of each triangular facet to generate a 3D real scene model with edge reference labels.

[0023] Step S1 of this invention involves simultaneously acquiring laser point clouds, multi-view images, and POS data of the measured area using a laser scanner, tilt camera, and integrated navigation device mounted on a UAV. The acquired multi-source data is then spatiotemporally aligned and initially registered to provide a unified data foundation for subsequent cross-modal edge processing. Specifically, the steps include: S101: Simultaneously collects laser point clouds, multi-view images, and POS data using a laser scanner, tilt camera, and integrated navigation device mounted on the drone.

[0024] First, the laser scanner, tilt camera, and integrated navigation device are respectively mounted and fixed on the drone body. Specifically, the laser scanner is mounted on the bottom of the drone, with its laser emission window facing vertically downwards or slightly tilted forwards to ensure that the laser pulse can effectively cover ground targets; The tilt camera is mounted below the center of the drone and connected to the body via a vibration-damping gimbal to reduce the impact of flight vibration on image quality. The integrated navigation device is installed near the center of gravity of the UAV, and its coordinate axis direction is kept consistent with the coordinate axis direction of the UAV body to facilitate subsequent attitude calculation and coordinate transformation.

[0025] After the three sensors are installed, their relative spatial positions are calibrated using high-precision measurement methods. These include the spatial offset vector between the laser scanner and the integrated navigation device, the spatial offset vector between the tilt camera and the integrated navigation device, and the attitude positioning angle error between the three. These calibration sensor installation parameters will be used for subsequent coordinate alignment between the sensors.

[0026] Before the flight mission, a time synchronization calibration is performed on the laser scanner, tilt camera, and integrated navigation device. The purpose is to ensure that the three sensors use the same time reference during subsequent data acquisition; otherwise, time misalignment will occur during data alignment, affecting the fusion accuracy.

[0027] Time synchronization is achieved by simultaneously connecting the laser scanner, tilt camera, and integrated navigation device to the same high-precision clock source. Specific methods include GPS second pulse timing or precise time protocol, and the choice between these two methods is made based on the interface type supported by the device hardware.

[0028] After takeoff, the integrated navigation system begins recording the UAV's flight status at a fixed frequency. Its output data includes the UAV's current three-dimensional coordinates, as well as its pitch, roll, and yaw angles. In addition, the integrated navigation system precisely timestamps each laser pulse emitted by the laser scanner and each exposure by the tilt camera, enabling the data from different sources to be linked together during subsequent spatiotemporal alignment.

[0029] The laser scanner continuously emits laser pulses towards the ground at a pre-set scanning frequency. Each pulse generates one or more echoes upon hitting the target surface, which the scanner receives and records. Each echo corresponds to a sampling point, which records the following information: the three-dimensional spatial coordinates of the laser point in the scanner's body coordinate system, the echo intensity value, and the echo number when multiple echoes are present.

[0030] All sampling points are aggregated to form the raw point cloud data. It should be noted that when there is vegetation or semi-transparent medium in the survey area, a single laser pulse may generate multiple echoes. The first echo usually corresponds to the vegetation canopy or the surface of the object, while the last echo may penetrate the gaps to reach the ground or the underlying structure.

[0031] Tilt cameras typically consist of multiple lenses, with a common configuration being five lenses. These five lenses face forward, backward, left, right, and directly below, respectively, and are exposed simultaneously at the same time, capturing images from five different angles in a single shot. The advantage of this approach is that it can simultaneously acquire texture information from the top surface and all four sides of the target, avoiding texture loss caused by a single viewpoint. When using a tilt camera, it's also necessary to record the timestamp of each exposure and the corresponding drone posture, saving this information along with the image itself.

[0032] It should be noted that the flight path must be planned before takeoff, and the path design must ensure that the scanning strip of the laser scanner and the exposure area of ​​the tilt camera can completely cover the entire survey area. The flight altitude and speed should be determined based on factors such as the terrain undulations and building heights of the survey area. The flight altitude directly determines the ground sampling spacing of the point cloud and the spatial resolution of the image: the lower the flight altitude, the denser the point cloud and the higher the image resolution, but the coverage width of a single flight strip decreases accordingly, increasing the operation time; the higher the flight altitude, the higher the coverage efficiency, but the data accuracy decreases accordingly.

[0033] In actual operations, a balance must be struck between data accuracy and operational efficiency to ensure that the point cloud density and image overlap meet the requirements of subsequent processing. In addition, the image overlap is usually set to a forward overlap rate of no less than 70% and a lateral overlap rate of no less than 60%, so that the same ground target will be captured in multiple adjacent images, making it easier to find corresponding image points during subsequent matching.

[0034] S102: Perform spatiotemporal alignment and initial coordinate alignment on the acquired laser point cloud, multi-view images and integrated navigation device data.

[0035] After data acquisition, the laser point cloud, multi-view imagery, and integrated navigation device data are imported into the processing system. Specifically, based on the timestamp carried by each data record, the point cloud and imagery are first associated with the corresponding integrated navigation data. Data segments with mismatched timestamps or obviously abnormal attitude information are directly removed, and the remaining data proceeds to subsequent processing steps.

[0036] Furthermore, initial coordinate alignment is performed based on the sensor installation parameters obtained through prior calibration.

[0037] Suppose the coordinates of a point measured by the laser scanner in its body coordinate system are... At the same time, the position in the navigation coordinate system output by the integrated navigation device is The attitude rotation matrix is Then the laser point is transformed to coordinates in the navigation coordinate system. The calculation formula is as follows: ; In the formula, d L This is the spatial offset vector of the laser scanner relative to the integrated navigation device; R err The rotation matrix corresponding to the attitude placement angle error between the laser scanner and the integrated navigation device; The meaning is: first, translate the laser point from the coordinate system of the laser scanner to the origin of the coordinate system of the integrated navigation device, and then use the rotation matrix R... err Transform it into the coordinate system of the integrated navigation device, and finally... Transform to the navigation coordinate system, and add the position of the integrated navigation device. This gives the coordinates of the laser point in the navigation coordinate system.

[0038] For images acquired by a tilted camera, the same coordinate transformation principle is used: Let the spatial direction vector of a pixel in the camera's own coordinate system be... Based on the pre-calibrated camera orientation parameters and installation parameters, they are transformed into direction vectors in the navigation coordinate system. Combined with the camera exposure position recorded by the integrated navigation device at that moment. This allows us to calculate the initial values ​​of the exterior orientation elements of the image, including the three-dimensional coordinates of the projection center and three attitude angles.

[0039] After the coordinate transformation described above, the laser point cloud and multi-view imagery are unified into the same navigation coordinate system, forming an initial fusion dataset. This initial fusion dataset contains the three-dimensional spatial coordinates and reflection intensity of each point cloud point, as well as the initial values ​​of the exterior orientation elements and pixel matrix of each image. This initial fusion dataset is then passed to the next step as input data for edge extraction and classification.

[0040] In step S2 of this invention, edge features are extracted from the laser point cloud and multi-view images in the initial fusion dataset, and the extracted edges are classified into types to provide a classification basis for the subsequent construction of the local elastic deformation field.

[0041] S201: Extract point cloud edge segments from the laser point cloud in the initial fused dataset.

[0042] The laser point cloud is obtained from the initial fusion dataset. The laser point cloud is used as input. The point cloud consists of several discrete point cloud points. Each point cloud point contains its three-dimensional spatial coordinates and reflection intensity information in the navigation coordinate system.

[0043] First, a neighborhood analysis algorithm based on the rate of change of the normal vector is used to extract edge points. Specifically, each point cloud point P is traversed. i Search for the set of neighboring points within its neighborhood radius r. Then, principal component analysis is used to calculate the normal vector n at that point. i : For neighboring point sets Construct the covariance matrix C i : ; In the formula, It is the geometric center of the set of neighboring points, that is, the average value of the coordinates of all neighboring points; This represents the number of neighboring points.

[0044] Furthermore, regarding the covariance matrix C i By performing eigenvalue decomposition, the eigenvector corresponding to the smallest eigenvalue is the normal vector n of that point. i .

[0045] Specifically, for each neighboring point within the neighborhood Calculate its normal vector n j With n i The included angle θ ij : ; If θ ij Exceeding the preset threshold θ th Then the neighboring point P j Marked as candidate edge points. Threshold θth The value ranges from 15 degrees to 30 degrees, and the specific value is determined based on the point cloud noise level and the topographic features of the survey area. The neighborhood search radius r is determined based on the average point spacing d of the point cloud, and is usually set to 10 to 20 times d, or directly set to 0.5 m to 2.0 m depending on the survey area.

[0046] It should be noted that, based on the above edge point detection, the candidate edge points are clustered. A region growing method is used to merge spatially adjacent edge points with consistent normal vector trends into the same edge segment. The seed point for region growing is selected from the candidate edge points with the smallest curvature. The growth criterion is that the Euclidean distance between the neighboring point and the current point is less than a threshold d. th And the angle between the normal vectors is less than the threshold θ growth ;d th θ is usually set to 2 to 3 times d, growth The temperature is typically set between 5 and 15 degrees. After growth is complete, fewer than N points are removed. min For excessively short line segments, retain the effective edge segments, N min Typically, 5 to 10 points are set.

[0047] Based on this, for each extracted point cloud edge segment E L Calculate the following edge properties respectively: ① Transition zone width w L Equivalent sampling is performed on both sides of the edge segment along the normal direction, and local planes on both sides of the edge are fitted respectively. Then, the vertical distance between the two planes is measured point by point along the normal direction from one end of the edge to the other end to obtain a distance value sequence. Then, the length interval corresponding to the change of distance value from the beginning to the stabilization of the sequence is taken, and the length of the interval is taken as the width of the transition zone.

[0048] ② Point cloud density ρ L : Count the total number N points in the point cloud contained on the edge segment. L Measure the actual length of the edge segment. The point cloud density is N. L and The ratio is calculated using the following formula: .

[0049] ③ Angle α between the two planes L : Fit the local planes on both sides of the edge to obtain the unit normal vectors n of the two planes respectively. L1 and n L2 The angle between the two planes is the angle between the two normal vectors, and its mathematical expression is: .

[0050] ④ Direction vector vL Principal component analysis is performed on all point cloud points on the edge segment. The specific steps are as follows: Let the point set be First, calculate the average coordinates of all points in the point set. : ; Then construct the covariance matrix C. L : ; Finally, the covariance matrix C is analyzed. L Eigenvalue decomposition yields three eigenvalues. And its corresponding eigenvector, the eigenvector corresponding to the largest eigenvalue λ1 is the direction vector v of the edge in three-dimensional space. L .

[0051] The transition band width w calculated above L Point cloud density ρ L α, the angle between the two planes L , directional vector v L The point cloud edge segments are associated with and stored to form a set of point cloud edge segments. The result is passed to step S203 for name matching.

[0052] S202: Extract image edge lines from multi-view images.

[0053] Multi-view images are acquired from the initial fusion dataset. Using these images as input, edge extraction is performed on each image separately. Edge extraction employs the Canny operator to detect edge pixels in the images. The specific steps are as follows: First, the image is smoothed by Gaussian filtering. Then, the gradient magnitude and gradient direction of each pixel are calculated, and non-maximum suppression is applied to the gradient magnitude. Finally, edge pixels are connected by double threshold detection to obtain a binarized edge image.

[0054] Based on this, a sub-pixel localization method is used to refine the edge position. Specifically, a quadratic surface is fitted within the neighborhood of the edge pixels, and the extreme points of the surface are used as the edge points with sub-pixel precision. These extreme points are then connected sequentially to form the image edge line E. I .

[0055] For each extracted image edge line E I Calculate the following edge properties respectively: ① Gray-scale step amplitude Sample several pixels perpendicular to the edge on both sides of the edge line, calculate the average gray level of the sampled points on both sides, and the gray level step amplitude is the absolute value of the difference between the average gray levels on both sides. The calculation formula is: ; In the formula, m is the number of sampling points on one side; and These are the gray values ​​of the u-th sampling points on the left and right sides of the edge, respectively.

[0056] ②Edge contrast direction Gray-level sampling is performed along the direction perpendicular to the image edge, and then the rate of change of gray level in that direction is calculated. Specifically, for any point on the edge, let the gray-level function at that point be... The directional derivative along the normal direction is: ; In the formula, θ is the angle of the edge tangent at that point.

[0057] Then, calculate the directional derivatives at that point in different directions, and take the direction with the largest absolute value of the directional derivative as the edge contrast direction at that point. Finally, take the average of the edge contrast directions calculated for all points along the entire edge line to obtain the edge contrast direction for the entire edge line. .

[0058] ③ Texture richness The distribution of pixel gray levels within the edge neighborhood is statistically analyzed and quantized using information entropy. Specifically, let p be the probability of gray level k appearing within the edge neighborhood. k K represents the total number of gray levels and the texture richness. The calculation formula is: .

[0059] The grayscale step amplitude calculated above Edge contrast direction Texture richness The image edge lines are stored together to form an image edge line set. The result is passed to step S203 for name matching.

[0060] S203: Perform name matching between point cloud edges and image edges, and perform adaptive classification based on data quality.

[0061] Specifically, the set of point cloud edge segments obtained from S201 The image edge line set obtained from S202 As input, the initial exterior orientation elements obtained in S1 are used for same-name matching.

[0062] First, edge projection and matching are performed on the set of point cloud edge segments. Each edge segment E in LBased on its corresponding exterior orientation elements, it is projected onto the image plane to obtain the projected two-dimensional edge line.

[0063] Then, the image edge lines are aggregated. Search for the image edge E that is closest to the projection line. I If the average distance between the two is less than the preset threshold d match Then, establish a candidate homonym correspondence between point cloud edges and image edges. .

[0064] Where, d match The distance between points is determined based on the ground sampling interval (GSD) of the image or the average point spacing (d) of the point cloud. It is usually set to 1 to 2 times the GSD, or 1 to 3 times the d. For example, when the GSD is 0.02m, it can be 0.02m to 0.04m.

[0065] Based on this, the point cloud confidence C is calculated for each pair of corresponding edges. L Image confidence C I .

[0066] The formula for calculating the confidence level of a point cloud is: ; In the formula, ρ L The point cloud density is the point cloud edge segment; w L The width of the transition zone for the point cloud edge segments; α L The angle between the two planes on either side of the point cloud edge segment.

[0067] ρ L with w L The ratio reflects the sharpness of the edges; the larger the ratio, the clearer the point cloud edges. The maximum value is taken when the angle between the two planes is close to 90°, at which point the edge features are most prominent.

[0068] The formula for calculating image confidence is: ; In the formula, The grayscale step amplitude of the image edge line; T I It represents the texture richness of image edge lines, quantified using information entropy, and reflects the complexity of textures within the edge neighborhood.

[0069] Therefore, the image confidence level C I It comprehensively reflects the visual salience of image edges; the higher the value, the more credible the edge is in the image.

[0070] Furthermore, determine the adaptive threshold for classification, the classification threshold. The ratio of the angle between the two planes on either side of the point cloud edge to the image grayscale gradient magnitude is dynamically determined, and its calculation formula is as follows: ; In the formula, α L The angle between the planes on both sides of the point cloud edge segment; Let be the gray-level gradient magnitude at the image edge, i.e., the rate of gray-level change. This is calculated using the Sobel operator. Specifically, let the pixel gray-level function at a point on the image edge be... Calculate the partial derivatives of this point in the x and y directions respectively: ; The gray-level gradient magnitude at that point is .

[0071] For the entire image edge line, the average value of the gray-level gradient magnitude of all points on the edge is taken as the gray-level gradient magnitude of that edge.

[0072] The ratio of the angle between the two planes on the edge of the point cloud to the gray-level gradient magnitude of the image is used as an adaptive threshold, so that the classification criteria are dynamically adjusted according to the local data quality: when the angle between the two planes on the edge of the point cloud is large and the gray-level gradient of the image is small, the threshold is increased and the judgment conditions of point cloud dominance are relaxed accordingly.

[0073] Finally, based on the confidence scores and adaptive thresholds, the edges are categorized into different types: When the ratio of point cloud confidence to image confidence exceeds a threshold multiplied by a preset coefficient k, that is... If so, the edge is determined to be point cloud dominant. When the ratio of image confidence to point cloud confidence exceeds a threshold multiplied by a preset coefficient k, that is... If the above two conditions are not met, it is determined to be image-dominant; if neither of the above two conditions is met, it is determined to be balanced.

[0074] The coefficient k ranges from 1.5 to 2.0 to avoid misclassification when confidence levels are similar.

[0075] The category label is associated with the edge homonym pair and stored to obtain the edge homonym pair set with category label, and then passed to step S3.

[0076] Step S3 of this invention involves constructing a local elastic deformation field based on a set of edge pairs with classification labels. Differentiated mapping strategies are employed for the three types of edges: point cloud-dominated, image-dominated, and balanced edges. The deformation field parameters are optimized through iterative solving driven by a unidirectional approach, and a cross-projection closed-loop feedback is introduced for self-consistency verification. Specifically, the steps include: S301: First, input the set of edge pairs with classification labels, and extract point cloud-dominant edges, image-dominant edges, and balanced edges from the edge pairs.

[0077] For point cloud-dominant edges, the point cloud edges are used as a reference, keeping the point cloud positions unchanged. Matching point pairs are extracted from the point cloud edges. Let P be the coordinates of the point cloud edge point corresponding to the i-th matching point pair in the control point set. i The pixel coordinates of the image edge points are q i A thin-plate spline interpolation model is established to map any pixel q on the image plane to a new position q'. The mapping function is: ; In the formula, For the radial basis functions of the thin plate spline, when Time definition .

[0078] This represents the distance from pixel q to the i-th control point q. i The Euclidean distance; w i The weight coefficient corresponding to the i-th control point is used to control the degree of influence of this control point on surrounding pixels. It is obtained by solving a system of linear equations. We obtain, where K is the kernel matrix, and its element in the i-th row and j-th column is... ; A is a 2×2 affine transformation matrix used to describe global rotation, scaling, and shearing deformation; t is a two-dimensional translation vector; To ensure the smoothness of the interpolation function, a regularization parameter λ is introduced. TPS The objective function is: ; Regularization parameter λ TPS The value is typically between 0.01 and 0.1, and is used to control the curvature of the interpolated surface.

[0079] The original coordinates are replaced with the corrected image texture coordinates q' to complete the texture correction.

[0080] For image-dominant edges, the image edge is used as a reference, while maintaining the image texture coordinates. Matching point pairs on the image edge are extracted. Let the coordinates of the image edge point corresponding to the j-th matching point pair in the control point set be q. j The coordinates of the point cloud edge point are P j A radial basis function network is used to describe the deformation field of the point cloud, transforming any point P in the point cloud space to a new position P'. The transformation function is: ; In the formula, For Gaussian kernel function, d is the kernel width parameter, used to control the influence range of the Gaussian kernel, d is the average point spacing of the point cloud, and the adjustment coefficient k is usually taken as 2 to 5; This represents the distance from point P in the point cloud to the j-th control point P. j The Euclidean distance; V j The weight coefficient corresponding to the j-th control point determines the displacement contribution of that control point to the surrounding point cloud points.

[0081] Replace the original coordinates with the deformed point cloud coordinates P' to complete the point cloud correction.

[0082] For balanced edges, both point cloud references and image references are preserved. Specifically, two implicit surfaces are constructed using the point cloud edges and image edges as inputs, respectively. The surface S spanned by the point cloud is... L Defined as the signed distance field to the edge of the point cloud: For any point x in space, its distance to the edge point set of the point cloud is... The shortest distance is ,but .

[0083] The surface S stretched by the image photometric uniformity L Defined as the gray-level gradient integral along the edge normal: Let the grayscale function of a point on the edge of the image be... Along the normal direction n I Calculate the gray-level gradient integral ,but .

[0084] Based on this, the normal distance field between the two surfaces is calculated. Its expression is: ; In the formula, x is a point on the point cloud surface; x s The point on the image surface corresponding to x along the normal direction; n L Let x be the unit normal vector of the point cloud surface at point x; Dot product It represents the projection length of the line connecting two points onto the normal direction.

[0085] This distance field is stored as a local offset vector for subsequent measurements to select a reference as needed.

[0086] The parameters of the thin plate spline interpolation model for point cloud-dominated edges, the parameters of the radial basis function network for image-dominated edges, and the parameters of the normal distance field for balanced edges are integrated to form an initial parameter set for the local elastic deformation field, which is then passed to step S302.

[0087] S302: Input the initial parameter set of the local elastic deformation field and initialize the deformation field to zero, meaning that there is no local offset between the point cloud and the image in the initial state. Based on this, proceed to the iterative optimization process, with the following specific steps: ① Fix the current deformation field, transform the point cloud edge and the image edge respectively through the deformation field, and then calculate the reprojection residual and backprojection residual after transformation. Suppose that a point P on the point cloud edge is transformed by the deformation field to obtain P', and then project it onto the image plane to obtain the projection point q. proj This point and the corresponding image edge point q I The Euclidean distance between them is the reprojection residual r. reproj : ; Simultaneously, a point q on the image edge is inversely transformed by the deformation field to obtain q', which is then back-projected onto the point cloud space to obtain the back-projected point P. back This point and its corresponding point cloud edge point P L The Euclidean distance between them is the backprojection residual r. back : ; ② Traverse all edges, calculate the maximum residual value for each type of edge, and determine the edge type with the largest residual in the current iteration.

[0088] ③ Determine the update object for this iteration based on the edge type with the largest residual: If the edge is point cloud-dominated, only update the parameters of the thin plate spline interpolation model corresponding to this type of edge, and keep the parameters of image-dominated and balanced edges unchanged; If it is an image-dominant edge, then only the radial basis function network parameters corresponding to this type of edge are updated; If it is a balanced edge, the normal distance field parameters are updated simultaneously, but the point cloud reference and the image reference are calculated independently and are not coupled.

[0089] ④ After the update is completed, the deformation field is regularized and smoothed. The Laplace smoothing method is used to constrain the displacement changes between adjacent control points to suppress high-frequency oscillations and overfitting.

[0090] Smoothed deformation field parameters This can be derived from the following formula: ; In the formula, Let i be the set of neighboring control points; λ is the smoothing coefficient, which ranges from 0 to 1.

[0091] ⑤ Determine if the iteration has converged. Specifically, when the reprojection residuals and backprojection residuals of all edges are less than a preset threshold... When convergence is reached, the iteration stops and the current deformation field parameters are passed to the next step.

[0092] Preset threshold The convergence accuracy of the deformation field is controlled by the average point spacing *d* of the point cloud or the ground sampling spacing *GSD* of the image. It is typically set to 0.5 to 1 times *d*, or 0.5 to 1 times *GSD*. For example, when *d* = 0.01 m, The value can be taken as 0.005 m to 0.01 m; when GSD = 0.02 m, A value of 0.01 m to 0.02 m can be used.

[0093] If the convergence condition is not met and the current iteration number does not exceed the preset maximum iteration number T. max If the number of iterations reaches T, then return to the first step and continue iterating; max If convergence is still not achieved, the iteration is stopped and the current deformation field parameters are passed to the next step, while the identifier of the non-converged edge is output.

[0094] S303: Obtain the deformation field parameters after iterative convergence and organize them into two storage formats: ① Radial basis function coefficient form, that is, the weight coefficient vector corresponding to each control point.

[0095] ② The offset vector form on regular grid nodes, that is, the space is divided into uniform grids according to the set grid spacing, and a three-dimensional offset vector is stored at each grid node.

[0096] Based on this, the three-dimensional offset vector of any point in space is calculated. Specifically, let x be any point in space, and its corresponding offset vector Δ(x) be calculated using radial basis function interpolation: ; In the formula, c i Let be the three-dimensional coordinate position of the i-th control point in space; w i The weight coefficient vector corresponding to the control point determines the degree of deformation contribution of the control point to the surrounding point cloud or image. These are radial basis functions used to calculate the distance from a spatial point x to a control point c. i The influence of the distance between them on the deformation weight; A is a 3×3 affine transformation matrix used to describe global rotation, scaling, and shearing deformation; t is a three-dimensional translation vector.

[0097] Furthermore, output edge reference labels: For point cloud-dominated edges, mark the reference label as the point cloud reference; For image-dominant edges, the reference label is marked as the image reference; For balanced edges, the reference label is marked as dual reference, with the offset distance between the two references.

[0098] The aforementioned deformation field parameters and edge reference labels are then passed to step S304.

[0099] S304: First, obtain the local elastic deformation field parameters and edge reference labels. Substitute the deformation field parameters into the point cloud and image data, and correct the edge of the point cloud and the edge of the image respectively.

[0100] Specifically, for point cloud-dominated edges, the point cloud position is kept unchanged, and the image texture coordinates are elastically distorted according to the deformation field; for image-dominated edges, the image texture coordinates are kept unchanged, and the point cloud position is non-rigidly deformed according to the deformation field; for balanced edges, both correction results are retained.

[0101] Based on this, cross-projection verification is performed through the following steps: ① Reproject the corrected point cloud edges onto the corrected image plane, and obtain the reprojection residual r for each edge according to the reprojection residual calculation method defined in step S302. reproj .

[0102] ② Backproject the corrected image edges onto the corrected point cloud space, and obtain the backprojection residual r for each edge according to the backprojection residual calculation method defined in step S302. back .

[0103] Furthermore, determine whether the residuals meet the preset threshold. Requirements. It should be noted that the preset threshold... Used for cross-projection verification, its value is determined based on the average point spacing d of the point cloud, and is usually set to 0.5 to 1.5 times d, or it can be combined with the convergence threshold in step S302. The values ​​are the same. For example, when d = 0.01m, A value of 0.005 m to 0.015 m can be used.

[0104] If all edges r reproj and r back All are less than the preset threshold If the deformation field meets the self-consistency requirement, the deformation field parameters and edge reference labels are passed to step S4.

[0105] If the residual at any edge exceeds the threshold If the edge region is not found, a local rematch is triggered. Specifically, the edge region is marked, and the edge matching in step S203 and the type mapping in step S301 are re-executed only for this region, without changing the deformation field parameters of other regions. After the rematch is completed, the iterative solution in step S302 and the cross-projection verification in this step are executed again until the residual of the edge meets the threshold requirement.

[0106] The deformation field parameters and edge reference labels that finally meet the self-consistency requirements are passed to step S4.

[0107] Step S4 of the present invention is to correct the point cloud and image according to the local elastic deformation field, reconstruct the triangular mesh model, and embed the edge reference label into the model attribute.

[0108] S401: Obtain the local elastic deformation field parameters and edge reference labels that meet the self-consistency requirements from step S3, and apply the deformation field parameters to the point cloud and image data respectively. The specific correction method is determined according to the edge type.

[0109] For point cloud-dominated edges, the point cloud position remains unchanged, and only the image texture coordinates are corrected.

[0110] Specifically, let the original coordinates of a pixel on the image corresponding to the edge be q. Calculate the offset vector Δ(q) of this point using the deformation field interpolation function established in step S303. Then, the corrected pixel coordinates q' are: ; Based on this, all pixels within the edge region are traversed, and the corrected coordinates are calculated according to the above formula to obtain the corrected image data after elastic distortion.

[0111] For image-dominant edges, the image texture coordinates are kept unchanged, and only the point cloud positions are corrected.

[0112] Specifically, let the original coordinates of a point in the point cloud corresponding to the edge be P. Calculate the offset vector Δ(P) of this point using the deformation field interpolation function established in step S303. Then, the corrected point cloud coordinates P' are: ; Based on this, all point cloud points within the edge region are traversed, and the corrected coordinates are calculated according to the above formula to obtain the corrected point cloud data after non-rigid deformation.

[0113] For balanced edges, both point cloud correction results and image correction results are retained. Point cloud correction is performed according to the image-dominant formula, and image correction is performed according to the point cloud-dominant formula. The two are calculated independently and stored in a dual-reference format.

[0114] The corrected point cloud data, the corrected image data, and the edge reference labels are transferred together to step S402.

[0115] S402: Obtain the corrected point cloud data, which has undergone non-rigid deformation based on the local elastic deformation field. Each point cloud point has three-dimensional spatial coordinates and reflection intensity information. Then, the Poisson surface reconstruction algorithm is used to generate the basic triangular mesh model. The specific steps are as follows: ① For each point cloud point P i Search for the set of neighboring points within its neighborhood radius, and calculate the normal vector n using principal component analysis. i The calculation method is the same as the calculation method of the normal vector in step S201.

[0116] ② Construct an octree structure, dividing the point cloud space into cubic units of different resolutions. Each unit corresponds to an octree node. For each node, define an indicator function. The value at this node indicates that the indicator function takes a value of 1 inside the object and a value of 0 outside the object, and there is a gradient change near the surface. The spatial partitioning result of the octree is used as the discretized grid for the subsequent Poisson equation.

[0117] ③ Based on the octree mesh, the point cloud normal vector n i Using the gradient direction at that point, the discrete normal vector is smoothly interpolated across the entire space using a Gaussian kernel function to construct a continuous vector field V(x): ; In the formula, The Gaussian kernel function; N is the total number of points in the point cloud.

[0118] The physical meaning of this vector field is: near a point in the point cloud, the direction of the vector field approaches the normal vector of that point; in regions far from the point cloud, the vector field approaches zero.

[0119] ④ Solve the Poisson equation The equation is discretized onto an octree grid of nodes, and the indicator function value at each node is obtained through iterative solution. The essence of this equation is to find a scalar field. This makes the divergence of its gradient equal to the divergence of the vector field, thus making the gradient of the indicator function equal to the divergence of the vector field. Approximate the vector field V as closely as possible.

[0120] ⑤ Based on the indicator function obtained from the solution Search for isosurfaces with a value of 0.5 in the octree mesh, extract the isosurfaces to generate triangular patches, and obtain the basic triangular mesh model.

[0121] The basic triangular mesh model is passed to step S403.

[0122] S403: Obtain the basic triangulation model and edge reference labels.

[0123] Based on this, traverse each triangular facet in the triangular mesh model. For the current triangular facet, calculate its geometric center coordinates using the average of the coordinates of its three vertices: ; In the formula, v1, v2, and v3 are the coordinates of the three vertices of the triangular facet.

[0124] After obtaining the geometric center, calculate the Euclidean distance between the geometric center and the location of each edge reference label, find the edge reference label with the closest distance, and record the minimum distance value.

[0125] If the minimum distance is less than the preset threshold d tag If the condition is met, the triangular facet is identified as an edge region, and the reference type, confidence vector, and local offset vector from the matched edge reference label are written into the attribute field of the facet.

[0126] If the minimum distance is greater than or equal to the preset threshold d tag If so, the triangular patch is identified as a fusion region and marked as a fusion region type in its attribute field. By default, the point cloud and the image are already aligned.

[0127] For the triangular facets marked as edge regions, further processing is performed based on the reference type: For point cloud-dominated edge regions, the reference type is recorded as point cloud reference in the attribute field, and the corresponding confidence level is stored. For the dominant edge region of the image, the reference type is recorded as the image reference, and the corresponding confidence level is stored; The patch of the edge region of the dual reference is recorded as a dual reference type, and the confidence of the point cloud reference and the image reference, as well as the offset distance between the two references are stored.

[0128] The triangular mesh model with the embedded edge reference labels is passed to step S404.

[0129] S404: Obtain the triangular mesh model with embedded edge reference labels and the corrected image data.

[0130] Iterate through all the triangular faces in the triangular mesh model and select the appropriate texture mapping method based on the baseline type in the attribute field of each face.

[0131] For triangular patches marked as point cloud-dominant edges, texture attachment is performed using corrected image data. Based on the elastically distorted image texture coordinates in step S401, the color values ​​of the corresponding pixels are mapped onto the triangular patches.

[0132] For triangular patches marked as image-dominant edges, texture is applied using the original image data, keeping the initial texture coordinates calculated in step S1 unchanged, and the image pixel colors are directly mapped onto the triangular patches.

[0133] For triangular patches marked as dual-reference points, two sets of textures are applied simultaneously. The first set of textures is based on the point cloud reference and is mapped using the corrected image data according to elastically distorted coordinates; the second set of textures is based on the image reference and is mapped using the original image data according to the initial coordinates. The two sets of textures are stored separately and can be switched by the user as needed in subsequent step S5.

[0134] For the triangular facets marked as fusion regions, a conventional texture mapping method is used. Based on the exterior orientation elements of each image obtained in step S1, the multi-view images are projected onto the surface of the triangular mesh. Each triangular facet may be covered by multiple images. Let there be a total of K images covering the triangular facet, and let A be the projected area corresponding to the k-th image. k The distance from the projection center to the center of the triangular facet is d. k The angular deviation between the image center and the projection point is θ. k Then the weight w of the image k The calculation formula is: ; The image weights are normalized to obtain normalized weights. : ; The image with the largest normalized weight is selected for texture mapping, and the color values ​​of the corresponding pixels in the image are mapped onto the triangular facets. At the seams between adjacent triangular facets, the texture color differences between adjacent facets on the same image are detected, and the Laplacian smoothing algorithm is used to correct the color differences, eliminating visual discontinuities.

[0135] After completing the texture attachment and seam smoothing of all triangular facets, the triangular mesh model, the attached texture image, and the reference label information in the attribute fields of each facet are integrated to obtain a complete 3D real scene model with reference labels, and then transferred to step S5.

[0136] In step S5 of the present invention, based on the three-dimensional real-scene model with reference labels, the type of area and measurement direction traversed by the user's measurement behavior are identified, the most suitable measurement reference is automatically selected, and a measurement result with credibility is output.

[0137] First, obtain a 3D reality model with reference labels from step S4. Each triangular facet in this model has a reference type label and corresponding confidence information.

[0138] After the user selects a measurement point or measurement path on the 3D model through the interactive interface, the system determines the set of triangular facets traversed by the measurement path based on the user's selected measurement start and end points, and analyzes the spatial orientation of the path.

[0139] When the measurement path is mainly located in the fusion zone, the system directly reads the three-dimensional coordinates of the starting point and the ending point from the model, calculates the Euclidean distance between the two points as the measurement value, and marks this measurement value as the fusion measurement value.

[0140] When the measurement path crosses an edge region, the system further determines the relationship between the measurement direction and the edge direction. It extracts the direction vector of the edge region traversed by the measurement path and calculates the angle between the measurement path direction and the edge direction. When the angle is close to 90 degrees, it is determined to be a measurement perpendicular to the edge; when the angle is close to 0 degrees or 180 degrees, it is determined to be a measurement parallel to the edge.

[0141] For measurements perpendicular to the edge, the system prioritizes image references, finds the corresponding image reference measurement value based on the edge reference label, reads the three-dimensional coordinates of the start and end points under the image reference, calculates the measurement distance, and labels the measurement value as the measurement value based on the image reference.

[0142] For measurements parallel to the edge, the system prioritizes point cloud references, finds the corresponding point cloud reference measurement value based on the edge reference label, reads the three-dimensional coordinates of the start and end points under the point cloud reference, calculates the measurement distance, and labels the measurement value as the measurement value based on the point cloud reference.

[0143] For edge regions marked as dual-reference areas, the system simultaneously outputs two measurements: one based on the point cloud reference and the other based on the image reference, and calculates the difference between the two measurements. If the difference is less than a preset threshold, the average of the two measurements is taken as the final measurement; if the difference exceeds the threshold, both measurements are presented to the user simultaneously, along with the confidence levels of each reference, allowing the user to choose which one to use based on their actual needs.

[0144] After each measurement is completed, the system generates a measurement report, which includes the three-dimensional coordinates of the start and end points of the measurement, the measured distance, the type of reference used, the confidence level of each reference, the angle between the measurement direction and the edge direction, and the fusion consistency index. If the measurement path involves multiple edge regions, the report will list the measurement results and corresponding references for each segment.

[0145] The above measurement results are output to the user interface for subsequent engineering analysis and decision-making.

[0146] The second objective of this invention is to provide a high-precision 3D modeling system for UAVs used in space measurement, including a data acquisition and registration module, an edge extraction and classification module, a deformation field construction and optimization module, a model reconstruction and benchmark embedding module, and an adaptive space measurement module.

[0147] The data acquisition and registration module includes a laser scanner, a tilt camera, and a combined navigation device, which are respectively installed and fixed on the UAV body. After time synchronization calibration, it synchronously acquires laser point cloud, multi-view images, and combined navigation data of the measured area.

[0148] The data acquisition and registration module also includes a spatiotemporal alignment unit and an initial coordinate alignment unit. It performs spatiotemporal alignment based on the timestamps carried by each data point and uses the pre-calibrated sensor installation parameters to unify the laser point cloud and multi-view images into the same navigation coordinate system to form an initial fusion dataset.

[0149] The edge extraction and classification module includes a point cloud edge extraction unit, an image edge extraction unit, and an edge matching and classification unit. The point cloud edge extraction unit extracts point cloud edge segments from the initial fused dataset and calculates the transition band width, point cloud density, the angle between the two planes, and the direction vector. The image edge extraction unit extracts sub-pixel precision image edge lines from multi-view images and calculates the grayscale step amplitude, edge contrast direction, and texture richness.

[0150] The edge matching classification unit performs same-name matching between point cloud edges and image edges. Based on the comparison results of point cloud confidence and image confidence and the adaptive threshold, the edges are divided into three categories: point cloud-dominated, image-dominated, and balanced. The unit outputs a set of edge same-name pairs with classification labels.

[0151] The deformation field construction and optimization module includes a typed mapping unit, an iterative solution unit, a deformation field output unit, and a closed-loop feedback verification unit. The typed mapping unit adopts differentiated mapping strategies for the three types of edges. Point cloud-dominated edges correct image texture coordinates based on point clouds, image-dominated edges correct point cloud positions based on images, and balanced edges retain both references and construct a normal distance field.

[0152] The iterative solution unit adopts a unidirectional driving principle. In each iteration, it only updates the deformation field parameters corresponding to the edge with the largest current residual and performs regularization constraints through Laplace smoothing. The deformation field output unit organizes the deformation field parameters after iterative convergence into two storage formats: radial basis function coefficients and regular grid offset vectors. The closed-loop feedback verification unit performs cross-projection verification on the corrected point cloud edges and image edges. If the residual exceeds the threshold, it triggers local re-matching until the self-consistency requirement is met.

[0153] The model reconstruction and reference embedding module includes a point cloud and image correction unit, a triangulation reconstruction unit, a reference label embedding unit, and a texture mapping unit. The point cloud and image correction unit performs elastic correction on the point cloud and image based on deformation field parameters. The triangulation reconstruction unit uses the Poisson surface reconstruction algorithm to generate the basic triangulation model.

[0154] The reference label embedding unit traverses each triangular facet in the triangular mesh, marks the facet as an edge region or a fusion region based on the distance between the geometric center and the edge reference label, and writes the reference type, confidence vector, and local offset vector into the facet attribute field.

[0155] The texture mapping unit selects the appropriate texture mapping method according to the reference type of the patch. Point cloud-dominated edges use the corrected image texture, image-dominated edges use the original image texture, dual reference edges are attached with two sets of textures at the same time, and the fusion area uses conventional weighted projection mapping to generate a 3D real scene model with reference labels.

[0156] The adaptive spatial measurement module includes a path analysis unit, a benchmark determination unit, and a measurement report generation unit. The path analysis unit determines the set of triangular facets and spatial orientation traversed by the measurement path based on the measurement start and end points selected by the user.

[0157] The benchmark determination unit automatically selects the measurement benchmark based on the type of area traversed by the path and the angle between the measurement direction and the edge direction. Measurements perpendicular to the edge prioritize the use of image benchmarks, while measurements parallel to the edge prioritize the use of point cloud benchmarks. Two measurement values ​​are output simultaneously for dual-benchmark areas. The measurement report generation unit compiles the measurement distance, the type of benchmark used, the confidence level of each benchmark, the angle between the measurement direction and the consistency index into a report and outputs it to the user interface.

[0158] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A high-precision 3D modeling method for unmanned aerial vehicles (UAVs) used for spatial measurement, characterized in that, Includes the following steps: S1. Collect laser point cloud and multi-view images of the measured area using drones; S2. Extract point cloud edges from laser point clouds, extract image edges from multi-view images, project each point cloud edge onto the image plane and perform nearest neighbor matching with the image edges to establish a corresponding relationship; Calculate the point cloud confidence and image confidence for each pair of edges with the same name, and classify the edges into three categories: point cloud-dominated, image-dominated, and balanced based on the ratio of the two. S3. For point cloud-dominated edges, establish a thin plate spline interpolation model based on the point cloud edges to correct the image texture coordinates. For image-dominant edges, a radial basis function network is constructed based on the image edges to correct the point cloud positions; For balanced edges, two implicit surfaces are constructed based on the point cloud and the image, respectively, and the normal distance field between the two is used as the local offset vector. Establish a local elastic deformation field and initialize it to zero; Calculate the reprojection and backprojection residuals for all edges, and update and smooth the constraints only for the deformation field parameters corresponding to the edge type with the largest residual. If there is a residual greater than or equal to the preset threshold, the residual is recalculated and the constraints are updated and smoothed until they are satisfied, at which point the local elastic deformation field and edge reference labels are output. S4. Based on the local elastic deformation field, perform geometric mapping correction on the point cloud and image, construct a triangular mesh model, and map the edge reference labels to the attribute fields of each triangular facet to generate a 3D real scene model with edge reference labels.

2. The high-precision 3D modeling method for UAVs used in space measurement according to claim 1, characterized in that: The laser point cloud is acquired by continuously emitting laser pulses at a preset scanning frequency and receiving echo signals through a laser scanner. Each sampling point records its three-dimensional spatial coordinates, echo intensity value, and echo number. The multi-view images are acquired by simultaneously exposing multiple lenses of a tilting camera in different directions. Furthermore, the three-dimensional coordinates and attitude angle data of the UAV are collected in real time at a fixed frequency through a combined navigation device to obtain combined navigation data.

3. The high-precision 3D modeling method for UAVs used in space measurement according to claim 2, characterized in that: The laser point cloud, multi-view images, and integrated navigation data are spatiotemporally aligned and transformed to form an initial fused dataset. The spatiotemporal alignment associates the laser point cloud and multi-view images with the integrated navigation data based on the timestamps carried by each data, and removes data segments whose timestamps do not match. The coordinate transformation uses pre-calibrated sensor installation parameters to transform the laser point cloud from the scanner body coordinate system to the navigation coordinate system, and at the same time calculates the initial value of the exterior orientation element of each image. The initial value of the exterior orientation element includes the three-dimensional coordinates of the projection center and three attitude angles.

4. The high-precision 3D modeling method for UAVs used in space measurement according to claim 1, characterized in that: The point cloud confidence score is calculated based on the point cloud density at the edge of the point cloud, the width of the transition zone, and the angle between the two planes. The higher the point cloud density, the narrower the transition zone, and the closer the angle between the two planes is to 90 degrees, the higher the point cloud confidence score. The image confidence score is calculated based on the gray-level step amplitude and texture richness at the edge of the image. The larger the gray-level step amplitude and the higher the texture richness, the higher the image confidence score.

5. The high-precision 3D modeling method for UAVs used in space measurement according to claim 1, characterized in that: The edges are classified into three types: point cloud-dominated, image-dominated, and balanced. Specifically, this is achieved by comparing the ratio of point cloud confidence to image confidence with a classification threshold. The classification threshold is dynamically determined by the ratio of the angle between the two planes on both sides of the point cloud edge to the gray-level gradient magnitude of the image edge. The angle between the two planes on both sides of the point cloud edge is calculated by fitting the normal vectors of the local planes on both sides of the edge, and the gray-level gradient magnitude of the image edge is calculated by calculating the gray-level change rate of the pixels at the edge.

6. The high-precision 3D modeling method for UAVs used in space measurement according to claim 1, characterized in that: The reprojection residual is obtained by projecting the point cloud edges onto the image plane after deformation field transformation and calculating the Euclidean distance between the projected point and the corresponding image edge point. The backprojection residual is obtained by projecting the image edges onto the point cloud space after inverse deformation field transformation and calculating the Euclidean distance between the projected point and the corresponding point cloud edge point.

7. The high-precision 3D modeling method for UAVs used in space measurement according to claim 1, characterized in that: The smoothing constraint employs the Laplace smoothing method to constrain the displacement variation of the deformation field parameters between adjacent control points.

8. The high-precision 3D modeling method for UAVs used in space measurement according to claim 1, characterized in that: The specific steps of embedding the edge reference label are as follows: calculate the geometric center coordinates of the triangular facet, search for the edge reference label that is closest to the geometric center, and write the reference type, confidence vector, and local offset vector into the attribute field of the facet.

9. The high-precision 3D modeling method for UAVs used in space measurement according to claim 1, characterized in that: It also includes obtaining the measurement path selected by the user in the 3D real-world model with reference labels, selecting the reference type according to the angle between the measurement direction and the edge direction when the measurement path crosses the edge area, and outputting a measurement report with reference type and confidence level.

10. A high-precision 3D modeling system for unmanned aerial vehicles (UAVs) used for spatial measurement, for executing the high-precision 3D modeling method for UAVs used for spatial measurement as described in any one of claims 1-9, characterized in that, It includes a data acquisition and registration module, an edge extraction and classification module, a deformation field construction and optimization module, a model reconstruction and benchmark embedding module, and an adaptive spatial measurement module; The data acquisition and registration module is used to simultaneously acquire laser point cloud, multi-view image and integrated navigation data, and form an initial fusion dataset after spatiotemporal alignment and coordinate transformation. The edge extraction and classification module is used to extract point cloud edges and image edges from the initial fusion dataset and establish a corresponding relationship, calculate the point cloud confidence and image confidence, and classify the edges into three categories according to the ratio of the two. The deformation field construction and optimization module is used to establish a local elastic deformation field based on the classification results, and output the local elastic deformation field and edge reference labels through iterative optimization. The model reconstruction and reference embedding module is used to correct the point cloud and image according to the local elastic deformation field, reconstruct the triangular mesh model and embed edge reference labels to generate a three-dimensional real scene model with reference labels. The adaptive spatial measurement module is used to select a measurement reference based on the relationship between the measurement path and the edge direction, and output a measurement report.

Images