Lidar-based adaptive reconstruction method and system for slanted measurement point clouds

Abstract

The present application relates to the field of three-dimensional reconstruction and spatial information processing, in particular to a laser radar-based adaptive reconstruction method and system for slant measurement point clouds. A reconstruction request is received to construct a request record containing a region identifier and a threshold parameter set; a laser radar point cloud is received to construct a laser record containing a point coordinate set; a slant photography image is received to construct a measurement record containing a camera pose set, a measurement point coordinate set, and a version identifier; an alignment transformation matrix is generated and a skeleton edge set and reliability are extracted; a parameter record is constructed and a weight-smoothed update is performed on a blocked region to obtain a reconstructed point cloud load, a residual error and a quality indicator are calculated, and a pass identifier is determined and written into an evidence library. The present application improves the reconstruction stability of blocked and edge regions and provides an auditable and traceable data closed loop.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional reconstruction and spatial information processing technology, specifically to an adaptive reconstruction method and system for tilt measurement point clouds based on lidar. Background Technology

[0002] Oblique photography can acquire geometric and texture information of a target scene through multi-view images. Combined with the photogrammetry process, it can generate a camera pose set and a set of measurement point coordinates, and further form a 3D point cloud model.

[0003] Due to factors such as occlusion, insufficient view coverage of facade and edge areas, matching of outliers, and uneven sampling density, the coordinate set of measurement points obtained by oblique photography is prone to holes, structural drift, and noise accumulation in local areas, resulting in local distortion of the reconstruction results in facade, edge, and occluded areas.

[0004] LiDAR point clouds offer stability advantages in terms of structural edges and facade geometry, making them a structural prior in photogrammetric point cloud reconstruction. However, in existing solutions, if LiDAR point clouds are only used for simple point filling, replacement, or general fusion, it remains difficult to establish a verifiable control link for the update range, update intensity, and structural consistency of occluded areas.

[0005] In addition, if the existing solutions fail to form a unified data loop for skeleton extraction, residual calculation, occlusion determination, weight control, quality auditing, and evidence storage, problems such as unclear parameter sources, unclear update basis, difficulty in reviewing quality results, and difficulty in tracing reconstruction results may easily arise. Summary of the Invention

[0006] The purpose of this invention is to provide a technical solution to address the aforementioned problems existing in the prior art. Specifically, this invention is achieved through the following technical solution: An adaptive reconstruction method for tilt measurement point clouds based on lidar includes: Step 1: Receive the lidar point cloud to obtain the lidar point coordinate set; receive oblique photogrammetry images to generate the camera pose set and the measurement point coordinate set; register the measurement point coordinate set with the lidar point coordinate set to obtain the alignment transformation matrix and the aligned measurement point coordinate set. Step 2: Perform voxelization statistics, planar patch segmentation, planar fitting, planar patch intersection line extraction, and connectivity statistics on the laser point coordinate set to obtain the skeleton edge set and the voxel density ratio, mean of planar fitting residuals, and edge connectivity corresponding to each skeleton edge. Then, through the weighted fusion of the monotonically increasing transformation of the voxel density ratio, the monotonically decreasing transformation of the mean of planar fitting residuals, and the monotonically increasing transformation of the edge connectivity, obtain the skeleton reliability value of each skeleton edge. Step 3: Perform nearest neighbor search based on the aligned set of measurement point coordinates and skeleton edge set to obtain the point-to-skeleton index set and the initial residual set. The initial residual is the shortest distance residual from the measurement point to the nearest skeleton edge. Step 4: Generate a view coverage grid based on the camera pose set, and generate a neighborhood missing rate based on the point-to-skeleton index set. The statistical window of the view coverage grid is configured in a consistent manner with the neighborhood scale of the neighborhood missing rate. When the coverage value of the grid where the measurement point is located is less than a preset coverage threshold and the neighborhood missing rate of the neighborhood corresponding to the skeleton edge of the measurement point in the point-to-skeleton index set is greater than a preset missing threshold, the measurement point is marked as an occluded point. Step 5: Generate weight coefficients based on the initial residuals and the skeleton reliability values ​​of the corresponding skeleton edges, and reduce the upper limit of the weight coefficients when the skeleton reliability value of the corresponding skeleton edge is less than the preset reliability threshold; update the coordinates of each occluded point by scaling the displacement vector of its nearest projection point to the corresponding skeleton edge using the weight coefficients, and form the reconstructed point cloud payload with the updated occluded points and the unupdated measurement points.

[0007] Furthermore, the process of obtaining the laser point coordinate set by receiving the lidar point cloud includes: Voxel downsampling is performed on the lidar point cloud, and outlier removal is performed based on k-nearest neighbor distance statistics to obtain the lidar point coordinate set.

[0008] Furthermore, the step of receiving oblique photographic images to generate a camera pose set and a measurement point coordinate set includes: Feature extraction and feature matching are performed on the oblique photogrammetric images to obtain a set of matching pairs; outlier culling and pose calculation are performed on the set of matching pairs to obtain the camera pose set; dense reconstruction is performed based on the camera pose set to obtain the set of measurement point coordinates; and the registration includes coarse registration and fine registration, wherein the alignment transformation matrix obtained by the registration is a rigid body transformation matrix or a similarity transformation matrix that aligns the set of measurement point coordinates to the coordinate system of the laser point coordinate set.

[0009] Furthermore, the process of performing voxelization statistics, planar patch segmentation, planar fitting, planar patch intersection extraction, and connectivity statistics on the laser point coordinate set to obtain the skeleton edge set and the voxel density ratio, mean planar fitting residual, and edge connectivity of each skeleton edge includes: Voxelization and statistical analysis are performed on the laser point coordinate set to generate a voxel density ratio; planar patch segmentation is performed on the laser point coordinate set to generate a set of planar patches; planar fitting is performed on the set of planar patches to generate the mean of the planar fitting residuals; planar patch intersection line extraction is performed on the set of planar patches to generate a skeleton edge set; connectivity statistics are performed on the skeleton edge set to generate edge connectivity; and skeleton reliability values ​​are generated for each skeleton edge.

[0010] Furthermore, the nearest neighbor search performed based on the aligned set of measurement point coordinates and the skeleton edge set to obtain the point-to-skeleton index set and the initial residual set includes: Based on k-nearest neighbor search, the candidate skeleton edge set corresponding to each measurement point is obtained. The point-to-skeleton distance residual from each measurement point to each candidate skeleton edge is calculated, and the minimum residual selection is performed. When no candidate skeleton edge is found for a measurement point, the measurement point does not enter the set of points to be updated, and the original measurement point coordinates are maintained to participate in the formation of the reconstructed point cloud load.

[0011] Furthermore, the weighting coefficient is the product of the output value of the piecewise monotonic mapping function with respect to the initial residual and the output value of the monotonic adjustment function with respect to the skeleton reliability value of the corresponding skeleton edge. When the skeleton reliability value of the corresponding skeleton edge is less than the system's preset reliability threshold, the weighting coefficient is truncated to an upper limit.

[0012] Furthermore, the weighting coefficient is the product of the output value of the piecewise monotonic mapping function with respect to the initial residual and the output value of the monotonic adjustment function with respect to the skeleton reliability value of the corresponding skeleton edge. When the skeleton reliability value of the corresponding skeleton edge is lower than the preset reliability threshold, the weighting coefficient is attenuated by the ratio of the skeleton reliability value to the preset reliability threshold.

[0013] Furthermore, it also includes: Based on the reconstructed point cloud payload and skeleton edge set, a reconstructed residual set is calculated. The reconstructed residual set is obtained by statistically analyzing the point-to-skeleton distance residuals from the reconstructed point cloud payload to the skeleton edge set. Based on the reconstructed point cloud payload and the reconstructed residual set, a quality index set is calculated. The quality index set includes residual statistics, density variance index, and hole ratio index. The residual statistics are compared with preset residual thresholds, the density variance index is compared with preset density variance thresholds, and the hole ratio index is compared with preset hole ratio thresholds to determine the pass / fail indicator.

[0014] Furthermore, it also includes: A version identifier is generated based on the input image batch identifier, timestamp, and reconstruction parameter summary; hash operations are performed on the laser point coordinate set and the reconstructed point cloud payload respectively to obtain point coordinate summary values ​​and payload summary values; the point coordinate summary values, the payload summary values, the alignment transformation matrix, and the version identifier are concatenated and then hashed using the SHA-256 hash algorithm to generate a model fingerprint value; furthermore, string concatenation and hash operations are performed based on the version identifier and the point coordinate summary value to generate a data source identifier, and the pass identifier, model fingerprint value, and data source identifier are written into the evidence storage library.

[0015] The adaptive reconstruction system for tilt measurement point clouds based on lidar is used to execute the adaptive reconstruction method for tilt measurement point clouds based on lidar, including: a lidar access preprocessing module, a measurement generation module, a registration module, a skeleton modeling module, an index and missing rate generation module, a weight generation module, a reconstruction module, an auditing module, an evidence storage module, and a data module. The laser access preprocessing module, measurement generation module, registration module, skeleton modeling module, index and missing rate generation module, weight generation module, reconstruction module, auditing module, and evidence storage module are respectively connected to the data processing module. The laser access preprocessing module is used to receive the laser radar point cloud and obtain the laser point coordinate set. The measurement generation module is used to receive oblique photographic images and generate a camera pose set and a measurement point coordinate set, and generate a version identifier when needed; The registration module is used to register the set of measurement point coordinates with the set of laser point coordinates to obtain an alignment transformation matrix and an aligned set of measurement point coordinates. The skeleton modeling module is used to extract the skeleton edge set and generate the skeleton reliability value; The index and missing rate generation module is used to generate a point-to-skeleton index set, an initial residual set, a view coverage raster, and a neighborhood missing rate, and to mark occluded points. The weight generation module is used to generate weight coefficients based on the initial residual and skeleton reliability value; The reconstruction module is used to update the occlusion points according to the weight coefficients and form a reconstructed point cloud payload; The aforementioned audit module is used to calculate the reconstructed residual set and quality indicator set and determine the pass / fail identifier; The aforementioned evidence storage module is used to generate point coordinate summary values ​​and load summary values ​​for the laser point coordinate set and the reconstructed point cloud load, respectively. It concatenates the point coordinate summary values, load summary values, alignment transformation matrix and version identifier and performs hash operation to generate model fingerprint values. It also performs string concatenation and hash operation based on version identifier and point coordinate summary values ​​to generate data source identifiers, and writes the identifier, model fingerprint value and data source identifier into the evidence storage library.

[0016] Compared with the prior art, the present invention has the following advantages and beneficial effects: First, the skeleton edge set is extracted using laser point cloud and a skeleton reliability value is generated, so that the reconstruction update is jointly constrained by structural priors and reliability. Second, occlusion points are determined by jointly using view coverage and neighborhood missing rate, and the update intensity is controlled by combining the initial residual and weight coefficients, so that the update basis, update range and update results are traceable. Third, a pass identifier is formed by reconstructing the residual set and quality index set, and written into the evidence storage library by combining the model fingerprint value and data source identifier, so that the reconstruction result has verifiable capability. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating the adaptive reconstruction method for tilt measurement point clouds based on lidar. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments and accompanying drawings. The illustrative embodiments and descriptions of this invention are for illustrative purposes only and are not intended to limit the invention. It should be noted that this invention is already in the actual research and development stage.

[0019] Example 1 like Figure 1 As shown, the adaptive reconstruction method for tilt measurement point clouds based on lidar includes: Step 1: Receive the lidar point cloud to obtain the lidar point coordinate set; receive oblique photogrammetry images to generate the camera pose set and the measurement point coordinate set; register the measurement point coordinate set with the lidar point coordinate set to obtain the alignment transformation matrix and the aligned measurement point coordinate set. Step 2: Perform voxelization statistics, planar patch segmentation, planar fitting, planar patch intersection line extraction, and connectivity statistics on the laser point coordinate set to obtain the skeleton edge set and the voxel density ratio, mean of planar fitting residuals, and edge connectivity corresponding to each skeleton edge. Then, through the weighted fusion of the monotonically increasing transformation of the voxel density ratio, the monotonically decreasing transformation of the mean of planar fitting residuals, and the monotonically increasing transformation of the edge connectivity, obtain the skeleton reliability value of each skeleton edge. Step 3: Perform nearest neighbor search based on the aligned set of measurement point coordinates and skeleton edge set to obtain the point-to-skeleton index set and the initial residual set. The initial residual is the shortest distance residual from the measurement point to the nearest skeleton edge. Step 4: Generate a view coverage grid based on the camera pose set, and generate a neighborhood missing rate based on the point-to-skeleton index set. The statistical window of the view coverage grid is configured in a consistent manner with the neighborhood scale of the neighborhood missing rate. When the coverage value of the grid where the measurement point is located is less than a preset coverage threshold and the neighborhood missing rate of the neighborhood corresponding to the skeleton edge of the measurement point in the point-to-skeleton index set is greater than a preset missing threshold, the measurement point is marked as an occluded point. Step 5: Generate weight coefficients based on the initial residuals and the skeleton reliability values ​​of the corresponding skeleton edges, and reduce the upper limit of the weight coefficients when the skeleton reliability value of the corresponding skeleton edge is less than the preset reliability threshold; update the coordinates of each occluded point by scaling the displacement vector of its nearest projection point to the corresponding skeleton edge using the weight coefficients, and form the reconstructed point cloud payload with the updated occluded points and the unupdated measurement points.

[0020] Specifically, the system receives a lidar point cloud and obtains a set of lidar point coordinates. Voxel downsampling and outlier removal are then performed on the lidar point cloud to obtain the set of lidar point coordinates for subsequent skeleton extraction. The outlier removal can employ a removal strategy based on k-nearest neighbor distance statistics, where k is the system's preset number of nearest neighbors.

[0021] It receives oblique photographic images and generates a camera pose set and a set of measurement point coordinates.

[0022] In one implementation, local feature points are extracted from the input oblique photographic images and corresponding feature descriptors are generated, denoted as the first... The feature point set of the image is ; in, For the first The first image The pixel coordinates of each feature point This corresponds to the feature descriptor. For two images with overlapping fields of view... and ,right and Perform nearest neighbor matching and second nearest neighbor ratio filtering to obtain a set of candidate matching pairs: ; Then, geometric consistency constraints are applied to remove outliers from the candidate matching pair set, retaining inliers that satisfy the multi-view geometric relationship, resulting in the matching pair set: ; Subsequently, the camera extrinsic parameters are jointly calculated based on the matching pair set of all image pairs to obtain the camera pose set: ; in, Indicates the first Rotation matrix of each camera, Indicates the first The translation vectors of each camera. Preferably, pose optimization is achieved by minimizing the reprojection error, and the objective function can be expressed as: ; in, Representing a three-dimensional point In the The coordinates of the observed pixels in the image. This represents the camera projection function. Based on the camera pose set, dense matching and triangulation reconstruction are performed on the multi-view images to obtain the measurement point coordinate set: ; The coordinate set of measurement points is registered with the coordinate set of laser points to obtain an alignment transformation matrix and an aligned coordinate set of measurement points. The registration includes coarse registration and fine registration, and the alignment transformation matrix is ​​a rigid body transformation matrix or similarity transformation matrix that aligns the coordinate set of measurement points to the coordinate system of the coordinate set of laser points.

[0023] Registration can be represented as: ; in, This represents a homogeneous vector of point coordinates in the set of measured point coordinates. This represents the homogeneous vector of the aligned point coordinates. This represents the alignment transformation matrix.

[0024] Voxelization statistics, planar patch segmentation, planar fitting, planar patch intersection extraction, and connectivity statistics are performed on the laser point coordinate set to obtain the skeleton edge set and the voxel density ratio, mean of planar fitting residuals, and edge connectivity of each skeleton edge. The skeleton reliability value of each skeleton edge is obtained by weighted fusion of the monotonically increasing transformation of the voxel density ratio, the monotonically decreasing transformation of the mean of planar fitting residuals, and the monotonically increasing transformation of edge connectivity.

[0025] In one implementation, the laser point coordinate set is set as follows: ; First of all Perform voxel mesh generation to obtain a non-empty voxel set. And count each voxel Number of points inside The reference point count is taken as the average count of non-empty voxels: ; The voxel density ratio of the corresponding voxels is: ; Subsequently, planar patch segmentation is performed on the laser point coordinate set to obtain a set of planar patches: ; For each planar piece Perform plane fitting on the points within the plane, and let the equation of the fitted plane be: ; The mean of the planar fitting residuals for this planar patch is: ; Next, the intersection lines of spatially adjacent planar pairs are extracted to obtain candidate intersection line segments; candidate intersection line segments with the same direction and spatial continuity are merged to form a skeleton edge set: ; For each skeleton edge Collect related voxel sets within its neighborhood. Associative planar slice set and the set of adjacent skeleton edges The voxel density ratio corresponding to the edge of the skeleton can then be taken as: ; The mean of the plane fitting residuals can be taken as: ; Edge connectivity can be defined as: ; Thus, the voxel density ratios corresponding to each skeleton edge are obtained. Mean of residuals from plane fitting and edge connectivity .

[0026] In one implementation, a skeleton reliability value is generated for each skeleton edge. To ensure that the voxel density ratio, mean plane fitting residual, and edge connectivity have a consistent dimensionless representation when participating in the fusion, let... For positive residual normalization parameters, then take the following values ​​respectively. ; ; ; in, ; and Both are monotonically increasing functions, which result in larger function output values ​​as the voxel density ratio and edge connectivity are respectively higher. The function is monotonically decreasing, such that the larger the mean of the plane fitting residuals, the smaller the function output value. Therefore, the skeleton reliability value corresponding to the skeleton edge can be expressed as... ; in, The system is pre-defined with fusion coefficients that satisfy the following conditions: ; This indicates that the result is restricted to a range. The truncation operator within. Therefore, the skeleton reliability value. The closer it is to 1, the more reliable the corresponding skeleton edge is.

[0027] Based on the aligned set of measurement point coordinates and skeleton edge set, a nearest neighbor search is performed to obtain a set of point-to-skeleton indexes and an initial residual set. Specifically, a set of candidate skeleton edges corresponding to each measurement point is obtained based on k-nearest neighbor search. The point-to-skeleton distance residual from each measurement point to each candidate skeleton edge is calculated, and the minimum residual selection is performed. The initial residual is the shortest distance residual from the measurement point to the nearest skeleton edge. If no candidate skeleton edge is found for a measurement point, the measurement point is not included in the set of points to be updated, and the original measurement point coordinates are retained to participate in the formation of the reconstructed point cloud payload.

[0028] A view coverage raster is generated based on the camera pose set, and a neighborhood missing rate is generated based on the point-to-skeleton index set. Specifically, a view coverage raster is generated based on the camera pose set, and each view coverage raster cell corresponds to its coverage value. Measurement points associated with the same skeleton edge are assigned to the neighborhood corresponding to that skeleton edge based on the point-to-skeleton index set. The neighborhood corresponding to that skeleton edge is divided into preset voxel cells with the same scale as the view coverage raster statistical window, and the presence of aligned measurement points within each preset voxel cell is counted. When at least one aligned measurement point exists in a preset voxel cell, the preset voxel cell is considered to be effectively covered by the measurement point. The proportion of preset voxel cells not effectively covered by the measurement point to the total number of preset voxel cells in the neighborhood corresponding to that skeleton edge is used as the neighborhood missing rate.

[0029] When the coverage value of the grid where the measurement point is located is less than a preset coverage threshold, and the neighborhood missing rate of the neighborhood corresponding to the skeleton edge of the measurement point in the skeleton index set is greater than a preset missing threshold, the measurement point is marked as an occluded point. Specifically, the statistical window of the view coverage grid is configured in a consistent manner with the neighborhood scale of the neighborhood missing rate to reduce misjudgment of edge regions.

[0030] Weighting coefficients are generated based on the initial residuals and the skeleton reliability values ​​of the corresponding skeleton edges, and the upper limit of the weighting coefficients is reduced when the skeleton reliability value of the corresponding skeleton edge is lower than a preset reliability threshold.

[0031] Specifically, let the initial residual from a certain measurement point to its corresponding skeleton edge be... The skeleton reliability value corresponding to the skeleton edge is Regarding the piecewise monotonic mapping function of the initial residual. Can be taken as ; in, and The preset residual segmentation points satisfy: ; and and Compared with the initial residual Having the same dimensions. A monotonically adjusting function for the reliability values ​​of the corresponding skeleton edges. Possible options are: ; Therefore, the weighting coefficient It can be represented as: ; Furthermore, let the preset reliability threshold be... And satisfy: ; When the skeleton reliability value of the corresponding skeleton edge is lower than the preset reliability threshold, the weight coefficient is decayed. It can be represented as: ; Let the coordinates of the current occlusion point be... The coordinates of its nearest projection point on the corresponding skeleton edge are Then the displacement vector is: ; The updated point coordinates are: ; Therefore, the final coordinate update is constrained by both the initial residual and the skeleton reliability value, and the update intensity is further suppressed when the skeleton reliability value is low.

[0032] In some specific implementations, quality auditing and evidence preservation steps may also be included, as follows: Based on the reconstructed point cloud payload and skeleton edge set formed above, the reconstructed residual set is calculated. The reconstructed residual set is obtained by statistically analyzing the point-to-skeleton distance residuals from the reconstructed point cloud payload to the skeleton edge set.

[0033] A quality index set is calculated based on the reconstructed point cloud payload and the reconstructed residual set. The quality index set includes residual statistics, density variance index, and hole ratio index. The residual statistics are compared with a preset residual threshold, the density variance index is compared with a preset density variance threshold, and the hole ratio index is compared with a preset hole ratio threshold to determine the pass / fail indicator.

[0034] In one implementation, after calculating the quality index set based on the reconstructed point cloud load and the reconstructed residual set, the residual statistics are obtained respectively. Density variance index and the ratio of holes Among them, residual statistics The density variance index is calculated from the reconstructed residual set according to the system's preset statistical caliber. The hole ratio index is calculated from the variance of the point number distribution of the reconstructed point cloud load on the preset voxel mesh. It is calculated from the proportion of empty voxels within the area covered by the skeleton edge set to the total number of voxels.

[0035] Let the preset residual threshold be... The preset density variance threshold is The preset threshold for the proportion of holes is Construct quality determination sub-markers respectively: ; ; ; Based on the quality judgment sub-mark, the identifier is determined. It can be represented as: ; when When, the pass flag is determined to be passed; when If the condition is met, the pass flag is determined as a fail flag. Preferably, the residual statistics, density variance index, void ratio index, corresponding threshold, and the pass flag are all written into the audit log for subsequent verification and traceability.

[0036] A version identifier is generated based on the input image batch identifier, timestamp, and reconstruction parameter summary. Hash operations are performed on the laser point coordinate set and the reconstructed point cloud payload according to the system's preset serialization rules to obtain point coordinate summary values ​​and payload summary values. The point coordinate summary values, payload summary values, alignment transformation matrix, and generated version identifier are concatenated and then hashed using the SHA-256 hash algorithm to generate a model fingerprint value. Furthermore, string concatenation and hash operations are performed based on the generated version identifier and the point coordinate summary value to generate a data source identifier. The pass identifier, model fingerprint value, and data source identifier are then written into the evidence storage database.

[0037] Through the above steps, a complete technical chain can be formed, from laser structure prior, occlusion determination, weight update, reconstruction results to quality audit and evidence fingerprinting.

[0038] Example 2 The adaptive reconstruction system for tilt measurement point clouds based on lidar is used to execute the adaptive reconstruction method for tilt measurement point clouds based on lidar, including: a lidar access preprocessing module, a measurement generation module, a registration module, a skeleton modeling module, an index and missing rate generation module, a weight generation module, a reconstruction module, an auditing module, an evidence storage module, and a data module. The laser access preprocessing module, measurement generation module, registration module, skeleton modeling module, index and missing rate generation module, weight generation module, reconstruction module, auditing module, and evidence storage module are respectively connected to the data processing module. The laser access preprocessing module is used to receive the laser radar point cloud and obtain the laser point coordinate set. The measurement generation module is used to receive oblique photographic images and generate a camera pose set and a measurement point coordinate set, and generate a version identifier when needed; The registration module is used to register the set of measurement point coordinates with the set of laser point coordinates to obtain an alignment transformation matrix and an aligned set of measurement point coordinates. The skeleton modeling module is used to extract the skeleton edge set and generate the skeleton reliability value; The index and missing rate generation module is used to generate a point-to-skeleton index set, an initial residual set, a view coverage raster, and a neighborhood missing rate, and to mark occluded points. The weight generation module is used to generate weight coefficients based on the initial residual and skeleton reliability value; The reconstruction module is used to update the occlusion points according to the weight coefficients and form a reconstructed point cloud payload; The aforementioned audit module is used to calculate the reconstructed residual set and quality indicator set and determine the pass / fail identifier; The aforementioned evidence storage module is used to generate point coordinate summary values ​​and load summary values ​​for the laser point coordinate set and the reconstructed point cloud load, respectively. It concatenates the point coordinate summary values, load summary values, alignment transformation matrix and version identifier and performs hash operation to generate model fingerprint values. It also performs string concatenation and hash operation based on version identifier and point coordinate summary values ​​to generate data source identifiers, and writes the identifier, model fingerprint value and data source identifier into the evidence storage library.

[0039] Example 3 Based on Example 2, the skeleton modeling module generates skeleton reliability values ​​according to the voxel density ratio, the mean of the plane fitting residuals, and the edge connectivity. It then obtains the skeleton reliability value corresponding to each skeleton edge by weighted fusion through the monotonically increasing transformation of the voxel density ratio, the monotonically decreasing transformation of the mean of the plane fitting residuals, and the monotonically increasing transformation of the edge connectivity.

[0040] The index and missing rate generation module performs k-nearest neighbor search, point-to-skeleton distance residual calculation, and minimum residual selection based on the aligned measurement point coordinate set and skeleton edge set to generate a point-to-skeleton index set and an initial residual set, and marks occluded points based on the view coverage raster and neighborhood missing rate.

[0041] When the skeleton reliability value of the corresponding skeleton edge is less than the system's preset reliability threshold, the weight generation module truncates the upper limit of the weight coefficient.

[0042] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An adaptive reconstruction method for tilt measurement point clouds based on lidar, characterized in that, include: Step 1: Receive the lidar point cloud to obtain the lidar point coordinate set, and receive the oblique photogrammetric image to generate the camera pose set and the measurement point coordinate set; Register the set of measurement point coordinates with the set of laser point coordinates to obtain the alignment transformation matrix and the aligned set of measurement point coordinates. Step 2: Perform voxelization statistics, planar patch segmentation, planar fitting, planar patch intersection line extraction, and connectivity statistics on the laser point coordinate set to obtain the skeleton edge set and the voxel density ratio, mean of planar fitting residuals, and edge connectivity corresponding to each skeleton edge. Then, through the weighted fusion of the monotonically increasing transformation of the voxel density ratio, the monotonically decreasing transformation of the mean of planar fitting residuals, and the monotonically increasing transformation of the edge connectivity, obtain the skeleton reliability value of each skeleton edge. Step 3: Perform nearest neighbor search based on the aligned set of measurement point coordinates and skeleton edge set to obtain the point-to-skeleton index set and the initial residual set. The initial residual in the initial residual set is the shortest distance residual from the measurement point to the nearest skeleton edge. Step 4: Generate a view coverage grid based on the camera pose set, with each view coverage grid cell corresponding to its coverage value; and classify the measurement points associated with the same skeleton edge into the neighborhood corresponding to that skeleton edge based on the point-to-skeleton index set; divide the neighborhood corresponding to that skeleton edge into preset voxel cells with the same scale as the view coverage grid statistical window, and count whether there are aligned measurement points in each preset voxel cell. When there is at least one aligned measurement point in a preset voxel unit, the preset voxel unit is considered to be effectively covered by the measurement point; the proportion of the number of preset voxel units not effectively covered by the measurement point to the total number of preset voxel units in the corresponding neighborhood of the skeleton edge is taken as the neighborhood missing rate. When the coverage value of the grid where the measurement point is located is less than the preset coverage threshold and the neighborhood missing rate of the neighborhood corresponding to the skeleton edge of the measurement point in the skeleton index set is greater than the preset missing threshold, the measurement point is marked as an occluded point; the statistical window of the view coverage grid is configured in a consistent manner with the neighborhood scale of the neighborhood missing rate. Step 5: Generate weighting coefficients based on the initial residuals and the skeleton reliability values ​​of the corresponding skeleton edges, and reduce the upper limit of the weighting coefficients when the skeleton reliability value of the corresponding skeleton edge is less than a preset reliability threshold. The coordinates of each occluded point are updated by scaling the displacement vector of the point to the nearest projection point on the corresponding skeleton edge using the weighting coefficient. The updated occluded points and the unupdated measurement points are then combined to form a reconstructed point cloud payload.

2. The adaptive reconstruction method for tilt measurement point clouds based on lidar according to claim 1, characterized in that, The laser point coordinate set obtained by receiving the lidar point cloud includes: Voxel downsampling is performed on the lidar point cloud, and outlier removal is performed based on k-nearest neighbor distance statistics to obtain the lidar point coordinate set.

3. The adaptive reconstruction method for tilt measurement point clouds based on lidar according to claim 1, characterized in that, The process of receiving oblique photographic images to generate a camera pose set and a measurement point coordinate set includes: Feature extraction and feature matching are performed on the oblique photogrammetric images to obtain a set of matching pairs; outlier culling and pose calculation are performed on the set of matching pairs to obtain the camera pose set; dense reconstruction is performed based on the camera pose set to obtain the set of measurement point coordinates; and the registration includes coarse registration and fine registration, wherein the alignment transformation matrix obtained by the registration is a rigid body transformation matrix or a similarity transformation matrix that aligns the set of measurement point coordinates to the coordinate system of the laser point coordinate set.

4. The adaptive reconstruction method for tilt measurement point clouds based on lidar according to claim 1, characterized in that, The process of performing voxelization statistics, planar patch segmentation, planar fitting, planar patch intersection extraction, and connectivity statistics on the laser point coordinate set yields the skeleton edge set and the voxel density ratio, mean planar fitting residual, and edge connectivity for each skeleton edge, including: Voxelization and statistical analysis are performed on the laser point coordinate set to generate a voxel density ratio; planar patch segmentation is performed on the laser point coordinate set to generate a set of planar patches; planar fitting is performed on the set of planar patches to generate the mean of the planar fitting residuals; planar patch intersection line extraction is performed on the set of planar patches to generate a skeleton edge set; connectivity statistics are performed on the skeleton edge set to generate edge connectivity; and skeleton reliability values ​​are generated for each skeleton edge.

5. The adaptive reconstruction method for tilt measurement point clouds based on lidar according to claim 1, characterized in that, The nearest neighbor search is performed based on the aligned set of measurement point coordinates and the skeleton edge set to obtain the point-to-skeleton index set and the initial residual set, including: Based on k-nearest neighbor search, the candidate skeleton edge set corresponding to each measurement point is obtained. The point-to-skeleton distance residual from each measurement point to each candidate skeleton edge is calculated, and the minimum residual selection is performed. When no candidate skeleton edge is found for a measurement point, the measurement point does not enter the set of points to be updated, and the original measurement point coordinates are maintained to participate in the formation of the reconstructed point cloud load.

6. The adaptive reconstruction method for tilt measurement point clouds based on lidar according to claim 1, characterized in that, The weighting coefficient is the product of the piecewise monotonic mapping function output value with respect to the initial residual and the monotonic adjustment function output value with respect to the skeleton reliability value of the corresponding skeleton edge. When the skeleton reliability value of the corresponding skeleton edge is less than the system preset reliability threshold, the upper limit of the weighting coefficient is truncated.

7. The adaptive reconstruction method for tilt measurement point clouds based on lidar according to claim 1, characterized in that, The weighting coefficient is the product of the output value of the piecewise monotonic mapping function with respect to the initial residual and the output value of the monotonic adjustment function with respect to the skeleton reliability value of the corresponding skeleton edge. When the skeleton reliability value of the corresponding skeleton edge is lower than the preset reliability threshold, the weighting coefficient is attenuated by the ratio of the skeleton reliability value to the preset reliability threshold.

8. The adaptive reconstruction method for tilt measurement point clouds based on lidar according to claim 1, characterized in that, Also includes: The reconstruction residual set is calculated based on the reconstructed point cloud payload and skeleton edge set. The reconstruction residual set is obtained by statistically analyzing the point-to-skeleton distance residuals from the reconstructed point cloud payload to the skeleton edge set. A quality index set is calculated based on the reconstructed point cloud payload and the reconstructed residual set. The quality index set includes residual statistics, density variance index and hole ratio index. The residual statistics are compared with a preset residual threshold, the density variance index is compared with a preset density variance threshold, and the hole ratio index is compared with a preset hole ratio threshold to determine the pass mark.

9. The adaptive reconstruction method for tilt measurement point clouds based on lidar according to claim 8, characterized in that, Also includes: A version identifier is generated based on the input image batch identifier, timestamp, and reconstruction parameter summary; hash operations are performed on the laser point coordinate set and the reconstructed point cloud payload respectively to obtain point coordinate summary values ​​and payload summary values; the point coordinate summary values, the payload summary values, the alignment transformation matrix, and the version identifier are concatenated and then hashed using the SHA-256 hash algorithm to generate a model fingerprint value; furthermore, string concatenation and hash operations are performed based on the version identifier and the point coordinate summary value to generate a data source identifier, and the pass identifier, model fingerprint value, and data source identifier are written into the evidence storage library.

10. An adaptive reconstruction system for tilt measurement point clouds based on lidar, characterized in that, The method for implementing the adaptive reconstruction method of tilt measurement point cloud based on lidar as described in any one of claims 1 to 9 includes: a lidar access preprocessing module, a measurement generation module, a registration module, a skeleton modeling module, an index and missing rate generation module, a weight generation module, a reconstruction module, an auditing module, an evidence storage module, and a data processing module. The laser access preprocessing module, measurement generation module, registration module, skeleton modeling module, index and missing rate generation module, weight generation module, reconstruction module, auditing module, and evidence storage module are respectively connected to the data processing module. The laser access preprocessing module is used to receive the laser radar point cloud and obtain the laser point coordinate set. The measurement generation module is used to receive oblique photographic images and generate a camera pose set and a measurement point coordinate set, and generate a version identifier when needed; The registration module is used to register the set of measurement point coordinates with the set of laser point coordinates to obtain an alignment transformation matrix and an aligned set of measurement point coordinates. The skeleton modeling module is used to extract the skeleton edge set and generate the skeleton reliability value; The index and missing rate generation module is used to generate a point-to-skeleton index set, an initial residual set, a view coverage raster, and a neighborhood missing rate, and to mark occluded points. The weight generation module is used to generate weight coefficients based on the initial residual and skeleton reliability value; The reconstruction module is used to update the occlusion points according to the weight coefficients and form a reconstructed point cloud payload; The aforementioned audit module is used to calculate the reconstructed residual set and quality indicator set and determine the pass / fail identifier; The aforementioned evidence storage module is used to generate point coordinate summary values ​​and load summary values ​​for the laser point coordinate set and the reconstructed point cloud load, respectively. It concatenates the point coordinate summary values, load summary values, alignment transformation matrix and version identifier and performs hash operation to generate model fingerprint values. It also performs string concatenation and hash operation based on version identifier and point coordinate summary values ​​to generate data source identifiers, and writes the identifier, model fingerprint value and data source identifier into the evidence storage library.

Images