Building reconstruction method based on confidence adaptive fusion and synthetic bias correction

Abstract

The application discloses a building reconstruction method based on confidence adaptive fusion and synthesis deviation correction, relates to the field of three-dimensional reconstruction, and solves the problem of insufficient three-dimensional reconstruction accuracy, comprising the following steps: S1, acquiring observation reference data; S2, dividing a scene space into units, performing consistency analysis in combination with the observation reference data, classifying local unit spaces, simultaneously performing weight assignment, and obtaining space generation parameters; S3, obtaining Gaussian primitives based on the space generation parameters, rendering the scene space, constructing a deviation compensation function, and optimizing the Gaussian primitives; S4, identifying a slender thin-walled structure, and recording Gaussian parameters of a slender structure region; and S5, generating an initial reconstruction result, comparing with independent laser radar verification points, and feeding back and optimizing the space generation parameters; the application can effectively improve the accuracy of building reconstruction and guarantee reconstruction accuracy.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional reconstruction, specifically involving multi-sensor data fusion technology, and more specifically, a building reconstruction method based on confidence-adaptive fusion and synthesis bias correction. Background Technology

[0002] Existing Gaussian splash reconstruction methods based on pure visual images rely on the convergence quality of photometric optimization for geometric accuracy. In weakly textured building facades and areas with repetitive textures, problems such as high noise in the initial sparse point cloud and inaccurate depth estimation are prone to occur.

[0003] Specifically, the existing technology has the following main shortcomings: 1. LiDAR point clouds can provide high-precision metric-level geometric measurements, but they usually lack color and texture information. In architectural details (such as railings, wires, and thin parapet walls), due to the limitations of point cloud density and multiple echo interference, it is difficult to generate realistic scene representations that can be directly used for rendering by relying solely on LiDAR point clouds.

[0004] 2. Existing vision and lidar fusion reconstruction methods often use lidar point clouds only as the initial source of sparse point clouds, or apply a depth supervision loss term with fixed weights directly to the rendering depth. They do not combine the consistency between the visual reconstruction results and lidar observations in the local area for confidence adaptive weighting, nor do they consider the inherent synthesis bias of the Gaussian splash rendering depth itself. This can easily introduce new systematic errors in areas of depth discontinuity, affecting the optimization convergence quality of complex geometric regions.

[0005] 3. Existing methods lack specific shape constraints for slender, thin-walled structures in the scene. Isotropic density adaptive strategies are prone to volume expansion or fracture artifacts on such structures. The geometric fidelity of detailed components such as railings, wires, and parapets in the reconstruction results is insufficient, making it difficult to meet the accuracy requirements of architectural applications for detailed components.

[0006] To address this, a building reconstruction method based on confidence-adaptive fusion and synthetic bias correction is proposed. Summary of the Invention

[0007] To address the shortcomings of existing technologies, the purpose of this invention is to provide a building reconstruction method based on confidence-adaptive fusion and synthetic bias correction, thereby improving the accuracy of building reconstruction.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: a building reconstruction method based on confidence-adaptive fusion and synthesis bias correction, the specific working process of each step of which is as follows: Step S1: Acquire multi-source observation data of the scene space, align the multi-source observation data, and obtain observation reference data; Step S2: Divide the scene space into units, combine the observation benchmark data, extract the visual sparse point cloud and lidar point cloud of the corresponding local unit space for consistency analysis, classify the local unit space and assign weights; statistically analyze the region type and corresponding fusion weight of the local unit space to obtain the space generation parameters. Step S3: Based on spatial generation parameters, fuse visual sparse point clouds and LiDAR point clouds to obtain Gaussian elements; render the scene space, analyze the uncertainty of the synthesis depth, correct the deviation of the synthesis depth, and construct a depth supervision loss; optimize the Gaussian elements using the depth supervision loss and record the optimized Gaussian parameters. Step S4: Based on the observation benchmark data, obtain the lidar point cloud, identify the slender thin-walled structure and its corresponding spatial range, apply anisotropic regularization terms to the Gaussian within the spatial range, and record the corresponding Gaussian parameters. Step S5: Statistically analyze the Gaussian elements and corresponding Gaussian parameters of the scene space to generate initial reconstruction results; obtain independent LiDAR verification points, compare the reconstruction results, obtain residual errors, and optimize the spatial generation parameters.

[0009] Furthermore, the specific steps of step S1 are as follows: Step S11: Based on multi-source observation data in the scene space, extract camera images and LiDAR point cloud data; perform motion reconstruction structure calculation on the camera images to obtain visual sparse point cloud and camera pose; perform point cloud registration on the LiDAR data, collect the LiDAR pose trajectory, and generate LiDAR point cloud. Step S12: Acquire the common observation area of ​​the camera image and the lidar point cloud data, map the visual sparse point cloud and the lidar point cloud, establish the correspondence between the visual sparse point cloud and the lidar point cloud, and obtain the relative external parameters between the camera coordinate system and the lidar coordinate system. Step S13: Based on the camera pose, reproject the visual sparse point cloud onto the camera image, record the pixel error between the reprojection result and the camera image to obtain the visual reprojection error; match the visual sparse point cloud with the lidar point cloud to obtain the lidar point cloud registration residual; use the visual reprojection error and the lidar point cloud registration residual as joint constraints, perform joint bundle adjustment to optimize the camera pose, lidar pose, and their relative extrinsic parameters, obtaining the camera pose, lidar pose, visual sparse point cloud, and lidar point cloud under a unified coordinate reference; statistically analyze the camera pose, lidar pose, visual sparse point cloud, and lidar point cloud to obtain the observation reference data.

[0010] Furthermore, the specific steps of step S2 are as follows: Step S21: The scene space is divided into multiple local unit spaces. The visual sparse point cloud and lidar point cloud of each local unit space are extracted by combining the observation benchmark data. The distribution of the visual sparse point cloud and lidar point cloud corresponding to the local unit space is statistically analyzed. Step S22: Calculate the distribution deviation based on the distribution of visual sparse point cloud and lidar point cloud, perform consistency analysis based on the distribution deviation, and label the type of local unit space; assign weights to the local unit space based on the local unit space type labeling to obtain the fusion weight coefficient; and statistically analyze the type labeling and fusion weight coefficient of the local spatial units to obtain the spatial generation parameters.

[0011] Furthermore, the specific steps of step S21 are as follows: Step S211: Based on the observation reference data, extract the visual sparse point cloud and lidar point cloud corresponding to the local unit space to obtain the local visual point cloud and local lidar point cloud; count the effective points of the local visual point cloud to obtain the number of effective visual points as, and collect the position sjw1 to sjw of each visual point. as The effective points of the local radar point cloud are counted to obtain the number of effective radar points bs, and the position of each radar point ldw1 to ldw is collected. bs ; Step S212: From the position of each visual point sjw1 to sjw as The mean value is calculated to obtain the visual midpoint szd; the local visual point cloud is fitted to obtain the optimal local visual plane, and the normal vector sfx of the visual plane is collected; each visual point is mapped to the optimal local visual plane, and the distance from the visual point to the plane is collected to obtain the visual mapping distance sjl. q Calculate the root mean square of all visual mapping distances to obtain the visual point dispersion (sls). From the location of each radar point ldw1 to ldw bs The mean value is calculated to obtain the radar midpoint lzd; the local radar point cloud is fitted to obtain the optimal local radar plane, and the radar plane normal lfx is collected; each radar point is mapped to the optimal local radar plane, and the distance from the radar point to the plane is collected to obtain the radar mapping distance ldl. q Calculate the root mean square of all radar mapping distances to obtain the radar point dispersion lls.

[0012] Furthermore, the specific steps of step S22 are as follows: Step S221: Collect the distance between the visual midpoint szd and the radar midpoint lzd to obtain the position deviation wpc; calculate the normal deviation fpc by combining the visual plane normal sfx and the radar plane normal lfx. To obtain building reconstruction standards, collect the position deviation threshold wpy and the normal deviation threshold fpy, and combine them with the position deviation wpc and the normal deviation fpc for judgment: If wpc≤wpy and fpc≤fpy, it indicates that the local visual point cloud and the local radar point cloud are highly consistent, and the local unit space is marked as a high-confidence region. If wpc > wpy or fpc > fpy, it indicates that there is an inconsistency between the local visual point cloud and the local radar point cloud. Collect the visual point dispersion sls and radar point dispersion lls of the local unit space. If sls ≥ lls, mark the local unit space as the radar dominant region; if sls < lls, mark the local unit space as the visual dominant region. Step S222: Traverse the local cell space, extract the local cell space labeled as high confidence region, collect the visual point dispersion sls and radar point dispersion lls of the corresponding local cell space, assign weights to the local visual point cloud and local radar point cloud of the region, and obtain the fusion weight coefficient. The spatial generation parameters are obtained by statistically analyzing the type labeling and fusion weight coefficients of local spatial units.

[0013] Furthermore, the specific steps of step S3 are as follows: Step S31: Traverse the local spatial units, collect the annotation types of the local spatial units, perform feature processing on each annotation type, fuse the visual sparse point cloud corresponding to different local spatial units with the lidar point cloud, and perform Gaussian generation in the corresponding region of the local spatial unit to obtain Gaussian elements. Step S32: Statistically analyze the Gaussian elements to obtain the Gaussian distribution in the scene space, render the scene space, collect the Gaussian elements participating in alpha synthesis along each ray during the rendering process, integrate the rendering status of the Gaussian elements, calculate the synthesis depth and depth uncertainty of the corresponding pixel of the ray, and correct the deviation of the synthesis depth based on the rendering status of the Gaussian elements; obtain the measured depth of the LiDAR, and construct the depth supervision loss based on the deviation-corrected synthesis depth to optimize the Gaussian distribution.

[0014] Furthermore, the specific steps of step S31 are as follows: Step S311: Extract the local cell space of the high-confidence region and obtain the fusion weight coefficient, visual midpoint, radar midpoint, visual plane normal, and radar plane normal corresponding to the local cell space; perform weighted fusion of the visual midpoint and radar midpoint according to the fusion weight coefficient to obtain the Gaussian initial position; perform weighted fusion of the visual plane normal and radar plane normal according to the fusion weight coefficient to obtain the Gaussian initial normal; obtain the lidar point cloud corresponding to the local cell space, collect the point cloud density of the local cell space, set the initial scale of the Gaussian, and obtain the Gaussian primitive from the initial position, initial normal, and initial scale; Step S312: Extract the local cell space marked as the radar-dominant region, obtain the radar midpoint and radar plane normal corresponding to the local cell space, and initialize the Gaussian position and normal; collect the point cloud density of the local cell space, set the initial scale of the Gaussian, and obtain the Gaussian cell by combining the Gaussian position and normal, while recording the visual sparse point cloud of the local cell space as an auxiliary reference. Step S313: Extract the local cell space marked as the visually dominant region, obtain the visual midpoint and visual plane normal corresponding to the local cell space, and initialize the Gaussian position and normal; collect the point cloud density of the local cell space, set the initial scale of the Gaussian, and obtain the Gaussian cell by combining the Gaussian position and normal, while recording the LiDAR point cloud of the local cell space as an auxiliary reference.

[0015] Furthermore, the specific steps of step S32 are as follows: Step S321: Collect each Gaussian element that participates in alpha compositing along each ray during the rendering process; calculate the opacity contribution ratio of each Gaussian element to obtain a ratio set; calculate the depth value of each Gaussian element to obtain a depth set; calculate the compositing depth by combining the ratio set and the depth set. The depth uncertainty is calculated from the proportion set, the depth set, and the composite depth. The composite depth is corrected by combining the depth uncertainty with the proportion set; Step S322: Obtain the measured depth of the lidar and the synthesized depth after deviation correction, record the difference between the measured depth of the lidar and the synthesized depth after deviation correction, and construct the depth supervision loss; update and optimize the Gaussian parameters using the depth supervision loss, and record the optimized Gaussian parameters.

[0016] Furthermore, the specific steps of step S4 are as follows: Step S41: Based on the lidar point cloud, identify the slender thin-walled structure and record the corresponding spatial range; collect the point cloud distribution of the corresponding space to obtain the local principal extension direction of the slender thin-walled structure, which is used as the principal axis constraint direction of the Gaussian covariance of the spatial range. Step S42: Collect Gaussian data within the spatial range, apply anisotropic regularization terms to the Gaussian data, constrain its covariance in the main extension direction and the cross-sectional scale perpendicular to the main extension direction, suppress the expansion of Gaussian data in the region along non-main extension directions during density adaptive clonal splitting, and record the Gaussian parameters of the slender structural region after anisotropic regularization.

[0017] Furthermore, the specific steps of step S5 are as follows: Step S51: Obtain the initial reconstruction results. Based on the multi-source observation data of the scene space, collect the LiDAR verification points that do not directly participate in the training supervision to obtain the LiDAR verification point set. Map the LiDAR verification point set pairs and count the residual errors in different local unit spaces. Step S52: Feedback is provided on the consistency analysis process between the visual sparse point cloud and the lidar point cloud in the local unit space using residual errors, the local unit space is adjusted, and the spatial generation parameters are optimized.

[0018] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. This invention solves the problem of inconsistent coordinate references between camera images and lidar point clouds acquired from different acquisition platforms or at different acquisition times by using cross-sensor spatiotemporal unified calibration and joint bundle adjustment; it ensures the stability of point cloud analysis, improves the accuracy of point cloud judgment, and guarantees reconstruction precision.

[0019] 2. This invention improves the accuracy of local construction by classifying local spatial units. At the same time, for high-confidence areas, it generates adaptive weights by using visual point dispersion and radar point dispersion to provide data support for subsequent fusion initialization, rather than using a fixed proportion of adaptive weighting, thereby improving fusion accuracy and stability. 3. This invention performs independent Gaussian initialization on different local unit spaces, and adaptively fuses high-confidence regions based on the classification results of local spatial units. For non-high-confidence regions, it prioritizes the use of LiDAR geometry for weak texture regions and visual geometry for detailed texture regions, thereby improving the problem of improper information selection caused by fixed-ratio fusion. Attached Figure Description

[0020] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0021] Figure 1 This is a schematic diagram of the method of the present invention; Figure 2 This is a schematic diagram of spatial classification according to the present invention; Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0023] This application provides a building reconstruction method based on confidence-adaptive fusion and synthesis bias correction. The execution entity of this building reconstruction method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the building reconstruction method based on confidence-adaptive fusion and synthesis bias correction can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0024] Reference Figure 1 The diagram shown is a flowchart illustrating a building reconstruction method based on confidence-adaptive fusion and synthesis bias correction according to an embodiment of the present invention. In this embodiment, the building reconstruction method based on confidence-adaptive fusion and synthesis bias correction includes: Step S1: Acquire multi-source observation data of the scene space, align the multi-source observation data, and obtain observation reference data; Step S11: Based on multi-source observation data in the scene space, extract camera images and LiDAR point cloud data; perform motion reconstruction structure calculation on the camera images to obtain visual sparse point cloud and camera pose; perform point cloud registration on the LiDAR data, collect the LiDAR pose trajectory, and generate LiDAR point cloud. Step S12: Acquire the common observation area of ​​the camera image and the lidar point cloud data, map the common observation area to the visual sparse point cloud and the lidar point cloud, establish the correspondence between the visual sparse point cloud and the lidar point cloud, and obtain the relative external parameters between the camera coordinate system and the lidar coordinate system. Step S13: Based on the camera pose, reproject the visual sparse point cloud onto the camera image, record the pixel error between the reprojection result and the camera image to obtain the visual reprojection error; match the visual sparse point cloud with the lidar point cloud, collect the distance error at the same matching point to obtain the lidar point cloud registration residual; use the visual reprojection error and the lidar point cloud registration residual as joint constraints, perform a joint bundle adjustment to optimize the camera pose, lidar pose, and the relative extrinsic parameters between them, to obtain the camera pose, lidar pose, visual sparse point cloud, and lidar point cloud under a unified coordinate reference; statistically analyze the camera pose, lidar pose, visual sparse point cloud, and lidar point cloud to obtain the observation reference data.

[0025] It should be noted that the joint bundle adjustment is a tightly coupled global nonlinear least squares optimization algorithm for expanding multi-source data from vision and lidar. It uses the visual reprojection error and lidar point cloud registration residual as joint constraints to simultaneously optimize three types of parameters: camera pose, lidar pose trajectory, and relative extrinsic parameters between the camera and lidar, thereby achieving global alignment and error elimination of data across sensors and across acquisition periods.

[0026] Step S2: Divide the scene space into units, combine the observation benchmark data, extract the visual sparse point cloud and lidar point cloud of the corresponding local unit space for consistency analysis, classify the local unit space and assign weights; statistically analyze the region type and corresponding fusion weight of the local unit space to obtain the space generation parameters. Please see Figure 2 Step S21: The scene space is divided into multiple local unit spaces. The visual sparse point cloud and lidar point cloud of each local unit space are extracted by combining the observation benchmark data. The distribution of the visual sparse point cloud and lidar point cloud corresponding to the local unit space is statistically analyzed. Step S211: Based on the observation reference data, extract the visual sparse point cloud and lidar point cloud corresponding to the local unit space to obtain the local visual point cloud and local lidar point cloud; count the effective points of the local visual point cloud to obtain the number of effective visual points as, and collect the position sjw1 to sjw of each visual point. as The effective points of the local radar point cloud are counted to obtain the number of effective radar points bs, and the position of each radar point ldw1 to ldw is collected. bs ; Step S212: From the position of each visual point sjw1 to sjw as The mean value is calculated to obtain the visual midpoint szd; the local visual point cloud is fitted using the least squares algorithm to obtain the optimal local visual plane, and the normal vector sfx of the visual plane is collected; each visual point is mapped to the optimal local visual plane, and the distance from the visual point to the plane is collected to obtain the visual mapping distance sjl. a Calculate the root mean square of all visual mapping distances to obtain the visual point dispersion (sls). ; It should be noted that by calculating the root mean square of the visual mapping distance, the noise level of the local single-view point cloud is judged, providing a classification basis for subsequent region classification, ensuring the accuracy of classification, and improving the accuracy of subsequent 3D Gaussian generation.

[0027] From the location of each radar point ldw1 to ldw bsThe mean value is calculated to obtain the radar midpoint lzd; the local radar point cloud is fitted using the least squares algorithm to obtain the optimal local radar plane, and the radar plane normal lfx is collected; each radar point is mapped to the optimal local radar plane, and the distance from the radar point to the plane is collected to obtain the radar mapping distance ldl. b Calculate the root mean square of all radar mapping distances to obtain the radar point dispersion lls; .

[0028] Step S22: Based on the distribution of visual sparse point cloud and lidar point cloud, calculate the distribution deviation, perform consistency analysis based on the distribution deviation, and label the type of local unit space; based on the local unit space type label, assign weights to the local unit space to obtain the fusion weight coefficient; statistically analyze the type label and fusion weight coefficient of the local spatial units to obtain the spatial generation parameters. Step S221: Collect the distance between the visual midpoint szd and the radar midpoint lzd to obtain the position deviation wpc; calculate the normal deviation fpc by combining the visual plane normal sfx and the radar plane normal lfx. ; To obtain building reconstruction standards, collect the position deviation threshold wpy and the normal deviation threshold fpy, and combine them with the position deviation wpc and the normal deviation fpc for judgment: If wpc≤wpy and fpc≤fpy, it indicates that the local visual point cloud and the local radar point cloud are highly consistent, and the local unit space is marked as a high-confidence region. If wpc > wpy or fpc > fpy, it indicates that there is an inconsistency between the local visual point cloud and the local radar point cloud. The visual point dispersion sls and radar point dispersion lls of the local unit space are collected. If sls ≥ lls, the local unit space is marked as the radar dominant region; if sls < lls, the local unit space is marked as the visual dominant region.

[0029] In this invention, space is classified based on deviation conditions. For spaces where both the positional deviation (wpc) and the normal deviation (fpc) ​​are below the corresponding deviation thresholds, the local visual point cloud and the local radar point cloud of this local unit space can be considered highly consistent, i.e., the data is stable and has high reliability, and are thus marked as high-confidence regions. High confidence primarily reflects that the data in the corresponding region is sufficiently reliable.

[0030] It should be noted that building reconstruction standards, such as the "Specifications for 3D Geographic Information Model Data Products" and the "Specifications for 3D Modeling of Oblique Photogrammetry," are relevant.

[0031] Step S222: Traverse the local cell space, extract the local cell space labeled as high confidence region, collect the visual point dispersion sls and radar point dispersion lls of the corresponding local cell space, assign weights to the local visual point cloud and local radar point cloud of the region, and obtain the fusion weight coefficient. ; Where: sqx is the weight coefficient corresponding to vision, and lqx is the weight coefficient corresponding to radar; based on the formula, the point cloud with greater point dispersion is assigned a smaller weight coefficient.

[0032] The spatial generation parameters are obtained by statistically analyzing the type labeling and fusion weight coefficients of local spatial units.

[0033] It should be noted that classifying local spatial units improves the accuracy of local construction. At the same time, for high-confidence areas, adaptive weights are generated using visual point dispersion and radar point dispersion to provide data support for subsequent fusion initialization, rather than using a fixed proportion of adaptive weighting, thus improving fusion accuracy and stability.

[0034] Step S3: Based on spatial generation parameters, fuse visual sparse point clouds and LiDAR point clouds to obtain Gaussian elements; render the scene space, analyze the uncertainty of the synthesis depth, correct the deviation of the synthesis depth, and construct a depth supervision loss; optimize the Gaussian elements using the depth supervision loss and record the optimized Gaussian parameters. Step S31: Traverse the local spatial units, collect the annotation types of the local spatial units, perform feature processing on each annotation type, fuse the visual sparse point cloud corresponding to different local spatial units with the lidar point cloud, and perform Gaussian generation in the corresponding region of the local spatial unit to obtain Gaussian elements. Step S311: Extract the local cell space of the high confidence region and obtain the fusion weight coefficient, visual midpoint szd, radar midpoint lzd, visual plane normal sfx and radar plane normal lfx corresponding to the local cell space; The visual midpoint and the radar midpoint are weighted and fused according to the fusion weight coefficient to obtain the Gaussian initial position gsw; ; Where: sqx is the weighting coefficient corresponding to vision, and lqx is the weighting coefficient corresponding to radar.

[0035] Based on the fusion weighting coefficient, the visual plane normal sfx and the radar plane normal lfx are weighted and fused to obtain the Gaussian initial normal gsf. ; The point cloud corresponding to the local unit space is obtained, the point cloud density of the local unit space is collected, the initial scale of the Gaussian is set, and the Gaussian element is obtained from the initial position, initial normal and initial scale of the Gaussian.

[0036] Step S312: Extract the local cell space marked as the radar-dominant region, obtain the radar midpoint and radar plane normal corresponding to the local cell space, and initialize the Gaussian position and normal; collect the point cloud density of the local cell space, set the initial scale of the Gaussian, and obtain the Gaussian cell by combining the Gaussian position and normal, while recording the visual sparse point cloud of the local cell space as an auxiliary reference.

[0037] Step S313: Extract the local cell space marked as the visually dominant region, obtain the visual midpoint and visual plane normal corresponding to the local cell space, and initialize the Gaussian position and normal; collect the point cloud density of the local cell space, set the initial scale of the Gaussian, and obtain the Gaussian cell by combining the Gaussian position and normal, while recording the LiDAR point cloud of the local cell space as an auxiliary reference.

[0038] It should be noted that: independent Gaussian initialization is performed on different local unit spaces, and the advantages of the two types of observation data are brought into play in their respective reliable areas, avoiding improper information selection caused by fixed-ratio fusion; thus improving the accuracy of building reconstruction and ensuring reconstruction precision.

[0039] Step S32: Statistically analyze the Gaussian elements to obtain the Gaussian distribution in the scene space, render the scene space, collect the Gaussian elements participating in alpha synthesis along each ray during the rendering process, integrate the rendering status of the Gaussian elements, calculate the synthesis depth and depth uncertainty of the corresponding pixel of the ray, and correct the synthesis depth deviation based on the rendering status of the Gaussian elements; obtain the measured depth of the LiDAR, and construct the depth supervision loss based on the deviation-corrected synthesis depth to optimize the Gaussian distribution. Step S321: Collect each Gaussian element that participates in alpha synthesis along each ray during the rendering process, and record the number of Gaussian elements as gs; count the proportion of opacity contribution of each Gaussian element to obtain the proportion set {tzb(1) to tzb(gs)}, count the depth value of each Gaussian element to obtain the depth set {sdj(1) to sdj(gs)}, and calculate the synthesis depth hsd by combining the proportion set {tzb(1) to tzb(gs)} and the depth set {sdj(1) to sdj(gs)}. ; The depth uncertainty sdq is obtained by calculating the proportion set {tzb(1) to tzb(gs)}, the depth set {sdj(1) to sdj(gs)} and the composite depth hsd; ; The composite depth is corrected by combining the depth uncertainty with the proportion set.

[0040] Step S322: Obtain the measured depth of the lidar and the synthesized depth after deviation correction, record the difference between the measured depth of the lidar and the synthesized depth after deviation correction, and construct the depth supervision loss; update and optimize the Gaussian parameters using the depth supervision loss, and record the optimized Gaussian parameters.

[0041] It should be noted that by modeling and correcting the inherent synthesis bias of the Gaussian splash rendering depth, and then comparing it with the LiDAR depth to construct the supervision loss, we avoid directly applying the original depth supervision and introducing new systematic errors.

[0042] Step S4: Based on the observation benchmark data, acquire the lidar point cloud, identify the slender thin-walled structure and its corresponding spatial range, and apply anisotropic regularization terms to the Gaussian region based on the spatial range; record the Gaussian parameters of the anisotropically regularized slender structure region. Step S41: Based on the lidar point cloud, identify the slender thin-walled structure and record the corresponding spatial range; collect the point cloud distribution of the corresponding space to obtain the local principal extension direction of the slender thin-walled structure, which is used as the principal axis constraint direction of the Gaussian covariance of the spatial range. Step S42: Collect Gaussian data within the spatial range, apply anisotropic regularization terms to the Gaussian data, constrain its covariance in the main extension direction and the cross-sectional scale perpendicular to the main extension direction, suppress the expansion of Gaussian data in the region along non-main extension directions during density adaptive clonal splitting, and record the Gaussian parameters of the slender structural region after anisotropic regularization.

[0043] Step S5: Statistically analyze the Gaussian primitives and corresponding Gaussian parameters of the scene space to generate initial reconstruction results; obtain independent LiDAR verification points, compare the reconstruction results to obtain residual errors, and optimize the spatial generation parameters based on the residual errors. Step S51: Obtain the initial reconstruction results. Based on the multi-source observation data of the scene space, collect the LiDAR verification points that do not directly participate in the training supervision to obtain the LiDAR verification point set. Map the LiDAR verification point set pairs and count the residual errors in different local unit spaces. Step S52: Feedback is provided on the consistency analysis process between the visual sparse point cloud and the lidar point cloud in the local unit space using residual errors, the local unit space is adjusted, and the spatial generation parameters are optimized.

[0044] Compared to the problems described in the background art, this invention solves the problem of inconsistent coordinate references between camera images and LiDAR point clouds acquired from different acquisition platforms or at different acquisition times by using cross-sensor spatiotemporal unified calibration and joint bundle adjustment; it ensures the stability of point cloud analysis, improves the accuracy of point cloud judgment, and guarantees reconstruction accuracy. Furthermore, this invention improves the accuracy of local construction by classifying local spatial units, and for high-confidence areas, it generates adaptive weights based on visual point dispersion and LiDAR point dispersion, providing data support for subsequent fusion initialization, rather than using a fixed-ratio adaptive weighting basis, thus improving fusion accuracy and stability. Finally, this invention performs independent Gaussian initialization on different local spatial units, and adaptively fuses high-confidence areas according to the classification results of local spatial units. For non-high-confidence areas, it prioritizes LiDAR geometry for weak texture areas and visual geometry for detailed texture areas, improving the problem of improper information selection caused by fixed-ratio fusion. Therefore, the building reconstruction method based on confidence-adaptive fusion and synthesis bias correction provided by this invention can improve the accuracy of building reconstruction.

[0045] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0046] Finally, it should be noted that deleting any one of the above embodiments does not affect the technical solutions of other embodiments. The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A building reconstruction method based on confidence-adaptive fusion and synthesis bias correction, characterized in that, include: Step S1: Acquire multi-source observation data of the scene space, align the multi-source observation data, and obtain observation reference data; Step S2: Divide the scene space into units, combine the observation benchmark data, extract the visual sparse point cloud and lidar point cloud of the corresponding local unit space for consistency analysis, classify the local unit space and assign weights; statistically analyze the region type and corresponding fusion weight of the local unit space to obtain the space generation parameters. Step S3: Based on the spatial generation parameters, fuse the visual sparse point cloud and the lidar point cloud to obtain Gaussian elements; The scene space is rendered, the composition depth uncertainty is analyzed, the composition depth is corrected for deviation, and a depth supervision loss is constructed. The Gaussian parameters are optimized by the depth supervision loss and the optimized Gaussian parameters are recorded. Step S4: Based on the observation benchmark data, obtain the lidar point cloud, identify the slender thin-walled structure and its corresponding spatial range, apply anisotropic regularization terms to the Gaussian within the spatial range, and record the corresponding Gaussian parameters. Step S5: Statistically analyze the Gaussian elements and corresponding Gaussian parameters of the scene space to generate initial reconstruction results; obtain independent LiDAR verification points, compare the reconstruction results, obtain residual errors, and optimize the spatial generation parameters.

2. The building reconstruction method based on confidence-adaptive fusion and synthesis bias correction according to claim 1, characterized in that, The specific steps of step S1 are as follows: Step S11: Based on multi-source observation data in the scene space, extract camera images and LiDAR point cloud data; perform motion reconstruction structure calculation on the camera images to obtain visual sparse point cloud and camera pose; perform point cloud registration on the LiDAR data, collect the LiDAR pose trajectory, and generate LiDAR point cloud. Step S12: Acquire the common observation area of ​​the camera image and the lidar point cloud data, map the visual sparse point cloud and the lidar point cloud, establish the correspondence between the visual sparse point cloud and the lidar point cloud, and obtain the relative external parameters between the camera coordinate system and the lidar coordinate system. Step S13: Based on the camera pose, reproject the visual sparse point cloud onto the camera image, record the pixel error between the reprojection result and the camera image to obtain the visual reprojection error; match the visual sparse point cloud with the lidar point cloud to obtain the lidar point cloud registration residual; use the visual reprojection error and the lidar point cloud registration residual as joint constraints, perform joint bundle adjustment to optimize the camera pose, lidar pose, and their relative extrinsic parameters, obtaining the camera pose, lidar pose, visual sparse point cloud, and lidar point cloud under a unified coordinate reference; statistically analyze the camera pose, lidar pose, visual sparse point cloud, and lidar point cloud to obtain the observation reference data.

3. The building reconstruction method based on confidence-adaptive fusion and synthesis bias correction according to claim 1, characterized in that, The specific steps of step S2 are as follows: Step S21: The scene space is divided into multiple local unit spaces. The visual sparse point cloud and lidar point cloud of each local unit space are extracted by combining the observation benchmark data. The distribution of the visual sparse point cloud and lidar point cloud corresponding to the local unit space is statistically analyzed. Step S22: Calculate the distribution deviation based on the distribution of visual sparse point cloud and lidar point cloud, perform consistency analysis based on the distribution deviation, and label the type of local unit space; assign weights to the local unit space based on the local unit space type labeling to obtain the fusion weight coefficient; and statistically analyze the type labeling and fusion weight coefficient of the local spatial units to obtain the spatial generation parameters.

4. The building reconstruction method based on confidence-adaptive fusion and synthesis bias correction according to claim 3, characterized in that, The specific steps of step S21 are as follows: Step S211: Based on the observation reference data, extract the visual sparse point cloud and lidar point cloud corresponding to the local unit space to obtain the local visual point cloud and local lidar point cloud; count the effective points of the local visual point cloud to obtain the number of effective visual points as, and collect the position sjw1 to sjw of each visual point. as The effective points of the local radar point cloud are counted to obtain the number of effective radar points bs, and the position of each radar point ldw1 to ldw is collected. bs ; Step S212: From the position of each visual point sjw1 to sjw as The mean value is calculated to obtain the visual midpoint szd; the local visual point cloud is fitted to obtain the optimal local visual plane, and the normal vector sfx of the visual plane is collected; each visual point is mapped to the optimal local visual plane, and the distance from the visual point to the plane is collected to obtain the visual mapping distance sjl. q Calculate the root mean square of all visual mapping distances to obtain the visual point dispersion (sls). From the location of each radar point ldw1 to ldw bs The mean value is calculated to obtain the radar midpoint lzd; the local radar point cloud is fitted to obtain the optimal local radar plane, and the radar plane normal lfx is collected; each radar point is mapped to the optimal local radar plane, and the distance from the radar point to the plane is collected to obtain the radar mapping distance ldl. q Calculate the root mean square of all radar mapping distances to obtain the radar point dispersion lls.

5. The building reconstruction method based on confidence-adaptive fusion and synthesis bias correction according to claim 3, characterized in that, The specific steps of step S22 are as follows: Step S221: Collect the distance between the visual midpoint szd and the radar midpoint lzd to obtain the position deviation wpc; calculate the normal deviation fpc by combining the visual plane normal sfx and the radar plane normal lfx. To obtain building reconstruction standards, collect the position deviation threshold wpy and the normal deviation threshold fpy, and combine them with the position deviation wpc and the normal deviation fpc for judgment: If wpc≤wpy and fpc≤fpy, it indicates that the local visual point cloud and the local radar point cloud are highly consistent, and the local unit space is marked as a high-confidence region. If wpc > wpy or fpc > fpy, it indicates that there is an inconsistency between the local visual point cloud and the local radar point cloud. Collect the visual point dispersion sls and radar point dispersion lls of the local unit space. If sls ≥ lls, mark the local unit space as the radar dominant region; if sls < lls, mark the local unit space as the visual dominant region. Step S222: Traverse the local cell space, extract the local cell space labeled as high confidence region, collect the visual point dispersion sls and radar point dispersion lls of the corresponding local cell space, assign weights to the local visual point cloud and local radar point cloud of the region, and obtain the fusion weight coefficient. The spatial generation parameters are obtained by statistically analyzing the type labeling and fusion weight coefficients of local spatial units.

6. The building reconstruction method based on confidence-adaptive fusion and synthesis bias correction according to claim 1, characterized in that, The specific steps of step S3 are as follows: Step S31: Traverse the local spatial units, collect the annotation types of the local spatial units, perform feature processing on each annotation type, fuse the visual sparse point cloud corresponding to different local spatial units with the lidar point cloud, and perform Gaussian generation in the corresponding region of the local spatial unit to obtain Gaussian elements. Step S32: Statistically analyze the Gaussian elements to obtain the Gaussian distribution in the scene space, render the scene space, integrate the rendering status of each Gaussian element participating in alpha synthesis along each ray, calculate the synthesis depth and depth uncertainty of the corresponding pixel of the ray, and correct the deviation of the synthesis depth based on the rendering status of the Gaussian elements; obtain the measured depth of the LiDAR, and construct the depth supervision loss based on the deviation-corrected synthesis depth to optimize the Gaussian distribution.

7. The building reconstruction method based on confidence-adaptive fusion and synthesis bias correction according to claim 6, characterized in that, The specific steps of step S31 are as follows: Step S311: Extract the local cell space of the high-confidence region and obtain the fusion weight coefficient, visual midpoint, radar midpoint, visual plane normal, and radar plane normal corresponding to the local cell space; perform weighted fusion of the visual midpoint and radar midpoint according to the fusion weight coefficient to obtain the Gaussian initial position; perform weighted fusion of the visual plane normal and radar plane normal according to the fusion weight coefficient to obtain the Gaussian initial normal; obtain the lidar point cloud corresponding to the local cell space, collect the point cloud density of the local cell space, set the initial scale of the Gaussian, and obtain the Gaussian primitive from the initial position, initial normal, and initial scale; Step S312: Extract the local cell space marked as the radar-dominant region, obtain the radar midpoint and radar plane normal corresponding to the local cell space, and initialize the Gaussian position and normal; collect the point cloud density of the local cell space, set the initial scale of the Gaussian, and obtain the Gaussian cell by combining the Gaussian position and normal, while recording the visual sparse point cloud of the local cell space as an auxiliary reference. Step S313: Extract the local cell space marked as the visually dominant region, obtain the visual midpoint and visual plane normal corresponding to the local cell space, and initialize the Gaussian position and normal; collect the point cloud density of the local cell space, set the initial scale of the Gaussian, and obtain the Gaussian cell by combining the Gaussian position and normal, while recording the LiDAR point cloud of the local cell space as an auxiliary reference.

8. The building reconstruction method based on confidence-adaptive fusion and synthesis bias correction according to claim 6, characterized in that, The specific steps of step S32 are as follows: Step S321: Collect each Gaussian element that participates in alpha compositing along each ray during the rendering process; calculate the opacity contribution ratio of each Gaussian element to obtain a ratio set; calculate the depth value of each Gaussian element to obtain a depth set; calculate the compositing depth by combining the ratio set and the depth set. The depth uncertainty is calculated from the proportion set, the depth set, and the composite depth. The composite depth is corrected by combining the depth uncertainty with the proportion set; Step S322: Obtain the measured depth of the lidar and the synthesized depth after deviation correction, record the difference between the measured depth of the lidar and the synthesized depth after deviation correction, and construct the depth supervision loss; update and optimize the Gaussian parameters using the depth supervision loss, and record the optimized Gaussian parameters.

9. The building reconstruction method based on confidence-adaptive fusion and synthesis bias correction according to claim 1, characterized in that, The specific steps of step S4 are as follows: Step S41: Based on the lidar point cloud, identify the slender thin-walled structure and record the corresponding spatial range; collect the point cloud distribution of the corresponding space to obtain the local principal extension direction of the slender thin-walled structure, which is used as the principal axis constraint direction of the Gaussian covariance of the spatial range. Step S42: Collect Gaussian data within the spatial range, apply anisotropic regularization terms to the Gaussian data, constrain its covariance in the main extension direction and the cross-sectional scale perpendicular to the main extension direction, suppress the expansion of Gaussian data in the region along non-main extension directions during density adaptive clonal splitting, and record the Gaussian parameters of the slender structural region after anisotropic regularization.

10. The building reconstruction method based on confidence-adaptive fusion and synthesis bias correction according to claim 1, characterized in that, The specific steps of step S5 are as follows: Step S51: Obtain the initial reconstruction results. Based on the multi-source observation data of the scene space, collect the LiDAR verification points that do not directly participate in the training supervision to obtain the LiDAR verification point set. Map the LiDAR verification point set pairs and count the residual errors in different local unit spaces. Step S52: Feedback is provided on the consistency analysis process between the visual sparse point cloud and the lidar point cloud in the local unit space using residual errors, the local unit space is adjusted, and the spatial generation parameters are optimized.

Images