A method for unified registration and reconstruction of multi-source point clouds of building indoor and outdoor

Abstract

The application provides a building indoor and outdoor multi-source point cloud unified registration and reconstruction method, and belongs to the technical field of three-dimensional reconstruction. The method comprises the following steps: acquiring outdoor point cloud, indoor point cloud and indoor point cloud collection tracks of a building, reducing to a unified coordinate system and adding confidence; obtaining first transformation parameters of each source point cloud; establishing a hole corresponding relationship; continuing iteration from the first transformation parameters to obtain second transformation parameters of each source point cloud; constructing absolute anchor point constraints of a pose graph, combining inter-frame constraints and closed loop constraints of the pose graph to optimize the pose graph, obtaining corrected collection tracks; obtaining corrected indoor point cloud and registered outdoor point cloud, and outputting an indoor and outdoor integrated three-dimensional model. The application automatically completes indoor and outdoor multi-source point cloud unified registration without manual control points, and simultaneously eliminates indoor track cumulative drift and makes the indoor and outdoor model splicing place geometrically continuous.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional reconstruction technology, and in particular to a method for unified registration and reconstruction of multi-source point clouds for indoor and outdoor buildings. Background Technology

[0002] An integrated 3D model of a building's interior and exterior needs to simultaneously reflect both the building's external form and internal spatial structure. Due to the limitations of a single sensor's field of view and operating method, such models typically require the joint construction of point clouds from multiple sources, including airborne laser scanning, oblique photography, terrestrial laser scanning, and indoor mobile laser scanning. Each source point cloud exists in its own acquisition coordinate system and must be registered and normalized to the same coordinate system before they can be merged for modeling.

[0003] Existing multi-source point cloud registration methods mostly rely on manually selecting corresponding control points or pre-setting targets to obtain coarse initial registration values. When the building base is obscured or the geometric features of the facade are scarce, there are insufficient corresponding features available, and the solution is prone to falling into an under-constrained state.

[0004] Indoor point clouds are accumulated frame by frame by a mobile scanning device. The pose error drifts as the travel distance accumulates, causing the indoor point cloud to deviate from the true state in the vertical direction and overall orientation. When merged with the outdoor point cloud, misalignment occurs at the stitching point.

[0005] Door and window openings are physical objects that are visible in both indoor and outdoor point clouds. Some studies have established the correspondence between indoor and outdoor point clouds based on this, but most studies calculate a coordinate transformation based on the openings during the final stitching stage. When an opening cannot be completely extracted from one side due to obstruction or voids in the point cloud, that opening must be discarded.

[0006] In summary, existing technologies rely on manual control points for indoor and outdoor point cloud registration, use opening clues only for one-time splicing, and cannot effectively eliminate indoor trajectory drift, resulting in misalignment of indoor and outdoor models and vertical drift. The splicing points often require manual checking and repair at each point. Summary of the Invention

[0007] This invention addresses the technical problems in existing technologies, such as reliance on manual control points for indoor and outdoor point cloud registration, lack of continuous participation of opening clues in the calculation, and lack of absolute outdoor reference constraints for indoor trajectory drift, leading to misalignment and vertical drift in indoor and outdoor model splicing. It provides a unified registration and reconstruction method for multi-source point clouds of buildings, the technical solution of which is as follows: This invention provides a method for unified registration and reconstruction of multi-source point clouds of buildings, including: acquiring outdoor point clouds, indoor point clouds, and the acquisition trajectory of the indoor point clouds; reducing them to a unified coordinate system and adding confidence levels; extracting building planar primitives from each source point cloud and establishing cross-source corresponding point pairs; constructing registration weights based on the confidence levels and normal vector consistency; iteratively solving the problem with the goal of minimizing the weighted residuals to obtain the first transformation parameters of each source point cloud; extracting external openings and internal openings from the outdoor and indoor point clouds after transformation by the first transformation parameters, and establishing an opening correspondence; and defining the boundaries of the opening correspondence. Points are incorporated as constraint point pairs into the weighted residual, and the solution continues iteratively from the first transformation parameter to obtain the second transformation parameters of each source point cloud. A pose map is constructed based on the acquisition trajectory, and absolute anchor point constraints of the pose map are constructed based on the correspondence of the openings. The pose map is optimized by combining the inter-frame constraints and closed-loop constraints of the pose map to obtain the corrected acquisition trajectory. The indoor point cloud is recalculated based on the corrected acquisition trajectory to obtain the corrected indoor point cloud. The outdoor point cloud is transformed based on the second transformation parameter to obtain the registered outdoor point cloud. After merging, surface reconstruction is performed to output an integrated indoor and outdoor 3D model.

[0008] Optionally, obtaining the first transformation parameters for each source point cloud includes: performing planar segmentation on each source point cloud, extracting the dominant plane of the building as the building planar primitive, and extracting the edges and corners of the building based on the orthogonality of adjacent building planar primitives; constructing a building topology graph for each source point cloud with the corners as nodes and the edges as edges, searching for corresponding nodes between the building topology graphs of each source point cloud, and obtaining initial transformation parameters based on the corresponding nodes; using the initial transformation parameters as initial values, performing iterative nearest-point solving according to the registration weights, and iterating until the weighted residual converges when the decrease in the weighted residual between two adjacent iterations is less than a first set proportion, thereby obtaining the first transformation parameters.

[0009] Optionally, in the outdoor point cloud and indoor point cloud transformed by the first transformation parameters, external openings and internal openings are extracted respectively, and a correspondence relationship between the openings is established. This includes: for the outdoor point cloud transformed by the first transformation parameters, taking the main plane of the exterior facade as a reference, extracting the area that is concave along the normal of the main plane of the exterior facade, with a concave depth greater than a set depth and the area of ​​the connected region within a set area range as the external opening; for the indoor point cloud transformed by the first transformation parameters, detecting continuous gaps in the horizontal projection of the wall point cloud along the wall direction, and extracting the area with a gap width within a set width range as the internal opening; and establishing a correspondence relationship between the openings based on the external openings and the internal openings.

[0010] Optionally, a correspondence relationship is established based on the external opening and the internal opening, including: calculating the horizontal distance between the projection centers of the external opening and the internal opening in the unified coordinate system, and the difference in the directional angle between the external opening and the internal opening; when the horizontal distance is less than a preset distance and the difference in directional angle is less than a preset angle difference, it is determined that the external opening and the internal opening are the same physical opening, and the external opening and the internal opening determined to be the same physical opening are recorded in the correspondence relationship.

[0011] Optionally, when only the external opening is extracted and no corresponding internal opening is extracted from the interior point cloud, the method further includes: constructing a tolerance prism that penetrates the wall using the center of the external opening, the normal of the main plane of the exterior facade, and the width and height of the external opening; retrieving the interior wall segments in the interior point cloud that intersect with the tolerance prism, recording the external opening and the interior wall segments as single-ended opening correspondences, and including them in the opening correspondence relationship; for the single-ended opening correspondence, replacing the constraint point pairs with positional consistency constraints along the tangential and vertical directions of the main plane of the exterior facade and incorporating them into the weighted residual; wherein, the covariance of the positional consistency constraint in the wall normal direction is greater than the covariance in the tangential and vertical directions.

[0012] Optionally, the boundary points of the corresponding hole relationships are used as constraint point pairs and incorporated into the weighted residual. The solution is iterated from the first transformation parameter to obtain the second transformation parameters for each source point cloud. This includes: extracting the boundary points of the external and internal holes in the corresponding hole relationships; pairing the boundary points of the external and internal holes according to their boundary orientation to obtain the constraint point pairs; assigning fixed weights to the constraint point pairs, which remain unchanged in each iteration; incorporating the constraint point pairs and the fixed weights into the weighted residual; retaining the constraint point pairs and the fixed weights in each iteration from the first transformation parameter; iterating until the weighted residual converges when the decrease in the weighted residual between two adjacent iterations is less than a second predetermined proportion, thus obtaining the second transformation parameters.

[0013] Optionally, when the correspondence of the openings on the same floor is not less than a preset logarithm, before continuing the iterative solution, the method further includes: using the projection centers and directions of each external opening and each internal opening in the opening correspondence to solve the overall orientation correction amount of the indoor point cloud relative to the outdoor point cloud; correcting the orientation of the indoor point cloud with the overall orientation correction amount, and using the corrected indoor point cloud as the input for continuing the iterative solution.

[0014] Optionally, constructing the absolute anchor point constraint of the pose graph based on the correspondence of openings includes: for each physical opening in the correspondence of openings, using the center of the external opening in the unified coordinate system as the absolute position reference; in each keyframe of the pose graph, retrieving the keyframe with the closest spatial distance to the center of the internal opening as the target keyframe; constructing the positional relationship between the target keyframe and the absolute position reference as the absolute anchor point constraint; wherein, the covariance of the absolute anchor point constraint is determined according to the number of boundary points of the internal opening, and the more boundary points there are, the smaller the covariance of the absolute anchor point constraint.

[0015] Optionally, the closed-loop constraint of the pose graph is adopted after double verification of the candidate closed-loop frame pairs. The candidate closed-loop frame pairs are two non-adjacent keyframes in the pose graph that are determined to correspond to the same location by loop closure detection. The double verification includes: performing registration on the local point clouds corresponding to each of the candidate closed-loop frame pairs, and calculating the overlap rate between the two local point clouds after registration. The local point clouds are the portions of the indoor point clouds collected by each of the candidate closed-loop frame pairs. The intensity reflectance histograms of the two local point clouds are calculated respectively, and the correlation coefficient of the two intensity reflectance histograms is calculated. When the overlap rate is not lower than a preset overlap rate and the correlation coefficient is not lower than a preset correlation coefficient, the candidate closed-loop frame pairs are adopted as the closed-loop constraint; otherwise, they are not adopted.

[0016] Optionally, when the building comprises multiple floors and the absolute anchor point constraints are distributed only on some floors, the method further includes: extracting vertically continuous structural segments from the indoor point clouds of each floor, wherein the vertically continuous structural segments are segments of the same structure that extend continuously between adjacent floors and can be identified across floors; selecting target vertically continuous structural segments from the vertically continuous structural segments, and constructing cross-floor constraints based on the geometric consistency of the target vertically continuous structural segments among segments on adjacent floors; adding the cross-floor constraints to the pose graph to reduce the number of segments in the pose graph during optimization. The relative relationship between the poses of adjacent floors is constrained, so that the pose of floors without the absolute anchor point constraint is subject to the absolute anchor point constraint through the cross-floor constraint; when the number of visible points of the target vertical through-structure segment is insufficient or the geometric consistency of the cross-floor constraint fails the verification, the remaining vertical through-structure segments are used as the target vertical through-structure segments, and the cross-floor constraint is reconstructed; wherein, the covariance of the cross-floor constraint increases with the increase of the plane fitting residual of the target vertical through-structure segment, and increases with the increase of the number of interval floors transmitted by the cross-floor constraint.

[0017] The beneficial effects of this invention are: The system acquires outdoor point clouds, indoor point clouds, and the acquisition trajectory of indoor point clouds of buildings. It then normalizes each source point cloud to a unified coordinate system and adds confidence levels, so that point clouds from different devices and with different precisions have comparable coordinate benchmarks and quantifiable credibility, providing a basis for subsequently treating each source point cloud differently according to data quality.

[0018] Building planar primitives are extracted from each source point cloud and cross-source corresponding point pairs are established. Stable geometric structures such as the building's roof, facade, and floor are used to replace manually laid control points and establish corresponding relationships with targets, thus obtaining the correspondence basis between cross sources without manual intervention. Registration weights are constructed based on confidence level and normal vector consistency, so that high-precision point clouds occupy a larger component in the solution and suppress erroneous correspondences with inconsistent surface orientations. Then, the solution is iteratively solved with the goal of minimizing the weighted residuals to obtain the first transformation parameters of each source point cloud, thereby initially reducing each source point cloud to the same coordinate system and providing a common position benchmark for the detection and comparison of openings.

[0019] In the outdoor and indoor point clouds after transformation by the first transformation parameters, external and internal openings are extracted and the corresponding relationship between the openings is established. Since the indoor and outdoor point clouds are now in the same coordinate system and the deviation has converged to a comparable range, the positions and orientations of the same door and window on both sides can be directly compared. Based on this, the corresponding relationship between the openings on both sides can be automatically determined without the need for manual identification of the named points.

[0020] The boundary points corresponding to the openings are used as constraint point pairs and incorporated into the weighted residuals. The solution is iterated from the first transformation parameter to obtain the second transformation parameters of each source point cloud. This ensures that the opening, as the only common physical element between the indoor and outdoor areas, continues to play a constraint role in each subsequent iteration, rather than participating in only one coordinate transformation. The degree of fit between the indoor and outdoor point clouds at the openings tightens continuously with each iteration.

[0021] A pose graph is constructed based on the acquired trajectory. Absolute anchor point constraints for the pose graph are constructed based on the correspondence between the openings. Since the external openings are located in the outdoor point cloud with absolute coordinates, their positions can be used as absolute references for the indoor trajectory. This allows the trajectory, which was originally closed only within the indoor data, to obtain an external reference from the outdoor coordinate system. The pose graph is optimized by combining the inter-frame constraints and closed-loop constraints of the pose graph to obtain the corrected acquired trajectory. This allows the pose error accumulated along the way to be redistributed and eliminated on the entire trajectory. The deformation caused by frame-by-frame iteration within the indoor point cloud is thus corrected. This correction is something that rigid transformation cannot achieve.

[0022] The indoor point cloud is recalculated based on the corrected acquisition trajectory to obtain the corrected indoor point cloud. At this time, the coordinates of the indoor point cloud are directly given by the trajectory constrained by the absolute anchor point, without the need for overall transformation. The outdoor point cloud is transformed based on the second transformation parameter to obtain the registered outdoor point cloud, so that the outdoor source point clouds are returned to the unified coordinate system. The two are merged and surface reconstruction is performed to output an integrated indoor and outdoor 3D model.

[0023] Therefore, this invention can automatically complete the unified registration of indoor and outdoor multi-source point clouds without the need for manual control points, and simultaneously eliminate the cumulative drift of indoor trajectories, so that the indoor and outdoor models are geometrically continuous at the splicing point, thus achieving the technical effect of improving the automation level and geometric consistency of integrated indoor and outdoor modeling of buildings. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating a method for unified registration and reconstruction of multi-source point clouds inside and outside buildings, provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the process for establishing the correspondence between openings provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the process for obtaining the second transformation parameter according to an embodiment of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] like Figure 1 As shown, this embodiment of the invention provides a method for unified registration and reconstruction of multi-source point clouds for indoor and outdoor buildings, including: S1: Obtain the outdoor point cloud, indoor point cloud, and acquisition trajectory of the indoor point cloud of the building, reduce them to a unified coordinate system, and add confidence level.

[0027] The building's external form and internal space were acquired using different methods. The coordinate references, point cloud densities, and measurement accuracy used in each acquisition were different. They needed to be incorporated into the same coordinate framework and their respective reliability levels indicated before subsequent steps could compare geometric relationships on a unified location reference and treat each source point cloud differently based on data quality.

[0028] Outdoor point clouds come from three sources: airborne laser scanning point clouds, oblique aerial imagery dense matching point clouds, and ground-based laser scanning point clouds.

[0029] Among them, the airborne laser scanning point cloud is acquired from above the building by a laser scanner carried by a drone, covering the roof and upper part of the facade. The point cloud density is relatively low, with about dozens of points per square meter. It contains position and attitude information recorded by satellite positioning and inertial navigation and is located in the geodetic coordinate system.

[0030] The oblique aerial image dense matching point cloud is generated by dense matching of images taken by UAV from multiple oblique perspectives. Its coverage is similar to that of airborne laser scanning point cloud, contains color information, and has slightly lower geometric accuracy.

[0031] Ground laser scanning point clouds are acquired by scanners mounted on tripods at substations around the building, covering the building base and lower part of the facade. The point cloud density can reach thousands of points per square meter, and the single-station ranging accuracy can reach the millimeter level. Each station is in a local coordinate system with the station as the origin.

[0032] Indoor point clouds are acquired using backpack-style or handheld mobile laser scanning equipment. Personnel carry the equipment as they walk through the building's interior passageways. During this movement, the equipment continuously scans and simultaneously calculates its own position and orientation, accumulating frame by frame to form the indoor point cloud. While calculating its own pose, the equipment records a pose sequence, which constitutes the acquisition trajectory. Each recording moment corresponds to a keyframe, and the pose of a keyframe includes three position components and three orientation components. The position components are the equipment's three-dimensional coordinates at that moment, and the orientation components are the equipment's rotation angles around the three coordinate axes, totaling six degrees of freedom, which together determine the equipment's spatial position and orientation at that moment. Each point in the indoor point cloud is determined by the pose of its corresponding keyframe and the observation value of that point in the equipment's coordinate system. Therefore, the overall shape of the indoor point cloud depends on the accuracy of the acquisition trajectory.

[0033] Coordinate reduction was performed separately for each source point cloud based on its coordinate reference. The point cloud from the airborne laser scanning imagery, densely matched with the oblique aerial imagery, contained geodetic coordinates and was transformed to the engineering coordinate system using a seven-parameter coordinate transformation model. This model included three translation parameters, three rotation parameters, and one scale parameter. The parameter values ​​were pre-calculated using least-squares adjustment from at least three evenly distributed known common points within the project area. The ground-based laser scanning point cloud was first stitched together from each station into a single entity, and then reduced to the engineering coordinate system based on the station's positioning records.

[0034] Indoor point cloud acquisition begins outside the building. Personnel carrying equipment walk a short distance outside the entrance before entering the building. The initial section scans the same area of ​​the first-floor facade, floor, and ground surface, allowing the starting position of the acquisition trajectory to be mapped to the engineering coordinate system, thus providing the initial coordinate reference for the indoor point cloud. If site conditions prevent starting from outside the building, the equipment positioning record at the starting point of the acquisition trajectory is used as the initial position. While this initial position is less accurate than the external one, it is corrected in subsequent registration and pose map optimization steps.

[0035] The above reduction only ensures that each source point cloud has a consistent coordinate reference. Due to limitations in the accuracy of transformation parameters, inter-station stitching accuracy, and equipment positioning accuracy, residual deviations ranging from centimeters to decimeters still exist between the source point clouds after reduction. These deviations are eliminated by registration in subsequent steps.

[0036] Differences in data quality and density among the source point clouds can also affect registration, so they need to be preprocessed before registration. Therefore, denoising and downsampling are performed on each source point cloud.

[0037] Noise denoising employs statistical filtering. For each point, its k nearest neighbors are retrieved (k = 30), and the average distance from the point to each neighbor is calculated. The average distance of all points in the same source point cloud is statistically analyzed, yielding the mean μ and standard deviation σ. Points with an average distance exceeding μ + 2.5σ are identified as outliers and removed. Multiple reflections from the glass curtain wall, airborne particles, and noise generated by pedestrians during construction have significantly larger average neighborhood distances than points on the building surface. These can be effectively identified and removed to prevent them from being mistakenly identified as corresponding points in subsequent correspondence searches. The removal threshold μ + 2.5σ is obtained from the statistical analysis of all points in the same source point cloud, reflecting the overall dispersion level of the source point cloud.

[0038] When the average neighborhood distance of points within a local area is significantly higher than the median level of this statistic in the source point cloud, or when the proportion of points in that area with fewer than a set number of effective neighbors is significantly increased, that area is classified as a low signal-to-noise ratio (SNR) area. Underground spaces with insufficient artificial lighting, glass curtain walls, and dark-colored surfaces with high surface absorptivity typically exhibit this characteristic. Point clouds within low SNR areas are inherently more discrete; if points are still removed at μ+2.5σ, true wall and column structural points will be deleted along with them. Therefore, the removal threshold for this area is relaxed to no more than μ+3σ, allowing effective structural points to be retained. Simultaneously, low-confidence markers are added to the retained points in this area to reduce their impact in subsequent calculations.

[0039] Noise denoising focuses on the reliability of individual points, while density differences between source point clouds are addressed through downsampling. The original density of ground-based laser scanning point clouds and indoor mobile laser scanning point clouds can be more than 10 times that of densely matched point clouds from airborne laser scanning and oblique aerial imagery. If all source point clouds are thinned using the same standard, a large number of points on the dense side will compete for a small number of points on the sparse side during the correspondence search, causing the solution result to shift towards the dense side. Therefore, downsampling needs to be performed at different levels according to the original density of each source point cloud. The downsampling method uses a voxel grid approach, dividing the space into cubic voxels with fixed side lengths, retaining only one representative point within each voxel. Airborne laser scanning point clouds and densely matched point clouds from oblique aerial imagery use 0.10 m voxels, while ground-based laser scanning point clouds and indoor mobile laser scanning point clouds use 0.03 m voxels. After this processing, the density difference between the source point clouds can be reduced to within 3 times.

[0040] The differences in measurement accuracy among the various source point clouds need to be treated differently in the solution process; therefore, a confidence level is assigned to each source point cloud. The confidence level is assigned based on the nominal accuracy of the equipment used for each source point cloud, and is taken as the single-point measurement error of that source point cloud: 0.005m for terrestrial laser scanning point clouds, 0.02m for indoor mobile laser scanning point clouds, 0.03m for oblique aerial imagery dense matching point clouds, and 0.05m for airborne laser scanning point clouds. While the density of indoor mobile laser scanning point clouds is similar to that of terrestrial laser scanning point clouds, their pose is obtained through frame-by-frame recursion, resulting in higher measurement uncertainty; therefore, a larger confidence level is assigned. The confidence level is passed along with the point cloud to subsequent steps. In S2, when constructing registration weights, the base weights are normalized using inverse variance. Point clouds with smaller single-point measurement errors receive larger base weights. For points with added low-confidence markers, their base weights are multiplied by a reduction factor of 0.5 based on the normalized result.

[0041] At this point, the point clouds from each source are in the same engineering coordinate system, their densities are close, and each point has a confidence level that characterizes its measurement uncertainty. The indoor point cloud is also accompanied by a corresponding acquisition trajectory, which can be used for subsequent steps to extract geometric primitives and establish cross-source correspondences.

[0042] S2: Extract building plane primitives from each source point cloud and establish cross-source corresponding point pairs. Construct registration weights based on the confidence level and normal vector consistency. Iterate and solve the problem with the goal of minimizing the weighted residual to obtain the first transformation parameters of each source point cloud.

[0043] After S1 reduction, although the source point clouds are now in the same engineering coordinate system, there are still residual deviations between them ranging from centimeters to decimeters. This step eliminates these residual deviations, ensuring that the source point clouds are truly geometrically aligned, providing a common positional reference for subsequent inspection of the opening and comparison of their positions.

[0044] Alignment is achieved based on the building's own geometric structure. Components such as the building's roof, facade, floor, interior walls, and floors appear as large and stable planes in the point cloud, which do not change with the acquisition method or viewing angle. They can be repeatedly extracted from point clouds from various sources, and thus extracted as building planar primitives, serving as the basis for establishing cross-source correspondences.

[0045] The intersection of adjacent building planar primitives forms building edges. The common intersection of multiple edges forms corner points. Corner points correspond to corners, eaves, and other turning points on the building. They are geometrically clear feature points that can be identified one by one in point clouds from different sources.

[0046] By establishing the correspondence between corner points among the source point clouds, a relatively coarse transformation parameter, called the initial transformation parameter, can be obtained. The corresponding registration residual is typically on the order of 0.3m. The extraction method of building planar primitives, the organization method of corner points, and the solution method of the initial transformation parameter will be explained in detail in the following sections.

[0047] The normal vector of each point is calculated simultaneously when extracting the building planar primitives. For each point, its 15 nearest neighbors are retrieved, and a local plane is fitted using the least squares method. The perpendicular direction of this local plane is taken as the normal vector of that point. The normal vector represents the orientation of the surface on which the point is located and is used to determine whether the correspondence is reasonable when constructing the registration weights.

[0048] The corner point correspondences between the three sources of outdoor point clouds can be directly established using the methods described above. However, this is not the case for indoor point clouds. The architectural planar primitives of indoor point clouds are interior walls, floors, and ceilings, which are not the same surfaces as the roofs, facades, and floors extracted from outdoor point clouds. Furthermore, the inner and outer sides of the same wall are separated by one wall thickness, and the corner point positions on both sides also differ by the same value. Therefore, the corner points of indoor point clouds do not directly correspond to the corner points of outdoor point clouds.

[0049] The initial transformation parameters of the indoor point cloud are obtained using the initial acquisition segment. As explained in S1, the acquisition of the indoor point cloud begins from the exterior of the building. The first-floor facade and ground level scanned in the initial segment are both within the coverage area of ​​the outdoor point cloud. By extracting the building planar primitives and corner points from the initial segment and establishing a correspondence with the ground laser scan point cloud, the initial transformation parameters of the indoor point cloud can be obtained. If the acquisition does not begin from the exterior of the building, the coarse matching results of the external and internal openings at the first-floor entrance are used instead, and the accuracy of the resulting initial transformation parameters is lower than that obtained based on the initial segment.

[0050] The number and distribution of corner points determine the reliability of the initial transformation parameters. When there are 4 to 6 pairs of corresponding corner points obtained between two source point clouds, although the number is sufficient for the solution, the distribution is often concentrated in the local area of ​​the building, resulting in a weakly constrained solution and easy divergence of the attitude solution. The walls of most buildings are arranged along two mutually perpendicular principal axes, and the direction of the principal axes does not change with local occlusion. These are stable and repeatable global features that can be used to supplement the constraints in such cases.

[0051] The building's principal axis direction is determined as follows: Near-vertical main wall surfaces are extracted from each source point cloud, and their horizontal normals are taken. All horizontal normals are clustered periodically at 180°, grouping walls with opposite orientations into the same cluster. The cluster center with the largest weight, weighted by the support area of ​​each wall surface and the fitted residual, is taken as the building's principal axis direction of that source point cloud. The difference between the building's principal axis directions of the two source point clouds is used as a yaw angle auxiliary constraint, which participates in the rotation solution together with the corner point residuals. The weight of the auxiliary constraint increases as the spatial coverage of the corresponding corner point decreases or the degree of collinearity increases. If the number of near-vertical main wall surfaces is insufficient to form an effective cluster, or if the difference between the yaw angle auxiliary constraint and the corner point solution exceeds a set consistency threshold, it indicates that one of them is incorrect. The auxiliary constraint is then removed, and coarse alignment is initiated.

[0052] When there are fewer than four pairs of corresponding corner points, the solution is no longer based on corner points, and coarse alignment is adopted. Specifically, the minimum bounding rectangle of the building for each of the two source point clouds is calculated separately. The horizontal rotation is determined by the difference in the major axis direction of the two, and the translation is determined by the prior pose recorded by the acquisition equipment using satellite navigation or inertial navigation. This yields the initial transformation parameters. The attitude error obtained by coarse alignment is usually within 5°. Although it is inferior to the result of solving based on corner points, it is sufficient as the initial value for iterative solution. In cases where the building base is obscured by greenery and vehicles, or the facade is a full glass curtain wall making it difficult to extract complete planar primitives, this process allows the solution process to continue without interruption.

[0053] After obtaining the initial transformation parameters, iterative calculations are performed using these as initial values ​​to further converge the residual bias. During the calculation, the point cloud with the smallest single-point measurement error among all source point clouds is designated as the reference point cloud, its position remaining unchanged. The remaining source point clouds are designated as the point clouds to be registered. Each iteration proceeds as follows: the point cloud to be registered is transformed to the location of the reference point cloud using the current transformation parameters; for each point in the point cloud to be registered, the nearest spatially distant point in the reference point cloud is retrieved, forming a cross-source corresponding point pair; registration weights are calculated for each cross-source corresponding point pair; and new transformation parameters are solved with the goal of minimizing the weighted residual. The weighted residual is the sum of the products of the squared distances between points in all cross-source corresponding point pairs and their registration weights. After one iteration, the point cloud position changes, and the nearest point relationship changes accordingly, requiring a re-retrieval of the correspondence and a re-solution. This process is repeated until convergence. The transformation parameters obtained after convergence are the first transformation parameters, and the corresponding registration residual in the overlapping area can reach the order of 0.03m. The phased execution method of iteration and the convergence criteria will be explained in detail in the following sections.

[0054] The registration weight consists of two parts: the base weight and the normal vector consistency weight, which are taken as their product. The base weight is obtained by normalizing the confidence level added in S1 according to the inverse variance. The smaller the error in the single-point measurement, the larger the base weight of the point cloud, so that high-precision data such as ground laser scanning point clouds will account for a larger proportion in the solution. For points with added low confidence markers, the base weight is reduced according to the reduction factor given in S1.

[0055] Cross-source corresponding point pairs are retrieved based on spatial distance. Two points located on adjacent but different walls may be retrieved as a pair due to their proximity. However, these point pairs are not actually corresponding points, and their participation in the solution will drag the transformation parameters in the wrong direction. Points on the same surface have the same normal vector orientation, while points on different surfaces have different normal vector orientations. This allows for the identification of mismatched point pairs. Normal vector consistency is determined by the dot product of the normal vectors of the two points in the cross-source corresponding point pair. Specifically, when two points are located on surfaces with the same orientation, the dot product is close to 1; when they are located on surfaces with different orientations, the dot product is close to 0 or negative. When a negative value is taken, the registration weight is reset to 0. The registration weight of mismatched point pairs is thus reduced or reset to 0, no longer affecting the solution results.

[0056] The base weights are assigned based on the nominal accuracy of the equipment, reflecting the overall accuracy level of each source point cloud. However, the point cloud quality in areas such as glass curtain walls and walls with weak textures is often lower than the overall level of the same source point cloud, and such local differences cannot be reflected by the base weights. Therefore, in the later stages of convergence, after the weighted residual decreases below the set proportion for the first time and the cross-source corresponding point pairs tend to stabilize, the reliability factor is activated to dynamically adjust the registration weights. Specifically, for cross-source corresponding point pairs whose residuals exceed the set residual threshold for multiple consecutive rounds, their reliability factor is continuously reduced according to the ratio of their residuals to the set residual threshold. At this time, the registration weight is the product of the base weights, the normal vector consistency, and the reliability factor. The reliability factor has a non-zero lower limit, and it gradually increases as the residuals fall back to within the set residual threshold, allowing point pairs with temporarily large residuals due to occlusion or other reasons to recover their weights. Activating this in the later stages of convergence is chosen because in the early stages of iteration, the source point clouds are not yet aligned, and the residuals are generally large. At this time, the reduction would misjudge normal point pairs as poor-quality data.

[0057] During the solution process, the residuals of corresponding point pairs across sources are recorded point by point to form point-by-point residual records, which are used in subsequent steps to locate areas with poor stitching quality and trigger local re-registration.

[0058] Through S2, the source point clouds are geometrically aligned after being transformed by the first transformation parameter, and the registration residuals of the overlapping areas converge to the centimeter level. The outdoor point cloud and the indoor point cloud are on the same location reference, and the positions and orientations of the same door and window on both sides can be directly compared.

[0059] S3: Extract the external openings and internal openings from the outdoor point cloud and indoor point cloud after transformation by the first transformation parameters, and establish the correspondence between the openings.

[0060] After S2 is completed, the outdoor and indoor point clouds are on the same location reference, but at this point, they only have a geometric alignment relationship and lack identifiable physical counterparts. The indoor and outdoor spaces are separated by walls, and their surfaces are not visible to each other. The only physical objects that appear simultaneously in both point clouds are door and window openings. A window, viewed from the outside, is a recessed area on the facade; viewed from the inside, it is a notch in the wall. Both belong to two surfaces of the same component. This step detects the openings in both point clouds, determines which external openings and which internal openings belong to the same door or window, and the result is the opening correspondence.

[0061] The detection of openings must be performed after the first transformation parameter transformation. The shapes of doors and windows appear differently in the point clouds on both sides, making direct attribution impossible based on shape alone. The usable criterion is that their positions and orientations in space should be consistent. For positions and orientations to be comparable, the point clouds on both sides must be in the same coordinate system and the deviation must converge to below the door / window scale. After S2, the registration residual is on the order of centimeters, far smaller than the dimensions of the door / window itself and the distance between adjacent doors / windows. Therefore, the detection results of the same door / window on both sides fall into adjacent positions.

[0062] External openings are detected using the building facade as a reference. The building facade appears as a large plane in the point cloud. Door and window components are installed within wall openings, with window frames and opening sidewalls located inside this plane. In the point cloud, these are represented as recessed areas along the normal to the main plane of the facade. Glass surfaces are typically difficult to obtain valid points for due to reflection and penetration. Specifically, the main plane containing the facade is extracted from the outdoor point cloud. The distance from each point to this main plane is calculated. Points with a recess depth exceeding a set depth are selected and clustered based on connectivity. Each connected region is a candidate region. The projected area of ​​each candidate region on the facade is calculated. Areas within a set range are identified as external openings.

[0063] By setting a depth, decorative moldings and grout lines on the facade are excluded, as the recessed area of ​​such structures is much smaller than the depth of door and window openings. By setting an area range, excessively small ventilation openings, reserved holes, and excessively large curtain wall partitions are excluded, thus limiting the detection results to the size range of conventional doors and windows.

[0064] The detection of internal openings is based on the continuity of the wall. In the point cloud, interior walls appear as a continuous band of points along the wall's orientation. Where doors and windows are located, openings disrupt the wall, creating gaps in the point band. Specifically, the wall surface is extracted from the interior point cloud, its orientation is determined, and the wall points are projected onto a horizontal plane. The distribution of projected points is statistically analyzed segment by segment along the wall's orientation, and continuous segments without points are considered candidate segments. Candidate segments whose width falls within a set width range are identified as internal openings. The width range is set according to the standard width of door and window openings, thus excluding narrow gaps caused by partial wall defects and large gaps caused by the loss of entire walls.

[0065] The orientation of an opening refers to the projection direction of the normal of the wall where the opening is located onto the horizontal plane, representing the orientation of the opening towards the outside of the building. The detection results of the openings on both sides each have their own orientation.

[0066] After detecting the openings on both sides, their affiliation is determined based on their location and orientation. For the same door or window, the external and internal openings are located on two different faces of the wall, with their projection centers on the horizontal plane no more than one wall thickness apart, and their orientations should be basically consistent. Two openings belonging to different doors or windows either have projection centers that are far apart or are located on walls with different orientations. The specific determination method is as follows: calculate the horizontal distance between the projection centers of the external and internal openings in a unified coordinate system, and the difference in their orientation angles. If both are within the set limits, they are considered the same physical opening and recorded in the opening correspondence relationship.

[0067] By establishing a physical correspondence between outdoor and indoor point clouds, with door and window openings as the carriers, a constraint basis is provided for subsequent operations.

[0068] S4: The boundary points corresponding to the openings are used as constraint point pairs and incorporated into the weighted residual. The solution is iterated from the first transformation parameter to obtain the second transformation parameters of each source point cloud.

[0069] The first transformation parameters obtained by S2 are solved with the objective of minimizing the weighted residuals of all cross-source corresponding point pairs. Each point pair involved in the solution affects the solution result according to its quantity and weight. The overlapping area between the outdoor point cloud and the indoor point cloud is limited to the door and window openings and their vicinity, accounting for a very small proportion of the total number of points in the overall solution. Its residuals are diluted by the numerous corresponding point pairs inside the outdoor point cloud. Therefore, although the first transformation parameters converge overall, the degree of fit between the indoor and outdoor sides at the openings may not be equal, and misalignment may still occur at the splicing point after merging.

[0070] Based on the correspondence between the openings established in S3, the boundary points at the openings are incorporated into the solution as constraint point pairs, so that the residuals at the openings are no longer diluted by the other point pairs in the solution.

[0071] The correspondence between openings records which external opening and which internal opening belong to the same door or window. Their boundaries are represented by a ring of contour points on each side of the point cloud, describing the same edge of the same component. Pairing the boundary points on both sides according to their boundary direction yields a batch of constraint point pairs. These pairs are similar in nature to the cross-source correspondence point pairs obtained in S2 based on nearest distance retrieval; both indicate that the two points correspond to the same physical location. The difference lies in the fact that the correspondence of the constraint point pairs originates from the entity affiliation of the door or window component, making them more reliable than the results obtained based on distance retrieval.

[0072] The constraint point pairs are incorporated into the weighted residuals, that is, residual terms consisting of constraint point pairs are added to the weighted residuals established in S2. Subsequent solutions still aim to minimize the weighted residuals, and the iteration process is the same as in S2.

[0073] The handling of constraint point pairs and cross-source corresponding point pairs differs during iteration. Cross-source corresponding point pairs are retrieved based on the nearest distance at the current position. After each iteration, the point cloud position changes, and the correspondence needs to be retrieved again. The correspondence of constraint point pairs is determined by the entity affiliation of the door and window components and does not change with the point cloud position. Therefore, it is always retained in each iteration and always participates in the solution with its fixed weight.

[0074] The iteration does not restart from the initial value, but continues from the first transformation parameter. The first transformation parameter has already converged the source point clouds to the centimeter level. From this point onward, the solution only requires adjustments within a small range, requiring fewer rounds to converge and avoiding the destruction of the already aligned remaining regions due to large adjustments. The transformation parameters obtained after convergence are the second transformation parameters.

[0075] After being transformed by the second transformation parameter, the source point clouds are aligned as a whole and fit together at the door and window openings, thus converging the misalignment of the indoor and outdoor splicing parts.

[0076] S5: Construct a pose graph based on the acquisition trajectory, construct absolute anchor point constraints for the pose graph based on the correspondence between the openings, optimize the pose graph by combining the inter-frame constraints and closed-loop constraints of the pose graph, and obtain the corrected acquisition trajectory.

[0077] After S4 is completed, the indoor point cloud is transformed by the second transformation parameter, and its overall position and orientation are consistent with the outdoor point cloud. The opening is also aligned. However, the shape of the indoor point cloud itself is still distorted.

[0078] As explained in S1, each point in the indoor point cloud is determined by the pose of its corresponding keyframe and the observation value of that point in the device coordinate system. The shape of the point cloud depends on the accuracy of the acquisition trajectory. In contrast, the pose of the mobile laser scanning device is obtained frame by frame through the matching relationship between adjacent frames. Each frame's calculation contains an error, which accumulates along the direction of travel, resulting in drift.

[0079] The drift increases with the distance traveled, becoming most significant in the middle of the trajectory, with the vertical component being dominant. This means that the indoor point cloud is not shifted as a whole, but rather the shift varies from point to point within it. The transformation parameters obtained by S2 and S4 are both rigid transformations, meaning that the displacement of each point is the same when applied to the point cloud. While they can correct the overall position and orientation, they cannot correct the distortion caused by unequal shifts at different points within the cloud. To eliminate this distortion, it is necessary to correct the pose of each keyframe in the acquisition trajectory and then recalculate the point cloud coordinates based on the corrected pose. This step completes the correction of the acquisition trajectory.

[0080] The corrections are performed on the pose graph. The pose graph is the rewritten graph structure of the acquired trajectory. Specifically, the poses of each keyframe in the acquired trajectory are taken as nodes of the graph, and the nodes are connected by edges. Each edge represents a constraint on the pose of the relevant node, which contains the relationship that these poses should satisfy and the reliability of the relationship.

[0081] The unknown quantity in the pose graph is the pose of all nodes. The relationships represented by the edges are not completely compatible. The optimization goal is to find a set of poses that minimizes the sum of the discrepancies of all constraints after weighting them according to their credibility. The credibility is represented by covariance. The smaller the covariance, the more credible the constraint is, and the lower its discrepancy is during optimization.

[0082] The pose graph contains three types of edges: inter-frame constraints, closed-loop constraints, and absolute anchor constraints.

[0083] Inter-frame constraints connect two adjacent keyframes, and their content is the change in position and attitude of the device from the previous frame to the next frame. It is obtained by registering the local point cloud obtained from the scanning of the two frames and is the basic basis for the formation of the acquisition trajectory.

[0084] Loop closure constraints are generated by loop closure detection. When the device reaches a previously visited location, loop closure detection compares the current keyframe with previously recorded keyframes. If they are determined to be likely to be at the same location, they are listed as candidate loop closure frame pairs. After the candidate loop closure frame pairs are verified and confirmed, loop closure constraints are established between the two keyframes. The constraints consist of the relative pose between the two frames, obtained by local point cloud registration of the two frames, and are used to mitigate the drift accumulated between the two frames.

[0085] Absolute anchor point constraints do not connect two nodes, but constrain the pose of a node to a known absolute position, providing an external reference for the entire pose graph.

[0086] The observables for inter-frame constraints and closed-loop constraints are the relative transformations between the poses of two keyframes. When the same rigid transformation is applied to all node poses, the relative relationship between any two poses remains unchanged, and the discrepancies of each edge also remain unchanged. Therefore, the normal equations have a null space corresponding to the overall six degrees of freedom, and the pose solution is not unique. The observables for absolute anchor point constraints are the absolute positions of a single keyframe pose in the engineering coordinate system. Their discrepancies change with the overall transformation, thus eliminating the null space, and the solution of the pose graph can be uniquely determined in the engineering coordinate system.

[0087] The absolute anchor point constraint originates from the opening correspondence established in S3. External openings of the same physical opening are located in the outdoor point cloud. After transformations in S2 and S4, the outdoor point cloud is placed in the engineering coordinate system, and its position has absolute significance. Internal openings are located in the indoor point cloud, allowing for the determination of keyframes adjacent to them on the acquisition trajectory. Constraining the pose of this keyframe to the absolute position determined by the external opening yields an absolute anchor point constraint. The keyframe recorded when a worker passes through a door or window has its position given by the observation results from the outdoor side, thus fixing the acquisition trajectory in the engineering coordinate system.

[0088] After constructing the three types of edges in the pose graph, optimization is performed. The optimization is a nonlinear least squares solution, which can be performed iteratively using the Levenberg-Marquardt algorithm. In each round, the discrepancy value of each edge is calculated based on the current pose. A normal equation is constructed using the weighted average of the edge covariances, and the pose update is calculated and applied to each node. Iteration continues until the update is less than a set limit, at which point convergence occurs. During the solution process, the structure of the pose graph remains unchanged; only the pose values ​​of each node change. The poses of each keyframe read after convergence are the corrected acquisition trajectory. Through this optimization, the pose at the anchor point is pulled to its absolute position, and the poses of each frame along the path are adjusted accordingly under the influence of inter-frame constraints and closed-loop constraints. The accumulated drift is redistributed and absorbed along the entire trajectory, and the vertical drift can converge from the order of 0.10m to the order of 0.02m.

[0089] The corrected acquisition trajectory is obtained through pose graph optimization. The pose of each keyframe in the acquisition trajectory becomes the absolute reference in the engineering coordinate system, and the drift accumulated by frame-by-frame recursion is eliminated. The corrected acquisition trajectory is used in S6 to recalculate the coordinates of the indoor point cloud.

[0090] S6: Based on the corrected acquisition trajectory, recalculate the indoor point cloud to obtain the corrected indoor point cloud. Transform the outdoor point cloud based on the second transformation parameters to obtain the registered outdoor point cloud. After merging, perform surface reconstruction to output an integrated indoor and outdoor 3D model.

[0091] All sources of the outdoor point cloud are statically acquired. Points obtained from a single station setup or flight are acquired in the same coordinate system at once. Their positional relationships do not contain cumulative errors. By applying a rigid transformation to the whole, their coordinates in the engineering coordinate system can be obtained. The parameters used are the parts of the second transformation parameters obtained by S4 corresponding to each outdoor point cloud source. The result obtained after the transformation is the registered outdoor point cloud.

[0092] The indoor point cloud is obtained by accumulating frame by frame. The coordinates of each point are determined by the pose of its key frame and the observation value of the point in the device coordinate system. S5 corrects the pose of each key frame. Therefore, the coordinates of the indoor point cloud are obtained in the following way: take the pose of each key frame in the corrected acquisition trajectory, and recombine it with the original recorded observation value of each point in the device coordinate system. Solve the coordinates of each point in the engineering coordinate system. The result is the corrected indoor point cloud.

[0093] The corrected acquisition trajectory is fixed to the engineering coordinate system by absolute anchor point constraints. The indoor point cloud obtained from this calculation is located in the engineering coordinate system, and there is no need to apply the corresponding part of the indoor point cloud in the second transformation parameters. The function of this part of the parameters is to place the indoor point cloud at the position where the opening constraint can be applied in S4, and its effect is already covered by the corrected acquisition trajectory. If it is applied repeatedly after the calculation, the indoor point cloud will be transformed twice and deviate from the correct position.

[0094] After obtaining the coordinates of the point clouds on both sides, their fit at the splicing points is checked. The area to be checked is the region where the indoor and outdoor point clouds meet, namely the area corresponding to the opening and the boundary between the wall and the floor. A 1m wide boundary buffer zone is constructed along the above area. When searching for nearest neighbors, only points existing on the initiating side are counted. If the initiating side has no points at a certain location, no statistics are generated for that location. Since unidirectional searching can only reflect the absence on one side, bidirectional searching is performed within the buffer zone. Specifically, the nearest point in the registered outdoor point cloud is searched using points from the corrected indoor point cloud, and the nearest point in the corrected indoor point cloud is searched using points from the registered outdoor point cloud. The distribution of nearest neighbor distances in both directions is then combined and counted. Through the above bidirectional searching, any misalignment or absence on either side can be reflected as an increase in the nearest neighbor distance. When the proportion of point pairs with a nearest neighbor distance greater than 0.05m exceeds 15%, the fit is deemed unqualified and marked as an area to be refined.

[0095] The processing of the area to be refined is divided into two levels. The first is local deformation correction. Specifically, using the point cloud within a 0.5m range outside the boundary buffer zone as a fixed anchor point, thin-plate spline deformation is applied to the point cloud within the buffer zone to make the point clouds on both sides fit together at that point. Thin-plate spline deformation can generate a continuous and smooth displacement field under the constraint of the fixed anchor point, without producing new wrinkles or abrupt changes at the correction point. The upper limit of the deformation amplitude is 0.08m. If it still cannot fit together after exceeding this amplitude, it indicates that the deviation at that point is not caused by the registration residual, but by missing data or the existence of systematic errors. Continuing to increase the deformation will change the true geometry of the building, so no further deformation is applied.

[0096] The second method is local re-registration. Specifically, the range of the region to be refined and the point-by-point residuals recorded in S2 are fed back to the registration solution. Only the point cloud within this region is re-solved using weighted iteration. The overall solution aims to minimize the weighted residuals of all point pairs. The residuals of the local region are averaged by the remaining regions. The local re-registration criterion is solved individually for the point pairs within that region, thus converging to a lower residual. It has a small range, good initial values, and requires fewer rounds to converge. The resulting transformation parameters only apply to the point cloud within this region, without changing the remaining regions or re-correcting the acquisition trajectory.

[0097] Areas that still fail the above processing will not be forcibly corrected but will be added to the checklist. The checklist records the bounding area, average normal, point cloud missing rate, and the floor and wall segment to which each area belongs, and is output along with the model. For areas requiring review due to missing data, further suggestions for supplementary data collection can be provided. For example, based on the existing point cloud construction occupying voxels, for missing areas on the exterior facade, candidate station areas and suggested observation directions are output from locations within sight of the area after occupancy detection and incident angle filtering; for missing indoor areas, supplementary data collection path segments and the area to be covered are output based on the free space connectivity of the floor. The supplementary data collection suggestions provide areas and directions, not unique coordinate points, which are used by the operators to select locations on-site.

[0098] After inspection and correction, the point clouds from both sides were merged, and duplicates were removed from overlapping areas. The indoor and outdoor point clouds overlapped at and near door and window openings, and there was also significant overlap between the various sources of the outdoor point cloud on the building facade. Consequently, the same surface recorded points from multiple sources. Direct merging would abnormally increase the point density at the overlapping areas, and since the measurement errors of each source differed, there was a centimeter-level misalignment between the superimposed points, which manifested as localized layering and roughness on the surface model. Specifically, the merged point cloud was divided into 0.02m voxels, and the point with the highest confidence score was retained within each voxel, while the rest were discarded. The confidence score is the value assigned in S1 based on the single-point measurement error of each source point cloud; selecting this score is equivalent to retaining the most accurate observation from multiple observations at the same location. After deduplication, the point density on the same surface tended to be consistent, providing a uniform input for surface reconstruction.

[0099] The deduplicated point cloud is a discrete set of points, not yet forming a continuous surface. It requires surface reconstruction to generate a surface model. Poisson surface reconstruction is used. Specifically, the point set with normal vectors is taken as input and treated as a sample of the target object's surface. A scalar field is solved to ensure its gradient best matches the point's normal vector field. The isosurface of this scalar field is then extracted as the reconstructed surface, and the output is a triangular mesh. The normal vectors of each point have already been calculated in S2 and can be directly used for reconstruction. The reconstruction depth is set to 11. This parameter controls the maximum level of the octree used in the solution. A larger value results in a finer mesh and more detailed restoration, but also increases computational and memory usage. A mesh edge length of 11 is comparable to the 0.02m voxels used for deduplication, which can restore structures such as the edges of door and window openings and wall transitions without excessive subdivision that would characterize point cloud noise as surface undulations.

[0100] A quality report is output along with the model, recording the quantitative results of each stage, including the registration residuals of each source point cloud, the number of corresponding holes and the detection status of each hole, the drift correction amount of the acquisition trajectory, the fitting statistics of each stitching part, the list of parts to be reviewed, and suggestions for supplementary acquisition. The quality report is stored in a structured format for use in results acceptance and subsequent review.

[0101] At this point, the indoor and outdoor point clouds are merged into a complete building point cloud in the engineering coordinate system, and then a unified indoor and outdoor 3D model is generated through surface reconstruction.

[0102] Furthermore, the first transformation parameters of each source point cloud are obtained, including: S21: Perform planar segmentation on each source point cloud, extract the dominant plane of the building as the building planar primitive, and extract the edge lines and corner points of the building from the orthogonal relationship of adjacent building planar primitives; S22: Construct building topology maps of each source point cloud using the corner points as nodes and the edges as sides, search for corresponding nodes among the building topology maps of each source point cloud, and obtain initial transformation parameters based on the corresponding nodes; S23: Using the initial transformation parameters as initial values, perform iterative nearest point solving according to the registration weights, and iterate until the weighted residual converges when the decrease in the weighted residual between two adjacent iterations is less than a first set proportion, thereby obtaining the first transformation parameters.

[0103] Plane segmentation employs a random sample consensus algorithm. Three points are randomly selected from the point cloud to determine a candidate plane. The number of points from all points whose distance to this candidate plane is less than an interior point threshold is counted. This process is repeated, and the candidate plane with the most interior points is identified as a plane, and its interior points are removed from the point cloud. This process is repeated for the remaining points until the number of interior points in the newly extracted plane is lower than the minimum support point count.

[0104] The interior point threshold is determined based on the measurement error of the point cloud and the surface flatness. If the value is too small, the same wall surface will be divided into multiple planes; if the value is too large, adjacent walls and eaves will be merged into the same plane. For example, a value of 0.05m is used. The minimum number of support points is used to filter out small-area planes fitted by non-building elements such as greenery, vehicles, and indoor furniture, so that the extraction results are limited to the dominant plane of the building body. For example, a value of 300 points is used.

[0105] Building components are mostly connected at right angles. Orthogonal planes correspond to defined structural positions such as wall corners and eaves transitions, and their intersection lines and points can be repeatedly extracted from each source point cloud. Non-orthogonal intersections mostly occur at pitched roofs and irregularly shaped components, and are fewer in number and their shapes vary depending on the viewing angle. Based on this, orthogonality is used as the criterion. For any two adjacent dominant planes, the angle between their normal vectors is calculated. An angle centered at 90° and within ±5° is considered orthogonal. The intersection line of the two planes is taken as the building edge, and the common intersection point of the three edges is the corner point. The ±5° tolerance is used to absorb construction deviations and plane fitting errors.

[0106] A building topology map is constructed using corner points as nodes and edges as edges. It records the relationships between corner points connected by edges. Each source point cloud constructs its own building topology map. The two parties involved in the comparison are not yet aligned. The absolute lengths of the edges in their respective coordinate systems cannot be directly compared. However, the ratio of the degree of a node to the length of its adjacent edges does not change with the coordinate system. Therefore, the ratio of degree to edge length is used as the descriptor of the node. The degree is the number of edges connected to the node, and the edge length ratio is the ratio of the length of each adjacent edge to the length of the shortest edge.

[0107] The search for corresponding nodes involves comparing descriptors one by one between nodes in the two building topology graphs. Nodes with the same degree and whose side length ratios differ within the tolerance are identified as corresponding nodes. The tolerance is determined by the amount of transfer of measurement error and plane fitting error along the edge length. If the value is too small, the same corner point may not be matched due to extraction errors from the two source point clouds. If the value is too large, different corner points with similar shapes may be misidentified as corresponding nodes. For example, a tolerance of 8% is used.

[0108] As explained in S2, the dominant plane of the indoor point cloud and the dominant plane of the outdoor point cloud are not from the same set of surfaces, and their building topology maps are not directly compared. The initial transformation parameters of the indoor point cloud are obtained using the acquisition start segment. Accordingly, the above search is performed between the sources of the outdoor point cloud, while the indoor point cloud constructs a building topology map using the corner points and edges extracted from its acquisition start segment, and searches for corresponding nodes between the indoor point cloud and the building topology map of the ground laser scan point cloud according to the same criteria.

[0109] When there are at least 4 pairs of corresponding nodes, control point pairs are formed by the corresponding nodes. Singular value decomposition is used to solve the similarity transformation containing rotation, translation and isotropic scaling factor. First, the two sets of corresponding points are decentered and the cross-covariance matrix is ​​constructed. After performing singular value decomposition on them, the rotation matrix is ​​obtained from the left and right singular vector matrices. The translation vector is obtained from the centroids of the two sets of points and the rotation matrix. The scaling factor is obtained from the ratio of the distances of the two sets of points relative to their respective centroids.

[0110] The two source point clouds scan the same building entity, and their scale difference should not exceed 2%. If the scale factor exceeds the range of 0.98 to 1.02, it indicates that the corresponding node is incorrect. In this case, the current solution is discarded and the corresponding node is searched again. The solution result is the initial transformation parameters. The auxiliary constraints when there are 4 to 6 pairs of corresponding nodes, and the coarse alignment when there are less than 4 pairs of corresponding nodes, are handled in the manner described in S2.

[0111] When the number of points involved in the solution reaches tens of millions, the scale factor is easily affected by noise and drifts slowly. Although overall scaling of the point cloud can slightly reduce the weighted residual, the result does not reflect the true proportional relationship, and the scaling amount accumulates over the entire building, causing significant deviations at distant points. Therefore, the iterative nearest point solution performed with initial transformation parameters as initial values ​​is divided into two stages. In the first stage, only feature points on the building's planar primitives and edges are used in the solution. These points are located on large-area planes or at plane intersections, have clear geometric meanings, and low noise levels. After convergence, the resulting scale factor is fixed and is no longer used as a parameter to be solved. In the second stage, the entire point cloud is introduced to perform point-by-point iteration, solving only the six degrees of freedom of rotation and translation.

[0112] The absolute level of the residual depends on the noise of the point cloud itself. When using an absolute threshold as the convergence criterion, noisy data may never reach the threshold and get stuck in infinite iteration. The decrease in residual between two adjacent iterations reflects whether continued iteration can bring further improvement, and is unrelated to data quality. Therefore, both stages use a decrease in weighted residual between two adjacent iterations that is less than a first set proportion as the convergence criterion, for example, the first set proportion is 0.5%. At the same time, it is required that the translational change between two adjacent iterations is less than 1 mm and the rotational change is less than 0.01°, in order to exclude the situation where the residual has stabilized but the pose is still fluctuating slightly. The transformation parameters obtained after convergence are the first transformation parameters.

[0113] Furthermore, such as Figure 2 As shown, in the outdoor point cloud and indoor point cloud after transformation by the first transformation parameters, external openings and internal openings are extracted respectively, and the correspondence between openings is established, including: S31: For the outdoor point cloud after transformation by the first transformation parameter, with the main plane of the exterior facade as a reference, extract the area that is concave along the normal of the main plane of the exterior facade, the concave depth is greater than the set depth, and the area of ​​the connected region is within the set area range as the external opening. S32: For the indoor point cloud after transformation by the first transformation parameter, detect the continuous gaps in the horizontal projection of the wall point cloud along the wall direction, and extract the area where the gap width is within the set width range as the internal opening. S33: Establish a correspondence between the external openings and the internal openings.

[0114] The main plane of the facade is extracted from the outdoor point cloud through planar segmentation. Its normal points outward, and the projection direction of this normal onto the horizontal plane is the orientation of the facade. Door and window components are installed within wall openings. The window frames and the side walls of the openings are located inside the main plane of the facade. Due to reflection and penetration, it is usually difficult to obtain effective points for the glass surface. Therefore, the openings appear as recessed areas along the normal of the main plane of the facade in the outdoor point cloud.

[0115] Based on this, the distance from each point to the main plane of the exterior facade is calculated as its concave depth. Points with a concave depth greater than the set depth are selected and clustered based on connectivity to obtain each connected region as a candidate region. The clustering radius is set according to the point spacing after sampling. Then, the projected area of ​​each candidate region on the main plane of the exterior facade is calculated, which is the area of ​​the connected region.

[0116] Decorative moldings, grout lines, and expansion joints on the facade are also recessed, with a depth between 0.02m and 0.05m, much smaller than the depth of door and window openings. Setting a depth higher than this range will exclude them; for example, 0.15m. The area connecting ventilation louvers and pre-drilled holes is smaller than that of conventional doors and windows, while the area of ​​the connecting area separated by the entire curtain wall is larger than that of conventional doors and windows. Setting the area range according to the opening size of conventional doors and windows will exclude both; for example, 1.2m. 2 Up to 6m 2 Candidate regions that simultaneously satisfy the above conditions regarding both concave depth and connected area are determined to be external openings.

[0117] Interior wall surfaces are extracted from the interior point cloud through planar segmentation. The projection direction of the normal onto the horizontal plane is the orientation of the wall surface. Where doors and windows are located, the wall surface is interrupted by openings, resulting in no points in the interior point cloud at those locations. After projecting the wall points onto the horizontal plane, the distribution of projected points is statistically analyzed segment by segment along the wall orientation. Continuous segments without points are considered candidate segments. Localized gaps in the wall surface due to occlusion or viewing angle limitations will create narrow gaps, while gaps where the entire wall is not scanned will create wider gaps. Setting a width range based on the standard door and window opening width will exclude both types of gaps. For example, a range of 0.6m to 3.0m can be used. Candidate segments with gap widths within this range are considered internal openings.

[0118] The detection results for openings include their boundary points, opening center, projection center, and orientation. Boundary points are points on the opening's outline. For external openings, the boundary points are taken from the outer edge of the candidate area; for internal openings, the boundary points are extracted from the 3D wall points corresponding to the missing segments, taking the outline points where the wall breaks at the opening. The opening center is the spatial geometric center of the boundary points, and the projection center is the geometric center of the boundary points projected onto the horizontal plane. The orientation is taken from the orientation of the wall where the opening is located.

[0119] After the openings on both sides are detected, their affiliation is determined according to the position and orientation criteria in S3. Those that are determined to be the same physical opening are recorded as corresponding openings.

[0120] Furthermore, such as Figure 2 As shown, a correspondence between the external openings and the internal openings is established, including: S331: Calculate the horizontal distance between the projection centers of the external opening and the internal opening in the unified coordinate system, and the difference in the azimuth angle between the external opening and the internal opening; S332: When the horizontal distance is less than a preset distance and the directional angle difference is less than a preset angle difference, it is determined that the external opening and the internal opening are the same physical opening, and the external opening and the internal opening that are determined to be the same physical opening are recorded in the opening correspondence relationship.

[0121] The external and internal openings of the same door or window are located on two different faces of the wall. The distance between their projection centers on the horizontal plane is mainly due to the wall thickness, and also affected by the detection error of the opening boundaries on both sides. Their orientations are the same for that wall segment. Two openings belonging to different doors or windows either have projection centers that are far apart, or are located on wall segments with different orientations. Therefore, using the difference between the horizontal distance and the orientation angle of the projection centers as criteria, the openings detected on both sides are paired and compared. If both are within the limits, they are considered to be the same physical opening.

[0122] The thickness of conventional masonry and concrete walls ranges from 0.2m to 0.4m. Considering the detection error at the boundaries of openings on both sides, the horizontal distance between the projected centers of the same door or window is typically no more than 0.4m. The spacing between adjacent doors and windows is generally more than 1m. A preset distance is chosen between these two values, for example, 0.5m, which accommodates wall thickness and detection errors while preventing adjacent doors and windows from being mistakenly matched.

[0123] The walls on both sides of the same door or window are parallel to each other, and their orientations are theoretically consistent. The actual differences in their values ​​come from the wall fitting error and the residual registration deviation of the point clouds on both sides, which is usually within 3°. The orientations of adjacent wall segments often differ by 90°, and the sloping walls at corners are no less than 15°. The preset angle difference is between these two values, for example, 10°.

[0124] When judging solely by the horizontal distance from the projection center, openings located at the same position but belonging to two adjacent wall segments may be mismatched. When judging solely by the difference in orientation angle, multiple doors and windows side by side on the same wall segment cannot be distinguished because they have the same orientation. Both conditions must be met simultaneously for an opening to be considered the same physical opening. The external opening and internal opening are then paired and recorded in the opening correspondence relationship, with one external opening paired with at most one internal opening.

[0125] On a wall section where doors and windows are arranged side by side with a small gap, the same external opening may satisfy the criteria with multiple internal openings. In this case, the one with the smallest horizontal distance is paired, and the others are not paired, so as to avoid the same external opening being paired repeatedly.

[0126] When one side of the opening is obscured by furniture, curtains, or piles of objects, the detected boundary points are incomplete, and the projection center obtained from them shifts accordingly, thus rendering the horizontal distance criterion ineffective. In this case, the shape descriptor of the opening is used as an auxiliary criterion.

[0127] The shape descriptor is derived from the boundary points of the opening and includes two terms: aspect ratio and boundary perimeter. The aspect ratio is calculated based on the distribution of the boundary points within the plane of the opening, with the width defined as the span of the boundary points along the wall direction and the height as the span in the vertical direction; the ratio of these two is the aspect ratio. The boundary perimeter is the length of the closed polygonal line formed by connecting the boundary points sequentially. Both terms are shape quantities of the opening itself and do not include the opening's position in the coordinate system; therefore, they do not change with the offset of the projection center.

[0128] When comparing, calculate the absolute value of the difference in the width-to-height ratio of the openings on both sides, as well as the ratio of the perimeters of the two sides' boundaries. The dimensions of the openings on both sides of the same door or window may vary slightly due to construction deviations in the wall opening and detection errors at the two sides' boundaries. The difference in width-to-height ratio is usually within 0.10, and the ratio of the perimeters is usually between 0.9 and 1.1. Between doors and windows of different specifications, the difference in width-to-height ratio is often above 0.3, and the ratio of the perimeters is often above 1.3 or below 0.75. Limits are taken between these two values; for example, a difference in width-to-height ratio of 0.15 and a ratio of the perimeters between 0.85 and 1.18.

[0129] When boundary points are incomplete, the shape descriptor itself is also affected, and its reliability is lower than when both boundaries are complete. Therefore, the auxiliary criterion is not used alone. When using the auxiliary criterion, the orientation angle difference criterion and the auxiliary criterion must be satisfied simultaneously to determine that it is the same physical opening. When multiple openings on the same wall segment satisfy the criteria simultaneously, the one with the smallest difference in width-to-height ratio is paired.

[0130] Furthermore, when only the external opening is extracted but no corresponding internal opening is extracted from the indoor point cloud, the method further includes: S333: Construct a tolerance prism that penetrates the wall, using the center of the external opening, the normal of the main plane of the exterior facade, and the width and height of the external opening; S334: Retrieve the indoor wall segments in the indoor point cloud that intersect with the tolerance prism, record the external openings and the indoor wall segments as single-end openings, and include them in the opening correspondence relationship; S335: For the single-end opening, replace the constraint point pair with the tangential and vertical positional consistency constraints along the main plane of the exterior facade and incorporate them into the weighted residual; Wherein, the covariance of the positional consistency constraint in the wall normal direction is greater than the covariance in the tangential and vertical directions.

[0131] The detection of an opening on the indoor side relies on the gaps in the wall point cloud at the opening location. When cabinets or shelves are placed in front of the window, curtains are hung on the window, or items are piled up inside the opening, the gap is filled by the points of these objects, and no internal opening can be detected at that location. In this case, the external opening cannot be matched. If it is discarded, there is no constraint between the indoor and outdoor point clouds near this wall segment, and their fit can only be indirectly affected by the constraint of adjacent wall segments. The detection of the external opening itself is not affected by indoor occlusion, and its position and orientation remain reliable. Based on this, the corresponding wall segment on the indoor side can still be identified, but the information that can be determined is less than when both sides are detected.

[0132] The tolerance prism is based on the retrieval range of the external opening. Starting from the center of the external opening, a prism with the same width and height as the external opening is constructed along the normal direction of the main plane of the exterior facade, pointing inwards. The cross-section is extended outwards in both width and height. This extension is used to accommodate the boundary detection error of the external opening and the residual registration deviation of the point clouds on both sides; for example, 0.2m. The length of the prism along the normal direction is set according to the upper limit of the conventional wall thickness, for example, 0.6m, to ensure that it penetrates the wall without extending into the interior depth and mistakenly retrieving other wall surfaces.

[0133] Retrieve the walls intersecting the tolerance prism in the indoor point cloud, and take the portion inside the prism as the indoor wall segment. For walls containing the same door or window, extend from the outer wall to the inner wall. The tolerance prism passes through this wall along its normal direction, and the intersecting wall surface is the inner surface of the wall segment containing that door or window. Record the external opening and the corresponding indoor wall segment as a pair; this is a single-end opening correspondence. This correspondence, along with those detected on both sides, is included in the opening correspondence. After inclusion, the single-end opening correspondence also participates in the subsequent construction of absolute anchor point constraints. Its absolute position reference is still the center of the external opening, and the target keyframe is the one closest to the indoor wall segment in the acquisition trajectory.

[0134] When openings are detected on both sides, the boundary points of the two rings can be paired one by one to form constraint point pairs. For a single-end opening, the interior side corresponds to a section of wall rather than a boundary ring; therefore, there are no points that can be paired one by one between the two, and no constraint point pairs are generated. Instead, a positional consistency constraint is incorporated into the weighted residual. The positional consistency constraint uses the center of the external opening as a reference, and its residual is the deviation of the position on the interior wall segment along the wall normal from the center of the opening in the specified direction.

[0135] The location of the center of the external opening within the plane of the facade is determined, and the location of the interior wall segment can be directly deduced along the tangential and vertical directions of the facade. The location along the wall normal depends on the wall thickness, which is unknown in the point cloud and cannot be estimated from the external opening alone.

[0136] The proportion of each constraint in the solution is determined by its covariance. The smaller the covariance, the greater the proportion of the residual of that constraint in the weighted residual, and the more the solution tends to satisfy that constraint. Accordingly, the covariance of the positional consistency constraint in the wall normal direction is taken to be greater than the covariance in the tangential and vertical directions, so that the deviation in this direction is basically unconstrained in the solution. The position of the interior wall segment along the wall thickness direction is determined by the other point pairs, while the tangential and vertical directions are constrained according to the position inferred from the external openings.

[0137] The reliability of a single-ended opening correspondence is lower than that of an opening detected from both sides. The indoor side is estimated by a tolerance prism rather than actual detection, and the estimation result is affected by the integrity of the external opening boundary and the tolerance margin value. The covariance of the positional consistency constraint in the tangential and vertical directions is therefore determined by the integrity of the external opening boundary; the more completely the boundary points cover the opening outline, the smaller the covariance. During the solution process, a robust kernel is applied to the positional consistency constraint. When the residual exceeds a set multiple of the mean square error, its influence is reduced proportionally to avoid individual incorrectly estimated single-ended opening correspondences dominating the solution results.

[0138] Furthermore, such as Figure 3 As shown, the boundary points corresponding to the openings are used as constraint point pairs and incorporated into the weighted residual. The solution is iteratively obtained from the first transformation parameters to obtain the second transformation parameters for each source point cloud, including: S41: Extract the boundary points of the external and internal openings in the opening correspondence relationship respectively, and pair the boundary points of the external and internal openings according to the boundary direction to obtain the constraint point pairs; S42: Assign a fixed weight to the constraint point pair, the fixed weight remains unchanged in each round of iterative solution, and incorporate the constraint point pair and the fixed weight into the weighted residual; S43: In each round of iterative solution from the first transformation parameter, retain the constraint point pair and the fixed weight, and iterate until the weighted residual converges when the decrease in the weighted residual between two adjacent iterations is less than the second set proportion, and obtain the second transformation parameter.

[0139] The external and internal openings recorded in the opening correspondence relationship belong to the same door or window, and their boundary points are located on the outlines of the door or window openings on the outer and inner wall surfaces, respectively. The two outlines are similar in shape and size, separated by one wall thickness along the wall normal, and travel in the same direction around the outlines, so they can be matched one by one according to the boundary direction.

[0140] During pairing, the starting points and directions of travel for the two contours are first determined. The point with the lowest elevation among their respective boundary points, and the one with the smallest coordinate along the wall, is chosen as the starting point. Starting from this point, the contours travel in the same direction, thus obtaining a consistent starting position and direction of travel for both contours. If one side of a contour is damaged, the starting points selected for the two contours may not be the same physical corner. Therefore, the orientations of the two starting points relative to the centroids of their respective contours are compared. If the orientation difference exceeds a set angle, the starting point of the side with intact boundary is projected along the wall normal to the other contour, and the boundary point with the closest projected position is chosen as the starting point for that side.

[0141] When navigating around the contours, each contour is divided into equal parts according to the arc length. Boundary points at corresponding parts are paired to obtain constraint point pairs. The number of boundary points on the two contours is usually unequal. Dividing them into equal parts according to the arc length ensures that the paired points are evenly distributed on the contours, preventing local concentration due to a larger number of points on one side. The number of equal parts is determined by the perimeter of the opening and the distance between points, for example, taking a pair every 0.05m.

[0142] Both the constraint point pairs and the cross-source corresponding point pairs obtained from the nearest distance retrieval in S2 are point pairs that indicate two points corresponding to the same physical location. They have the same form in the weighted residuals, the difference being the method of determining the weights. The weights of the cross-source corresponding point pairs are constructed from the confidence level and the consistency of the normal vector, changing with the correspondence during iterations. The correspondence of the constraint point pairs comes from the entity affiliation of the door and window components and does not change with the point cloud position; their weights are fixed values ​​that remain unchanged in each iteration. The fixed weights are higher than the conventional weights of the cross-source corresponding point pairs, for example, 2.0, so that the solution prioritizes meeting the fitting requirements at the opening.

[0143] At the start of each iteration, cross-source corresponding point pairs are re-searched, while constraint point pairs and their fixed weights are retained as is. These, along with the newly searched cross-source corresponding point pairs, constitute the weighted residual for that round. The obtained transformation parameters are used for the point cloud transformation in the next round, and this process is repeated. The iteration starts from the first transformation parameter. At this point, the point clouds of each source have converged to the centimeter level, and the adjustment amount after adding constraint point pairs is limited to the vicinity of the hole. The number of rounds required for convergence is less than the number of iterations in S2 starting from the initial transformation parameter.

[0144] The convergence criterion is the same as in S2, where the decrease in the weighted residual between two adjacent iterations is less than a second predetermined proportion. Since the adjustment amount in this iteration is smaller than that in S2, if the same decrease threshold is used, the iteration will be considered convergent before the hole is fully closed. Therefore, the second predetermined proportion is less than the first predetermined proportion, for example, 0.2%. The transformation parameters obtained after convergence are the second transformation parameters.

[0145] Furthermore, when the correspondence between the openings on the same floor is not less than a preset logarithm, the process further includes the following steps before continuing the iterative solution: S71: Using the projection centers and orientations of each external and internal opening in the opening correspondence, solve for the overall orientation correction of the indoor point cloud relative to the outdoor point cloud. S72: Correct the orientation of the indoor point cloud using the overall orientation correction amount, and use the corrected indoor point cloud as input for further iterative solution.

[0146] The pose of the indoor point cloud is obtained frame by frame. During the recursion process, the pose error and position error accumulate together. The accumulated pose error is manifested as the entire indoor point cloud rotating around the vertical axis by a small angle relative to the outdoor point cloud. This kind of deviation acts on all points, and the displacement of each point varies with its distance from the rotation axis. The displacement of the far end can be several times that of the near end.

[0147] The iterative nearest point solution adjusts the transformation parameters round by round. The adjustment amount in each round is constrained by the distribution of corresponding point pairs in that round. The convergence speed of this type of overall rotation is slow, and it is easy to get stuck in a local optimum when the distribution of openings is uneven. When the number of corresponding openings detected on the same floor reaches a certain number, the turning angle can be obtained and corrected for each of these openings first, and then the iteration with constrained point pairs can be performed.

[0148] The solution is performed in the horizontal plane. The projection centers of the external and internal openings in the correspondence of each opening are taken to form two sets of corresponding points: outdoor and indoor. Simultaneously, their respective orientations are taken to form two sets of corresponding directions. The angle calculation combines two types of observations: one minimizes the positional residuals of the two sets of corresponding points after rotation around the vertical axis, and the other minimizes the angular residuals between the two sets of corresponding directions. These two types of residuals are normalized according to their dimensions and then combined. The angle that minimizes the combined residual is taken as the overall orientation correction.

[0149] When only the projection center is considered, the openings are distributed along the same wall, resulting in nearly collinear points and unstable solutions for corners. When only the orientation is considered, the openings on the same wall have the same orientation, and multiple openings do not increase constraints. Combining the two types of observations can avoid underdeterminacy in both of these situations.

[0150] The preset logarithm is determined based on the number of observations required for the solution. A pair of openings can only provide one positional and one directional correspondence, and the solved angles lack redundant checks. The checking capability of two pairs of openings is still limited; if one pair is incorrectly matched, it cannot be identified. For three or more pairs, each pair of openings can be solved and compared separately, eliminating those that deviate too much from the majority of results. Therefore, a preset logarithm of 3 pairs is preferred. If the number of opening correspondences on the same floor is insufficient, this process is not performed, and the overall orientation deviation will converge automatically in subsequent iterations.

[0151] The correction is applied to both the indoor point cloud and its acquisition trajectory. Using the geometric center of the projection center of each external opening on that floor as the axis of rotation, the overall orientation correction is applied to the indoor point cloud of that floor, and simultaneously to the keyframe pose of that floor on the acquisition trajectory, ensuring consistency between the two. If only the point cloud is corrected without correcting the acquisition trajectory, the effect of this correction will be overwritten when the indoor point cloud coordinates are recalculated based on the acquisition trajectory later.

[0152] The corrected indoor point cloud is used as input for further iterative solution. At this point, the orientation deviation of the indoor point cloud relative to the outdoor point cloud has been eliminated, and the iteration only needs to deal with the position deviation and local deformation. The constraint point pairs at the opening can converge in a few rounds.

[0153] Furthermore, the absolute anchor point constraints of the pose graph are constructed based on the correspondence between the openings, including: S51: For each physical opening in the opening correspondence, the center of the external opening in the unified coordinate system is used as the absolute position reference. S52: Among the keyframes of the pose diagram, retrieve the keyframe that is closest in space to the center of the internal opening and use it as the target keyframe. S53: Construct the positional relationship between the target keyframe and the absolute position reference as the absolute anchor point constraint; The covariance of the absolute anchor point constraint is determined by the number of boundary points of the internal opening; the more boundary points there are, the smaller the covariance of the absolute anchor point constraint.

[0154] The external and internal openings of the same physical opening are located on opposite sides of the wall, with their centers separated by one wall thickness in the engineering coordinate system. The external opening is situated within the outdoor point cloud, which, after transformation using the first and second transformation parameters, is located in the engineering coordinate system. The coordinates of its opening center are directly provided by observations from the outdoor side, without the cumulative error from frame-by-frame iterations, and can be used as an absolute position reference. The internal opening is situated within the indoor point cloud, and its coordinates are calculated from the pose of its respective keyframe. Containing cumulative error, it cannot be used as a reference; it is only used to determine the corresponding position of the opening on the acquisition trajectory.

[0155] The retrieval of target keyframes is performed within each keyframe of the pose graph. When the worker approaches a door or window, the internal opening of that door or window enters the scanning range, and its boundary points are thus acquired from several keyframes at that location. The center of the internal opening is taken, and the keyframe with the closest spatial distance among the pose positions of each keyframe is retrieved. This keyframe corresponds to the worker's position when the opening was acquired and is used as the target keyframe. The distance between the retrieved keyframe and the internal opening is usually within a few meters.

[0156] The absolute anchor point constraint defines the positional relationship that the pose of the target keyframe and the absolute position datum must satisfy. The pose of the target keyframe indicates the location of the operator, while the absolute position datum indicates the location of the door or window opening. These two locations are separated by a distance given by the observations taken from the acquisition device to the opening at the time of acquisition; specifically, the coordinates of the center of the internal opening in the device coordinate system of the target keyframe. The constraint is that the position obtained after applying the pose of the target keyframe to the observations should match the absolute position datum; the difference between the two is the residual of the constraint. During the solution process, the pose of the target keyframe is adjusted to reduce this residual, thereby shifting the acquisition trajectory to the absolute position in the engineering coordinate system.

[0157] The covariance of the absolute anchor point constraint is determined by the number of boundary points of the internal opening. The number of boundary points reflects the completeness of the detection; the more completely the boundary points cover the opening outline, the closer the obtained opening center is to the true location of the opening, and the smaller the error of the observation. When one side of the opening is partially obscured by furniture or curtains, the boundary points only cover a part of the outline, and the obtained opening center deviates accordingly. If a small covariance is still assigned as if the opening were completely detected, the constraint will lead the acquisition trajectory to the wrong location. Therefore, values ​​are tiered according to the number of boundary points. For example, when the number of boundary points is not less than 200, the variance of the covariance matrix in each direction is taken as 0.005m. 2 When the number of points is less than 200, take 0.02m. 2 .

[0158] For single-ended openings, absolute anchor point constraints are also constructed. The absolute position reference is taken as the center of the external opening, and the target keyframe is taken as the closest to the indoor wall segment in the acquisition trajectory. The indoor side of such anchor points is estimated by a tolerance prism, and its covariance is determined according to the number of boundary points of the external opening, with a value greater than the corresponding value when both sides are detected.

[0159] Furthermore, the loop closure constraints of the pose graph are adopted after double verification of candidate loop closure frame pairs. The candidate loop closure frame pairs are two non-adjacent keyframes in the pose graph that are determined to correspond to the same location through loop closure detection. The double verification includes: S54: Perform registration on the local point cloud corresponding to each candidate closed-loop frame, and calculate the overlap rate between the two local point clouds after registration. The local point cloud is the part of the indoor point cloud collected by the candidate closed-loop frame. S55: Calculate the intensity reflectance histograms of the two local point clouds respectively, and calculate the correlation coefficient between the two intensity reflectance histograms; S56: When the overlap rate is not lower than the preset overlap rate and the correlation coefficient is not lower than the preset correlation coefficient, the candidate closed-loop frame pair is adopted as the closed-loop constraint; otherwise, it is not adopted.

[0160] Loop closure detection encodes the local point cloud of the current keyframe into a feature descriptor, which is then compared with the descriptors of each previous keyframe. Frames with a similarity exceeding a threshold are considered candidate loop closure frames. Corridors, rows of rooms, and standard floor plans within a building are geometrically highly similar, and the descriptors encoded from the local point clouds at different locations are also similar. Therefore, loop closure detection may mistakenly identify two keyframes located at different locations as the same location. The resulting loop closure constraint requires two non-overlapping poses to coincide, which will pull the trajectory to the wrong position during optimization. Therefore, candidate loop closure frames must be validated before being adopted.

[0161] The first verification step focuses on geometric shape. For each candidate closed-loop frame, its acquired local point cloud is used. Registration is performed with the current pose relationship between the two as the initial value. After registration, the overlap rate between the two local point clouds is calculated. The overlap rate is the proportion of points in one local point cloud where neighbors within a set distance can be found in the other local point cloud out of its total number of points. The set distance can be 0.1m. Two keyframes located at the same location, whose local point clouds describe the same space, should largely overlap after registration. Two keyframes located at different locations but with similar shapes, although their local point clouds are similar in overall outline, differences in furniture placement, door and window positions, and wall openings will still result in large areas of non-overlapping after registration. An overlap rate not lower than a preset overlap rate passes this verification. The preset overlap rate can be 60%. Two spaces with similar geometric shapes may simultaneously reach the overlap rate threshold, especially areas with identical floor plans within the same building.

[0162] The second verification item therefore targets surface material. Laser scanning records the echo intensity at each point along with its coordinates. Echo intensity is related to the reflectivity of the surface material; the values ​​differ for dark finishes, metal components, and light-colored painted walls. The echo intensity of each point in the local point cloud is statistically analyzed into an intensity-reflectivity histogram, thus revealing the material distribution characteristics of that space. Two keyframes located at the same site, whose local point clouds capture the same batch of surfaces, will have identical histogram shapes. Two floors with the same floor plan but different surface materials will show significant differences in their histograms. The correlation coefficient between the two histograms is calculated; a correlation coefficient not lower than a preset correlation coefficient passes this verification. The preset correlation coefficient can be set to 0.7.

[0163] Candidate closed-loop frames that pass both checks are adopted as closed-loop constraints; those that fail either check are not adopted. Geometry and surface material are two independent types of features. The probability that two spatial locations in different positions are close to each other on both types of features is much lower than the probability that they are close to each other on a single feature. Therefore, dual checks can significantly reduce the number of closed-loop constraints generated by misjudgments.

[0164] The relative pose of the closed-loop constraint after adoption is the result of the above registration. The covariance is estimated according to the registration residual. The smaller the registration residual, the smaller the covariance.

[0165] Furthermore, when the building comprises multiple floors and the absolute anchor point constraints are distributed only on some floors, the method further includes: S541: Extract vertically continuous structural segments from the indoor point clouds of each floor, wherein the vertically continuous structural segments are segments of the same structure that extend continuously between adjacent floors and can be identified across floors. S542: Select a target vertical through-structure segment from the vertical through-structure segments, and construct cross-floor constraints based on the geometric consistency of the target vertical through-structure segment between segments on adjacent floors; S543: Add the cross-layer constraint to the pose graph to constrain the relative relationship between the poses of adjacent floors in the optimization of the pose graph, so that the pose of the floor without the absolute anchor point constraint is subject to the effect of the absolute anchor point constraint through the cross-layer constraint. S544: When the number of visible points of the target vertical through-structure segment is insufficient or the geometric consistency of the cross-layer constraint fails the verification, the remaining vertical through-structure segments are used as the target vertical through-structure segments, and the cross-layer constraint is reconstructed. The covariance of the cross-layer constraint increases with the increase of the plane fitting residual of the target vertical through-structure segment, and also increases with the increase of the number of interval floors transmitted by the cross-layer constraint.

[0166] The construction of absolute anchor point constraints is based on the detection of openings on both sides, and their distribution is therefore concentrated on the ground floor of the building. When workers enter the building through the entrance, the detection conditions on the interior side of the doors and windows on the ground floor are relatively good. As the floors rise, the interior side of the doors and windows is affected by the height of the windowsill and the furniture arrangement, making it difficult to completely detect the boundary points. As a result, the number of anchor points constructed based on this is small and the covariance is large.

[0167] When anchor points are distributed only on some floors, the poses of the other floors are only connected to the closed-loop constraints within the floor through inter-frame constraints in the pose diagram. The only connection between the floor containing anchor points and the floor is a segment of inter-frame constraints along the acquisition trajectory when going up and down the stairs. The error of this segment of constraints accumulates with the number of floors, and the higher the floor, the weaker the influence of the anchor points on its pose.

[0168] Staircases, elevator shafts, pipe shafts, and atriums in buildings extend through several floors from bottom to top. A segment of the same structure can be scanned in the indoor point cloud of each floor. These segments belong to the same entity, and their geometric relationships do not change with floor level. Based on this, constraints can be established between segments on adjacent floors, providing an additional connection between floors outside the data acquisition trajectory. The plane containing these structures is extracted from the indoor point cloud of each floor through planar segmentation. The structure's affiliation is determined by its vertical extension and its positional relationship with adjacent structures of the same name on adjacent floors. The resulting segments are the vertically connected structural fragments.

[0169] There may be more than one vertically connected structural segment extracted from each floor. Select one of these segments as the target vertically connected structural segment to construct cross-floor constraints. When selecting, prioritize segments that can be scanned from all floors, have a large number of visible points, and have small planar fitting residuals, as the constraints established from such segments are highly reliable.

[0170] Geometric consistency is determined by structural type. When the target vertical continuous structural segment is a staircase structure, the planes of the steps or platforms on adjacent floors are taken. Both belong to the same staircase, and their plane normals should be consistent. The height difference between platforms of the same name on adjacent floors should be equal to the floor height of that floor. The deviation between the difference in normals and the height difference constitutes the geometric consistency discrepancy. When the target vertical continuous structural segment is a shaft wall structure, two segments of the same vertical wall on adjacent floors are taken. Both segments belong to the same wall in the same shaft, and their plane normals should be parallel. The distance between the planes containing the two segments along the normal should be 0. The discrepancy between the included angle of the normals and the plane distance constitutes the geometric consistency discrepancy.

[0171] The constraints established for vertical shaft wall structures are generally stronger than those for staircase structures. Staircases have small, numerous treads with overlapping surfaces, resulting in larger fitting residuals for the extracted planes. Elevator shafts and pipe shafts, on the other hand, have walls that extend from the ground floor to the top floor; each individual wall is a large, continuous plane on each floor, leading to higher accuracy in extracting their normals and positions. When a building possesses both types of structures, vertical shaft wall structures are prioritized as the target vertically continuous structural segment.

[0172] Cross-layer constraints use the keyframe poses of two adjacent floors as the constraint objects. The discrepancy value is taken as the deviation of the geometric consistency mentioned above. After being added to the pose graph, it participates in the optimization together with inter-frame constraints, closed-loop constraints, and absolute anchor point constraints. The pose of a floor with an anchor point is pulled to the absolute position by the anchor point. The pose of the adjacent floors is adjusted accordingly by the cross-layer constraints. The next floor is adjusted again by the cross-layer constraints between it and the current floor. The effect of the anchor point is thus passed up layer by layer.

[0173] If a vertically continuous structural segment is obscured by a car or stored objects on a certain floor, or is not scanned in time, the number of visible points for that segment on that floor is insufficient, and the fitted plane is unreliable. Before constructing cross-floor constraints, the quality of the segment itself is checked by taking the fitting residuals and normal stability of the two plane segments. If the fitting residuals exceed the normal level of similar segments, or if the normal obtained by fitting the segment segment within the same floor is significantly discrete, the check is deemed to have failed.

[0174] The verification only considers the geometric quality of the segment itself, without taking into account the deviation between the poses of adjacent floors, so as not to use deviations that should be corrected by the constraint as a reason for rejection. If the number of visible points is insufficient or the verification fails, other vertically continuous structural segments are used as the target vertically continuous structural segments for reconstruction. For example, if the original elevator shaft wall surface was not swept, the stair tread plane is used instead. If there are no other available vertically continuous structural segments in the building, no cross-floor constraints are established for that floor. Its pose is solved by the inter-frame constraints and closed-loop constraints of that floor, and it is marked as a floor without cross-floor constraints in the quality report.

[0175] The covariance of cross-layer constraints is determined by two factors. First, the plane fitting residual of the target vertical continuous structural segment. A larger residual indicates less accurate positioning and normals of the extracted plane, resulting in lower reliability of the discrepancies and consequently, a larger covariance. Second, the number of floors the constraint transmits, i.e., the number of floors between the current floor and the nearest floor containing the anchor point. The anchor point introduces error into the constraint itself each time it is transmitted across layers. A larger number of floors results in a greater cumulative error and a larger covariance. For example, each floor adds 0.01m of positional uncertainty.

[0176] In this embodiment, the multi-source point cloud required for integrated indoor and outdoor modeling of a building is incorporated into the same engineering coordinate system. First, cross-source correspondence is established between the building's own planar primitives and corner points, and the first transformation parameter is obtained through weighted iteration. Then, the correspondence between the door and window openings, which are common physical objects in both indoor and outdoor areas, is established. The boundary points of the door and window openings are used as constraint point pairs and incorporated into the weighted residual calculation to obtain the second transformation parameter. At the same time, the parameter is converted into the absolute anchor point constraint of the pose graph. Together with the inter-frame constraint and closed-loop constraint, the pose graph is optimized to obtain the corrected acquisition trajectory. Finally, the indoor point cloud is recalculated according to the corrected acquisition trajectory, and the outdoor point cloud is transformed according to the second transformation parameter. After merging, the indoor and outdoor integrated 3D model is output through surface reconstruction.

[0177] Therefore, the unified registration of indoor and outdoor multi-source point clouds can be completed automatically without the need for manual control points, and the cumulative drift of indoor trajectories can be eliminated simultaneously, so that the indoor and outdoor models are geometrically continuous at the splicing point, thereby improving the automation level and geometric consistency of integrated indoor and outdoor modeling of buildings.

[0178] Although preferred embodiments of the invention have been described, those skilled in the art, once they have learned the basic inventive concept, can make other changes and modifications to these embodiments.

[0179] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of this invention and its equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for unified registration and reconstruction of multi-source point clouds inside and outside buildings, characterized in that, The method includes: Acquire the outdoor point cloud, indoor point cloud, and acquisition trajectory of the indoor point cloud of the building, reduce them to a unified coordinate system, and add confidence level; Extract building plane primitives from each source point cloud and establish cross-source corresponding point pairs. Construct registration weights based on the confidence level and normal vector consistency. Iterate and solve the solution with the goal of minimizing the weighted residual to obtain the first transformation parameters of each source point cloud. In the outdoor point cloud and indoor point cloud after transformation by the first transformation parameter, external openings and internal openings are extracted respectively, and the corresponding relationship of openings is established. The boundary points corresponding to the openings are used as constraint point pairs and incorporated into the weighted residual. The solution is iterated from the first transformation parameter to obtain the second transformation parameters of each source point cloud. A pose graph is constructed based on the acquisition trajectory. Absolute anchor point constraints of the pose graph are constructed based on the correspondence between the openings. The pose graph is optimized by combining the inter-frame constraints and closed-loop constraints of the pose graph to obtain the corrected acquisition trajectory. The indoor point cloud is recalculated based on the corrected acquisition trajectory to obtain the corrected indoor point cloud. The outdoor point cloud is then transformed based on the second transformation parameters to obtain the registered outdoor point cloud. After merging, surface reconstruction is performed to output an integrated indoor and outdoor 3D model.

2. The method according to claim 1, characterized in that, The first transformation parameters of each source point cloud are obtained, including: Perform planar segmentation on each source point cloud, extract the dominant plane of the building as the building planar primitive, and extract the edge lines and corner points of the building by the orthogonal relationship of adjacent building planar primitives; Construct building topology maps of each source point cloud using the corner points as nodes and the edges as sides, search for corresponding nodes among the building topology maps of each source point cloud, and obtain initial transformation parameters based on the corresponding nodes; Using the initial transformation parameters as initial values, perform iterative nearest point solving according to the registration weights, and iterate until the weighted residual converges when the decrease in the weighted residual between two adjacent iterations is less than a first set proportion, thus obtaining the first transformation parameters.

3. The method according to claim 1, characterized in that, From the outdoor point cloud and indoor point cloud after transformation by the first transformation parameters, external and internal openings are extracted respectively, and the correspondence between the openings is established, including: For the outdoor point cloud after transformation by the first transformation parameter, with the main plane of the exterior facade as a reference, the region that is concave along the normal of the main plane of the exterior facade, has a concave depth greater than a set depth, and has a connected area within a set area range is extracted as the external opening. For the indoor point cloud after transformation by the first transformation parameter, detect the continuous gaps in the horizontal projection of the wall point cloud along the wall direction, and extract the area where the gap width is within the set width range as the internal opening. Establish a correspondence between the external openings and the internal openings.

4. The method according to claim 3, characterized in that, Based on the external opening and the internal opening, establish the correspondence between the openings, including: Calculate the horizontal distance between the projection centers of the external opening and the internal opening in the unified coordinate system, and the difference in the azimuth angle between the external opening and the internal opening; When the horizontal distance is less than a preset distance and the directional angle difference is less than a preset angle difference, it is determined that the external opening and the internal opening are the same physical opening, and the external opening and the internal opening that are determined to be the same physical opening are recorded in the opening correspondence relationship.

5. The method according to claim 4, characterized in that, When only the external opening is extracted but no corresponding internal opening is extracted from the indoor point cloud, the method further includes: A tolerance prism is constructed through the wall, using the center of the external opening, the normal of the main plane of the exterior facade, and the width and height of the external opening. Retrieve the indoor wall segments in the indoor point cloud that intersect with the tolerance prism, and record the external openings and the indoor wall segments as single-end openings, and include them in the opening correspondence relationship; For the single-end opening, the constraint point pair is replaced by the positional consistency constraint along the tangential and vertical directions of the main plane of the exterior facade and incorporated into the weighted residual; Wherein, the covariance of the positional consistency constraint in the wall normal direction is greater than the covariance in the tangential and vertical directions.

6. The method according to claim 1, characterized in that, The boundary points corresponding to the openings are used as constraint point pairs and incorporated into the weighted residual. The solution is then iteratively obtained from the first transformation parameters to obtain the second transformation parameters for each source point cloud, including: Extract the boundary points of the external and internal openings in the opening correspondence, and pair the boundary points of the external and internal openings according to the boundary direction to obtain the constraint point pairs; The constraint point pairs are assigned fixed weights, which remain unchanged in each iteration of the solution process. The constraint point pairs and the fixed weights are then incorporated into the weighted residual. In each iteration of solving the first transformation parameter, the constraint point pair and the fixed weight are retained. The iteration continues until the weighted residual decreases by less than a second set proportion between two adjacent iterations, and then converges to obtain the second transformation parameter.

7. The method according to claim 1, characterized in that, When the correspondence between the openings on the same floor is not less than a preset logarithm, the process before continuing the iterative solution also includes: Using the projection centers and orientations of each external and internal opening in the opening correspondence, the overall orientation correction of the indoor point cloud relative to the outdoor point cloud is calculated. The orientation of the indoor point cloud is corrected using the overall orientation correction amount, and the corrected indoor point cloud is used as input for further iterative solution.

8. The method according to claim 1, characterized in that, The absolute anchor point constraints of the pose graph are constructed based on the correspondence between the openings, including: For each physical opening in the aforementioned opening correspondence, the center of the external opening in the unified coordinate system is used as the absolute position reference. Among the keyframes of the pose graph, the keyframe that is closest in spatial distance to the center of the internal opening is selected as the target keyframe. The positional relationship between the target keyframe and the absolute position reference is constructed as the absolute anchor point constraint; The covariance of the absolute anchor point constraint is determined by the number of boundary points of the internal opening; the more boundary points there are, the smaller the covariance of the absolute anchor point constraint.

9. The method according to claim 1, characterized in that, The loop closure constraints of the pose graph are adopted after double verification of candidate loop closure frame pairs. The candidate loop closure frame pairs are two non-adjacent keyframes in the pose graph that are determined to correspond to the same location through loop closure detection. The double verification includes: Registration is performed on the local point clouds corresponding to the candidate closed-loop frames, and the overlap rate between the two local point clouds after registration is calculated. The local point clouds are the parts of the indoor point clouds collected by the candidate closed-loop frames. The intensity reflectance histograms of the two local point clouds are statistically analyzed, and the correlation coefficient between the two intensity reflectance histograms is calculated. When the overlap rate is not lower than the preset overlap rate and the correlation coefficient is not lower than the preset correlation coefficient, the candidate closed-loop frame pair is adopted as the closed-loop constraint; otherwise, it is not adopted.

10. The method according to claim 8, characterized in that, When the building comprises multiple floors and the absolute anchor point constraints are distributed only on some floors, the method further includes: Vertically continuous structural segments are extracted from the indoor point clouds of each floor. These vertically continuous structural segments are segments of the same structure that extend continuously between adjacent floors and can be identified across floors. Select a target vertical through-structure segment from the vertical through-structure segments, and construct cross-floor constraints based on the geometric consistency of the target vertical through-structure segment between segments on adjacent floors; The cross-layer constraint is added to the pose graph to constrain the relative relationship between the poses of adjacent floors in the optimization of the pose graph, so that the pose of the floor without the absolute anchor point constraint is subject to the effect of the absolute anchor point constraint through the cross-layer constraint. When the number of visible points of the target vertical through-structure segment is insufficient or the geometric consistency of the cross-layer constraint fails the verification, the remaining vertical through-structure segments are used as the target vertical through-structure segments, and the cross-layer constraint is reconstructed. The covariance of the cross-layer constraint increases with the increase of the plane fitting residual of the target vertical through-structure segment, and also increases with the increase of the number of interval floors transmitted by the cross-layer constraint.

Images