Intelligent Stitching Method and System for Building Elements Based on 3D Dynamic Scanning Imaging

Abstract

This invention discloses an intelligent stitching method and system for building elements based on 3D dynamic scanning imaging, relating to the field of computer vision technology. The method includes: acquiring multi-source sensor data of a building scene, performing temporal correction and spatial alignment on the point cloud data to generate a point cloud data sequence with a unified time series and initial pose relationship; based on the initial pose relationship of the point cloud data sequence, incrementally stitching the point cloud data in chronological order under dynamic scanning conditions, estimating and correcting the relative poses between point clouds in real time during the stitching process, generating a point cloud stitching result; and performing closed-loop detection on the point cloud stitching result, constructing closed-loop constraints and performing global optimization when spatial backtracking is detected, outputting a continuously fused point cloud of the building scene. This invention solves the problems of asynchronous time between point cloud and pose data under dynamic scanning conditions, difficulty in correcting cumulative pose errors, and insufficient efficiency and accuracy of closed-loop detection in large-scale building scenes.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically, to a method and system for intelligent stitching of architectural elements based on 3D dynamic scanning imaging. Background Technology

[0002] With the development of 3D laser scanning, structured light scanning, and multi-sensor fusion imaging technologies, digital modeling of building scenes based on 3D point clouds has been widely applied in fields such as architectural surveying, digital twins, and smart cities. Especially in large or complex building scenes, it is often necessary to continuously acquire 3D point cloud data while the scanning equipment is moving in a dynamic scanning state, and then fuse multiple frames of point clouds into a complete building scene model through point cloud stitching technology.

[0003] In existing technologies, common point cloud stitching methods mainly include stitching methods based on geometric feature matching, stitching methods based on device pose (such as IMU, odometry) assistance, and stitching methods based on simultaneous localization and mapping (SLAM). These methods typically assume that there is a relatively accurate time synchronization relationship and initial pose estimation between point cloud data, and rely on the local matching results between adjacent frame point clouds to achieve frame-by-frame stitching.

[0004] However, in actual dynamic scanning processes, since point cloud sensors and pose sensors are usually independent devices, there are often problems such as time asynchrony, inconsistent sampling frequencies, or communication delays between them. This leads to deviations in the temporal correspondence between point cloud data and device pose data, thus introducing initial pose errors. These errors may not be obvious in the early stages of point cloud stitching, but they accumulate gradually during long-term dynamic scanning, seriously affecting the overall stitching accuracy and spatial consistency. In addition, existing incremental point cloud stitching methods mostly rely on local matching constraints between adjacent frames. In the absence of effective global constraints, when the scanning device performs a spatial revisit (i.e., returns to a scanned area), it is difficult to detect the closed-loop relationship in a timely and accurate manner, resulting in the inability to effectively correct the accumulated errors. Ultimately, this manifests as distortion of the overall architectural scene structure, ghosting, or pose drift. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a method and system for intelligent stitching of building elements based on 3D dynamic scanning imaging. By integrating multi-source sensing time correction, incremental point cloud stitching, and closed-loop constraint optimization based on spatial partitioning and semantic perception, the method solves the problems of asynchronous time between point cloud and pose data under dynamic scanning conditions, difficulty in correcting cumulative pose errors, and insufficient efficiency and accuracy of closed-loop detection in large-scale building scenes.

[0006] To achieve the above objectives, the present invention provides the following technical solution: Firstly, this application provides an intelligent stitching method for building elements based on 3D dynamic scanning imaging. The method includes: acquiring multi-source sensor data of the building scene, performing time correction and spatial alignment on continuously acquired point cloud data to generate a point cloud data sequence with a unified time series and initial pose relationship; based on the initial pose relationship of the point cloud data sequence, incrementally stitching the point cloud data in time order under dynamic scanning conditions; during the stitching process, fusing multi-source sensor data to estimate and correct the relative pose between point clouds in real time, generating continuously updated point cloud stitching results; performing closed-loop detection on the point cloud stitching results; constructing closed-loop constraints and performing global optimization when spatial backtracking is detected; and outputting a continuously fused point cloud of the building scene.

[0007] In one embodiment, time correction and spatial alignment are performed on continuously acquired point cloud data to generate a point cloud data sequence with a unified time series and initial pose relationship. This includes: extracting the acquisition time of each frame of point cloud data and device pose-related data within the corresponding time period from multi-source sensor data; setting a time offset parameter for the device pose-related data based on the acquisition time; correcting the time correspondence of the device pose-related data based on the time offset parameter to establish an initial time correspondence between the device pose-related data and the point cloud data; mapping each frame of point cloud data to a unified spatial reference coordinate system using the corrected device pose data based on the initial time correspondence to obtain an initial spatial alignment result; constructing an optimization objective function including spatial error terms and time constraint terms based on the initial spatial alignment result, and iteratively jointly optimizing the point cloud spatial pose parameters and time offset parameters; and performing time correction and spatial alignment on the multi-source sensor data based on the optimized time offset parameter and spatial pose parameter to form a point cloud data sequence with a unified time series and initial pose relationship.

[0008] In one embodiment, the temporal correspondence of device pose-related data is corrected based on a time offset parameter to establish an initial temporal correspondence between the device pose-related data and the point cloud data. This includes: obtaining the original time stamps corresponding to each pose data in the device pose-related data and constructing a device pose time sequence in chronological order; correcting the original time stamps in the device pose time sequence based on the time offset parameter to obtain a corrected device pose time sequence, and determining the corresponding time position of the acquisition time of each frame of 3D point cloud data in the device pose time sequence; when there is device pose data in the corrected device pose time sequence corresponding to the acquisition time of a certain frame of 3D point cloud data, selecting that device pose data as the initial pose information of that frame of 3D point cloud data; when there is no device pose data in the corrected device pose time sequence that is consistent with the acquisition time of 3D point cloud data, generating corresponding initial pose information based on the device pose data adjacent to that acquisition time in the device pose time sequence; and based on the initial pose information, associating the corresponding device pose data with each frame of 3D point cloud data to establish an initial temporal correspondence between the corrected device pose-related data and the 3D point cloud data.

[0009] In one embodiment, based on the initial pose relationship of the point cloud data sequence, incremental stitching of the point cloud data is performed in chronological order under dynamic scanning conditions. During the stitching process, multi-source sensor data is fused to estimate and correct the relative poses between point clouds in real time, generating continuously updated point cloud stitching results. This includes: selecting the first frame of 3D point cloud data in the point cloud data sequence as the initial reference for the incremental point cloud stitching model based on the initial pose relationship of the point cloud data sequence, and establishing an initial state; sequentially acquiring 3D point cloud data from the point cloud data sequence, excluding the first frame, according to the acquisition time order, to form a point cloud data sequence to be incrementally added; and targeting... For each frame of 3D point cloud data to be incrementally added, initial pose information is extracted and transformed into a unified spatial reference coordinate system to achieve preliminary spatial alignment. The spatially aligned current frame of 3D point cloud data is then incrementally fused with the existing point cloud set in the incremental point cloud stitching model. After incremental fusion, the relative pose between the current frame point cloud and the stitched point cloud is estimated and corrected in real time based on multi-source sensor data. Based on the incremental fusion and relative pose correction, the global pose state of the incremental point cloud stitching model is updated until all the point cloud data sequences to be incrementally added are added to the incremental point cloud stitching model to form a real-time point cloud stitching result.

[0010] In one embodiment, the incremental fusion process includes: determining the spatial distribution relationship between the current frame's 3D point cloud data and the point cloud set in a unified spatial reference coordinate system based on the preliminary spatial alignment results, and dividing the point cloud overlapping region; within the point cloud overlapping region, extracting candidate fusion point sets from the current frame's 3D point cloud data, and extracting reference point sets from the point cloud set; for each candidate point in the candidate fusion point set, determining whether the candidate point meets the fusion consistency condition based on its spatial distance from the corresponding reference point set, local geometric distribution features, and point cloud density; and performing differentiated fusion update processing on the candidate fusion points according to the determination result to obtain the updated point cloud set.

[0011] In one embodiment, the relative pose between the current frame point cloud and the stitched point cloud is estimated and corrected in real time based on multi-source sensor data, including: determining the spatially overlapping region based on the updated point cloud set; extracting spatial feature information for pose estimation within the spatially overlapping region; constructing a relative pose error based on the spatial feature information; acquiring device pose-related data corresponding to the acquisition time of the current frame point cloud and constructing device motion constraints; and jointly correcting the relative pose of the current frame point cloud relative to the stitched point cloud through the relative pose error and the device motion constraints.

[0012] In one embodiment, loop closure detection is performed on the point cloud stitching results. When spatial backtracking is detected, loop closure constraints are constructed and global optimization is performed to output a continuously fused point cloud of the building scene. This includes: constructing a hierarchical historical point cloud index structure based on the point cloud stitching results and a hierarchical historical point cloud indexing method based on spatial partitioning and semantic awareness, and filtering a candidate set of historical point clouds; for the current frame point cloud, based on its spatial position and pose information in a unified spatial reference coordinate system, filtering candidate historical point clouds with spatial backtracking relationships from the candidate set of historical point clouds; performing spatial matching verification between the current frame point cloud and the candidate historical point clouds to determine that a loop closure relationship has been detected; after detecting a loop closure relationship, constructing loop closure constraints based on the spatial matching results, and forming a global constraint set together with the device motion constraints formed during the incremental stitching of the point cloud; performing global optimization on the point cloud stitching results based on the global constraint set, and jointly adjusting the spatial pose parameters of each frame point cloud; updating the point cloud data of each frame according to the optimized spatial pose parameters, and continuously fusing the building scene point cloud to output a fused point cloud of the building scene.

[0013] In one embodiment, based on the point cloud stitching results, a hierarchical historical point cloud indexing method based on spatial partitioning and semantic awareness is used to construct a hierarchical historical point cloud index structure and filter a candidate set of historical point clouds. This includes: dividing the three-dimensional spatial range of the building scene into spatial partitioning units based on the point cloud stitching results; determining the historical point cloud index structure for each stitched frame of historical point cloud data; performing semantic analysis on the building elements represented by each historical point cloud data to obtain semantic identification information; binding the semantic identification information with the corresponding historical point cloud data and its corresponding spatial partitioning unit to form a hierarchical historical point cloud index structure; determining the spatial partitioning unit corresponding to the current frame point cloud and using the spatial partitioning unit and its adjacent spatial partitioning units as candidate retrieval areas; and within the candidate retrieval areas, filtering historical point cloud data whose semantic attributes match those of the current frame point cloud from the hierarchical historical point cloud index structure to form a candidate set of historical point clouds.

[0014] Secondly, this application provides an intelligent stitching system for building elements based on 3D dynamic scanning imaging. The system includes: a correction and alignment module, used to acquire multi-source sensor data of the building scene and perform time correction and spatial alignment on continuously acquired point cloud data to generate a point cloud data sequence with a unified time series and initial pose relationship; a point cloud stitching module, used to incrementally stitch the point cloud data in time order under dynamic scanning conditions based on the initial pose relationship of the point cloud data sequence, and to estimate and correct the relative pose between point clouds in real time by integrating multi-source sensor data during the stitching process, generating continuously updated point cloud stitching results; and a closed-loop detection module, which performs closed-loop detection on the point cloud stitching results, constructs closed-loop constraints and performs global optimization when spatial backtracking is detected, and outputs a continuously fused point cloud of the building scene.

[0015] As can be seen from the above technical solutions, the embodiments of this application have the following advantages: By introducing a time synchronization and spatial alignment mechanism for multi-source sensor data under dynamic scanning conditions, a point cloud data foundation with a unified time series and reliable initial pose relationships is established. Based on this, an incremental point cloud stitching model is adopted, combined with consistency judgment of overlapping areas, differentiated fusion updates, and joint correction of relative pose constrained by device motion. This achieves stable, real-time, and high-precision stitching of point clouds during continuous acquisition. At the same time, by constructing a hierarchical historical point cloud index structure that integrates spatial partitioning and semantic awareness, the candidate range for loop closure detection is effectively narrowed. When spatial revisit is detected, loop closure constraints are introduced for global optimization, systematically suppressing the cumulative error and global pose drift generated during dynamic scanning. Thus, while ensuring the spatial continuity, geometric consistency, and smoothness of point clouds, the robustness, accuracy, and scalability of point cloud stitching and 3D reconstruction in complex architectural scenes are significantly improved. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the intelligent stitching method for building elements based on 3D dynamic scanning imaging provided in an embodiment of this application.

[0018] Figure 2 A schematic diagram of the intelligent splicing system for building elements based on 3D dynamic scanning imaging provided in this application embodiment. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "a plurality of" or "several" means two or more, unless otherwise explicitly specified.

[0021] It should also be noted that, in this document, terms such as “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase “comprising one…” does not exclude the presence of other identical elements in the article or device that includes the aforementioned element.

[0022] Reference Figure 1 As shown in the diagram, the intelligent stitching method for building elements based on 3D dynamic scanning imaging provided by this invention includes the following steps: S1. Acquire multi-source sensor data of the building scene, wherein the multi-source sensor data includes three-dimensional point cloud data and device pose-related data. Based on the multi-source sensor data, perform time synchronization and spatial alignment on the continuous point cloud data acquired during the scanning process to form a point cloud data sequence with a unified time sequence and initial pose relationship.

[0023] The acquisition of multi-source sensor data of the building scene includes: setting the scanning device to be in continuous motion, allowing it to move and scan along the interior or exterior of the building scene; during the movement of the scanning device, continuously imaging the building scene through its onboard 3D scanning unit, and collecting 3D point cloud data reflecting the spatial morphology of the building surface in real time; simultaneously, using the pose acquisition unit set on the scanning device, synchronously collecting device pose-related data reflecting the spatial position and attitude changes of the scanning device at each acquisition moment; wherein, by synchronously recording the 3D point cloud data and the device pose-related data under a unified time reference, each frame or each moment of acquired 3D point cloud data corresponds to the corresponding device pose information, thereby forming multi-source sensor data acquired under continuous motion scanning conditions, providing basic data support for subsequent spatial alignment and stitching processing of point cloud data.

[0024] In this embodiment, based on the multi-source sensor data, the continuous point cloud data acquired during the scanning process is time-synchronized and spatially aligned to form a point cloud data sequence with a unified time sequence and initial pose relationship, including: Extract the acquisition time corresponding to each frame of 3D point cloud data from the multi-source sensing data, and the device pose-related data obtained within the time range corresponding to the acquisition time, wherein the device pose-related data includes information used to characterize the spatial position and attitude changes of the scanning device. Based on the acquisition time, a time offset parameter is set for the device pose-related data. The time offset parameter is used to characterize the offset of the device pose-related data on the time axis relative to a preset unified time reference. Based on the time offset parameter, the time correspondence of the device pose-related data is corrected so that the corrected device pose-related data and the three-dimensional point cloud data establish an initial time correspondence. Based on the initial time correspondence, the corrected device pose data and the device pose data corresponding to each frame of 3D point cloud data are used to map the 3D point cloud data to the same spatial reference coordinate system to obtain the initial spatial alignment result. Based on the initial spatial alignment results, an optimization objective function reflecting the spatial consistency between multiple frames of 3D point cloud data is constructed. The optimization objective function comprehensively measures the alignment consistency of multiple frames of 3D point cloud data in a unified spatial reference coordinate system. This objective function includes a spatial error term and a temporal constraint term. The spatial error term characterizes the degree of geometric deviation between different frames of 3D point cloud data within the spatially overlapping region, reflecting the spatial matching error between point clouds. The temporal constraint term limits the range of variation of the time offset parameter, ensuring that the time offset parameter maintains temporal continuity and physical rationality while satisfying the spatial consistency of the point clouds. By comprehensively optimizing the spatial error term and the temporal constraint term, the spatial positions of multiple frames of 3D point cloud data tend to be consistent, and the corresponding time offset parameter matches the actual time relationship of the scanning process.

[0025] The objective function is jointly optimized by iteratively updating the spatial pose parameters and time offset parameters obtained from the parameterization of the device pose data, so that the spatial error term and time constraint term meet the preset optimization conditions, thereby obtaining the optimized spatial pose parameters and time offset parameters. Based on the optimized time offset parameters and spatial pose parameters, the multi-source sensor data is time-corrected and spatially aligned to form a point cloud data sequence with a unified time sequence and initial pose relationship.

[0026] Specifically, the acquisition time of multi-source sensor data is corrected by using the optimized time offset parameter, and the 3D point cloud data of each frame is mapped to a unified spatial reference coordinate system by combining the optimized spatial pose parameter, so as to obtain a point cloud data sequence that is temporally continuous and spatially aligned, ensuring that each frame of point cloud has an initial pose relationship under a unified time and spatial reference.

[0027] Furthermore, based on the time offset parameter, the time correspondence of the device pose-related data is corrected, so that the corrected device pose-related data establishes an initial time correspondence with the 3D point cloud data, including: The original timestamps corresponding to each pose data in the device pose-related data are obtained and the device pose time series are formed in chronological order. The device pose time series is used to characterize the continuous pose changes of the scanning device during the scanning process. Based on the time offset parameter, the original time markers in the device pose time series are corrected to obtain the corrected device pose time series, wherein the time correction includes overall translation or time axis adjustment of the original time markers; Based on the corrected device pose time series, determine the corresponding time position of each frame of 3D point cloud data acquisition time in the corrected device pose time series; For each frame of 3D point cloud data, when there is device pose data in the corrected device pose time series that corresponds to the acquisition time of the 3D point cloud data of that frame, the device pose data is directly selected as the initial pose information of the 3D point cloud data of that frame. The device pose data refers to a single pose information corresponding to a specific time point selected from or generated based on the device pose related data, which is used to describe the spatial position and attitude of the scanning device at that moment. When there is no time marker in the corrected device pose time series that is completely consistent with the acquisition time of the three-dimensional point cloud data, the initial pose information corresponding to the three-dimensional point cloud data is generated based on the device pose data corresponding to the adjacent time markers in the corrected device pose time series. Based on the initial pose information, the corresponding device pose data is associated with each frame of 3D point cloud data, and the initial time correspondence between the corrected device pose-related data and the 3D point cloud data is established for subsequent spatial mapping processing of point cloud data.

[0028] It should be noted that by using time correction and spatial alignment of multi-source sensor data, parameterized optimization of device pose data, and joint adjustment of time offset and spatial pose, the 3D point cloud data under continuous scanning is highly consistent in time and space, thereby improving the point cloud stitching accuracy and ensuring the real-time performance of dynamic scanning.

[0029] S2, based on the initial pose relationship of the point cloud data sequence, incrementally stitching the point cloud data in chronological order under dynamic scanning conditions, fusing multi-source sensor data during the stitching process to estimate and correct the relative poses between point clouds in real time, generating continuously updated point cloud stitching results, including: Based on the initial pose relationship of the point cloud data sequence, the first frame of the three-dimensional point cloud data of the point cloud data sequence is selected as the initial reference of the incremental point cloud stitching model, and the spatial position and pose of the point cloud in the unified spatial reference coordinate system are located to establish the initial state of the incremental point cloud stitching model. The incremental point cloud stitching model is a data structure used to progressively integrate multiple frames of 3D point clouds under dynamic scanning conditions. Its core components include: a currently stitched point cloud set, used to store all historical frame point cloud data after initial alignment and incremental fusion processing; a global spatial reference coordinate system, used to provide a unified spatial positioning benchmark for all point clouds, enabling precise mapping and fusion of point clouds across frames within the same coordinate system; and global pose state parameters, used to describe the overall spatial position, orientation, and interrelationships of the entire point cloud set, supporting incremental addition, relative pose estimation, and correction of subsequent new frame point clouds. During the incremental point cloud stitching process, the model continuously updates the point cloud set and global pose state, achieving real-time accumulation of point cloud data, maintenance of spatial consistency, and continuous 3D reconstruction of the dynamic scanning scene.

[0030] According to the acquisition time sequence, the three-dimensional point cloud data of each frame except the first frame are sequentially acquired from the point cloud data sequence to form a point cloud data sequence to be added incrementally. For each frame of 3D point cloud data to be incrementally added, its corresponding initial pose information is extracted, and based on the initial pose information, the frame of 3D point cloud data is transformed from the local coordinate system at the time of acquisition to the unified spatial reference coordinate system, so as to achieve preliminary spatial alignment between the frame of point cloud and the spliced ​​point cloud in the current incremental point cloud splicing model. The initially aligned 3D point cloud data of the current frame is incrementally fused with the existing point cloud set in the incremental point cloud stitching model. The incremental fusion process includes: Based on the preliminary spatial alignment results, the spatial distribution relationship between the current frame 3D point cloud data and the point cloud set in a unified spatial reference coordinate system is determined, and the overlapping and non-overlapping areas of the point clouds are divided accordingly. The overlapping area of ​​the point clouds is used to characterize the area where the current frame point cloud and the spliced ​​point cloud have a spatial correspondence. Within the overlapping area of ​​the point cloud, a candidate fusion point set is extracted from the current frame's 3D point cloud data, and a reference point set that is spatially adjacent to the candidate fusion point set is extracted from the point cloud set for subsequent point cloud consistency judgment. It should be noted that the candidate fusion point set refers to the set of point cloud data selected from the current frame point cloud that may represent the same building structure as the existing point cloud in terms of spatial location within the area where there is spatial overlap between the current frame 3D point cloud and the stitched point cloud, and is used for subsequent consistency judgment and fusion processing.

[0031] For each candidate point in the candidate fusion point set, based on its spatial distance relationship with the corresponding reference point set, local geometric distribution features, and point cloud density changes, it is determined whether the candidate point satisfies the fusion consistency condition. The fusion consistency condition is used to characterize whether the current frame point cloud and the spliced ​​point cloud represent the same spatial structure at this position. It should be noted that by calculating whether the spatial distance (Euclidean distance) between the candidate point and the nearest neighbor point in the corresponding reference point set is less than a preset distance threshold, and simultaneously determining whether the candidate point and the reference point set are consistent in local geometric distribution direction and whether the point cloud density change range is within a preset allowable range, if the above conditions are met simultaneously, the candidate point is determined to meet the fusion consistency condition.

[0032] Based on the consistency determination results, differentiated fusion update processing is performed on the candidate fusion points to obtain the updated point cloud set, including: When a candidate point meets the fusion consistency condition, the candidate point and the corresponding reference point are merged or their attributes are updated to enhance the spatial stability of the point cloud in the region. When a candidate point does not meet the fusion consistency condition, the candidate point is added to the point cloud set as a new spatial sampling point to supplement the spatial information of the existing point cloud in that region. After completing the differentiated fusion update, the updated point cloud set is obtained.

[0033] After incremental fusion processing, the relative pose between the current frame point cloud and the stitched point cloud is estimated and corrected in real time based on multi-source sensor data, including: Based on the updated point cloud set, the spatial overlap region between the current frame point cloud and the stitched point cloud in a unified spatial reference coordinate system is determined, and the spatial overlap region is used as the reference region for real-time estimation of relative pose. Within the spatially overlapping region, spatial feature information for pose estimation is extracted from the current frame point cloud and the stitched point cloud, respectively. The spatial feature information includes local geometric distribution features and point cloud spatial structure features, which are used to characterize the relative spatial relationship between the two within the region. Based on the spatial feature information, a relative pose error quantity is constructed to reflect the spatial difference between the current frame point cloud and the stitched point cloud. The relative pose error quantity is used to characterize the deviation relationship between the two in position, translation direction, attitude and rotation direction. Acquire device pose-related data corresponding to the current frame point cloud acquisition time, and construct device motion constraints based on the device pose-related data to limit the variation range of the relative pose error, so that the relative pose error conforms to the physical characteristics of continuous motion of the scanning device. The process involves acquiring device pose-related data corresponding to the current frame point cloud acquisition time. Specifically, this includes extracting device pose data corresponding to the current frame point cloud timestamp from multi-source sensor data. Based on this, device motion constraints are constructed according to the displacement and attitude changes between adjacent device poses. These constraints limit the relative pose error of the current frame point cloud relative to the stitched point cloud to a preset range, ensuring that the relative pose error conforms to the physical characteristics of velocity, acceleration, or attitude changes under continuous motion conditions of the scanning device in both translation and rotation directions. This avoids non-physical pose jumps caused by mismatched point clouds or noise, and improves the stability and reliability of the relative pose correction results.

[0034] Under the combined effect of the relative pose error and the device motion constraint, the relative pose of the current frame point cloud relative to the stitched point cloud is jointly corrected so that the corrected relative pose simultaneously meets the requirements of point cloud spatial consistency and device motion continuity. Specifically, after obtaining the relative pose error, which characterizes the spatial deviation between the current frame point cloud and the stitched point cloud, and the device motion constraints used to limit the rationality of device motion, the relative pose error is used as the basis for spatial consistency optimization, and the device motion constraints are used as physical continuity constraints to jointly correct the relative pose of the current frame point cloud. Specifically, during the correction process, the spatial position and orientation of the current frame point cloud are gradually adjusted within the range of translation and rotation changes limited by the device motion constraints, so that the geometric deviation between it and the stitched point cloud in the spatial overlap area is continuously reduced, while avoiding non-physical changes in relative pose that exceed the continuous motion capability of the device. This results in a corrected relative pose that simultaneously meets the requirements of point cloud spatial consistency and scanning device motion continuity, and this corrected relative pose is used to update the spatial alignment result of the current frame point cloud.

[0035] Based on the corrected relative pose, the spatial position and orientation of the current frame's 3D point cloud data are updated, and the updated current frame point cloud data is fed back into the incremental fusion processing flow to update the point cloud set and generate real-time updated point cloud stitching results.

[0036] After completing the incremental fusion and relative pose correction of the current frame point cloud, the global pose state of the incremental point cloud stitching model is updated to ensure spatial continuity and geometric consistency, and to provide a spatial reference benchmark for the addition of the next frame point cloud data. After completing the incremental fusion and relative pose correction of the current frame point cloud, the global pose state parameters in the incremental point cloud stitching model are updated based on the corrected pose of each frame point cloud in the updated point cloud set under a unified spatial reference coordinate system. Specifically, by summarizing and weighting the spatial positions and orientations of all stitched point clouds, a global translation, rotation, and orientation state reflecting the entire point cloud set under a unified coordinate system is generated, so that the global pose state parameters can accurately represent the overall spatial relationship of all currently stitched point clouds. The updated global pose state parameters are then used to provide a spatial reference benchmark for the next frame of point clouds to be added, thereby ensuring the continuity, consistency, and overall geometric stability of the point clouds in spatial position and orientation during subsequent incremental stitching processes.

[0037] The incremental point cloud data sequence to be added is sequentially added to the incremental point cloud stitching model in chronological order, forming a real-time point cloud stitching result that is continuously updated and spatially consistent under dynamic scanning conditions, thus realizing the incremental spatial integration of multi-frame 3D point cloud data.

[0038] It should be noted that by establishing a unified spatial reference benchmark with the first frame of point cloud during dynamic scanning, and by incrementally adding subsequent point clouds in chronological order, combining multi-source sensor data to perform real-time estimation and joint correction of the relative pose between point clouds under physical constraints, this not only effectively suppresses the cumulative registration deviation and non-physical pose jumps caused by continuous device movement, sensor errors, or point cloud noise, but also achieves stable enhancement of repeating structures and adaptive supplementation of new structures through consistency judgment and differentiated fusion strategies based on overlapping areas. Thus, while ensuring overall spatial continuity, geometric consistency, and smoothness, it improves the real-time performance, robustness, and accuracy of point cloud stitching, providing high-quality, continuously updated basic point cloud data for 3D reconstruction and intelligent analysis in complex architectural scenes.

[0039] S3, perform closed-loop detection on the point cloud stitching result. When spatial backtracking is detected, construct closed-loop constraints and perform global optimization to output a continuous fused point cloud of the building scene, realizing real-time automatic stitching in dynamic scanning state, including: Based on the point cloud stitching results, a hierarchical historical point cloud indexing method based on spatial partitioning and semantic awareness is used to construct a hierarchical historical point cloud indexing structure and select a candidate set of historical point clouds for loop closure detection. For the point cloud of the latest frame that has been stitched together at the current moment, based on its spatial position and attitude information in a unified spatial reference coordinate system, the historical point cloud candidate set is filtered according to a preset spatial distance threshold and a pose difference threshold. Historical point clouds with spatial distance values ​​less than the preset spatial distance threshold and pose differences (including translation difference and rotation difference) less than the preset pose difference threshold are selected as candidate historical point clouds that may have spatial revisit relationships. The current frame point cloud and the candidate historical point cloud are spatially matched and verified. By comparing the geometric similarity between the two in a unified spatial reference coordinate system, if the geometric similarity is greater than the preset similarity threshold, the above matching result satisfies the preset closed-loop judgment condition, and it is determined that spatial revisit has been detected, thus forming a closed-loop relationship. After detecting the closed-loop relationship, a closed-loop constraint is constructed based on the spatial matching result between the current frame point cloud and the corresponding historical point cloud. The closed-loop constraint is used to describe the relative pose consistency relationship that the current frame point cloud and the historical point cloud should satisfy in a unified spatial reference coordinate system. The closed-loop constraint and the device motion constraint formed during the incremental stitching of the point cloud are combined to form a global constraint set. Based on the global constraint set, the point cloud stitching result is globally optimized. The spatial pose parameters corresponding to each stitched point cloud are used as optimization variables. Under the condition of simultaneously satisfying the device motion constraints and closed-loop constraints of adjacent frames, the spatial pose parameters are jointly adjusted so that the stitching result meets the spatial consistency requirements on a global scale, thereby eliminating the global pose drift caused by the cumulative error during dynamic scanning. The global optimization process based on the global constraint set involves unifying the spatial pose parameters (including position, translation, attitude, and rotation parameters) of each stitched point cloud frame in a unified spatial reference coordinate system as variables to be optimized. Using the adjacent frame device motion constraints and closed-loop constraints included in the global constraint set, a global optimization objective is constructed to measure the spatial consistency between the point clouds in each frame. The adjacent frame device motion constraints maintain the stability of the local relative relationships of the point clouds during continuous temporal scanning, while the closed-loop constraints force the satisfaction of pose consistency requirements between non-adjacent frames at the detected spatial revisit locations. During optimization, the spatial pose parameters of each frame point cloud are jointly adjusted globally to ensure that all constraints are satisfied simultaneously as much as possible. This redistributes and corrects accumulated pose errors without disrupting local continuity, resulting in a consistent and coordinated spatial structure in the overall point cloud stitching. This effectively eliminates the global pose drift problem caused by the gradual accumulation of errors during dynamic scanning.

[0040] After completing the global optimization, based on the optimized spatial pose parameters of each frame of point cloud, the position and attitude of each frame of point cloud data in the point cloud stitching result are updated in a unified manner, so that all point cloud data maintain a consistent spatial relationship under a unified spatial reference coordinate system, and the point cloud stitching result after closed-loop constraint correction is obtained. After completing global optimization and obtaining the optimized spatial pose parameters corresponding to each frame of point cloud, the optimized spatial pose parameters are used as the basis for the final spatial transformation. The position and attitude of each frame of point cloud data already included in the point cloud stitching result are uniformly updated. Specifically, according to their respective optimized translation and rotation parameters, they are transformed from their original alignment state and mapped to a unified spatial reference coordinate system. Through this unified update process, all frame point clouds share a consistent spatial reference benchmark on a global scale, and their relative spatial position relationship is strictly consistent with the global optimization result. This ensures that the point clouds are continuous at spatial connections and aligned consistently within closed loop regions. Finally, a point cloud stitching result with global spatial consistency and geometric continuity is obtained under the closed loop constraint, eliminating cumulative errors and avoiding the problem of inconsistent overall spatial relationships caused by only local correction.

[0041] Based on the point cloud stitching results optimized by closed-loop constraints, all stitched point clouds in the building scene are continuously fused according to the optimized spatial pose parameters, and the resulting building scene fused point cloud data is continuous, complete and spatially consistent under a unified spatial reference coordinate system.

[0042] Furthermore, based on the point cloud stitching results, a hierarchical historical point cloud index structure is constructed using a spatial partitioning and semantic awareness-based hierarchical historical point cloud indexing method, and a candidate set of historical point clouds for loop closure detection is selected, including: Based on the point cloud stitching results, the three-dimensional spatial range of the building scene is divided under a unified spatial reference coordinate system. The three-dimensional space is divided into multiple spatial partition units according to the preset spatial partitioning rules, where each spatial partition unit is used to represent a local spatial region in the building scene. Among them, the preset spatial division rules refer to the rules for dividing three-dimensional space according to the geometric positional relationship under a unified spatial reference coordinate system, based on fixed-scale voxel division, regular grid division, or according to the building structure level (such as the range of floors, rooms, or corridors), and are used to divide the architectural scene into several local areas with clear spatial ranges.

[0043] For each frame of historical point cloud data that has been stitched together, the spatial partition unit covered by the historical point cloud data is determined based on its spatial location distribution in a unified spatial reference coordinate system, and the historical point cloud data is associated with the corresponding spatial partition unit to form a historical point cloud index structure based on spatial partition. After completing the construction of the historical point cloud index, semantic analysis is performed on the building elements represented by each historical point cloud data to obtain semantic identification information used to characterize the building structure type corresponding to the point cloud. The semantic identification information includes wall, ground, facade, corridor or other building component categories. The semantic identification information is bound to the corresponding historical point cloud data and its spatial partition unit, so that the historical point cloud data in each spatial partition unit has both spatial partition attributes and semantic attributes. Thus, a hierarchical historical point cloud index structure that integrates spatial partition information and semantic information is formed on the basis of the historical point cloud index structure. When performing loop closure detection, the spatial partition unit corresponding to the current frame point cloud is determined based on the spatial position of the current frame point cloud in a unified spatial reference coordinate system, and the spatial partition unit and its adjacent spatial partition units are used as candidate retrieval regions. Within the candidate retrieval area, based on the semantic features of the building elements corresponding to the current frame point cloud, historical point cloud data whose semantic attributes match those of the current frame point cloud are selected from the hierarchical historical point cloud index structure and used as a candidate historical point cloud set. Within the candidate retrieval area, semantic feature information that characterizes the building structure type is first extracted from the current frame point cloud, including the dominant building element category, the proportion of each element in the point cloud, and its spatial distribution characteristics. This semantic feature information is then mapped to corresponding semantic identifiers or semantic vectors. Subsequently, in the hierarchical historical point cloud index structure, only historical point cloud data belonging to the candidate retrieval area are read one by one, and their associated semantic identifier information is calculated. The semantic identifier information of the historical point cloud is then compared with the semantic feature information of the current frame point cloud. If the semantic similarity is higher than a preset threshold, the historical point cloud is determined to be comparable to the current frame point cloud at the semantic level. It is then selected and added to the candidate historical point cloud set for subsequent spatial matching and loop closure detection processing, thereby preventing semantically irrelevant point clouds from entering the candidate set and improving retrieval efficiency and accuracy.

[0044] It should be noted that by introducing a hierarchical historical point cloud indexing mechanism of "spatial partitioning + semantic awareness" on the basis of point cloud stitching results, the originally disordered and massive historical point cloud data is organized into a hierarchical index structure that has both spatial location attributes and architectural semantic attributes. In the loop closure detection stage, spatial partitioning is first used to limit the search range, and then candidate point clouds are further filtered through semantic feature similarity. This significantly compresses the size of the candidate set without relying on high-precision global pose, reduces the computational overhead of irrelevant point clouds participating in matching, and effectively reduces the risk of false detection caused by repeated structures or geometric similarity. This improves the efficiency, stability and accuracy of loop closure detection, and is particularly suitable for dynamic scanning and long-term point cloud stitching applications in large-scale and complex architectural scenes.

[0045] Reference Figure 2 As shown in the diagram, the intelligent building element stitching system based on 3D dynamic scanning imaging provided by this invention includes a correction and alignment module, a point cloud stitching module, and a closed-loop detection module, with connections between the modules; specifically: The calibration and alignment module is used to acquire multi-source sensor data of the building scene and perform time correction and spatial alignment on the continuously acquired point cloud data to generate a point cloud data sequence with a unified time series and initial pose relationship. The point cloud stitching module is used to incrementally stitch point cloud data in chronological order under dynamic scanning conditions, based on the initial pose relationship of the point cloud data sequence. During the stitching process, multi-source sensor data is integrated to estimate and correct the relative poses between point clouds in real time, generating continuously updated point cloud stitching results. The closed-loop detection module performs closed-loop detection on the point cloud stitching results. When spatial backtracking is detected, closed-loop constraints are constructed and global optimization is performed to output a continuous fused point cloud of the building scene.

[0046] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0047] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0048] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0049] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0050] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent stitching of building elements based on 3D dynamic scanning imaging, characterized in that, Includes the following steps: Acquire multi-source sensor data of the building scene, and perform time correction and spatial alignment on the continuously collected point cloud data to generate a point cloud data sequence with a unified time series and initial pose relationship; Based on the initial pose relationship of the point cloud data sequence, the point cloud data is incrementally stitched together in time order under dynamic scanning conditions. During the stitching process, multi-source sensor data is integrated to estimate and correct the relative pose between point clouds in real time, generating continuously updated point cloud stitching results. Closed-loop detection is performed on the point cloud stitching results. When spatial backtracking is detected, closed-loop constraints are constructed and global optimization is performed to output a continuous fused point cloud of the building scene.

2. The intelligent stitching method for building elements based on 3D dynamic scanning imaging according to claim 1, characterized in that, The step of performing time correction and spatial alignment on continuously acquired point cloud data to generate a point cloud data sequence with a unified time sequence and initial pose relationship includes: Extract the acquisition time of each frame of point cloud and the device pose-related data within the corresponding time period from multi-source sensor data; Based on the acquisition time, set a time offset parameter for the device pose-related data; Based on the time offset parameter, the time correspondence of the device pose-related data is corrected, and the initial time correspondence between the device pose-related data and the point cloud data is established. Based on the initial time correspondence, the point cloud of each frame is mapped to a unified spatial reference coordinate system using the corrected device pose data to obtain the initial spatial alignment result. Based on the initial spatial alignment results, an optimization objective function including spatial error terms and temporal constraint terms is constructed, and the spatial pose parameters and temporal offset parameters of the point cloud are iteratively and jointly optimized. Based on the optimized time offset parameters and spatial pose parameters, time correction and spatial alignment are performed on multi-source sensor data to form a point cloud data sequence with a unified time series and initial pose relationship.

3. The intelligent stitching method for building elements based on 3D dynamic scanning imaging according to claim 2, characterized in that, The step of correcting the time correspondence of device pose-related data based on time offset parameters and establishing an initial time correspondence between device pose-related data and point cloud data includes: Obtain the original timestamps corresponding to each pose data in the device pose-related data, and construct the device pose time series in chronological order; The original time stamps in the device pose time series are corrected based on the time offset parameter to obtain the corrected device pose time series, and the corresponding time position of the acquisition time of each frame of 3D point cloud data in the device pose time series is determined. When there is device pose data in the corrected device pose time series that corresponds to the acquisition time of a certain frame of 3D point cloud data, the device pose data is selected as the initial pose information of the 3D point cloud data for that frame. When there is no device pose data in the corrected device pose time series that is consistent with the acquisition time of the 3D point cloud data, the corresponding initial pose information is generated based on the device pose data in the device pose time series that is adjacent to the acquisition time. Based on the initial pose information, the corresponding device pose data is associated with each frame of 3D point cloud data, and the initial time correspondence between the corrected device pose-related data and the 3D point cloud data is established.

4. The intelligent stitching method for building elements based on 3D dynamic scanning imaging according to claim 1, characterized in that, The initial pose relationship based on the point cloud data sequence is used to incrementally stitch the point cloud data in chronological order under dynamic scanning conditions. During the stitching process, multi-source sensor data is fused to estimate and correct the relative poses between point clouds in real time, generating continuously updated point cloud stitching results, including: Based on the initial pose relationship of the point cloud data sequence, the first frame of 3D point cloud data in the point cloud data sequence is selected as the initial reference of the incremental point cloud stitching model, and the initial state is established. According to the acquisition time sequence, the three-dimensional point cloud data of each frame except the first frame are obtained from the point cloud data sequence in sequence to form the point cloud data sequence to be added incrementally; For each frame of 3D point cloud data to be added incrementally, the initial pose information is extracted and transformed into a unified spatial reference coordinate system to achieve preliminary spatial alignment. Incremental fusion processing is performed on the current frame's 3D point cloud data that has undergone preliminary spatial alignment and the existing point cloud set in the incremental point cloud stitching model. After incremental fusion processing, the relative pose between the current frame point cloud and the stitched point cloud is estimated and corrected in real time based on multi-source sensor data. Based on incremental fusion processing and relative pose correction, the global pose state of the incremental point cloud stitching model is updated until all point cloud data sequences to be added are added to the incremental point cloud stitching model to form a real-time point cloud stitching result.

5. The intelligent stitching method for building elements based on 3D dynamic scanning imaging according to claim 4, characterized in that, The incremental fusion process includes: Based on the preliminary spatial alignment results, the spatial distribution relationship between the current frame's 3D point cloud data and the point cloud set under a unified spatial reference coordinate system is determined, and the overlapping area of ​​the point clouds is delineated. Within the overlapping area of ​​point clouds, a candidate fusion point set is extracted from the current frame's 3D point cloud data, and a reference point set is extracted from the point cloud set; For each candidate point in the candidate fusion point set, based on its spatial distance from the corresponding reference point set, local geometric distribution features, and point cloud density, it is determined whether the candidate point satisfies the fusion consistency condition. Based on the judgment results, differentiated fusion update processing is performed on the candidate fusion points to obtain the updated point cloud set.

6. The intelligent stitching method and system for building elements based on 3D dynamic scanning imaging according to claim 4, characterized in that, The real-time estimation and correction of the relative pose between the current frame point cloud and the stitched point cloud based on multi-source sensor data includes: Based on the updated point cloud set, determine the spatial overlapping region; Within the spatially overlapping region, spatial feature information for pose estimation is extracted; Based on spatial feature information, a relative pose error is constructed; Acquire device pose-related data corresponding to the current frame point cloud acquisition time, and construct device motion constraints; The relative pose of the current frame point cloud relative to the stitched point cloud is jointly corrected by using the relative pose error and device motion constraints.

7. The intelligent stitching method for building elements based on 3D dynamic scanning imaging according to claim 1, characterized in that, The process of performing closed-loop detection on the point cloud stitching results, constructing closed-loop constraints and performing global optimization when spatial backtracking is detected, and outputting a continuous fused point cloud of the building scene includes: Based on the point cloud stitching results, a hierarchical historical point cloud indexing method based on spatial partitioning and semantic awareness is used to construct a hierarchical historical point cloud indexing structure and filter the candidate set of historical point clouds. For the current frame point cloud, based on its spatial position and attitude information in a unified spatial reference coordinate system, candidate historical point clouds with spatial revisit relationships are selected from the candidate set of historical point clouds. Spatial matching verification is performed between the current frame point cloud and the candidate historical point cloud to confirm that a closed loop relationship has been detected. After detecting the closed-loop relationship, the closed-loop constraint is constructed based on the spatial matching result, and together with the device motion constraint formed during the incremental stitching of the point cloud, they form a global constraint set. Based on the global constraint set, the point cloud stitching result is globally optimized, and the spatial pose parameters of each frame of point cloud are jointly adjusted. The point cloud data of each frame is updated based on the optimized spatial pose parameters, and the point cloud of the building scene is continuously fused to output the fused point cloud of the building scene.

8. The intelligent stitching method for building elements based on 3D dynamic scanning imaging according to claim 7, characterized in that, The method for hierarchical historical point cloud indexing based on spatial partitioning and semantic awareness, according to the point cloud stitching results, constructs a hierarchical historical point cloud index structure and filters the candidate set of historical point clouds, including: Based on the point cloud stitching results, the three-dimensional spatial range of the architectural scene is divided into spatial partition units; For each frame of historical point cloud data that has been stitched together, determine the historical point cloud index structure; Semantic analysis is performed on the building elements represented by each historical point cloud data to obtain semantic identification information; Semantic identifier information is bound to the corresponding historical point cloud data and its spatial partition unit to form a hierarchical historical point cloud index structure. Determine the spatial partition unit corresponding to the point cloud of the current frame, and use the spatial partition unit and its adjacent spatial partition units as candidate search areas; Within the candidate retrieval area, historical point cloud data whose semantic attributes match those of the current frame point cloud are selected from the hierarchical historical point cloud index structure and used as a candidate historical point cloud set.

9. A system using the intelligent stitching method for building elements based on 3D dynamic scanning imaging as described in any one of claims 1-8, characterized in that, include; The calibration and alignment module is used to acquire multi-source sensor data of the building scene and perform time correction and spatial alignment on the continuously acquired point cloud data to generate a point cloud data sequence with a unified time series and initial pose relationship. The point cloud stitching module is used to incrementally stitch point cloud data in chronological order under dynamic scanning conditions, based on the initial pose relationship of the point cloud data sequence. During the stitching process, multi-source sensor data is integrated to estimate and correct the relative poses between point clouds in real time, generating continuously updated point cloud stitching results. The closed-loop detection module performs closed-loop detection on the point cloud stitching results. When spatial backtracking is detected, closed-loop constraints are constructed and global optimization is performed to output a continuous fused point cloud of the building scene.

Images