A real scene three-dimensional modeling method and system fusing indoor and outdoor point cloud data

Abstract

This invention belongs to the field of modeling technology, specifically relating to a method and system for real-scene 3D modeling that integrates indoor and outdoor point cloud data. The modeling method includes the following steps: collecting indoor and outdoor point cloud data; preprocessing, standardizing the format, and controlling the quality of the collected point cloud data; extracting significant features from the point cloud data, performing coarse registration, and constructing a spatial relationship vector model with constraints for preliminary alignment; performing fine registration of the point cloud data and graph optimization to correct errors; and combining semantic tags to perform refined geometric modeling and fitting to form a fused real-scene 3D model. This invention addresses the problem of poor indoor and outdoor modeling results. By utilizing feature extraction and preliminary alignment techniques, it successfully performs preliminary registration of indoor and outdoor point cloud data, and eliminates errors through fine registration and graph optimization, ensuring the fusion of data from different regions. Through semantic analysis and accurate 3D modeling, a high-precision 3D model with semantic tags is finally constructed.

Description

Technical Field

[0001] This invention belongs to the field of modeling technology, specifically relating to a real-scene 3D modeling method and system that integrates indoor and outdoor point cloud data. Background Technology

[0002] With the rapid development of smart city and Building Information Modeling (BIM) technologies, real-world 3D modeling is becoming increasingly important in fields such as urban planning, architectural design, and facility management. 3D modeling technology collects data from the real world and transforms it into digital models, enabling the intuitive representation of complex spatial structures and environments. However, existing 3D modeling technologies often rely on a single data source, which frequently leads to registration errors between indoor and outdoor data, thus affecting modeling accuracy and subsequent application effectiveness.

[0003] To achieve more accurate and efficient 3D modeling, especially in urban construction and smart city management, the integration of indoor and outdoor point cloud data and the elimination of registration errors have become urgent technical requirements. By integrating indoor and outdoor point cloud data, spatial information can be reconstructed more comprehensively, improving the breadth and accuracy of data applications.

[0004] Although existing 3D modeling techniques have been widely applied in various scenarios, they still face challenges in fusing indoor and outdoor point cloud data. Traditional point cloud data acquisition techniques are usually carried out independently, and indoor and outdoor data may have significant differences in spatial and geometric structures, causing errors during registration. Existing preliminary registration methods often cannot handle such geometric differences between data, resulting in poor registration effects and ultimately affecting the accuracy and continuity of modeling. In the process of point cloud data processing, in practical applications, existing methods still have limited effectiveness in areas such as hole repair and boundary smoothing, and cannot guarantee perfect fusion of the model structure, especially in large-scale urban models, where efficient and accurate transitions cannot be achieved. Summary of the Invention

[0005] To address the shortcomings of existing technologies, a real-scene 3D modeling method and system that integrates indoor and outdoor point cloud data is provided to solve the problem of poor indoor and outdoor modeling results.

[0006] On the one hand, the technical solution of the present invention to solve the above-mentioned technical problems is as follows: a real-scene 3D modeling method that integrates indoor and outdoor point cloud data, which includes the following steps: Step S1: Collect indoor and outdoor point cloud data, and perform preprocessing, format standardization, and quality control on the collected point cloud data; Step S2: Extract salient features from point cloud data, perform coarse registration, and construct a spatial relationship vector model with constraints for preliminary alignment; Step S3: Perform fine registration of the point cloud data and optimize the graph to correct errors; Step S4: Combine semantic tags to perform refined geometric modeling and fit to form a fused real-world 3D model.

[0007] In one embodiment, step S1 includes: S1.1 Collect indoor and outdoor point cloud data from multiple sources; S1.2 Perform preprocessing operations such as denoising, thinning, and registration on the collected indoor and outdoor point cloud data; S1.3 Unify the point cloud format of all indoor and outdoor point cloud data into a standard framework; S1.4 Identify missing regions and outliers in the preprocessed point cloud data and perform interpolation repair.

[0008] In one embodiment, step S2 includes: S2.1 Extract salient features from point cloud data of indoor and outdoor public areas; S2.2. Based on the extracted feature information, the indoor and outdoor point cloud data are initially aligned using the ICP algorithm; S2.3 Construct a mathematical model with constraints to optimize the initial alignment results; S2.4 Fine-tune the initially aligned point cloud data by adjusting the parameters in the transformation matrix to reduce the registration error between point clouds and obtain the initial registration result.

[0009] In one embodiment, step S3 includes: S3.1 Establish a pose graph optimization problem model, integrating the position information of multiple scanning stations into a unified optimization framework; S3.2 Incorporate shared feature points from indoor and outdoor point cloud data into the optimization framework for visual overlay and fusion; S3.3. Apply the graph optimization algorithm to minimize the error, obtain a globally consistent scanning station pose, and iterate to optimize the point cloud registration results; S3.4 Real-time monitoring of registration errors and convergence of abnormal errors in the target area.

[0010] In one embodiment, step S4 includes: S4.1. Use a deep learning model to perform semantic segmentation on the fused point cloud, identify different architectural elements and assign semantic labels; S4.2. Based on the semantic segmentation results, perform geometric modeling, and combine semantic information to perform hierarchical optimization and reconstruction of the geometric structure of point clouds of different categories, so as to fit a more realistic scene. S4.3 Based on the optimized point cloud data and semantic information, each part of the semantic information classification model is used to fit the spatial pose and size ratio of the scene and the geometry in the scene based on semantic attributes, and then outputs the fused real-world 3D model with semantic labels.

[0011] On the other hand, this embodiment also discloses a real-scene 3D modeling system that integrates indoor and outdoor point cloud data, which includes: The data acquisition and preprocessing module is used to acquire indoor and outdoor point cloud data, and to preprocess, standardize, and control the quality of the acquired point cloud data. The coarse alignment module is used to extract significant features from point cloud data, perform coarse registration, and construct a spatial relationship vector model with constraints for initial alignment. The graph optimization module is used to perform fine registration of point cloud data and to perform graph optimization to correct errors. The semantic-guided reconstruction module is used to combine semantic tags to perform refined geometric modeling and fit a fused real-world 3D model.

[0012] In one embodiment, the data acquisition preprocessing module includes: The outdoor point cloud acquisition module is used to collect outdoor point cloud data from multiple sources; The indoor point cloud acquisition module is used to collect indoor point cloud data from multiple sources; The data cleaning and preprocessing module is used to perform preprocessing operations such as denoising, thinning, and registration on the collected indoor and outdoor point cloud data. The point cloud registration module is used to unify the point cloud format of all indoor and outdoor point cloud data into a standard framework.

[0013] In one embodiment, the coarse alignment module includes: The feature extraction module is used to extract salient features from point cloud data of indoor and outdoor public areas; The coarse registration module is used to initially align indoor and outdoor point cloud data based on the extracted feature information and the ICP algorithm. The modeling module is used to construct a mathematical model containing constraints to optimize the initial alignment results; The fine-tuning module is used to fine-tune the initially aligned point cloud data. By adjusting the parameters in the transformation matrix, the registration error between point clouds is reduced to obtain the initial registration result.

[0014] In one embodiment, the graph optimization module includes: The graph optimization and integration module is used to establish a pose graph optimization problem model, integrating the position information of multiple scanning stations into a unified optimization framework; The overlay and fusion module is used to incorporate common feature points from indoor and outdoor point cloud data into an optimization framework for visual overlay and fusion. The algorithm optimization module is used to apply graph optimization algorithms to minimize errors, obtain globally consistent scanning station poses, and iterate to optimize the point cloud registration results. The monitoring module is used to monitor registration errors in real time and converge abnormal errors in the target area.

[0015] In one embodiment, the semantic-guided reconstruction module includes: The semantic segmentation module is used to perform semantic segmentation on the fused point cloud using a deep learning model, identify different architectural elements and assign semantic labels. The boundary repair module is used to perform geometric modeling based on semantic segmentation results. It combines semantic information to perform hierarchical optimization and reconstruction of the geometric structure of point clouds of different categories, and fits a more realistic scene. The refined fusion module is used to classify each part of the model based on the optimized point cloud data and semantic information, and fit the scene and the geometry in the scene according to the semantic attributes to fit the spatial posture and size ratio of the real scene, and then output the fused real scene 3D model with semantic labels.

[0016] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. Through precise acquisition, preprocessing, and quality control, the accuracy and reliability of point cloud data were ensured. By utilizing feature extraction and preliminary alignment techniques, indoor and outdoor point cloud data were successfully preliminarily registered. Errors were eliminated through fine registration and graph optimization, ensuring the fusion of data from different regions. Through semantic analysis and precise 3D modeling, a 3D model with high precision and semantic labels was finally constructed, which can effectively support applications in fields such as architecture and urban planning, and enhance the practical value and application effect of data in multiple scenarios.

[0017] 2. By efficiently collecting, processing, registering, and reconstructing indoor and outdoor point cloud data, high-precision 3D model construction was achieved. Data quality was optimized through denoising, point cloud simplification, and coarse registration to ensure spatial alignment of indoor and outdoor data. Semantic segmentation, hole repair, and boundary smoothing were performed using deep learning to ensure perfect integration of model accuracy and structure. Graph optimization and error minimization techniques were used to further improve registration accuracy and ensure the transition of point cloud data. This technology can be applied to path planning and dynamic updates in smart cities. Attached Figure Description

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the method flow of the present invention.

[0020] Figure 2 This is a schematic diagram of the system modules of the present invention. Detailed Implementation

[0021] The embodiments of the technical solution of the present invention will now be described in detail with reference to the accompanying drawings. These embodiments are merely illustrative of the technical solution of the present invention and are therefore intended to limit the scope of protection of the present invention. It should be noted that, unless otherwise stated, the technical or scientific terms used in this application should have the ordinary meaning as understood by those skilled in the art to which this invention pertains.

[0022] In the description of this application, it should be understood that the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting the present invention.

[0023] Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly defined.

[0024] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0025] Example 1 like Figure 1 As shown, a real-scene 3D modeling method that integrates indoor and outdoor point cloud data includes the following steps: Step S1: Collect indoor and outdoor point cloud data, and perform preprocessing, format standardization, and quality control on the collected point cloud data; Step S2: Extract salient features from point cloud data, perform coarse registration, and construct a spatial relationship vector model with constraints for preliminary alignment; Step S3: Perform fine registration of the point cloud data and optimize the graph to correct errors; Step S4: Combine semantic tags to perform refined geometric modeling and fit to form a fused real-world 3D model.

[0026] Step S1 includes: S1.1 Collect indoor and outdoor point cloud data from multiple sources; Outdoor data is typically collected using vehicle-mounted or airborne LiDAR systems, which can cover a wide area and accurately capture external terrain features. Indoor data is collected using backpack or handheld LiDAR devices, which can provide rich point cloud information in complex indoor environments. Specifically, the choice of different devices can be determined according to actual needs, such as acquisition accuracy, range, and environmental conditions.

[0027] S1.2 Preprocessing of the collected indoor and outdoor point cloud data, including denoising, thinning, and registration; The denoising process is used to eliminate noise points in point cloud data to ensure the accuracy and reliability of the data. The thinning operation reduces the density of the point cloud to ensure computational efficiency during data processing. The registration operation mainly uses an optimized algorithm to align point cloud data at different scanning positions.

[0028] S1.3 Unified format includes unifying the point cloud format of all indoor and outdoor point cloud data into a standard framework; Point cloud format unification can be achieved by converting point cloud data to LAS format through operations such as format conversion. Point cloud coordinates can be converted based on the seven-parameter coordinate system calculated from existing local control points, so that data from different sources can be unified under a standard framework, which is helpful for subsequent registration and fusion.

[0029] S1.4 Quality control includes identifying missing regions and outliers in the preprocessed point cloud data and performing interpolation repair. To ensure data integrity and accuracy, quality checks include identifying missing regions or outliers in the preprocessed point cloud data, assessing their impact on the overall model. Specifically, this involves extracting point cloud data surrounding the missing regions, including x, y, and z coordinates. Then, a backpropagation (BP) neural network is used to train the existing point cloud data around the missing regions. This is achieved by extracting point cloud data from the vicinity of the missing regions as training samples and inputting the x, y, and z coordinates of the point clouds into the network z = ax2 In +bx+c, the model parameters are iteratively optimized by comparing the residual with the true value, calculating the error backpropagation, and training to obtain the final nonlinear point cloud prediction model.

[0030] The model is trained to predict the terrain change trend reflected by the point cloud, and the point cloud in the blank area is interpolated or repaired to ensure that the data meets the requirements of subsequent modeling.

[0031] Step S2 includes: S2.1 The salient features include the corner points of doors and windows in the common edge area, and all target feature points within the visible area where indoor and outdoor point clouds intersect; Before coarse registration, salient features are extracted from the point cloud data of indoor and outdoor public areas. The salient features mainly include the corner points of doors and windows in the public border area and all target feature points within the line of sight between indoor and outdoor point clouds. Specifically, image feature points are automatically extracted using the SIFT operator based on the collected panoramic image data.

[0032] Next, the target feature points of indoor and outdoor public areas are compared with the existing point cloud data of the public areas, and the intersection is taken to extract the point cloud data with obvious features. This point cloud data can reflect obvious features. Then, it is combined with the joint mode selected by manual judgment and screening to select and refine the point cloud with obvious features again.

[0033] S2.2 Coarse registration includes: using the ICP algorithm to perform initial alignment of indoor and outdoor point cloud data; Based on the extracted feature information, the indoor and outdoor point cloud data are initially aligned using the ICP algorithm (Iterative Proximity Method) or its variants or other feature matching algorithms. Specifically, by matching common feature points or boundary points, a rough transformation matrix can be calculated. This transformation matrix contains homography matrices for spatial translation and rotation. Using one of the point clouds as a reference, the matrix translates and rotates the point clouds to be registered, so that the indoor point cloud is roughly aligned with the outdoor point cloud in space, thus completing the coarse registration.

[0034] S2.3. Based on coarse registration, construct a spatial relationship vector model containing constraints. The constraints can come from the geometric constraints of indoor and outdoor point clouds, such as wall boundaries and doors and windows, as well as known information about the relative positions between scanning devices, to ensure the accuracy of the alignment process. Specifically, the constraints can be set by setting thresholds such as distance and angle through the common buffer constructed between scanning points, the proximity distance between adjacent point clouds after coarse registration, and the spatial angle between the line clouds formed by adjacent point clouds, so as to make the point clouds used for registration fit better, thus optimizing the initial alignment results.

[0035] S2.4 also includes fine-tuning the initially aligned point cloud data. By actively adjusting the parameters in the transformation matrix, the parameter values ​​are adaptively adjusted in reverse according to the point cloud registration error, so as to gradually reduce the registration error between point clouds and obtain a more accurate preliminary registration result, ensuring that the transition area between indoor and outdoor point clouds can be more accurately integrated.

[0036] Step S3 includes: S3.1 In the fine registration stage, the problem is modeled as a pose graph optimization problem, where nodes represent the positions of scanning stations and edges represent geometric constraints such as the relative positions between scanning stations. By establishing this graph structure, the position information of multiple scanning stations can be integrated into a unified framework to form a framework graph containing nodes and edges, which provides support for subsequent optimization calculations. S3.2 Incorporate shared feature points from indoor and outdoor point cloud data into the optimization framework for visual overlay and fusion; To achieve accurate fusion, cross-regional constraints on shared feature points between indoor and outdoor areas are incorporated into the optimization framework. Using the features and common feature points extracted from step 2, visual overlay fusion is performed on areas containing common features. Spatial and semantic relationships are established for pairs of common feature points. Specifically, for example, whether the distance is less than a threshold and whether they belong to the same feature. If the conditions are met, a semantic relationship is established. The indoor and outdoor point cloud data are optimized through mutual correlation to ensure the connection of transition areas. S3.3. Apply graph optimization algorithms, such as g2o or Ceres Solver, to minimize the error. Treat the cloud position and pose of each scanning station as nodes, and the ICP registration results between scanning stations as constraint edges. The constraint form is as follows: The error is .in, and These are the spatial poses of the i-th and j-th scanning stations. It represents the relative spatial relationship between i and j, indicating that a certain matching calculation relationship has been established at the i-th scanning station and the result has been transferred to the j-th scanning station. Essentially, it establishes the discrepancy between the "actual pose difference" and the "observed pose difference." This represents the actual relative pose of station i and station j. It is the "observed relative pose" obtained through registration, that is, the true value in the ideal. This is the deviation between the "estimated relative pose" and the "observed relative pose". If the optimized matching is completely correct, the pose deviation will be infinitely close to 0.

[0037] Adjust the parameters of all nodes to minimize the sum of errors of all edges, thereby obtaining a globally consistent pose solution. Optimize the constraint relationship between all nodes and edges to obtain a globally consistent scanning station pose. Then, through multiple iterations, gradually optimize the point cloud registration results, eliminate accumulated errors and inconsistencies, and improve the fusion accuracy of the point cloud. S3.4 Real-time monitoring of registration errors and convergence of abnormal errors in the target area; During the optimization process, the registration error is monitored in real time, and corresponding correction measures are taken. If abnormal errors are found in the target area, the constraints or local optimization strategies and related parameters are adjusted in a timely manner to avoid significant deviations in the global model.

[0038] Not only are overlapping areas detected between adjacent scanning stations, but overlapping areas are also detected between some non-adjacent scanning stations. At this time, global constraints are used to match the point clouds between scanning stations. If overlapping areas between non-adjacent scanning stations are detected, for example, returning to the starting position, the global constraints obtained through ICP are used to eliminate the cumulative error over long distances, and finally eliminate the weighted sum of squared errors of all weighted edges.

[0039] Specifically, this is achieved through the following formula: ; in, Let E be the set of the positions and orientations of all nodes, and let E be the set of edges. It is the error weight matrix, which is adaptively adjusted based on the ICP registration results. The more accurate the registration, the greater the weight.

[0040] Step S4 includes: S4.1. Use a deep learning model to perform semantic segmentation on the fused point cloud, identify different architectural elements and assign semantic labels; After data optimization, a deep learning model is trained using existing point cloud data of different types to perform semantic segmentation on the fused point cloud. The model automatically identifies different building elements, such as exterior walls, interior walls, doors, and windows, giving the point cloud data semantic labels. This process makes the spatial layout of the model clearer and provides key support for subsequent modeling and applications. S4.2. Based on the semantic segmentation results, perform geometric modeling, and combine semantic information to perform hierarchical optimization and reconstruction of the geometric structure of point clouds of different categories, so as to fit a more realistic scene. Specifically, based on the semantic segmentation results, a refined geometric model is then performed. The geometric structure of the point cloud is optimized under the guidance of semantic information. The geometric structure of different types of point clouds is optimized and reconstructed in a hierarchical manner. Combined with its semantic attribute constraints, a planar, curved, or regular geometric body that is more in line with the actual scene is fitted. The geometric deviations caused by noise and occlusion in the original point cloud are filtered and corrected, and its regularity and integrity are enhanced.

[0041] Targeted geometric fitting algorithms are employed to fit planar, curved, or regular geometric shapes to various target regions in the original point cloud: First, for semantic regions labeled as walls, ground, or other planar regions, non-planar interference points within the region are eliminated using planar fitting algorithms such as RANSAC, combined with semantic boundary constraints.

[0042] In the process of conducting refined geometric modeling based on semantic segmentation results, semantic attribute constraints will run through the entire process of point cloud geometric structure optimization, becoming the core guiding principle for fitting accurate geometric shapes, correcting geometric deviations, and improving the regularity and integrity of the model.

[0043] First, for the different categories of targets labeled after semantic segmentation, such as building walls, ground, doors and windows, vegetation, and furniture, the geometric morphological rules corresponding to each type of target are clarified by combining their inherent semantic attribute features. For example, building targets, such as walls, ground, and roof, usually have regular planar or regular curved surface features. Furniture targets, such as tables and chairs, are mostly composed of standard geometric shapes, such as cuboids, cylinders, and prisms. Natural scene targets, such as vegetation and terrain, present irregular curved surface features with inherent morphological logic. Based on this, exclusive geometric fitting constraints are set for each type of target.

[0044] S4.3 Based on the optimized point cloud data and semantic information, each part of the semantic information classification model is used to fit the spatial pose and size ratio of the scene and the geometry in the scene based on semantic attributes, and then outputs the fused real-world 3D model with attached semantic labels. Based on the above semantic attribute constraints, a precise plane that fits the actual scene's spatial posture and size ratio is fitted to ensure the plane's flatness and spatial positioning accuracy. For semantic regions labeled as roofs, cylindrical components, or other curved surfaces, fitting methods such as B-spline surfaces and NURBS surfaces can be used. By relying on the region range defined by the semantic labels, the curvature variation law of the surface can be captured, and a smooth surface structure that conforms to the actual shape of the target can be fitted, taking into account both the continuity of the surface and the authenticity of the shape. For semantic regions labeled as tables, chairs, pipes, etc., which are composed of regular geometric shapes, the relative spatial constraint relationship between the geometric dimension parameters and the physical components is estimated, and corresponding standard geometric shapes such as cuboids, cylinders, and cones are fitted to ensure that the side length, radius, angle and other parameters of the geometric shapes are consistent with the target in the actual scene, so as to achieve the standardized geometric expression of regular targets.

[0045] Meanwhile, during the acquisition process, raw point cloud data is susceptible to factors such as environmental noise, object occlusion, and equipment precision limitations, resulting in geometric deviation problems. For example, noise causes the point cloud to be discretely distributed and the boundaries to be blurred; occlusion causes the target to have local point cloud missing and geometric shape broken; and equipment errors cause the fitted geometric structure to deviate from the actual target.

[0046] Combining semantic attribute constraints will provide a clear direction for correcting these geometric deviations.

[0047] For example, to address the bias caused by noise, effective point clouds of various targets are selected by combining semantic tags, noise points and isolated points are removed, and the point cloud distribution is optimized by point cloud smoothing algorithm to make the point cloud more closely match the true geometric shape of the target. For example, to address deviations caused by occlusion, based on the geometric feature patterns of similar semantic targets, missing point cloud regions can be repaired and geometric breaks can be filled through interpolation completion, morphological reasoning, and other methods to ensure the integrity of the target's geometric structure. For example, to address deviations caused by equipment precision, the parameters of the fitted planes, curved surfaces, and regular geometric bodies are corrected based on the actual geometric scale and spatial posture corresponding to the semantic attributes, so that the geometric modeling results are consistent with the scale and position of the actual scene.

[0048] Finally, the fused 3D model of the real scene is output, along with detailed semantic tags.

[0049] This model can be used in a variety of application scenarios. Its high accuracy and semantic richness make it more widely applicable, especially in cross-domain integrated utilization.

[0050] In the above embodiments, the accuracy and reliability of point cloud data are ensured through precise acquisition, preprocessing, and quality control. The method utilizes feature extraction and preliminary alignment techniques to successfully perform preliminary registration of indoor and outdoor point cloud data. Errors are eliminated through fine registration and graph optimization, ensuring the fusion of data from different regions. Through semantic analysis and precise 3D modeling, a 3D model with high precision and semantic labels is finally constructed, which can effectively support applications in fields such as architecture and urban planning, and enhance the practical value and application effect of data in multiple scenarios.

[0051] Example 2 like Figure 1-2As shown, a real-scene 3D modeling system integrating indoor and outdoor point cloud data includes: The data acquisition and preprocessing module is used to acquire indoor and outdoor point cloud data, and to preprocess, standardize, and control the quality of the acquired point cloud data. The coarse alignment module is used to extract significant features from point cloud data, perform coarse registration, and construct a spatial relationship vector model with constraints for initial alignment. The graph optimization module is used to perform fine registration of point cloud data and to perform graph optimization to correct errors. The semantic-guided reconstruction module is used to combine semantic tags to perform refined geometric modeling and fit a fused real-world 3D model. And an application integration module, which is used to apply the processed 3D model to real-world scenarios to support smart city functions.

[0052] The data acquisition and preprocessing module includes: The outdoor point cloud acquisition module is used to collect outdoor point cloud data from multiple sources; The indoor point cloud acquisition module is used to collect indoor point cloud data from multiple sources; The data cleaning and preprocessing module is used to perform preprocessing operations such as denoising, thinning, and registration on the collected indoor and outdoor point cloud data. The point cloud registration module is used to unify the point cloud format of all indoor and outdoor point cloud data into a standard framework.

[0053] The outdoor point cloud acquisition module collects outdoor point cloud data through equipment such as vehicle-mounted lidar or UAV oblique photography, and provides outdoor point cloud data for the indoor point cloud acquisition module. The indoor point cloud acquisition module collects indoor point clouds using a backpack or handheld lidar device, providing indoor point cloud data for the data cleaning and preprocessing module. The data cleaning and preprocessing module processes indoor and outdoor point cloud data through noise reduction and point cloud simplification algorithms to provide processed point cloud data for the point cloud registration module. The point cloud registration module performs preliminary registration of indoor and outdoor point cloud data, providing preliminary aligned point cloud data for the coarse alignment module.

[0054] The coarse alignment module includes: The feature extraction module is used to extract salient features from point cloud data of indoor and outdoor public areas; The coarse registration module is used to initially align indoor and outdoor point cloud data based on the extracted feature information and the ICP algorithm. The modeling module is used to construct a mathematical model containing constraints to optimize the initial alignment results; The fine-tuning module is used to fine-tune the initially aligned point cloud data. By adjusting the parameters in the transformation matrix, the registration error between point clouds is reduced to obtain the initial registration result.

[0055] The feature extraction module extracts significant building features from indoor and outdoor point clouds, providing extracted feature data for the coarse registration module. The coarse registration module uses the extracted features to calculate a preliminary transformation matrix, providing the modeling module with coarsely aligned point cloud data. The graph optimization module includes: The graph optimization and integration module is used to establish a pose graph optimization problem model, integrating the position information of multiple scanning stations into a unified optimization framework; The overlay and fusion module is used to incorporate common feature points from indoor and outdoor point cloud data into an optimization framework for visual overlay and fusion. The algorithm optimization module is used to apply graph optimization algorithms to minimize errors, obtain globally consistent scanning station poses, and iterate to optimize the point cloud registration results. The monitoring module is used to monitor registration errors in real time and converge abnormal errors in the target area.

[0056] The semantic-guided reconstruction module includes: The semantic segmentation module is used to perform semantic segmentation on the fused point cloud using a deep learning model, identify different architectural elements and assign semantic labels. The boundary repair module is used to perform geometric modeling based on semantic segmentation results. It combines semantic information to perform hierarchical optimization and reconstruction of the geometric structure of point clouds of different categories, and fits a more realistic scene. The refined fusion module is used to classify each part of the model based on the optimized point cloud data and semantic information, and fit the scene and the geometry in the scene according to the semantic attributes to fit the spatial posture and size ratio of the real scene, and then output the fused real scene 3D model with semantic labels.

[0057] The semantic segmentation module performs semantic segmentation on the fused point cloud data using a deep learning model, providing semantic information for the boundary repair module. The boundary repair module uses semantic segmentation results to repair holes in the point cloud and smooth the boundaries, providing processed point cloud data for the fine fusion module; The lightweight modeling module provides a lightweight model for the refined fusion module; The refined fusion module converts point cloud data into a mesh model or BIM model based on semantic information, ensures the transition between indoor and outdoor point clouds by processing transition areas, and provides a processed 3D model.

[0058] The application integration module includes: The model output display module is used to generate a real-world 3D model with semantic labels based on the processed point cloud data and output it to the user or application system. The application integration module may also include: a virtual reality and augmented reality module, used to use the generated 3D model for VR or AR applications, supporting indoor and outdoor navigation and architectural design scenarios, and achieving an immersive experience by integrating VR and AR devices; The intelligent route planning module is used for route planning based on a fused indoor and outdoor model and to support indoor and outdoor navigation functions.

[0059] In the above embodiments, high-precision 3D model construction is achieved by collecting, processing, registering, and reconstructing indoor and outdoor point cloud data. The system uses equipment such as vehicle-mounted LiDAR, UAV photography, and handheld LiDAR to collect indoor and outdoor point cloud data. Data quality is optimized through denoising, point cloud simplification, and coarse registration to ensure spatial alignment of indoor and outdoor data. Semantic segmentation, hole repair, and boundary smoothing are performed using deep learning to ensure perfect integration of model accuracy and structure. Graph optimization and error minimization techniques further improve registration accuracy and ensure the transition of point cloud data. The generated 3D model supports VR / AR applications, enhances the immersive experience, and can be applied to path planning and dynamic updates in smart cities.

[0060] Working Principle: This invention utilizes efficient data acquisition, preprocessing, registration, optimization, and modeling techniques to achieve precise fusion of indoor and outdoor data and high-precision construction of 3D models. During the data acquisition phase, the method and system employ various devices, including vehicle-mounted LiDAR, UAV photography, and handheld LiDAR, to collect outdoor and indoor point cloud data. This point cloud data undergoes preprocessing steps, including denoising, thinning, registration, and quality inspection, to ensure data accuracy and reliability. Through feature extraction and preliminary alignment techniques, using the ICP algorithm or other matching algorithms, the indoor and outdoor point cloud data are initially registered, providing a foundation for subsequent fine registration and fusion.

[0061] In the fine registration and graph optimization stages, the method and system integrate the positional information of multiple scanning stations into a global framework by constructing a pose graph. Graph optimization algorithms, such as g2o and Ceres Solver, are used to minimize errors and further optimize the registration results. Especially in the transition area between indoor and outdoor point clouds, semantic segmentation and deep learning models are used to automatically identify and separate different building elements, such as walls, doors, and windows, thereby achieving semantic analysis and geometric modeling of the point cloud data. Fine geometric modeling is based on these semantic tags to generate BIM models or mesh models, so that the final 3D model has rich semantic information and can accurately reflect the spatial layout of the real scene.

[0062] Through refined fusion technology, the system ensures seamless integration of indoor and outdoor point cloud data in transitional areas, generating a high-precision 3D model of the real-world scene. This model not only supports virtual reality and augmented reality applications for immersive experiences but can also be used for path planning and dynamic updates in smart cities. The system can identify and update changes in buildings or indoor environments in real time, supporting incremental data fusion without the need to rebuild the entire 3D model. This efficient modeling and application integration enhances the practical application effects in multiple fields such as architectural design, urban planning, and management.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A real-scene 3D modeling method integrating indoor and outdoor point cloud data, characterized in that, Includes the following steps: Step S1: Collect indoor and outdoor point cloud data, and perform preprocessing, format standardization, and quality control on the collected point cloud data; Step S2: Extract salient features from point cloud data, perform coarse registration, and construct a spatial relationship vector model with constraints for preliminary alignment; Step S3: Perform fine registration of the point cloud data and optimize the graph to correct errors; Step S4: Combine semantic tags to perform refined geometric modeling and fit to form a fused real-world 3D model.

2. The real-scene 3D modeling method according to claim 1, characterized in that, Step S1 includes: S1.1 Collect indoor and outdoor point cloud data from multiple sources; S1.2 Perform preprocessing operations such as denoising, thinning, and registration on the collected indoor and outdoor point cloud data; S1.3 Unify the point cloud format of all indoor and outdoor point cloud data into a standard framework; S1.4 Identify missing regions and outliers in the preprocessed point cloud data and perform interpolation repair.

3. The real-scene 3D modeling method according to claim 1, characterized in that, Step S2 includes: S2.1 Extract salient features from point cloud data of indoor and outdoor public areas; S2.

2. Based on the extracted feature information, the indoor and outdoor point cloud data are initially aligned using the ICP algorithm; S2.3 Construct a mathematical model with constraints to optimize the initial alignment results; S2.4 Fine-tune the initially aligned point cloud data by adjusting the parameters in the transformation matrix to reduce the registration error between point clouds and obtain the initial registration result.

4. The real-scene 3D modeling method according to claim 1, characterized in that, Step S3 includes: S3.1 Establish a pose graph optimization problem model, integrating the position information of multiple scanning stations into a unified optimization framework; S3.2 Incorporate shared feature points from indoor and outdoor point cloud data into the optimization framework for visual overlay and fusion; S3.

3. Apply the graph optimization algorithm to minimize the error, obtain a globally consistent scanning station pose, and iterate to optimize the point cloud registration results; S3.4 Real-time monitoring of registration errors and convergence of abnormal errors in the target area.

5. The real-scene 3D modeling method according to claim 1, characterized in that, Step S4 includes: S4.

1. Use a deep learning model to perform semantic segmentation on the fused point cloud, identify different architectural elements and assign semantic labels; S4.

2. Based on the semantic segmentation results, perform geometric modeling, and combine semantic information to perform hierarchical optimization and reconstruction of the geometric structure of point clouds of different categories, so as to fit a more realistic scene. S4.3 Based on the optimized point cloud data and semantic information, each part of the semantic information classification model is used to fit the spatial pose and size ratio of the scene and the geometry in the scene based on semantic attributes, and then outputs the fused real-world 3D model with semantic labels.

6. A real-scene 3D modeling system integrating indoor and outdoor point cloud data, characterized in that, include: The data acquisition and preprocessing module is used to acquire indoor and outdoor point cloud data, and to preprocess, standardize, and control the quality of the acquired point cloud data. The coarse alignment module is used to extract significant features from point cloud data, perform coarse registration, and construct a spatial relationship vector model with constraints for initial alignment. The graph optimization module is used to perform fine registration of point cloud data and to perform graph optimization to correct errors. The semantic-guided reconstruction module is used to combine semantic tags to perform refined geometric modeling and fit a fused real-world 3D model.

7. The real-scene 3D modeling system according to claim 6, characterized in that, The data acquisition and preprocessing module includes: The outdoor point cloud acquisition module is used to collect outdoor point cloud data from multiple sources; The indoor point cloud acquisition module is used to collect indoor point cloud data from multiple sources; The data cleaning and preprocessing module is used to perform preprocessing operations such as denoising, thinning, and registration on the collected indoor and outdoor point cloud data. The point cloud registration module is used to unify the point cloud format of all indoor and outdoor point cloud data into a standard framework.

8. The real-scene 3D modeling system according to claim 6, characterized in that, The coarse alignment module includes: The feature extraction module is used to extract salient features from point cloud data of indoor and outdoor public areas; The coarse registration module is used to initially align indoor and outdoor point cloud data based on the extracted feature information and the ICP algorithm. The modeling module is used to construct a mathematical model containing constraints to optimize the initial alignment results; The fine-tuning module is used to fine-tune the initially aligned point cloud data. By adjusting the parameters in the transformation matrix, the registration error between point clouds is reduced to obtain the initial registration result.

9. The real-scene 3D modeling system according to claim 6, characterized in that, The graph optimization module includes: The graph optimization and integration module is used to establish a pose graph optimization problem model, integrating the position information of multiple scanning stations into a unified optimization framework; The overlay and fusion module is used to incorporate common feature points from indoor and outdoor point cloud data into an optimization framework for visual overlay and fusion. The algorithm optimization module is used to apply graph optimization algorithms to minimize errors, obtain globally consistent scanning station poses, and iterate to optimize the point cloud registration results. The monitoring module is used to monitor registration errors in real time and converge abnormal errors in the target area.

10. The real-scene 3D modeling system according to claim 6, characterized in that, The semantic-guided reconstruction module includes: The semantic segmentation module is used to perform semantic segmentation on the fused point cloud using a deep learning model, identify different architectural elements and assign semantic labels. The boundary repair module is used to perform geometric modeling based on semantic segmentation results. It combines semantic information to perform hierarchical optimization and reconstruction of the geometric structure of point clouds of different categories, and fits a more realistic scene. The refined fusion module is used to classify each part of the model based on the optimized point cloud data and semantic information, and fit the scene and the geometry in the scene according to the semantic attributes to fit the spatial posture and size ratio of the real scene, and then output the fused real scene 3D model with semantic labels.

Images