Urban building three-dimensional automatic modeling and visualization method

Abstract

The invention discloses an urban building three-dimensional automatic modeling and visualization method, and belongs to the technical field of building three-dimensional modeling. The method comprises the steps that point cloud data, high-resolution images and geographic information system data of urban buildings are acquired, data cleaning, registration and alignment are carried out, and preliminary building digital representation is formed; accurately segmenting each building, and identifying the contour and main structural features of the building; based on the data integrity and the building complexity, adaptively selecting a proper reconstruction strategy to carry out three-dimensional reconstruction; in the reconstruction process, the geometric structure is analyzed and optimized in real time, and potential topological problems are repaired; automatically generating missing details based on a predefined architectural style library and a component library, and performing material inference and texture mapping; a graph structure is used for representing the relation between the buildings, and the positions and orientations of the buildings are adjusted through a global optimization algorithm; a rendering engine supporting multi-level detail switching is developed, and smooth visualization and interaction of a large-scale city scene are achieved.

Description

Technical Field

[0001] This application relates to the field of 3D building modeling technology, and more specifically, to a method for automatic 3D modeling and visualization of urban buildings. Background Technology

[0002] In modern urban planning and management, accurate 3D modeling and visualization of urban buildings are crucial. This not only aids in urban design, planning, and maintenance but also plays a vital role in fields such as virtual reality, augmented reality, and historical heritage preservation. As cities continue to expand, the complexity and volume of data associated with buildings are rapidly increasing; therefore, traditional modeling methods are struggling to meet the demands for accuracy and efficiency.

[0003] Traditional 3D building modeling methods primarily rely on manual measurement and modeling, typically requiring significant manpower and time. Furthermore, these methods are susceptible to errors when dealing with complex buildings and large-scale urban data, resulting in insufficient accuracy and realism in the models. While existing technologies have incorporated techniques such as laser scanning and photogrammetry to improve modeling efficiency and accuracy, they still face numerous challenges in areas such as handling sparse data, automated segmentation, and feature extraction.

[0004] In recent years, with advancements in sensor technology, computer vision, and deep learning, automated 3D modeling and visualization technologies have gradually become a research hotspot. For example, laser scanners and high-resolution cameras can collect large amounts of 3D point cloud data and image information, but how to effectively integrate this data, accurately identify the outlines and structural features of buildings, and how to handle geometric and semantic consistency issues in large-scale urban environments remain pressing technical challenges.

[0005] Furthermore, 3D modeling of urban buildings involves the automatic identification of architectural styles and the generation of details, which requires a deep understanding and modeling of the building's history, cultural background, and environment. While current technology can perform basic modeling of buildings, its level of automation in handling architectural details and material inference is still low, often requiring significant manual intervention.

[0006] Therefore, developing an efficient and accurate method for automatic 3D modeling and visualization of urban buildings, capable of comprehensively integrating point cloud data, image information, and geographic information system data, and achieving automated feature recognition, 3D reconstruction, model optimization, and visualization, is an important direction in current research and application. This can not only significantly improve modeling efficiency and accuracy but also provide strong support for urban management, planning, and virtual reality applications. Summary of the Invention

[0007] To overcome a series of shortcomings in existing technologies, the purpose of this application is to provide a method for automatic 3D modeling and visualization of urban buildings, including the following steps: Step 1: Acquire point cloud data, high-resolution images, and geographic information system data of urban buildings; perform data cleaning, registration, and alignment to form a preliminary digital representation of the buildings. Step 2: Construct an intelligent recognition module that adapts to data sparsity to accurately segment each building and identify its outline and main structural features; Step 3: Based on data completeness and building complexity, adaptively select an appropriate reconstruction strategy for 3D reconstruction; Step 4: During the reconstruction process, analyze and optimize the geometry in real time, fix potential topological problems, and ensure the integrity of the model; Step 5: Based on the predefined architectural style library and component library, automatically generate missing details and perform material inference and texture mapping to enhance the realism of the model; Step 6: Use a graph structure to represent the relationships between buildings, and adjust the position and orientation of buildings through a global optimization algorithm to ensure the geometric and semantic consistency of the entire urban scene; Step 7: Develop a rendering engine that supports multi-level detail switching to achieve smooth visualization and interaction of large-scale urban scenes.

[0008] Furthermore, step 1 includes the following steps: The drone, equipped with a high-resolution camera, takes a grid-like picture of the target area along a preset path to obtain high-resolution orthophotos and oblique photographs. Vehicles equipped with high-precision laser scanners are used to scan major urban roads to obtain high-density 3D point cloud data. For areas that are difficult to cover by vehicle-mounted scanning and aerial photography, high-resolution cameras are used for supplementary shooting to obtain high-resolution close-up images; The 3D point cloud data is denoised, filtered, and semantically segmented to extract building point clouds. Orthophotos, oblique photographic images, and close-up images are used to color the building point clouds, generating an accurate color 3D point cloud model. The motion structure restoration and multi-view stereo matching techniques were applied to process orthophotos, oblique photographs and close-up images to generate dense point clouds and further construct a preliminary three-dimensional mesh model. Integrate existing geographic information system data as auxiliary information and verification basis for reconstruction; The color 3D point cloud model, 3D mesh model and geographic information system data are registered and fused through coordinate transformation and iterative nearest point algorithm; Preliminary digital representations of buildings are generated based on fused data, laying the foundation for subsequent intelligent recognition and modeling processes.

[0009] Furthermore, through coordinate transformation and iterative nearest-point algorithm, the color 3D point cloud model, 3D mesh model, and geographic information system data are registered and fused, including the following steps: The color 3D point cloud model, 3D mesh model and geographic information system data are registered and fused through coordinate transformation and iterative nearest point algorithm; Preprocessing is performed on the color 3D point cloud model, 3D mesh model, and geographic information system: noise and outliers in the color 3D point cloud model are cleaned up, the topology of the 3D mesh model is optimized, and the coordinate system and projection method of the geographic information system data are kept consistent. Convert the color 3D point cloud model, 3D mesh model, and geographic information system to the same coordinate system; A feature-based registration method is used to perform preliminary alignment of a color 3D point cloud model, a 3D mesh model, and a geographic information system. The ICP algorithm is applied to perform fine registration of point cloud data and 3D mesh model. Specifically, the mapping relationship between corresponding points in point cloud data and 3D mesh model is continuously found and optimized through iterative methods to minimize the distance error between them. The registered point cloud data and 3D mesh model are integrated with the geographic information system data, including: associating objects in the point cloud and mesh model with geographic features in the geographic information system data, adding attribute information, and ensuring the consistency of spatial relationships. The accuracy of the fusion results is evaluated, and the registration error and data consistency index are calculated.

[0010] Furthermore, the intelligent recognition module includes: The multi-scale feature extraction unit uses a convolutional neural network to extract multi-scale features from point cloud and image data, enhancing the ability to represent sparse data. A cross-modal attention fusion mechanism is used to achieve adaptive fusion of point cloud and image features; The semantic segmentation unit, based on the U-Net architecture, achieves pixel-level segmentation of buildings, laying the foundation for subsequent contour extraction; The contour extraction and optimization unit combines traditional image processing techniques and deep learning methods to accurately extract building contours and optimize boundaries. The structural feature recognition unit uses a graph neural network to identify the main structural features of a building. The sparse data augmentation unit uses generative adversarial networks to complete and augment data in sparse regions; The consistency constraint unit uses physical rules and geometric constraints to ensure the rationality and consistency of the identification results.

[0011] Furthermore, step 3 includes the following steps: Analyze the completeness and complexity of building data, and dynamically select appropriate reconstruction strategies based on the analysis results; For regular buildings, a parametric model library is used, and the optimal parameter values ​​are determined by fitting point cloud data and contour information through optimization algorithms. At the same time, the identified structural features are added to generate a complete parametric model. For complex buildings, a voxel-based progressive reconstruction method is adopted, which involves: converting point cloud data into voxel representations and using a 3D convolutional neural network for preliminary classification to further construct a rough 3D model of the building; using an adaptive subdivision strategy to gradually improve the level of detail of the rough 3D model; and integrating the identified structural features into the 3D model to ensure geometric consistency and realism. The reconstructed model is optimized and simplified to reduce geometric complexity and ensure computational efficiency and visualization performance.

[0012] Furthermore, step 4 includes the following steps: Continuously monitor the geometric characteristics of the reconstructed model to quickly identify potential geometric anomalies or discontinuities; Perform topology checks to identify topology problems in the model and automatically repair them using mesh repair techniques; Based on predefined architectural rules and constraints, the structural integrity of the reconstructed model is evaluated; at the same time, heuristic algorithms are used to automatically complete missing structural elements to ensure the physical rationality of the model. Adaptive mesh refinement and smoothing algorithms are applied to improve surface quality while preserving important geometric features.

[0013] Furthermore, step 5 includes the following steps: Analyze the building's geometry to identify stylistic features and missing details; Based on the identified style features, appropriate detail elements are selected from the component library and seamlessly integrated into the reconstruction model; By combining architectural features and environmental information, the material of each part is inferred, the corresponding texture is extracted, and it is accurately mapped onto the surface of the reconstructed model; Implement global shape optimization to balance local details with the overall shape and ensure that the reconstructed model reflects the overall characteristics and stylistic consistency of the building; Based on material properties and environmental information, rendering parameters are automatically optimized to improve the realism of individual building models.

[0014] Furthermore, step 6 includes the following steps: Step 6.1: Use graph structures to represent the spatial and semantic relationships between buildings to capture the overall characteristics of the urban structure; Step 6.2: Adjust the position and orientation of buildings using a global optimization algorithm to ensure geometric consistency across the entire urban scene; Step 6.3: Ensure that the geometric structure of each building in the urban scene is consistent with the recognized semantic information, and make adjustments as necessary to reflect the correct urban semantics; Step 6.4: Integrate environmental elements to enhance the integrity and realism of the urban scene; Step 6.5: Based on scene complexity and visualization requirements, optimize the rendering parameters of the entire city scene to prepare for large-scale visualization.

[0015] Compared with the prior art, this application has the following beneficial effects: This application constructs an intelligent recognition module using high-precision point cloud data, high-resolution images, and geographic information system data to perform accurate segmentation and feature recognition. Combined with dynamically selected reconstruction strategies and real-time optimization technology, it achieves high-quality 3D model generation. In addition, it adjusts the position and orientation of buildings through a global optimization algorithm to ensure the geometric and semantic consistency of the city model. At the same time, it supports a rendering engine with multi-level detail switching to achieve smooth visualization of large-scale urban scenes. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a method for automatic 3D modeling and visualization of urban buildings disclosed in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be described in more detail below with reference to the accompanying drawings. In the drawings, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The described embodiments are some embodiments of this invention, but not all embodiments.

[0018] Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0019] The embodiments and directional terms described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0020] like Figure 1 As shown, a method for automatic 3D modeling and visualization of urban buildings includes the following steps: Step 1: Acquire point cloud data, high-resolution images, and geographic information system data of urban buildings; perform data cleaning, registration, and alignment to form a preliminary digital representation of the buildings. Step 2: Construct an intelligent recognition module that adapts to data sparsity to accurately segment each building and identify its outline and main structural features; Step 3: Based on data completeness and building complexity, adaptively select an appropriate reconstruction strategy for 3D reconstruction; Step 4: During the reconstruction process, analyze and optimize the geometry in real time, fix potential topological problems, and ensure the integrity of the model; Step 5: Based on the predefined architectural style library and component library, automatically generate missing details and perform material inference and texture mapping to enhance the realism of the model; Step 6: Use a graph structure to represent the relationships between buildings, and adjust the position and orientation of buildings through a global optimization algorithm to ensure the geometric and semantic consistency of the entire urban scene; Step 7: Develop a rendering engine that supports multi-level detail switching to achieve smooth visualization and interaction of large-scale urban scenes.

[0021] Step 1 is the foundation of the entire modeling process, constructing an initial building model by integrating multiple data sources. Point cloud data, typically generated by LiDAR or multi-view stereo vision systems, provides the building's 3D coordinate information. High-resolution images capture the building's detailed textures, providing material for subsequent texture mapping. Geographic Information System (GIS) data contains the building's geographic location, outline, and other relevant information, helping to ensure data accuracy and geographic consistency of the model. Data cleaning and registration steps primarily eliminate noisy data, correct errors between multi-source data, and ensure they are accurately aligned in the same coordinate system. Step 1 not only improves data quality but also provides a reliable foundation for subsequent model construction.

[0022] Step 2 is crucial for semantic segmentation and feature extraction of the building using machine learning or deep learning algorithms. The success of the intelligent recognition module directly impacts the accuracy of subsequent 3D reconstruction. If the building's outline and structural features can be accurately identified in this step, the building's shape and structure can be reproduced more precisely during 3D modeling.

[0023] Step 3 is the process of adaptively selecting a 3D reconstruction strategy. Different buildings have varying levels of data completeness and complexity; therefore, a "one-size-fits-all" approach cannot be used during reconstruction. Instead, the most suitable reconstruction strategy must be selected based on the specific circumstances. For example, for buildings with relatively complete data and simple structures, traditional 3D reconstruction algorithms can be used; while for buildings with sparse data or complex structures, rule-based models or deep learning generative models are needed. The core of adaptive strategy selection lies in balancing reconstruction quality and computational complexity, ensuring that high-quality 3D modeling is completed as efficiently as possible with limited resources.

[0024] 3D reconstruction is not simply a matter of converting point clouds into 3D models; it also requires real-time analysis and optimization of the geometry to correct potential topological issues. Topological problems, such as self-intersections, holes, and non-manifold edges, can severely impact model integrity and rendering quality. Therefore, these issues must be detected and corrected in real time during reconstruction. Geometric optimization may involve surface smoothing, edge repair, and other operations to ensure the geometric accuracy and visual quality of the final model. The technical challenge of step 4 lies in the need to process and optimize large amounts of data in real time while maintaining model consistency and integrity.

[0025] In step 5, to enhance the realism of the model, detail generation, material inference, and texture mapping are required. Based on a predefined library of architectural styles and components, automatically generating missing details can significantly reduce manual intervention and improve modeling efficiency. Furthermore, by inferring building materials and performing texture mapping, the model can more closely resemble the appearance of real buildings. Texture mapping not only requires accurately matching the building's exterior image but also needs to consider the effects of lighting and viewing angle, ensuring the model maintains a realistic visual effect under different viewing conditions.

[0026] In step 6, a graph structure is used to represent the relationships between buildings, which facilitates the description of their spatial locations and interrelationships. Next, a global optimization algorithm adjusts the positions and orientations of the buildings to ensure that the positions and orientations of all buildings in the entire city scene conform to reality, thereby improving the overall harmony and realism of the scene. This process involves not only geometric adjustments but also semantic consistency, such as factors like roads and greenery between buildings.

[0027] In step 7, to achieve smooth visualization and interaction of large-scale urban scenes, developing a rendering engine that supports multi-level detail switching is essential. Large-scale urban scenes typically contain numerous buildings and complex environmental details; directly rendering all details would result in a huge computational load and latency. Therefore, multi-level detail (LOD) technology is needed to dynamically adjust the model's level of detail based on the user's viewing distance and interaction needs, thereby improving rendering efficiency and smoothness while ensuring visual quality. Such a rendering engine can not only handle large-scale data but also support real-time interaction, providing users with an immersive experience.

[0028] Furthermore, step 1 includes the following steps: Step 1.1: Use a drone equipped with a high-resolution camera to take grid-like pictures of the target area along a preset path to obtain high-resolution orthophotos and oblique photographs. Step 1.2: Use a vehicle equipped with a high-precision laser scanner to scan the main roads of the city to obtain high-density three-dimensional point cloud data; Step 1.3: For areas that are difficult to cover by vehicle-mounted scanning and aerial photography, a high-resolution camera is used to take supplementary photos to obtain high-resolution close-up images; Step 1.4: Denoise, filter and semantic segment the 3D point cloud data, extract building point clouds, and use orthophotos, oblique photographic images and close-up images to color the building point clouds to generate an accurate color 3D point cloud model. The motion structure restoration and multi-view stereo matching techniques were applied to process orthophotos, oblique photographs and close-up images to generate dense point clouds and further construct a preliminary three-dimensional mesh model. Step 1.5: Integrate existing geographic information system data as auxiliary information and verification basis for reconstruction; Step 1.6: Through coordinate transformation and iterative nearest point algorithm, the color 3D point cloud model, 3D mesh model and geographic information system data are registered and fused; Step 1.7: Generate a preliminary digital representation of the building based on the fused data, laying the foundation for the subsequent intelligent recognition and modeling process.

[0029] In step 1.1, using drones for grid-based photography can cover large urban areas and acquire detailed information on building rooftops and facades. This method is highly flexible and efficient, adaptable to complex urban environments. Orthophotos provide distortion-free planar images, suitable for extracting building planar contours, while oblique photogrammetry images provide rich information on building facade details; these data form the basis for subsequent 3D reconstruction.

[0030] In step 1.2, vehicle-mounted laser scanning can quickly and accurately capture 3D information of buildings and other urban facilities along city roads, generating high-density point cloud data containing the precise geometry of buildings. This point cloud data plays a crucial role in subsequent building recognition and 3D modeling, especially when dealing with complex urban street scenes, where vehicle-mounted scanning can provide more detailed local 3D data than aerial photography. Furthermore, vehicle scanning can cover low-altitude and ground-level perspectives, supplementing the limitations of high-altitude drone photography.

[0031] In step 1.3, due to the complexity of the urban environment, some areas may not be able to obtain complete data through drone or vehicle-mounted scanning due to terrain obstacles or dense buildings. In this case, close-up ground-based photography can fill these gaps, providing more detailed information about building facades. High-resolution close-up images can capture the texture, material, and other fine structural details of buildings, providing necessary detailed data for subsequent 3D reconstruction, thereby improving the accuracy and realism of the model.

[0032] The key to step 1.4 lies in improving the quality of point cloud data through data processing techniques and incorporating color information from multiple data sources. Noise reduction and filtering techniques remove noisy points and redundant data from the point cloud, while semantic segmentation is used to accurately identify and extract point cloud data of buildings. By mapping images at different angles and resolutions onto the point cloud data, the generated colored point cloud model not only contains the geometric information of buildings but also possesses the visual characteristics of surface materials, laying a solid foundation for subsequent 3D modeling.

[0033] In step 1.5, Structure-of-Motion (SfM) reconstructs the 3D structure of the scene by analyzing the relative motion between multiple 2D images; while multi-view stereo matching further accurately calculates the depth information in the images, generating a dense 3D point cloud. These techniques can effectively integrate image data from different perspectives to construct a more detailed and accurate 3D mesh model, providing a solid foundation for the 3D reconstruction of complex scenes.

[0034] In step 1.6, GIS data typically includes attribute information such as the building's geographical location, purpose, and height. This data can provide additional reference and verification for the 3D modeling process. For example, during model generation, GIS data can help calibrate the building's geographical location and height, ensuring the model's accuracy and consistency. Furthermore, GIS data can be used to verify the correctness of the reconstruction results; by comparing them with the model, potential errors can be identified and corrected.

[0035] In step 1.7, coordinate transformation unifies the coordinate systems of different data sources into a global coordinate system, ensuring that all data are processed under the same spatial reference. The ICP algorithm then uses iterative calculations to precisely align different datasets in three-dimensional space, achieving model fusion. Through this process, all data sources are unified and integrated into a complete three-dimensional model, generating a preliminary digital representation of the building. This preliminary model not only includes the building's geometric shape and color information but also incorporates geographical location and other auxiliary information, providing comprehensive foundational data for subsequent intelligent recognition and further modeling processes.

[0036] Furthermore, through coordinate transformation and iterative nearest-point algorithm, the color 3D point cloud model, 3D mesh model, and geographic information system data are registered and fused, including the following steps: By employing coordinate transformation and the Iterative Closest Point (ICP) algorithm, the colored 3D point cloud model, 3D mesh model, and Geographic Information System (GIS) data are registered and fused. This step aims to align the multi-source datasets into a unified coordinate system, ensuring seamless integration of various data sources during subsequent processing. The ICP algorithm repeatedly adjusts the positions and angles of the point cloud and mesh models to achieve a high degree of consistency with the geographic features in the GIS data. This precise registration significantly improves the accuracy of data fusion, providing a reliable data foundation for subsequent building modeling. Preprocessing is performed on the color 3D point cloud model, 3D mesh model, and geographic information system (GIS): Noise and outliers in the color 3D point cloud model are removed, the topology of the 3D mesh model is optimized, and the coordinate system and projection method of the GIS data are ensured to be consistent. Preprocessing improves the accuracy of the point cloud data and reduces errors that may be introduced in subsequent processing. The topology of the 3D mesh model is optimized, including repairing cracks, deleting redundant faces, and smoothing the mesh surface, ensuring the geometric integrity and smoothness of the model. Ensuring the consistency of the coordinate system and projection method of the GIS data avoids misalignment or distortion during data fusion. Through this series of preprocessing steps, the quality of various types of data can be significantly improved, laying a solid foundation for registration and fusion. The key technical aspect of transforming color 3D point cloud models, 3D mesh models, and geographic information systems (GIS) into the same coordinate system lies in precise coordinate transformation, which ensures the spatial consistency of all datasets. Whether it's point cloud data, mesh models, or GIS data, they may exhibit displacement, rotation, or scale differences in different coordinate systems. Therefore, a unified transformation is essential to place them within the same reference frame. Only under the same coordinate system can various data be effectively fused and aligned, thereby guaranteeing the spatial accuracy and consistency of the model. A feature-based registration method is used to initially align a color 3D point cloud model, a 3D mesh model, and a geographic information system. This method utilizes key building features, such as corners, edges, and structural lines, to establish preliminary correspondences between different datasets. This approach can quickly and coarsely align datasets without relying on global information, laying the foundation for subsequent fine-grained registration. Although the initial alignment has not yet achieved the final high-precision registration effect, it has significantly reduced the errors between data, providing better initial conditions for the ICP algorithm in subsequent steps. The ICP algorithm is applied to perform fine registration of point cloud data and 3D mesh models. Specifically, it iteratively searches for and optimizes the mapping relationship between corresponding points in the point cloud data and the 3D mesh model to minimize the distance error between them. Specifically, the ICP algorithm first searches for possible matching point pairs between the point cloud data and the mesh model based on the preliminary alignment results. Then, it calculates the transformation matrix using the least squares method to adjust the position and orientation of the model, so that the error is gradually reduced. After multiple iterations, the final registration result can achieve sub-pixel accuracy, ensuring perfect alignment between the point cloud and the mesh model. This step significantly improves the spatial accuracy of the data and provides high-quality input for subsequent data fusion and attribute addition.

[0037] The process of fusing registered point cloud data and 3D mesh models with geographic information system (GIS) data includes: associating objects in the point cloud and mesh models with geographic features in the GIS data, adding attribute information, and ensuring spatial consistency. This step involves associating objects in the point cloud and mesh models with geographic features in the GIS data and adding attribute information. In this way, the point cloud and mesh models not only acquire geometric shape and color information but also rich semantic information, such as the purpose, name, and height of buildings. Ensuring spatial consistency is crucial in this process; that is, the spatial location and topological relationships of all geographic features must match the actual geographic situation. This fusion of multi-source data greatly enhances the practicality and information integrity of the model, providing a reliable foundation for applications such as urban planning and building management. The accuracy assessment of the fusion results, including the calculation of registration error and data consistency indices, aims to verify the quality of the data fusion and ensure high accuracy and consistency across all datasets. Calculating the registration error quantifies the geometric deviations between point clouds, grid models, and GIS data, determining the accuracy of the registration. Data consistency indices evaluate the spatial and attribute consistency of the fused data, ensuring that the model accurately reflects the real world in both geometry and semantic information. Accuracy assessment allows for the timely identification and correction of potential problems during the fusion process, ensuring the high reliability and usability of the final digital representation of buildings.

[0038] Furthermore, a feature-based registration method is used to perform preliminary alignment of the color 3D point cloud model, the 3D mesh model, and the geographic information system, including the following steps: Feature points are extracted from color 3D point cloud models, 3D mesh models, and geographic information systems. Match feature points in different models by calculating the similarity between features; The initial transformation matrix is ​​estimated using the random sample consensus algorithm. ; Transformation matrix It is applied to color 3D point cloud models, 3D mesh models, and geographic information systems for initial alignment; Assess the initial alignment error to ensure the accuracy of the initial alignment. The specific calculation formula is as follows: ,in, This is the alignment error, representing the distance error between feature points before and after registration; n is the total number of feature points. It is the first match 1 feature point.

[0039] Furthermore, the intelligent recognition module includes: The multi-scale feature extraction unit uses a convolutional neural network to extract multi-scale features from point cloud and image data, enhancing the ability to represent sparse data. A cross-modal attention fusion mechanism is used to achieve adaptive fusion of point cloud and image features; The semantic segmentation unit, based on the U-Net architecture, achieves pixel-level segmentation of buildings, laying the foundation for subsequent contour extraction; The contour extraction and optimization unit combines traditional image processing techniques and deep learning methods to accurately extract building contours and optimize boundaries. The structural feature recognition unit uses a graph neural network to identify the main structural features of a building. The sparse data augmentation unit uses generative adversarial networks to complete and augment data in sparse regions; The consistency constraint unit uses physical rules and geometric constraints to ensure the rationality and consistency of the identification results.

[0040] Furthermore, step 3 includes the following steps: The process involves analyzing the completeness and complexity of building data and dynamically selecting an appropriate reconstruction strategy based on the analysis results. This step aims to intelligently analyze data quality and the structural complexity of buildings to choose the most suitable reconstruction method, ensuring both accurate structural reconstruction and efficient use of computational resources. Completeness analysis determines the density and coverage of point cloud data, while complexity analysis assesses the building's geometry and detail richness. Based on these analysis results, the reconstruction strategy can be dynamically adjusted; for example, a fast parametric reconstruction method can be chosen for simple structures, while a more refined progressive reconstruction method can be selected for complex structures. This adaptive strategy selection mechanism significantly improves the efficiency and effectiveness of the reconstruction process, ensuring high-quality and highly realistic models even with limited resources.

[0041] For regular buildings, a parametric model library is used, and an optimization algorithm is employed to fit point cloud data and contour information to determine the optimal parameter values. Simultaneously, identified structural features are added to generate a complete parametric model. The purpose of this step is to efficiently reconstruct geometrically regular buildings, such as rectangular high-rises or simple industrial structures. The parametric model library provides a series of predefined geometric models, which can typically be fully described by a few parameters. By fitting point cloud data and contour information to these parametric models using an optimization algorithm, the precise shape of the building can be quickly determined. The introduction of the optimization algorithm allows the fitting process to automatically adjust parameters and find the model that best matches the actual data. At the same time, identified structural features, such as windows, doors, and floors, can also be automatically added to the model, generating a more complete and detailed 3D building model. This method not only improves the speed of modeling but also ensures the accuracy and structural integrity of the model.

[0042] For complex buildings, a voxel-based progressive reconstruction method is adopted. Specifically, point cloud data is converted into voxel representations, and a 3D convolutional neural network is used for initial classification to further construct a rough 3D model of the building. An adaptive subdivision strategy is employed to progressively improve the level of detail in the rough 3D model. Identified structural features are integrated into the 3D model to ensure geometric consistency and realism. Voxel representations divide 3D space into a series of small cubes, allowing point cloud data of complex buildings to be processed in a structured manner. The 3D convolutional neural network, by classifying the voxel data, can identify the main components of the building and construct a rough 3D model. Based on this, the adaptive subdivision strategy further improves the level of detail in the model, gradually adding detail while maintaining the overall shape. Finally, previously identified structural features, such as curved windows and complex roofs, are integrated into the model to ensure that the model is not only geometrically accurate but also highly realistic. This progressive method can effectively handle buildings with large amounts of data and complex structures, ultimately generating a highly detailed and realistic 3D model.

[0043] The reconstructed model is optimized and simplified to reduce geometric complexity, ensuring computational efficiency and visualization performance. This step aims to make the generated 3D model more lightweight without significantly sacrificing visual appeal by reducing the number of polygons and simplifying the geometry. The optimization process typically includes techniques such as mesh simplification, redundant facet removal, and edge smoothing, designed to reduce the computational burden on the model and thus improve real-time rendering performance in various application scenarios. Especially in large-scale city models, this optimization can significantly reduce memory and processor usage, allowing complex urban scenes to run smoothly on ordinary hardware. Furthermore, the optimized model is more efficient in data transmission and storage, providing a better user experience for visualization and interaction in practical applications. Through this optimization and simplification process, the generated 3D model achieves an optimal balance between computational efficiency and visualization performance while retaining key details and accuracy.

[0044] Furthermore, step 4 includes the following steps: Continuously monitor the geometric properties of the reconstructed model to quickly identify potential geometric anomalies or discontinuities. This step aims to promptly detect problems that may affect model quality, such as surface depressions, protrusions, discontinuous surfaces, or irregular shapes, by monitoring the model's geometry in real time. This monitoring is typically based on automated detection algorithms that analyze the geometric properties of the model's vertices, edges, and faces to identify anomalous regions. Rapidly identifying geometric problems can prevent the accumulation of major errors in the early stages of model building, thereby reducing the difficulty and cost of subsequent repairs. By promptly identifying and addressing these geometric anomalies, the final generated model ensures higher geometric consistency and accuracy, avoiding uncontrollable errors in further processing.

[0045] A topology check is performed to identify topological issues in the model and then automatically repairs them using mesh repair techniques. The purpose of this step is to identify potential topological errors in the model through detailed topology analysis, such as cracks, overlapping surfaces, inconsistent normals, and non-manifold edges. Topological problems directly affect the connectivity and integrity of the model, leading to errors during rendering or physical simulation. Mesh repair techniques include automated topology repair algorithms, such as hole filling, patch merging, and normal adjustment, to ensure the model's topological structure is correct. Automated repair effectively improves the overall stability and reliability of the model, ensuring smooth subsequent geometric processing and physical simulations, and reducing the workload of manual repairs.

[0046] Based on predefined architectural rules and constraints, the structural integrity of the reconstructed model is evaluated. Simultaneously, heuristic algorithms are used to automatically complete missing structural elements, ensuring the model's physical rationality. This step aims to assess the model's rationality and automatically complete it by introducing architectural knowledge and structural engineering constraints. During the modeling process, some architectural details may not be fully reconstructed due to missing data or acquisition errors. Heuristic algorithms can automatically infer and fill in these missing parts based on architectural rules, such as repairing missing walls or completing window or door details. This structural evaluation and completion mechanism ensures that the model is not only geometrically accurate but also physically rational, which is particularly important for simulation, analysis, and functional verification of buildings, improving the model's practicality and accuracy.

[0047] Adaptive mesh refinement and smoothing algorithms are applied to improve surface quality while preserving important geometric features. This step aims to eliminate rough surfaces and irregular textures that may have been introduced during the initial modeling process by refining and smoothing the model's surface. Adaptive mesh refinement selectively increases the mesh resolution based on the model's local geometric complexity, resulting in more accurate representation of detailed areas while maintaining a lower mesh density in flatter or more regular areas to conserve computational resources. Simultaneously, the smoothing algorithm removes surface noise and improves visual appeal, but it takes special care to preserve important geometric features, such as sharp edges or corners, to prevent them from being over-smoothed and distorted. Through this approach, the final model not only boasts high-quality surface representation but also retains important structural details, ensuring excellent performance in visualization and analysis applications.

[0048] Furthermore, step 5 includes the following steps: Analyzing the building's geometry identifies stylistic features and missing details. This step aims to identify unique architectural stylistic features, such as Gothic pointed arches or Baroque ornamentation, and any details lost due to incomplete data or modeling processes, through in-depth analysis of the building's geometry. Identifying these stylistic features ensures that the model is not only geometrically accurate but also reflects the building's historical and cultural context. This analysis provides a foundation for subsequent detail completion and stylistic consistency processing, while also making the model more visually realistic and consistent with the building's original design intent and artistic style.

[0049] Based on identified stylistic features, appropriate detail elements are selected from a component library and seamlessly integrated into the reconstructed model. The purpose of this step is to quickly add detailed elements, such as doors, windows, decorative sculptures, and columns, to the model using a predefined component library. These libraries typically contain typical elements from various architectural styles, ensuring that the selected details are consistent with the overall style of the building. Seamlessly integrating these elements is a key challenge; it is essential to ensure that the newly added details perfectly match the existing model geometrically and texturally, avoiding unnatural seams or discontinuities. Through this detail enhancement, the model becomes not only structurally complete but also visually richer and more realistic, better reflecting the complexity and aesthetics of the actual architecture.

[0050] By combining architectural features and environmental information, the material of each part is inferred, and the corresponding texture is extracted and accurately mapped onto the surface of the reconstructed model. The purpose of this step is to intelligently infer the material type of each part of the building, such as stone, glass, metal, or wood, through analysis of architectural features and environmental information. Material inference depends not only on the building's appearance but also on the influence of its surrounding environment, such as lighting conditions and the style of surrounding buildings. Extracting the corresponding high-resolution texture and mapping it onto the model surface ensures that the model's surface texture and color are consistent with the real building. Precise texture mapping technology avoids problems such as texture stretching and distortion, ensuring that every detail is correctly presented. Through this material and texture processing, the realism of the model is greatly enhanced, providing a lifelike visual experience in visualization applications.

[0051] Global shape optimization is implemented to balance local details with the overall shape, ensuring that the reconstructed model reflects the building's overall characteristics and stylistic consistency. This step aims to resolve the conflict between local details and the overall shape through global optimization. While detail enhancement can increase the model's accuracy and richness, without global shape optimization, the model may exhibit inconsistencies or stylistic discrepancies. Global shape optimization automatically adjusts the model's proportions, contours, and curves using algorithms, ensuring that details harmonize with the overall design style and that the model's shape and structure accurately reflect the building's original design intent. Through this optimization, the model is not only rich in detail but also more harmonious and consistent overall, perfectly reproducing the building's stylistic features.

[0052] Based on material properties and environmental information, rendering parameters are automatically optimized to enhance the realism of individual building models. This step aims to automatically adjust rendering parameters by comprehensively considering material properties (such as reflectivity, transparency, and roughness) and environmental information (such as lighting conditions and shadows), resulting in a more lifelike representation of the building model in the virtual environment. The optimized rendering parameters ensure realistic material reflections, lighting effects, and performance under different lighting conditions, significantly improving the model's visual realism. Through this automated rendering optimization, the model can be displayed at its best on various visualization platforms, providing a high level of visual impact and realism for architectural design presentations, urban planning, and the digital preservation of historical buildings.

[0053] Furthermore, step 6 includes the following steps: Step 6.1 uses a graph structure to represent the spatial and semantic relationships between buildings, capturing the overall characteristics of the urban structure. The purpose of this step is to represent the buildings and their relationships in the city as nodes and edges using this abstract method. Each building can be considered a node, and the edges between nodes represent spatial or semantic connections, such as street connections, functional similarities, or historical links. This graph structure not only effectively captures the layout of urban space but also reveals the complex relationships between buildings, providing fundamental data support for subsequent geometric and semantic adjustments. This method makes urban modeling more systematic and hierarchical, providing a clear framework for macro-structural analysis and micro-detail processing of the entire city.

[0054] Step 6.2 involves adjusting the position and orientation of buildings using a global optimization algorithm to ensure geometric consistency across the entire urban scene. The purpose of this step is to use an optimization algorithm to globally adjust the position and orientation of buildings in the city, eliminating geometric inconsistencies caused by data acquisition errors or independent modeling. The optimization algorithm can automatically detect and correct issues such as positional offsets and rotation errors between buildings, ensuring a reasonable and harmonious spatial layout of buildings within the urban environment. Through this global adjustment, the geometric structure of the urban scene becomes more consistent, and the arrangement and orientation of buildings better conform to actual urban planning and design, improving the overall quality and realism of the scene.

[0055] Step 6.3 ensures that the geometric structure of each building in the urban scene is consistent with the identified semantic information, making adjustments as necessary to reflect the correct urban semantics. The purpose of this step is to ensure the model's semantic accuracy by matching and verifying the geometric form of buildings with their corresponding semantic information. For example, if a building is semantically labeled as a "historical building," its geometric form and details should match this semantic information to avoid the inaccurate inclusion of modern elements. If a discrepancy is found between the geometric structure and the semantic information, adjustments will be made automatically, such as modifying architectural details, materials, or structures to conform to the expected semantic description. In this way, the urban scene is not only geometrically accurate but also semantically accurate, ensuring that the model can correctly convey the various functions and historical and cultural information of the city.

[0056] Step 6.4, Integrating Environmental Elements to Enhance the Completeness and Realism of the Urban Scene; the purpose of this step is to make the model more complete and realistic by integrating natural and man-made environmental elements such as greenery, roads, and rivers into the urban scene. These environmental elements not only provide background and context for buildings but also greatly enhance the visual expressiveness and realism of the scene. For example, trees lining the streets, railings along the river, and the city center square can all add vitality and energy to the scene. By integrating these environmental elements, the urban scene is not only improved at the building level but also significantly enhanced in terms of the overall urban atmosphere and visual effects, forming a more realistic virtual city.

[0057] Step 6.5 optimizes the rendering parameters of the entire city scene based on scene complexity and visualization requirements, preparing for large-scale visualization. The purpose of this step is to optimize and adjust the rendering parameters of various elements in the scene, ensuring the entire city model runs efficiently on various display devices and rendering engines. As the complexity of the city scene increases, the computational burden of rendering also increases. Therefore, it is necessary to optimize the rendering parameters according to actual needs, such as simplifying unnecessary details, adjusting lighting and shadow effects, and optimizing texture resolution, to ensure smooth rendering of the scene on large display systems or in real-time interactions. Through these optimizations, the final city scene can not only be presented efficiently but also automatically adjust rendering quality and performance according to different application scenarios, meeting various needs from daily visualization to professional simulation analysis.

[0058] Furthermore, in step 6.2, the objective of the global optimization algorithm is to minimize the geometric inconsistencies between buildings, expressed by the formula: ,in, The objective function is used to measure the geometric inconsistencies between buildings. It is a building The set of transformation parameters, including the rotation matrix. Translation vector ; It is a building The point in the middle; It is the weight, representing the building and The strength of the correlation between them; M and N are the number of buildings, respectively; In step 6.2, the position constraint is: ,in, This is the adjusted point position; It is the allowable position error threshold; In step 6.2, the orientation constraint is: ,in, It is a building The main direction vector; It is the reference direction vector; It is the allowed orientation error threshold.

[0059] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for automatic 3D modeling and visualization of urban buildings, characterized in that, Includes the following steps: Step 1: Acquire point cloud data, high-resolution images, and geographic information system data of urban buildings, perform data cleaning, registration, and alignment to form a preliminary digital representation of the buildings; Step 2: Construct an intelligent recognition module that adapts to data sparsity to accurately segment each building and identify its outline and main structural features; Step 3: Based on data completeness and building complexity, adaptively select an appropriate reconstruction strategy for 3D reconstruction; Step 4: During the reconstruction process, analyze and optimize the geometry in real time, fix potential topological problems, and ensure the integrity of the model; Step 5: Based on the predefined architectural style library and component library, automatically generate missing details and perform material inference and texture mapping to enhance the realism of the model; Step 6: Use a graph structure to represent the relationships between buildings, and adjust the position and orientation of buildings through a global optimization algorithm to ensure the geometric and semantic consistency of the entire urban scene; Step 7: Develop a rendering engine that supports multi-level detail switching to achieve smooth visualization and interaction of large-scale urban scenes.

2. The method for automatic 3D modeling and visualization of urban buildings according to claim 1, characterized in that, Step 1 includes the following steps: The drone, equipped with a high-resolution camera, takes a grid-like picture of the target area along a preset path to obtain high-resolution orthophotos and oblique photographs. Vehicles equipped with high-precision laser scanners are used to scan major urban roads to obtain high-density 3D point cloud data. For areas that are difficult to cover by vehicle-mounted scanning and aerial photography, high-resolution cameras are used for supplementary shooting to obtain high-resolution close-up images; The 3D point cloud data is denoised, filtered, and semantically segmented to extract building point clouds. Orthophotos, oblique photographic images, and close-up images are used to color the building point clouds, generating an accurate color 3D point cloud model. The motion structure restoration and multi-view stereo matching techniques were applied to process orthophotos, oblique photograms and close-up images to generate dense point clouds and further construct a preliminary three-dimensional mesh model. Integrate existing geographic information system data as auxiliary information and verification basis for reconstruction; The color 3D point cloud model, 3D mesh model and geographic information system data are registered and fused through coordinate transformation and iterative nearest point algorithm; The fused data generates a preliminary digital representation of the building, laying the foundation for subsequent intelligent recognition and modeling processes.

3. The method for automatic 3D modeling and visualization of urban buildings according to claim 2, characterized in that, The color 3D point cloud model, 3D mesh model, and geographic information system data are registered and fused using coordinate transformation and iterative nearest-point algorithm, including the following steps: The color 3D point cloud model, 3D mesh model and geographic information system data are registered and fused through coordinate transformation and iterative nearest point algorithm; Preprocessing is performed on the color 3D point cloud model, 3D mesh model, and geographic information system: noise and outliers in the color 3D point cloud model are cleaned up, the topology of the 3D mesh model is optimized, and the coordinate system and projection method of the geographic information system data are kept consistent. Convert the color 3D point cloud model, 3D mesh model, and geographic information system to the same coordinate system; A feature-based registration method is used to perform preliminary alignment of a color 3D point cloud model, a 3D mesh model, and a geographic information system. The ICP algorithm is applied to perform fine registration of point cloud data and 3D mesh model. Specifically, the mapping relationship between corresponding points in point cloud data and 3D mesh model is continuously found and optimized through iterative methods to minimize the distance error between them. The registered point cloud data and 3D mesh model are integrated with the geographic information system data, including: associating objects in the point cloud and mesh model with geographic features in the geographic information system data, adding attribute information, and ensuring the consistency of spatial relationships. The accuracy of the fusion results is evaluated, and the registration error and data consistency index are calculated.

4. The method for automatic 3D modeling and visualization of urban buildings according to claim 1, characterized in that, The intelligent recognition module includes: The multi-scale feature extraction unit uses a convolutional neural network to extract multi-scale features from point cloud and image data, enhancing the ability to represent sparse data. A cross-modal attention fusion mechanism is used to achieve adaptive fusion of point cloud and image features; The semantic segmentation unit, based on the U-Net architecture, achieves pixel-level segmentation of buildings, laying the foundation for subsequent contour extraction; The contour extraction and optimization unit combines traditional image processing techniques and deep learning methods to accurately extract building contours and optimize boundaries. The structural feature recognition unit uses a graph neural network to identify the main structural features of a building. The sparse data augmentation unit uses generative adversarial networks to complete and augment data in sparse regions; The consistency constraint unit uses physical rules and geometric constraints to ensure the rationality and consistency of the identification results.

5. The method for automatic 3D modeling and visualization of urban buildings according to claim 1, characterized in that, Step 3 includes the following steps: Analyze the completeness and complexity of building data, and dynamically select appropriate reconstruction strategies based on the analysis results; For regular buildings, a parametric model library is used, and the optimal parameter values ​​are determined by fitting point cloud data and contour information through optimization algorithms. At the same time, the identified structural features are added to generate a complete parametric model. For complex buildings, a voxel-based progressive reconstruction method is adopted, which involves: converting point cloud data into voxel representations and using a 3D convolutional neural network for preliminary classification to further construct a rough 3D model of the building; using an adaptive subdivision strategy to gradually improve the level of detail of the rough 3D model; and integrating the identified structural features into the 3D model to ensure geometric consistency and realism. The reconstructed model is optimized and simplified to reduce geometric complexity and ensure computational efficiency and visualization performance.

6. The method for automatic 3D modeling and visualization of urban buildings according to claim 1, characterized in that, Step 4 includes the following steps: Continuously monitor the geometric characteristics of the reconstructed model to quickly identify potential geometric anomalies or discontinuities; Perform topology checks to identify topology problems in the model and automatically repair them using mesh repair techniques; Based on predefined architectural rules and constraints, the structural integrity of the reconstructed model is evaluated; at the same time, heuristic algorithms are used to automatically complete missing structural elements to ensure the physical rationality of the model. Adaptive mesh refinement and smoothing algorithms are applied to improve surface quality while preserving important geometric features.

7. The method for automatic 3D modeling and visualization of urban buildings according to claim 1, characterized in that, Step 5 includes the following steps: Analyze the building's geometry to identify stylistic features and missing details; Based on the identified style features, appropriate detail elements are selected from the component library and seamlessly integrated into the reconstruction model; By combining architectural features and environmental information, the material of each part is inferred, the corresponding texture is extracted, and it is accurately mapped onto the surface of the reconstructed model; Implement global shape optimization to balance local details with the overall shape and ensure that the reconstructed model reflects the overall characteristics and stylistic consistency of the building; Based on material properties and environmental information, rendering parameters are automatically optimized to improve the realism of individual building models.

8. The method for automatic 3D modeling and visualization of urban buildings according to claim 1, characterized in that, Step 6 includes the following steps: Step 6.1: Use graph structures to represent the spatial and semantic relationships between buildings to capture the overall characteristics of the urban structure; Step 6.2: Adjust the position and orientation of buildings using a global optimization algorithm to ensure geometric consistency across the entire urban scene; Step 6.3: Ensure that the geometric structure of each building in the urban scene is consistent with the recognized semantic information, and make adjustments as necessary to reflect the correct urban semantics; Step 6.4: Integrate environmental elements to enhance the integrity and realism of the urban scene; Step 6.5: Based on scene complexity and visualization requirements, optimize the rendering parameters of the entire city scene to prepare for large-scale visualization.

Images