Multi-level texture instantiation and lightweight processing method for building three-dimensional point cloud

Abstract

The present application relates to the field of three-dimensional point cloud reconstruction and computer graphics, and more particularly to a multi-level texture instantiation and lightweight processing method for building three-dimensional point clouds. In view of the problems of large-scale building point cloud data, high geometric redundancy, low texture reuse efficiency, and difficulty in balancing compression rate and visual fidelity, a double instantiation compression strategy is adopted. Through a B-Seg network, coarse-grained semantic segmentation is completed. Combined with a 3D sliding window, density clustering, and a strategy from trunk to crown, multi-granularity segmentation of buildings and vegetation is realized. Non-parametric geometric feature coding and bidirectional similarity verification are used to complete instance grouping and deduplication. A low-polygon proxy mesh is constructed, and texture is generated through bidirectional projection of ray marching. Combined with SAM segmentation and DALL-E enhancement, component detection is realized. Texture reuse is realized by fusing image and semantic similarity, and decoupled independent texture storage is adopted. The present application has high compression rate and good fidelity, supports dynamic updating, and adapts to the rendering requirements of various urban scenes.

Description

Technical Field

[0001] This invention relates to the field of 3D point cloud reconstruction and computer graphics technology, and more specifically to a method for multi-level texture instantiation, deduplication, and lightweighting of architectural 3D point clouds. Background Technology

[0002] Building point clouds are a core data source for fields such as digital twins and smart cities, typically acquired through laser scanning or photogrammetry. However, raw building point clouds suffer from massive data volumes (tens of gigabytes for a single building) and high geometric redundancy (such as repetitive regular structures like walls and roofs), posing significant challenges for lightweight scenarios like Web3D transmission and mobile device rendering. Traditional building point cloud reconstruction methods (such as Poisson surface reconstruction and depth Marching Cubes) can generate high-precision mesh models, but their high computational complexity and lack of texture information make them unsuitable for real-time interaction in large-scale scenes.

[0003] The existing technology has the following problems: 1. Limitations of traditional surface reconstruction methods (1) High computational cost: Reconstruction methods based on implicit fields or deep learning (such as Deep Marching Cubes) require a lot of computational resources and are difficult to process large-scale point cloud data.

[0004] (2) Texture loss: Existing methods focus on geometric reconstruction and ignore the need for texture reuse of repeated components on the building surface (such as windows, doors and columns), resulting in a lack of visual details in the model.

[0005] (3) High model redundancy: The directly generated mesh model contains redundant triangular patches (such as walls being divided into a large number of triangles), which cannot be adapted to lightweight rendering frameworks such as WebGL.

[0006] 2. Shortcomings in architectural texture reconstruction (1) Low efficiency of texture atlas: Traditional texture mapping methods (such as XAtlas) package all components into a single atlas, but updating a single component requires reconstructing the entire atlas, resulting in high costs for dynamic modification.

[0007] (2) Missing processing of repetitive components: A large number of repetitive components in the building facade (such as standardized windows) are not uniformly modeled, resulting in redundant texture data (such as storing independent textures for each window).

[0008] (3) Imbalance between visual fidelity and compression rate: Existing proxy models often sacrifice texture details while reducing data volume (e.g., PSNR < 28dB when compression rate is only 50%), which cannot meet the requirements of visual consistency.

[0009] 3. Poor adaptability to large-scale scenarios (1) Insufficient LOD (Level of Detail) support: Models generated by traditional methods are difficult to adapt quickly to different rendering requirements (e.g., Web3D requires extremely low polygon models, while VR requires high-precision meshes).

[0010] (2) Weak dynamic update capability: The textures and meshes of the existing proxy model are tightly bound. Modifying local components (such as changing the window material) requires reconstructing the entire model, which limits the possibility of secondary creation.

[0011] In response to the above problems, the academic community has proposed some improvement solutions: Geometric simplification: Mesh simplification methods based on Mesh Decimation (such as QEM error-driven simplification) can reduce the number of polygons, but they destroy the regularity of the building structure (such as jagged window edges).

[0012] Neural compression: Neural texture compression techniques (such as Nerf) reduce the amount of data through implicit representation, but rely on high-dimensional vector storage, making it difficult to support direct rendering on the web.

[0013] Texture reuse: Some works (such as Adaptive Texture Atlas) attempt to optimize storage through texture block reuse, but do not take into account the geometric repetitive features of building components.

[0014] Therefore, how to provide a method for multi-level texture instantiation, deduplication, and lightweight processing of 3D point clouds in architecture is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0015] In view of this, the present invention provides a method for multi-level texture instantiation deduplication and lightweight processing of architectural 3D point clouds, aiming to solve the above-mentioned technical problems.

[0016] To achieve the above objectives, the present invention adopts the following technical solution: A method for deduplication and lightweighting of multi-level texture instantiation in 3D architectural point clouds includes the following steps: S1. Preprocess the large-scale building point cloud input, use the B-Seg network to complete coarse-grained semantic instance segmentation, and classify the instances into building and vegetation categories. Use a top-down 3D sliding window strategy to segment building instances into independent sub-building units, and combine density clustering and local maximum filtering to segment vegetation instances into single trees, thus achieving coarse-to-fine multi-granularity instance segmentation. S2. Extract the global geometric feature vector of each fine-grained instance through a non-parametric geometric feature encoder, determine the geometric consistency between instances through bidirectional similarity verification, divide highly similar instances into similar instance groups through a graph grouping method based on disjoint set, and select representative instances of each group. S3. For each representative instance, perform low-polygon mesh reconstruction to generate a simplified proxy geometry model and establish initial UV mapping to form the basic mesh structure for texture projection. S4. A bidirectional projection strategy based on ray-tracing algorithm is adopted to accurately map the voxelized point cloud to the texture pixel space, generating a color texture map and a displacement texture map. S5. Perform component-level segmentation on the color texture map and displacement texture map respectively, and optimize the segmentation results through multi-strategy mask fusion; combine DALL-E image enhancement and SAM re-segmentation to complete the detection of building facade components with high robustness. S6. After deduplication of the component mask, the image similarity and semantic similarity are fused to calculate the comprehensive similarity of the components. The components are grouped according to the preset threshold and representative textures are selected to achieve texture-level instantiation. S7. Based on component layout, the two-dimensional texture space is re-meshed and restored to three-dimensional space by UV mapping; the component texture is stored independently to record the relative coordinates of the UV space, and a decoupled texture architecture is constructed. S8. Construct a lightweight textured proxy model that includes a low-poly mesh, decoupled component textures, and instance transformation matrices. Conduct multi-view reconstruction quality evaluation using compression ratio, PSNR, and SSIM metrics. If the performance does not meet the standards, iteratively adjust the parameters until a balance between high compression ratio and high visual fidelity is achieved.

[0017] Furthermore, in S1, the specific process of the top-down 3D sliding window strategy is as follows: For each coarse-grained building instance, calculate its directed bounding box and establish a local coordinate system aligned with the building's main axis. Define a 3D sliding window with dimensions aligned with the directed bounding box and move it layer by layer vertically from top to bottom, starting from the top of the bounding box. When the window moves to each layer, extract the point cloud data within that layer and perform connected component analysis. Through a cross-layer tracking and merging mechanism, connect the spatially overlapping connected components in adjacent layers, decomposing the overall building into multiple independent sub-building units.

[0018] Furthermore, in S1, for dense forest point clouds, a stem-to-canopy segmentation method is adopted, with the specific steps as follows: First, a digital terrain model is constructed using cloth simulation filtering to separate the ground. Then, a two-stage DBSCAN clustering combined with RANSAC circular fitting is used to detect and filter out the real tree trunks at a specific height level. Finally, using the tree trunks as seeds, a height-weighted region growing algorithm is used to assign the branch and leaf point clouds to the corresponding tree trunks to complete the single tree segmentation.

[0019] Furthermore, in S2, the specific process by which the non-parametric geometric feature encoder extracts the global geometric feature vector is as follows: For each point in the instance, the 3D coordinates are embedded, and the points are expanded into a high-dimensional vector using sine and cosine functions. The geometric center of the instance is calculated, and the same embedding process is performed on the center point. Then, the features of the center point and its neighboring points are fused and aggregated using the farthest point sampling and K-nearest neighbor algorithm. Max pooling and average pooling operations are performed on the features of all points, and the results are concatenated to generate a fixed-length global geometric feature vector.

[0020] Furthermore, in S2, the specific process of bidirectional similarity verification and graph grouping is as follows: Calculate the cosine similarity between the global geometric feature vectors of all instances. Only when the similarity between two instances exceeds a preset threshold is it considered a valid match. Construct a similarity graph based on the verification results, where nodes represent instances and edges represent valid matching relationships. Use a graph grouping method based on disjoint-set data structure to aggregate all instances connected by edges into multiple disconnected similar instance groups. Within each similar instance group, comprehensively consider the completeness of instances, point cloud density, and geometric clarity, and select the instance with the most complete structure and richest details as the representative instance.

[0021] Furthermore, in S3, the specific process of low-polygon mesh reconstruction is as follows: RANSAC plane segmentation is performed on the input RGB building point cloud. The dominant planar structure in the point cloud is identified by the random sampling consensus algorithm to generate candidate patches. Based on the extracted key planes, the PolyFit method is used for mesh reconstruction. Closed polygonal boundaries are generated by analyzing the pairwise intersection relationships between planes, and a manifold polygonal surface model without redundant boundaries is constructed. At the same time, the connectivity and integrity of the mesh topology are ensured by optimizing vertex indices and edge connectivity. The XAtlas tool is used to perform preliminary texture mapping on the low-polygon mesh to generate an initial texture atlas.

[0022] Furthermore, in S5, the specific process of enhancing component detection and mask fusion is as follows: Point cloud information is projected into two basic textures: color texture and displacement texture. The color texture is a continuous color texture map generated by projecting the RGB color information of the point cloud onto the mesh surface through voxelization, and the displacement texture is a texture map generated by calculating the vertical height difference between the point cloud and the mesh surface. The Segment Anything Model is used to segment the color texture, displacement texture, and enhanced texture generated by DALL-E, respectively, and corresponding initial masks are generated. By using three fusion conditions—symmetry preservation, co-occurrence of color and displacement masks, and latent symmetry judgment—multiple sets of masks are merged to generate a preliminary fused mask set, and small-area noise masks are removed through morphological operations. The fused mask is used as an input prompt to guide DALL-E to generate a higher resolution enhanced texture. The enhanced texture is then segmented again using the Segment Anything Model to generate an optimized mask. The optimized mask is then matched with the fused mask set using IoU, and the old mask with an overlap higher than the threshold is replaced to obtain the final high-precision component mask.

[0023] Furthermore, in S6, the specific process of component mask deduplication and similarity grouping is as follows: For each component mask, calculate its geometric center, area, and minimum bounding rectangle to complete mask rectangularization; traverse all mask pairs to calculate IoU value, and if the IoU exceeds a preset threshold, mark the smaller mask as redundant and discard it; For each component, a perceptual hashing algorithm is applied to the texture image to generate a 64-bit binary fingerprint. The Hamming distance between any two component fingerprints is calculated to obtain image similarity. Semantic labels for the components are generated using GPT-4, and the labels are embedded into vectors using the CLIP model. The cosine similarity between the vectors is calculated to obtain semantic similarity. A weighted fusion of image similarity and semantic similarity is used to obtain a comprehensive similarity score. A similarity threshold is set, and components that exceed the threshold are grouped together. For each group of components, the texture with the richest detail is selected as the representative texture, and the UV coordinates of the representative texture are mapped to the corresponding positions of all components in the group.

[0024] Furthermore, in S7, the specific process of re-meshing and independent storage of component textures is as follows: The component mask contours of the extracted texture image are used as the boundaries of the quadrilateral mesh to construct a two-dimensional quadrilateral mesh. The quadrilateral mesh is converted into a triangular mesh by constrained Delaunay triangulation based on the Chew algorithm. The maximum triangle area threshold and minimum interior angle constraint are set to avoid geometric distortion caused by narrow triangles. By using a predefined 3D mesh to 2D texture mapping relationship, the coordinates of the vertices of the two-dimensional triangular mesh in three-dimensional space are calculated based on the centroid interpolation, and a complete three-dimensional mesh model of the building is generated by splicing them together. Each component's texture file is stored separately, the relative position of the component's vertices in the 2D texture is calculated, and the mapping relationship between component textures and vertex indices is stored through a hash table, supporting local replacement and dynamic loading of component textures.

[0025] Furthermore, in S8, the specific process of reconstruction quality assessment and iterative optimization is as follows: A multi-view rendering comparison method was used to calculate the compression ratio, PSNR and SSIM quantization index to verify the visual consistency between the reconstructed proxy model and the original point cloud. If the evaluation results do not meet the standards, optimization can be achieved by adjusting the SAM segmentation parameters, optimizing the DALL-E text prompts, and adjusting the component similarity threshold and weights. Through multiple iterations of gradual optimization, convergence is determined when the rate of change of the index is less than 1% in two consecutive iterations; when the compression rate is ≥90% and PSNR ≥30dB and SSIM ≥0.9, the final surrogate model is output; if the maximum number of iterations is exceeded and the target is still not met, the building is marked as a non-traditional building, and a higher resolution texture or a neural texture optimization strategy is introduced.

[0026] Compared to existing technologies, this invention uses multimodal fusion of RGB color textures, displacement textures, and DALL-E enhanced textures to accurately detect repetitive components (such as windows and doors) in building facades and generate shared textures; it stores component textures independently rather than integrating them into a single atlas, and achieves flexible component replacement (such as modifying only the window texture on a certain floor) through local UV coordinate mapping; it uses a comprehensive similarity score (image similarity + semantic similarity) to group components, and combines bidirectional ray tracing and voxelization alignment to achieve a proxy model compression rate of ≥90% and a PSNR of ≥30dB. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0028] Figure 1 This is a flowchart illustrating the overall process of instantiation, deduplication, texture extraction, and multi-level lightweight reconstruction of building point clouds in this invention.

[0029] Figure 2 This is a schematic diagram of the original scenario of the present invention and existing compression methods.

[0030] Figure 3 This is a schematic diagram of the data compression process of the present invention.

[0031] Figure 4 This invention provides a method for processing large-scale building point clouds by combining large-scale point cloud deduplication instantiation with building texture extraction. Detailed Implementation

[0032] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] The purpose of this invention is to provide a method for multi-level texture instantiation deduplication and lightweight processing of architectural 3D point clouds. To make the above-mentioned objectives, features and advantages of this invention more apparent and understandable, the invention will be further described in detail below with reference to specific embodiments.

[0034] 1. Multi-granularity instance segmentation from coarse-grained to fine-grained S1.1. Coarse-grained segmentation using a B-Seg network: This step aims to perform preliminary semantic and instance segmentation on the input large-scale building point cloud. For example... Figure 3 As shown in the upper part, specifically, the raw point cloud data is input into a deep learning instance segmentation network called B-Seg, which can identify and separate objects of different semantic categories. After processing, the system outputs a coarse-grained set of instances, each representing a major object category.

[0035] S1.2. Top-Down 3D Sliding Window Segmentation Strategy: Addressing the common issue of multiple interconnected sub-buildings within a building instance, a top-down 3D sliding window segmentation strategy is proposed. First, the directed bounding box (OBB) of each coarse-grained building instance is calculated to establish a local coordinate system aligned with the building's principal axis. Then, a small-height 3D sliding window, aligned with the OBB in length and width, is defined and moves vertically layer by layer from the top of the OBB. As the window moves to each layer, point cloud data within that layer is extracted, and connected component analysis is performed to identify separated building structures at that height. Finally, through a cross-layer tracking and merging mechanism, spatially overlapping connected components in adjacent layers are connected, thereby decomposing the overall building into multiple independent, complete sub-building units.

[0036] S1.3. Single Tree Segmentation Based on Density Clustering and Local Maxim Filtering: To address the issue of interconnected single trees in vegetation instances, a two-stage segmentation strategy is employed. First, a density clustering algorithm (such as DBSCAN) is used to initially segment coarse-grained vegetation instances, separating dense tree clusters from sparse open areas, forming several point cloud clusters containing multiple connected trees. Then, based on the assumption that "each tree has one and only one highest point," local maxima filtering is performed within each cluster to search for and extract "seed points" representing the crown vertices of single trees. Finally, all points in the original vegetation point cloud are assigned to the nearest "seed point" based on Euclidean distance, thereby accurately segmenting the entire vegetation area into multiple independent single tree instances.

[0037] S1.4. Large-scale dense tree point cloud segmentation from "trunk" to "crown": For dense forest point clouds, the core idea of ​​processing is "from trunk to crown", which is achieved in three steps: First, a digital terrain model (DTM) is constructed using cloth simulation filtering (CSF) to separate the ground; then, two-stage DBSCAN clustering combined with RANSAC circular fitting is used to accurately detect and filter out the real tree trunks at a specific height level (utilizing the geometric feature that the tree trunk is cylindrical); finally, using the tree trunk as "seed", the branch and leaf point clouds are robustly assigned to the corresponding tree trunks through a height-weighted region growing algorithm, thereby completing the segmentation of individual trees.

[0038] 2. Unsupervised similarity detection and instance grouping S2.1. Non-parametric Geometric Feature Encoding and Global Feature Generation: A global feature vector representing the overall geometry of each finely segmented instance (e.g., a sub-building or a single tree) is generated. To this end, a non-parametric geometric feature encoder is designed. First, drawing inspiration from the positional encoding concept of the Transformer model, the 3D coordinates of each point in the instance are "point-embedded," and expanded into a high-dimensional vector using sine and cosine functions to capture fine-grained information about the local structure. Then, the geometric center of the instance is calculated, and the center point undergoes the same embedding process. Next, the features of the center point and its neighbors are fused and aggregated using farthest point sampling (FPS) and K-nearest neighbor (KNN) algorithms. Finally, max pooling and average pooling operations are performed on the features of all points, and the results are concatenated to generate a fixed-length global geometric feature vector, providing the core basis for subsequent similarity matching and grouping.

[0039] S2.2. Similar Instance Grouping Based on Bidirectional Validation and Union-Find: This step aims to effectively group fine-grained instances with highly similar geometric shapes. First, the cosine similarity between the global geometric feature vectors of all instances is calculated to measure their geometric consistency. To ensure the symmetry and reliability of the matching, a bidirectional similarity validation mechanism is introduced, meaning that a valid match is only determined when the similarity between two instances exceeds a preset threshold. Based on this validation result, a "similar graph" is constructed, where nodes represent instances and edges represent valid matching relationships. Finally, a graph grouping method based on Union-Find is used to efficiently aggregate all nodes (instances) connected by edges into multiple disconnected "similar instance groups," laying the foundation for subsequent instantiation compression.

[0040] S2.3. Selecting a Representative Instance: After obtaining all similar instance groups using the disjoint-set data structure algorithm, one most representative instance needs to be selected from each group. This representative instance will serve as the "template" for all members of the group. The selection criteria typically consider the instance's completeness, point cloud density, and geometric clarity, prioritizing instances with the most complete structure and richest details. In subsequent scene lightweighting, this representative instance will be fully preserved, while all other instances in the group will be deleted, retaining only a 4×4 transformation matrix that can restore it to its original position, pose, and size, thereby achieving significant data compression.

[0041] like Figure 2 As shown, the left side represents the original scene, showcasing a large-scale urban building point cloud scene containing multiple buildings and vegetation. However, due to the similar appearance of many buildings, the data exhibits severe redundancy, failing to achieve a lightweight effect. The right side illustrates the accuracy loss issues inherent in existing compression methods (sampling and compression methods). We propose and demonstrate a solution: by identifying similar instances and retaining only one representative instance, a lightweight proxy model is generated. In this model, only one representative instance is retained for each group of similar instances, and the remaining instances are restored using a transformation matrix, significantly reducing the data volume.

[0042] Detailed process as follows Figure 3 As shown, efficient data compression was achieved through a series of steps. The input point cloud size is N. 10 6 After compression, the output point cloud size is significantly reduced (N). 10 5(or less). During compression, the system first performs coarse-grained segmentation using a B-Seg network, then employs a top-down 3D sliding window segmentation strategy and density clustering and local maximum filtering methods to further subdivide the coarse-grained instances into independent sub-building units and individual trees. Next, a non-parametric geometric feature encoder extracts the local features of each instance, and max pooling and average pooling are used to generate global geometric feature vectors. Based on these feature vectors, a bidirectional similarity verification mechanism is used to determine the geometric consistency between instances, and a graph grouping method based on disjoint-set data structure is used to organize highly similar instances into similar instance groups, with one representative instance selected from each group. Finally, a lightweight proxy model is generated, where only one representative instance is retained from each similar instance group, and the remaining instances are restored using a 4×4 transformation matrix. This method not only significantly reduces the amount of data but also maintains good visual consistency. The compressed data is suitable for transmission and can be restored to near-original size (N) after decompression. 10 6 This method of point cloud processing solves the data redundancy problem in large-scale urban point clouds, while maintaining high-quality rendering. It provides strong support for online transmission and real-time rendering.

[0043] 3. Input Preprocessing and Low-Polygon Mesh Reconstruction S3.1. Point Cloud Preprocessing and Plane Segmentation: RANSAC plane segmentation is performed on the input RGB building point cloud. The dominant planar structures in the point cloud are identified using the random sampling consensus algorithm, generating candidate patches. During this process, the algorithm iteratively selects point sets and fits planar models to filter out planar structures that highly match the point cloud data.

[0044] S3.2. Low-Polygon Mesh Reconstruction: Based on the key planes extracted in S1.1, the PolyFit method is used for mesh reconstruction. This method first generates closed polygonal boundaries through pairwise intersection analysis between planes and constructs a manifold polygonal surface model without redundant boundaries. The final generated mesh model expresses the building geometry in a minimalist form, while optimizing vertex indices and edge connectivity to ensure the connectivity and integrity of the mesh topology, avoiding isolated faces or non-manifold edges.

[0045] S3.3. Initial Texture Atlas Generation: The XAtlas tool is used to perform initial texture mapping on the low-poly mesh obtained in S1.2 to generate an initial texture atlas.

[0046] 4. Precise mapping from point cloud to texture pixels S4.1. Voxel and Texture Pixel Alignment: To ensure spatial resolution matching between the point cloud and texture pixels, the input point cloud is first voxelized. By setting the voxel side length to match the physical size of the texture pixels, the point cloud is divided into uniform cubic voxel grids. Only one point (or approximately) is retained within each voxel to avoid texture blurring caused by oversampling. Subsequently, the number of voxels in each texture image in the horizontal and vertical directions is counted to estimate the total number of pixels in the required texture atlas, and the final texture resolution (e.g., 1024×1024 or 2048×2048) is determined accordingly. This process provides the basis for spatial scale alignment for subsequent projection, ensuring that the geometric details of the point cloud are consistent with the precision of the texture pixels.

[0047] S4.2. Bidirectional Ray Tracing and Intersection Matching: Based on the established UV coordinate mapping relationship, starting from the UV coordinates corresponding to each texture pixel, rays are emitted in the normal direction of the mesh surface and its opposite direction, respectively, to find the nearest intersection point with the point cloud surface as the final corresponding point.

[0048] 5. Enhanced component detection and mask fusion S5.1. Color and Displacement Texture Generation: To fully explore the visual and geometric features of the building facade components, the point cloud information is first projected into two basic textures: (1) Color texture (Tc): The RGB color information of the point cloud is projected onto the mesh surface through voxelization to ensure that each texture pixel corresponds to the color value in the point cloud, thus generating a continuous color texture map.

[0049] (2) Displacement texture (Td): Calculate the vertical height difference between the point cloud and the mesh surface to generate displacement texture.

[0050] S5.2. Multi-stage component segmentation and enhanced texture generation: The Segment Anything Model (SAM) is used to segment the color texture (Tc), displacement texture (Td), and enhanced texture (Te) generated by DALL-E respectively. (1) SAM basic segmentation: Set the prompting strategy (such as full frame prompting, point prompting) for Tc and Td respectively, adjust the number of prompting points of SAM (such as 2 to 3 points per component) and the segmentation threshold (such as IoU threshold of 0.7); generate two sets of initial masks (Mc and Md), which correspond to the component segmentation results of color and displacement features respectively.

[0051] (2) DALL-E Enhanced Texture Generation: Input Tc into DALL-E to generate high-resolution enhanced texture Te. Apply SAM to Te again to generate enhanced mask Me, which supplements the missing details in Mc and Md (such as occluded symmetrical components).

[0052] S5.3. Masking Strategy To integrate multi-source segmentation results and eliminate noise, three mask fusion conditions are proposed. Masks that meet the above conditions in Mc, Md, and Me are merged to generate a preliminary fused mask set. Small-area noise masks are removed by morphological operations (such as opening and hole filling) (the area threshold is set to 0.1% of the total texture area).

[0053] S5.4. Enhanced Segmentation and Boundary Optimization To improve the accuracy of component boundaries, the fused mask was further optimized: (1) DALL-E secondary enhancement: The fusion mask is used as an input prompt to guide DALL-E to generate a higher resolution enhanced texture Te2, focusing on repairing the blurred edge areas; (2) SAM re-segmentation: Apply high-precision SAM segmentation to Te2, adjust the hint strategy (such as hinting at each mask boundary point), and generate an optimized mask Mo; perform IoU matching between Mo and the fused mask set, replace the old mask with an overlap of >0.7, and retain the boundary with richer details.

[0054] (3) Final mask selection: After multi-stage segmentation and mask fusion, a set of high-precision, low-redundancy building facade component masks are generated, covering repeated structures such as windows, doors, and columns, providing a reliable basis for texture reuse and compression.

[0055] 6. Component overlap removal and similarity grouping S6.1. Component Boundary Simplification and Redundancy Removal: First, the boundary of the segmented component mask is simplified. This process is achieved through the following steps: (1) Mask rectangularization: Calculate the geometric center, area and minimum bounding rectangle of each mask to ensure that the bounding box fits the original mask outline tightly; (2) IoU calculation and filtering: Traverse all mask pairs and calculate the IoU value (defined as the ratio of the intersection area of ​​two bounding boxes to the union area). If the IoU exceeds the preset threshold (such as 0.7), mark the smaller mask as redundant. (3) Redundancy removal: Delete all masks marked as redundant and keep only the master mask, thereby eliminating the interference area in the texture mapping and ensuring the independence of subsequent grouping results.

[0056] S6.2. Comprehensive Similarity Scoring and Component Grouping: The simplified mask is grouped based on similarity. This process comprehensively considers two dimensions: image similarity and semantic similarity, and quantifies the similarity using a weighted scoring formula: (1) Image feature extraction and hash encoding: Apply the perceptual hash algorithm to the texture image corresponding to each component to generate a 64-bit binary fingerprint; calculate the Hamming distance (dH) between any two component fingerprints to reflect the differences in their image content (dH∈[0,1], the smaller the value, the more similar they are).

[0057] (2) Semantic feature extraction and similarity calculation: Use GPT-4 to generate semantic labels for components (such as “window”, “door”, “pillar”), and embed the labels into 768-dimensional vectors through CLIP model; calculate the cosine similarity (SemanticSimilarity,SL∈[0,1]) between vectors to quantify the semantic association of components.

[0058] (3) Comprehensive scoring and threshold screening: Combine image and semantic similarity, use a weighted formula to calculate the comprehensive score, and prioritize the balance between image details and semantic consistency; set a similarity threshold (e.g., St≥0.7), and group components that are higher than the threshold together to ensure that the components in the group are highly consistent in visual and semantic terms.

[0059] (4) Representative texture selection and texture reuse: For each group of components, select the texture with the richest details (such as components with clear edges and uniform color) as the representative texture; map the UV coordinates of the representative texture to the corresponding positions of all components in the group to achieve texture reuse and significantly reduce the amount of data.

[0060] S6.3. Similarity Threshold Optimization and Iterative Adjustment: To adapt to the differences in component characteristics across different architectural scenarios, the similarity threshold needs to be dynamically adjusted to achieve the optimal grouping effect. (1) Threshold adaptive mechanism: The threshold is adjusted based on the building type (such as high / low symmetry, component density) (e.g., the threshold for symmetrical buildings is set to 0.75, and the threshold for asymmetrical buildings is set to 0.65), and the redundancy rate and mismatch rate are evaluated by visualizing the grouping results, and the threshold is dynamically corrected.

[0061] (2) Manual intervention and post-processing: For cases where automatic grouping fails (such as mismatch of irregular windows), corrections are made by manually adjusting weights w1 / w2 or reselecting representative textures; finally, a lightweight component texture set is output, which supports instantiated rendering and dynamic updates.

[0062] 7. Remeshing and Retexturing S7.1. 2D Quadrilateral Mesh Generation and Triangulation: A 2D quadrilateral mesh is generated based on the component mask layout, and then converted into a triangular mesh using constrained Delaunay triangulation (CDT). The specific process is as follows: (1) Quadrilateral mesh construction: Extract the component mask contour of the texture image as the boundary of the quadrilateral mesh, and ensure that the mesh boundary is highly consistent with the component shape; add component quadrilaterals in the detection box, and adjust the position of the quadrilateral vertices according to the mask boundary to match the component geometric features (such as the rectangular contour of the window).

[0063] (2) Constrained Delaunay triangulation: The Chew algorithm is used to triangulate the quadrilateral mesh to generate a triangular mesh without redundant boundaries; The parameter settings include a maximum triangle area threshold (e.g., 0.01m²) and a minimum interior angle constraint (e.g., 20°) to avoid geometric distortion caused by elongated triangles; the generated triangular mesh preserves the component topology to ensure the geometric continuity of subsequent 3D reconstruction.

[0064] S7.2. 3D Mesh Reconstruction and UV Coordinate Mapping: Through the UV mapping relationship established by XAtlas, the vertices of the 2D triangular mesh are reconstructed to 3D space, and the independent storage and efficient mapping of component textures are achieved. (1) Calculation of three-dimensional vertex positions: For each component vertex, obtain its UV coordinates and corresponding barycentric coordinates in the two-dimensional texture. Based on the 3D mesh to 2D texture mapping relationship predefined by XAtlas, calculate the coordinates of the vertex in three-dimensional space by barycentric interpolation; by traversing the triangular mesh of all two-dimensional textures, finally stitch together to generate a complete three-dimensional mesh model of the building.

[0065] (2) Independent texture storage and coordinate binding: Instead of packaging all component textures into a single atlas, each component's texture file (such as PNG format) is stored separately. The relative position of the component's vertices in the 2D texture is calculated to determine its associated components and corresponding corners. Since the component shapes are all rectangular, only four UV coordinate combinations are needed to cover all cases, avoiding the global remapping cost of traditional texture atlas updates.

[0066] S7.3. Dynamic Texture Update and Lightweight Optimization: To support subsequent dynamic scene editing, the texture management and mesh update mechanism have been further optimized. (1) Texture replacement interface design: The corresponding texture file can be quickly located by the component ID index, allowing users to directly replace the texture of the specified component (such as changing the window material); after replacement, only the UV coordinate mapping of the associated vertices needs to be updated, without reconstructing the overall mesh.

[0067] (2) Lightweight data structure: Hash table is used to store the mapping relationship between component textures and vertex indices, supporting fast lookup with O(1) time complexity; low-resolution textures are used for non-critical components (such as decorative details) to further reduce memory usage (such as 128×128 resolution).

[0068] 8. Reconstruction Quality Assessment and Iterative Optimization S8.1. Multi-view quantitative evaluation and index calculation: The visual consistency between the reconstructed proxy model and the original point cloud is verified by combining multi-view rendering comparison with quantitative index evaluation.

[0069] S8.2. Parameter Adjustment and Optimization Strategy: If the evaluation results show that the compression rate or visual fidelity does not meet the standards (e.g., PSNR < 30dB or SSIM < 0.9), optimization should be performed using the following strategies: (1) SAM segmentation parameter adjustment: Increase the number of cue points (e.g., from 2 points per component to 5 points) to improve the segmentation integrity of complex structures (e.g., irregular windows); at the same time, adjust the IoU threshold of SAM (e.g., from 0.7 to 0.6) to retain more potential effective masks.

[0070] (2) Texture enhancement strategy optimization: When generating enhanced textures in DALL-E, add more specific text prompts (such as "symmetrical window" and "high-resolution brick wall texture") to reduce interference from irrelevant details; manually filter the generated enhanced images and select the version with the richest details for subsequent segmentation (such as selecting the best one after comparing 10 generation results).

[0071] (3) Component similarity threshold adjustment: dynamically adjust the weights in the comprehensive scoring formula according to the building type (e.g., w1=0.7, w2=0.3 for symmetrical buildings, w1=0.5, w2=0.5 for asymmetrical buildings), prioritize balancing image or semantic similarity; reduce the similarity threshold (e.g., from 0.7 to 0.65), increase the texture reuse ratio to further compress the amount of data.

[0072] S8.3. Iterative Optimization and Convergence Judgment: Through multiple rounds of iteration, the dual goals of compression ratio and visual fidelity are gradually approached. If the rate of change of the metric is less than 1% for two consecutive iterations (e.g., PSNR improvement < 0.1dB), then convergence is determined and optimization is terminated. Rendering results for key perspectives (e.g., the front view) are manually reviewed, and obviously distorted components (e.g., mismatched window textures) are removed. 9. Final output determination When CR≥90% and PSNR≥30dB and SSIM≥0.9, output the final proxy model; If the maximum number of iterations is exceeded and the target is still not met, the building is marked as a "non-traditional building," and it is recommended to use a higher resolution texture or introduce a neural texture optimization strategy.

[0073] Figure 4 This paper demonstrates a method for processing large-scale architectural point clouds by combining large-scale point cloud deduplication and instantiation with building texture extraction. First, the point cloud is segmented into instances with different semantic meanings. Then, representative instances are selected for texture generation. PolyFit is used to generate a simplified low-polygon mesh from the input RGB architectural point cloud, and XAtlas is used to generate initial UV mappings, providing a foundation for subsequent processing. Next, a bidirectional projection strategy based on ray-mapping algorithm is employed to accurately map the voxelized point cloud onto the texture pixels of the mesh, generating high-fidelity color (Tc) and displacement (Td) texture maps. Then, Segment Anything Model (SAM) is used to perform component-level segmentation on these texture maps, identifying facade components such as windows and doors. DALL-E is introduced to enhance low-quality or blurry areas, and SAM is used for re-segmentation to obtain more robust component masks. Subsequently, texture compression is performed on the segmented components: image similarity calculated using perceptual hashing and semantic similarity calculated using semantic tags generated by GPT-4 analysis are combined to group similar components and select a representative texture. Finally, by re-meshing and re-texturing, the 2D component layout is restored to 3D space. Instead of the traditional method of packaging texture atlases, the texture of each component is stored independently. This creates a lightweight proxy model that includes low-poly meshes, decoupled component textures, and instance transformation matrices. This approach not only greatly reduces the amount of data processed but also has a good visual effect.

[0074] The technical effects achieved by this invention compared to the prior art are as follows: (1) Pioneering a lightweight city-level point cloud instantiation framework: For the first time, a complete automated process from large-scale city point clouds to lightweight texture proxy models was proposed and implemented. This framework integrates two core technologies: multi-level instance segmentation and instantiated texture proxy reconstruction. By segmenting buildings from top to bottom using 3D sliding windows, separating individual trees using density clustering and local maximum filtering, the problem of connecting substructures in coarse-grained instances is solved, laying a solid foundation for subsequent deduplication and compression.

[0075] (2) Data Compression Driven by Dual Instantiation: A dual instantiation compression strategy is proposed to achieve dual deduplication of geometry and texture. At the instance level, through non-parametric geometric feature encoding and bidirectional similarity verification, geometrically similar building and vegetation instances are grouped, retaining only one representative instance and its transformation matrix, significantly reducing duplicate geometry. At the component level, using mask fusion strategies such as symmetry preservation and color-displacement co-occurrence, textures are instantiated for facade components such as windows and doors, enabling cross-instance reuse of a single texture. The dual compression mechanism works synergistically to compress the point cloud data volume to less than 2.8% of the original data while maintaining visual consistency.

[0076] (3) A decoupled model supporting dynamic updates and efficient interaction: A lightweight proxy model was constructed, which includes low-poly meshes, decoupled component textures, and instance transformation matrices. The traditional method of packaging texture atlases was abandoned, and instead, the textures of each component were stored independently. The components and transformation matrices were associated through efficient data structures such as hash tables. This decoupled design not only significantly reduced the cost of texture maintenance, but also supported efficient local updates and dynamic loading on the Web. Users can replace the materials of components such as door signs and windows online in real time, with a response latency of less than 0.1 seconds, which greatly improved the interactive experience.

[0077] (4) Cross-platform and adaptive multi-scenario application capabilities: The generated lightweight proxy model has strong cross-scenario adaptability. By calculating metrics such as compression ratio, PSNR, and SSIM for quality evaluation, segmentation and deduplication parameters can be dynamically adjusted to achieve a balance between high compression ratio and high visual fidelity. The model can seamlessly adapt to downstream applications with different needs: In scenarios such as Web3D where transmission speed is extremely important, high compression ratio and low latency are prioritized; in scenarios such as VR / AR and digital twins where visual quality is critical, high-fidelity LOD (Level of Detail) models can be generated to meet the application needs of different levels.

[0078] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0079] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for deduplication and lightweight processing of multi-level texture instantiation in architectural 3D point clouds, characterized in that, Includes the following steps: S1. Preprocess the large-scale building point cloud input, use the B-Seg network to complete coarse-grained semantic instance segmentation, and classify the instances into building and vegetation categories. Use a top-down 3D sliding window strategy to segment building instances into independent sub-building units, and combine density clustering and local maximum filtering to segment vegetation instances into single trees, thus achieving coarse-to-fine multi-granularity instance segmentation. S2. Extract the global geometric feature vector of each fine-grained instance through a non-parametric geometric feature encoder, determine the geometric consistency between instances through bidirectional similarity verification, divide highly similar instances into similar instance groups through a graph grouping method based on disjoint set, and select representative instances of each group. S3. For each representative instance, perform low-polygon mesh reconstruction to generate a simplified proxy geometry model and establish initial UV mapping to form the basic mesh structure for texture projection. S4. A bidirectional projection strategy based on ray-tracing algorithm is adopted to accurately map the voxelized point cloud to the texture pixel space, generating a color texture map and a displacement texture map. S5. Perform component-level segmentation on the color texture map and displacement texture map respectively, and optimize the segmentation results through multi-strategy mask fusion; combine DALL-E image enhancement and SAM re-segmentation to complete the detection of building facade components with high robustness. S6. After deduplication of the component mask, the image similarity and semantic similarity are fused to calculate the comprehensive similarity of the components. The components are grouped according to the preset threshold and representative textures are selected to achieve texture-level instantiation. S7. Based on component layout, the two-dimensional texture space is re-meshed and restored to three-dimensional space by UV mapping; the component texture is stored independently to record the relative coordinates of the UV space, and a decoupled texture architecture is constructed. S8. Construct a lightweight textured proxy model that includes a low-poly mesh, decoupled component textures, and instance transformation matrices. Conduct multi-view reconstruction quality evaluation using compression ratio, PSNR, and SSIM metrics. If the performance does not meet the standards, iteratively adjust the parameters until a balance between high compression ratio and high visual fidelity is achieved.

2. The method for multi-level texture instantiation deduplication and lightweight processing of architectural 3D point clouds according to claim 1, characterized in that, In S1, the specific process of the top-down 3D sliding window strategy is as follows: For each coarse-grained building instance, calculate its directed bounding box and establish a local coordinate system aligned with the building's main axis. Define a 3D sliding window with dimensions aligned with the directed bounding box and move it layer by layer vertically from top to bottom, starting from the top of the bounding box. When the window moves to each layer, extract the point cloud data within that layer and perform connected component analysis. Through a cross-layer tracking and merging mechanism, connect the spatially overlapping connected components in adjacent layers, decomposing the overall building into multiple independent sub-building units.

3. The method for multi-level texture instantiation deduplication and lightweight processing of architectural 3D point clouds according to claim 1, characterized in that, In S1, for dense forest point clouds, a stem-to-canopy segmentation method is adopted, and the specific steps are as follows: First, a digital terrain model is constructed using cloth simulation filtering to separate the ground. Then, a two-stage DBSCAN clustering combined with RANSAC circular fitting is used to detect and filter out the real tree trunks at a specific height level. Finally, using the tree trunks as seeds, a height-weighted region growing algorithm is used to assign the branch and leaf point clouds to the corresponding tree trunks to complete the single tree segmentation.

4. The method for multi-level texture instantiation deduplication and lightweight processing of architectural 3D point clouds according to claim 1, characterized in that, In S2, the specific process of the non-parametric geometric feature encoder extracting the global geometric feature vector is as follows: For each point in the instance, the 3D coordinates are embedded, and then extended into a high-dimensional vector using sine and cosine functions. The geometric center of the instance is calculated, and the center point is subjected to the same embedding process. Then, the features of the center point and its neighboring points are fused and aggregated using farthest point sampling and the K-nearest neighbor algorithm. Max pooling and average pooling operations are performed on the features of all points, and the results are concatenated to generate a fixed-length global geometric feature vector.

5. The method for multi-level texture instantiation deduplication and lightweight processing of architectural 3D point clouds according to claim 1, characterized in that, In S2, the specific process of bidirectional similarity verification and graph grouping is as follows: Calculate the cosine similarity between the global geometric feature vectors of all instances. Only when the similarity between two instances exceeds a preset threshold is it considered a valid match. Construct a similarity graph based on the verification results, where nodes represent instances and edges represent valid matching relationships. Use a graph grouping method based on disjoint-set data structure to aggregate all instances connected by edges into multiple disconnected similar instance groups. Within each similar instance group, comprehensively consider the completeness of instances, point cloud density, and geometric clarity, and select the instance with the most complete structure and richest details as the representative instance.

6. The method for multi-level texture instantiation deduplication and lightweight processing of architectural 3D point clouds according to claim 1, characterized in that, In S3, the specific process of low-polygon mesh reconstruction is as follows: RANSAC plane segmentation is performed on the input RGB building point cloud. The dominant planar structure in the point cloud is identified by the random sampling consensus algorithm to generate candidate patches. Based on the extracted key planes, the PolyFit method is used for mesh reconstruction. Closed polygonal boundaries are generated by analyzing the pairwise intersection relationships between planes, and a manifold polygonal surface model without redundant boundaries is constructed. At the same time, the connectivity and integrity of the mesh topology are ensured by optimizing vertex indices and edge connectivity. The XAtlas tool is used to perform preliminary texture mapping on the low-polygon mesh to generate an initial texture atlas.

7. The method for multi-level texture instantiation deduplication and lightweight processing of architectural 3D point clouds according to claim 1, characterized in that, In S5, the specific process of enhancing component detection and mask fusion is as follows: Point cloud information is projected into two basic textures: color texture and displacement texture. The color texture is a continuous color texture map generated by projecting the RGB color information of the point cloud onto the mesh surface through voxelization, and the displacement texture is a texture map generated by calculating the vertical height difference between the point cloud and the mesh surface. The Segment Anything Model is used to segment the color texture, displacement texture, and enhanced texture generated by DALL-E, respectively, and corresponding initial masks are generated. By using three fusion conditions—symmetry preservation, co-occurrence of color and displacement masks, and latent symmetry judgment—multiple sets of masks are merged to generate a preliminary fused mask set, and small-area noise masks are removed through morphological operations. The fusion mask is used as an input prompt to guide DALL-E to generate a higher resolution enhanced texture. The enhanced texture is then segmented again using the Segment Anything Model to generate an optimized mask. The optimized mask is matched with the fused mask set by IoU, and the old mask with an overlap of more than the threshold is replaced to obtain the final high-precision component mask.

8. The method for multi-level texture instantiation deduplication and lightweight processing of architectural 3D point clouds according to claim 1, characterized in that, In S6, the specific process of component mask deduplication and similarity grouping is as follows: For each component mask, calculate its geometric center, area, and minimum bounding rectangle to complete mask rectangularization; traverse all mask pairs to calculate IoU value, and if the IoU exceeds a preset threshold, mark the smaller mask as redundant and discard it; For each component, a perceptual hashing algorithm is applied to the texture image to generate a 64-bit binary fingerprint. The Hamming distance between any two component fingerprints is calculated to obtain image similarity. Semantic labels for the components are generated using GPT-4, and the labels are embedded into vectors using the CLIP model. The cosine similarity between the vectors is calculated to obtain semantic similarity. A weighted fusion of image similarity and semantic similarity is used to obtain a comprehensive similarity score. A similarity threshold is set, and components that exceed the threshold are grouped together. For each group of components, the texture with the richest detail is selected as the representative texture, and the UV coordinates of the representative texture are mapped to the corresponding positions of all components in the group.

9. The method for multi-level texture instantiation deduplication and lightweight processing of architectural 3D point clouds according to claim 1, characterized in that, In S7, the specific process of re-meshing and independent storage of component textures is as follows: The component mask contours of the extracted texture image are used as the boundaries of the quadrilateral mesh to construct a two-dimensional quadrilateral mesh. The quadrilateral mesh is converted into a triangular mesh by constrained Delaunay triangulation based on the Chew algorithm. The maximum triangle area threshold and minimum interior angle constraint are set to avoid geometric distortion caused by narrow triangles. By using a predefined 3D mesh to 2D texture mapping relationship, the coordinates of the vertices of the two-dimensional triangular mesh in three-dimensional space are calculated based on the centroid interpolation, and a complete three-dimensional mesh model of the building is generated by splicing them together. Each component's texture file is stored separately, the relative position of the component's vertices in the 2D texture is calculated, and the mapping relationship between component textures and vertex indices is stored through a hash table, supporting local replacement and dynamic loading of component textures.

10. The method for multi-level texture instantiation deduplication and lightweight processing of architectural 3D point clouds according to claim 1, characterized in that, In S8, the specific process of reconstruction quality assessment and iterative optimization is as follows: A multi-view rendering comparison method was used to calculate the compression ratio, PSNR and SSIM quantization index to verify the visual consistency between the reconstructed proxy model and the original point cloud. If the evaluation results do not meet the standards, optimization can be achieved by adjusting the SAM segmentation parameters, optimizing the DALL-E text prompts, and adjusting the component similarity threshold and weights. Through multiple iterations of gradual optimization, convergence is determined when the rate of change of the index is less than 1% in two consecutive iterations; when the compression rate is ≥90% and PSNR ≥30dB and SSIM ≥0.9, the final surrogate model is output; if the maximum number of iterations is exceeded and the target is still not met, the building is marked as a non-traditional building, and a higher resolution texture or a neural texture optimization strategy is introduced.

Images