GeoSemantic-gs framework and application for scene surface reconstruction

Abstract

The application discloses a kind of GeoSemantic-GS framework for scene surface reconstruction, comprising: semantic prior generation module, multiple-source perception data stream is converted into multimodal semantic prior and physical semantic prior;Space-time filtering density control module is connected to semantic prior generation module, shields dynamic interference data and adaptively controls element density;Asymmetric Gaussian generation module is connected to semantic prior generation module and space-time filtering density control module respectively, executes multimodal semantic driven asymmetric Gaussian parameter optimization, and outputs generated structured geometric representation;And laser vision fusion module is connected to semantic prior generation module and asymmetric Gaussian generation module, provides physical level geometric anchoring, realizes macrostructure flatness and micro detail fidelity dynamic balance, compresses representation element order of magnitude, inhibits geometric parameter drift and dynamic interference artifact, meets the construction needs of high-precision lightweight scene representation.

Description

Technical Field

[0001] This invention relates to the field of computer modeling technology, specifically to a GeoSemantic-GS framework and its application for scene surface reconstruction. Background Technology

[0002] With the rapid evolution of engineering applications such as digital twins, smart cities, real-world 3D rendering, and building asset digitization, the efficient reconstruction of high-fidelity, lightweight 3D scenes from multi-view images has become a core requirement in the fields of computer vision and graphics. Traditional Neural Radiation Field (NeRF) has achieved breakthroughs in novel perspective synthesis tasks through implicit volumetric rendering, but its massive multilayer perceptron query overhead severely limits training efficiency and real-time rendering capabilities. In recent years, 3D Gaussian Splashing (3DGS), as an explicit radiation field representation paradigm, has achieved real-time rendering frame rates while maintaining or even surpassing NeRF rendering quality thanks to its differentiable rasterization technology. This combination of high fidelity and high efficiency makes it demonstrate enormous application potential in the 3D digitization of large-scale industrial scenes.

[0003] However, when applying existing 3DGS technology directly to the structured reconstruction of complex real-world scenes, at least three key technical bottlenecks still exist:

[0004] First, there's the issue of unstructured primitive representation and geometric redundancy. Original 3DGS uses three-dimensional volumetric ellipsoids for scene modeling. This disordered, physically undefined discrete point cloud generates millions of redundant primitives when representing artificial structures (such as large, flat walls and floors). This not only leads to extremely high video memory and storage overhead but also causes severe surface distortion and depth noise in areas lacking texture.

[0005] Second, there is the problem of "global blindness" in existing geometric regularization strategies. To improve surface quality, some existing studies attempt to introduce two-dimensional surface elements or smoothing regularization terms to constrain the distribution of Gaussian elements. However, these methods typically employ global and isotropic constraint strategies, failing to perceive the underlying physical topology of the scene. Applying a uniform smoothing constraint to the entire scene often inevitably erases high-frequency geometric features (such as carvings and window frames) while repairing flat areas, making it difficult to achieve an effective balance between smoothness and detail fidelity.

[0006] Third, there is the vulnerability to dynamic interference. Unconstrained data acquisition in the real world inevitably includes dynamic interference such as pedestrians and moving vehicles. Existing 3DGS methods, lacking spatiotemporal consistency screening mechanisms, will incorrectly fit these transient objects as permanent floating artifacts, severely compromising the geometric purity of digital assets.

[0007] To address the geometric accuracy and efficiency challenges in surface reconstruction, existing technologies have explored various approaches. For example, some studies have attempted to introduce alignment losses or monocular depth and normal priors to anchor diffuse primitives to the real physical surface; others have further improved geometric stability under sparse observations through anchor-point-based representations. To fundamentally eliminate the "thickness expansion" defect caused by 3D ellipsoidal representations, subsequent research has driven the paradigm evolution towards 2D Gaussian surface elements, utilizing ray-surface element intersection and unbiased depth rendering techniques to ensure topological consistency and fidelity across multiple views. For the complexity of large-scale architectural scenes, some existing technologies utilize multi-view consistency criteria to achieve efficient primitive density management or introduce structured geometric constraints to enhance the regularity of planes in urban environments.

[0008] However, a common limitation of the aforementioned existing technologies is that their geometric regularization strategies are typically global and isotropic, relying on photometric consistency or pre-trained depth estimation models to impose uniform smoothing constraints on the entire scene. This makes them unable to adaptively handle the extremely uneven geometric complexity in the real world, creating an inherent contradiction between restoring flat regions and preserving high-frequency geometric details.

[0009] On the other hand, in terms of the fusion of semantics and radiation fields, existing technologies have transferred the capabilities of visual foundational models such as contrastive language-image pre-trained models and arbitrary object segmentation models to three-dimensional radiation fields. Early research demonstrated that neural radiation fields can achieve robust propagation of semantic labels in three-dimensional implicit space through multi-view consistency constraints. Subsequently, some studies have achieved three-dimensional spatial queries based on natural language instructions by embedding high-dimensional CLIP features in NeRF. With the development of 3DGS technology, research focus has further shifted to utilizing the explicit representation of Gaussian elements to carry complex semantic features. For example, combining mask prediction with SAM to achieve instance-level decoupling of Gaussian elements, or using feature distillation techniques to efficiently compress two-dimensional visual features into Gaussian element attributes.

[0010] However, the limitations of existing semantic radiation field technologies lie in their focus on "semantic feature enhancement," i.e., how to losslessly map features extracted from a two-dimensional base model to three-dimensional space to support downstream segmentation or retrieval tasks. In these frameworks, semantic labels are merely considered additional attributes attached to geometric representations, without participating in or interfering with the optimization and reconstruction process of the underlying geometry. Furthermore, traditional visual base models primarily perform pixel-level pattern matching, lacking the ability to perform deep logical reasoning about the three-dimensional physical properties of a scene (such as surface flatness and spatial structure orientation).

[0011] In summary, existing technologies for scene surface reconstruction lack a solution that can overcome the limitations of traditional low-level geometric optimization and effectively integrate macroscopic structural priors with physical common sense. They struggle to achieve an optimal balance between preserving complex details and simplifying large-area planes, and cannot proactively utilize semantic information to guide the geometric reconstruction process. Therefore, there is an urgent need in this field for a novel, high-fidelity, and lightweight framework for scene surface reconstruction. Summary of the Invention

[0012] The purpose of this invention is to at least solve one of the technical problems existing in the prior art, and to provide a GeoSemantic-GS framework and its application for scene surface reconstruction.

[0013] To achieve the above objectives, the technical solution adopted by this invention is as follows: a GeoSemantic-GS framework for scene surface reconstruction, comprising:

[0014] A semantic prior generation module is used to receive multi-source perceptual data streams, convert the data streams into a prior set, and output multimodal semantic priors and physical semantic priors;

[0015] The spatiotemporal filtering density control module has its input end connected to the output end of the semantic prior generation module. It is used to receive the multimodal semantic prior and physical semantic prior, and combine semantic masking and multi-view consistency evaluation to shield and remove dynamic interference data. At the same time, it adaptively controls the primitive density to curb the growth of redundant primitives and outputs the characterization data after dynamic interference removal and density optimization.

[0016] An asymmetric Gaussian generation module, whose input is connected to the output of the semantic prior generation module and the output of the spatiotemporal filter density control module, is used to receive the multimodal semantic prior, physical semantic prior and the optimized representation data, and perform asymmetric Gaussian surface parameter optimization driven by multimodal semantics based on the two-dimensional Gaussian surface element representation, and output the generated three-dimensional spatial data.

[0017] The laser vision deep fusion module has its input end connected to the output end of the semantic prior generation module, and its output end connected to the input end of the asymmetric Gaussian generation module, for providing physical-level geometric anchoring to the asymmetric Gaussian generation module.

[0018] Furthermore, the semantic prior generation module includes:

[0019] A multi-granularity scene deconstruction unit is used to convert multi-source sensing data streams into a tree-like data structure with spatial nesting relationships. It includes a multi-granularity mask generation subunit and an association tree construction subunit. The multi-granularity mask generation subunit is used to extract mask sets across granularities; the association tree construction subunit is used to construct a spatial association tree based on the area inclusion function to determine the parent node relationship between masks. ;

[0020] The multimodal physical attribute reasoning unit, connected to the multi-granularity scene deconstruction unit, is used to receive the association tree. It generates a structured prior for each node, which includes a joint input construct subunit for generating a hint for each node containing probability distribution features of the global 3D space, density field parameters of the local spatial neighborhood, and pose metadata.

[0021] The attribute reasoning decoding subunit uses the Qwen3.5 multimodal large model to output the structured attribute data of Gaussian surface elements;

[0022] The semantic propagation subunit performs top-down high-confidence prior inheritance along the parent-child edges of the association tree to correct the inference results of the bottom-level nodes;

[0023] Furthermore, the multi-granularity scene deconstruction unit provides a spatial skeleton and topological constraints for the multimodal physical attribute reasoning unit, and the multimodal physical attribute reasoning unit injects semantic priors into the multi-granularity scene deconstruction unit. The two work together to output a structured prior, which is indexed by the association tree, and each node in the association tree carries a set of multimodal attributes.

[0024] Furthermore, the multi-granularity mask generation subunit is configured to: employ a Semantic-SAM zero-shot segmentation model to extract a mask set spanning six granularity levels from macroscopic to microscopic, denoted as . ;

[0025] The association tree construction subunit is configured to: construct a multi-granularity spatial association tree. Among them, vertex set Corresponding mask set ;

[0026] For any two masks and Directed edge The weights are the area coverage function. :

[0027] ;

[0028] In the formula, A measure of the pixel area of ​​the mask; For mask and The intersection region; The preset area coverage threshold is determined if and only if and The level to which it belongs is higher than At that time, establish for The direct parent node.

[0029] Furthermore, the joint input construct subunit is configured as follows: for the association tree any node in Construct a joint input suggestion that includes global references and local details. ;in, Provides a global spatial reference in three-dimensional space. For based on Extracted local spatial feature set with background set to zero. This refers to the area ratio and parent node identifier extracted from the association tree;

[0030] The attribute reasoning decoding subunit is configured to: provide the joint input hints Inputting the data into a Qwen3.5 multimodal large model with a parameter size of 9B, conditional probability maximization is performed to obtain a structured attribute feature set. :

[0031] ;

[0032] In the formula, The optimal structured attribute feature set obtained is denoted as the corresponding output structured attribute feature set. ; For candidate structured attribute feature sets; Provide a prompt for the combined input; These are the pre-trained parameters for Qwen3.5; It is the conditional probability distribution function; , , , The structured property data of Gaussian surface elements;

[0033] The semantic propagation subunit is configured to utilize the association tree. The topological connectivity is used to perform Markov property coverage on micro-level nodes, where: if child nodes parent node satisfy And confidence level Then let the child nodes Directly inherit from the parent node The physical flatness prior.

[0034] Furthermore, the spatiotemporal filter density control module includes:

[0035] The semantic masking and gradient truncation unit is configured to: based on temporal sampling data obtained from multimodal semantic priors, use the Qwen3.5 model to determine transient interference of data points in the feature space, and generate a spatiotemporally consistent binary masking mask. ,Should ;

[0036] In the differentiable rendering and backpropagation stages of 3D Gaussian splashing, the mask is directly applied to the photometric loss function. Calculate the photometric loss function after shielding. :

[0037] ;

[0038] In the formula, A spacetime consistent binary mask; This is the original photometric loss function; Represents the Hadama product;

[0039] The density control unit, whose input is connected to the output of the semantic mask and gradient truncation unit, is used to receive the clean rendering features after masking and the corresponding photometric residual information, and is configured to perform multi-view photometric residual accumulation densification and perform primitive pruning with rendering contribution awareness.

[0040] Furthermore, performing multi-view photometric residual accumulation densification includes: for any candidate Gaussian element in space... Random sampling during training Spatial observation pose parameter points ; Calculate the primitive's projected footprint Multi-view average photometric residual contribution :

[0041]

[0042] In the formula, The number of randomly sampled spatial observation pose parameter points; For spatial observation pose parameter point index; For projected footprints Projection data sampling points within; For the candidate Gaussian element Projected footprints; It is an indicator function; For the first Pixels from each viewpoint Normalized high-error distribution mask; This is the error sensitivity threshold; For the first Sampling points of projection data under spatial observation pose parameter points of Photometric reconstruction error; The density threshold is set only when the cumulative residual... Strictly greater than the denser threshold At that time, it triggers primitive splitting;

[0043] Introducing physical semantic priors as an asymmetric threshold modulator: punitive threshold increases are applied to regions judged as FLAT_SURFACE. To suppress primitive growth in planar regions; while maintaining the original threshold for COMPLEX_GEOMETRY regions. To capture high-frequency geometric details.

[0044] Furthermore, the rendering contribution-aware primitive pruning includes defining a comprehensive impact factor for each Gaussian primitive on the global multi-view reconstruction quality. :

[0045] ;

[0046] In the formula, It is a nonlinear normalization operator; The number of randomly sampled spatial observation pose parameter points; For spatial observation pose parameter point index; For projected footprints Projection data sampling points within; The projected footprint of the candidate Gaussian element; It is an indicator function; For the first Sampling points of projection data under spatial observation pose parameter points Normalized high-error distribution mask; This is the error sensitivity threshold; For the first Sampling points of projection data under spatial observation pose parameter points of Photometric reconstruction error; The preset pruning threshold is used; when the contribution of the primitive... Below the preset pruning threshold At that time, a forced removal will be performed.

[0047] Furthermore, the asymmetric Gaussian generation module includes: a two-dimensional Gaussian surface element representation unit, configured to: employ dimensionality-reduced 2D Gaussian surface elements in three-dimensional space, such that each surface element degenerates into an oriented planar disk; wherein, the first... The geometric state of a Gaussian surface element is determined by its center point. Normal vector Two-dimensional scale vector And the joint definition of a local coordinate system composed of two orthogonal tangent vectors;

[0048] For any point in space The impact is expressed as follows:

[0049] ;

[0050] In the formula, For the first The center point of a Gaussian surface element; For the first The rotation matrix of Gaussian surface elements, and Its third column strictly corresponds to the normal vector of the surface element. ; Flattened covariance matrix The inverse matrix, and ,in and Two-dimensional scale vector The amount, It is a very small positive number used to represent the surface element thickness to ensure the invertibility and numerical stability of the covariance matrix;

[0051] The oriented plane regularization unit, whose input is connected to the output of the two-dimensional Gaussian surface element representation unit, is used to obtain the normal vector of each surface element. and semantic tags And it is configured to: introduce a global gravity direction. Construct directional hard-constraint operators based on semantic categories Apply orthogonal constraints to vertical surface elements Apply parallel constraints to horizontal surface elements ;

[0052] Calculate the directional plane regularization loss function :

[0053] ;

[0054] In the formula, It is a set of Gaussian surface elements; Let be the index of the Gaussian surface element, and ; For the first Semantic mask weights of Gaussian surface elements; For the first Confidence level of physical flatness of a Gaussian surface element; A directional hard constraint operator constructed based on semantic categories;

[0055] The constraint relaxation and topology detail preservation unit has its first input connected to the output of the two-dimensional Gaussian surface element representation unit to obtain geometric state parameters and radiation properties, and its second input connected to the output of the oriented plane regularization unit to obtain the oriented plane regularization loss function. Confidence of flatness Its third input terminal receives multi-view photometric error signals. Structural similarity loss function and the local normal consistency loss function This unit is configured to construct the joint objective function. :

[0056] ;

[0057] In the formula, Let be the directional plane regularization loss function; For use in balancing Preset weighting coefficients for photometric error loss and structural similarity loss; The loss weights are used to control the strength of local normal consistency constraints. These are the loss weight coefficients used to adjust the strength of the plane regularization constraint;

[0058] For component regions identified as COMPLEX_GEOMETRY by the multimodal large model, their corresponding flatness confidence scores are used. Spontaneous blocking The effect of the term on this region frees the Gaussian elements in this region from orthogonality / parallelism constraints, making them completely driven by multi-view photometric errors; for non-complex geometric regions, it maintains complete planar regularization constraints.

[0059] Furthermore, the laser vision deep fusion module includes a multi-source point cloud registration and geometric attribute joint initialization unit, configured to: uniformly align the visual sparse point cloud estimated by COLMAP and the LiDAR dense point cloud to an absolute metric coordinate system, and construct a fused point cloud set. For any point in the set Utilizing its local spatial neighborhood Constructing the covariance matrix :

[0060] ;

[0061] In the formula, For any point in the merged point cloud set; For point The local spatial neighborhood; | | represents the number of neighboring points in the local spatial neighborhood; These are the neighboring points within the local spatial neighborhood; Represents the transpose operation of a matrix or vector;

[0062] Principal component analysis (PCA) was used to select... The eigenvector corresponding to the smallest eigenvalue is used as the initial normal vector. And construct the rotation matrix to make its local The axis is strictly parallel to the normal. Simultaneously, the initial two-dimensional scale vector of the Gaussian surface element is... The constraints are:

[0063] ;

[0064] In the formula, and The initial two-dimensional scale vector Scale components; For the local spatial neighborhood The number of neighboring points in the array, and satisfying ; For any point in the fused point cloud set; These are the neighboring points within the local spatial neighborhood; express Norm;

[0065] The forced pruning unit has its input connected to the output of the multi-source point cloud registration and geometric attribute joint initialization unit, and periodically receives the center point positions of all current Gaussian units during training. It is configured as: the center of each primitive in the computation space To LiDAR physical surface point cloud Shortest absolute Euclidean distance :

[0066] ;

[0067] In the formula, For the current number The center point position vector of each Gaussian element; This is the reference point position vector in the LiDAR physical surface point cloud; Point cloud of the physical surface of the LiDAR; express Norm;

[0068] Based on this shortest absolute Euclidean distance, a hard space elimination indicator function is defined. :

[0069] ;

[0070] In the formula, It is an indicator function; The preset physical tolerance threshold; once The framework ignores the contribution of this primitive to the multi-view photometric residual and performs a forced deletion operation.

[0071] Based on the above application of the GeoSemantic-GS framework for scene surface reconstruction, the GeoSemantic-GS framework is applied to the structured reconstruction of complex real-world scenes to generate a structured 3D Gaussian scene model.

[0072] Compared with the prior art, the technical solution of this application has the following beneficial effects:

[0073] 1. Significantly Improved Reconstruction Accuracy and Physical Realism: This application deeply integrates the high-order spatial perception and causal reasoning capabilities of the Qwen3.5 model. The framework can perform spatial coordinate calculation in a two-dimensional projection space with zero samples, accurately resolving the spatial orientation, occlusion relationships, and physical flatness of components in the Atlanta-world model (absolute plane vs. complex topology). This analytical result is injected as an explicit prior into the subsequent generation process, fundamentally avoiding the orientation ambiguity and geometric distortion caused by traditional methods that rely solely on geometric features or simple label matching, making the geometric surface more closely resemble the real physical structure.

[0074] 2. Achieving "Macroscopic Smoothness, Microscopic Fidelity" Adaptive Production: This application's framework innovatively transforms semantic priors into an asymmetric regularization weight matrix: Strict parallel / orthogonal constraints are applied to regions determined to be flat, achieving primitive flattening and adaptive pruning, effectively suppressing the proliferation of redundant primitives on flat surfaces; geometric constraints are released for complex detail regions, preserving the degree of freedom for primitives to capture high-frequency topology. This mechanism breaks through the limitations of traditional global smoothing regularization's "one-size-fits-all" approach, ensuring the simplicity and purity of large-scale structures while not losing or even enhancing the ability to restore local details.

[0075] 3. Significantly reduced number of 3D primitives, improving representation compactness: This application benefits from asymmetric optimization and multi-view density control (an adaptive primitive addition and deletion strategy that integrates photometric residuals and semantic confidence). The GeoSemantic-GS framework compresses the number of 3D primitives in the scene by several orders of magnitude without sacrificing rendering quality. This significantly reduces memory overhead and computational complexity, providing a foundation for efficient storage, transmission, and real-time rendering of large-scale digital twin scenes.

[0076] 4. Effectively eliminates geometric artifacts caused by dynamic interference: The framework of this application incorporates a spatiotemporal filtering density control module, which eliminates the negative impact of transient interference sources by truncating feature-level gradients during the initial training stage. Compared to traditional methods that rely on later filtering or robust loss functions, this mechanism prevents transient objects from contaminating the geometric parameter estimation at the source, significantly improving the representation accuracy and stability of static structures.

[0077] 5. Supports industrial-grade absolute accuracy requirements (pluggable LiDAR module): This application framework provides a high-precision extension module based on joint penalty of tangential plane distance and primitive anisotropic shape for multimodal data equipped with LiDAR. This module effectively eliminates the absolute scale geometric drift common in monocular or weakly supervised methods, meeting the stringent requirements for physical scale accuracy in fields such as industrial digital twins and architectural surveying. Furthermore, this module can be flexibly plugged in and out according to the data source, exhibiting excellent scalability.

[0078] 6. Achieving highly automated and efficient semantic attribute allocation: This application utilizes a multi-granularity spatial association tree and the hierarchical topology propagation mechanism of Qwen3.5. The framework can automatically propagate the physical flatness and structural orientation priors resolved from multimodal semantic priors to all child nodes along the tree structure from top to bottom. This process eliminates the need for manual annotation of each component, significantly reducing the cost and time of semantic attribute acquisition in large-scale scene reconstruction and achieving end-to-end automated prior generation. Attached Figure Description

[0079] Figure 1 This is a schematic diagram of the GeoSemantic-GS framework for scene surface reconstruction according to the present invention;

[0080] Figure 2 This is a sparse point cloud and camera pose parameter map of a typical scene from a self-built dataset (Geo-Wild dataset) obtained by COLMAP using the framework of this invention.

[0081] Figure 3 This is a qualitative comparison chart of the framework of this invention on the Mip-NeRF360, Tanks and Temples datasets;

[0082] Figure 4This is a qualitative comparison graph of the framework of this invention on the Geo-Wild dataset;

[0083] Figure 5 This is an overall effect diagram of the 3D mesh reconstruction of 15 typical scenes using the framework of this invention on the DTU dataset;

[0084] Figure 6 This is a qualitative comparison of the 3D mesh reconstruction of the framework of this invention and the comparison method on the Mip-NeRF360 dataset;

[0085] Figure 7 This is a qualitative comparison of the framework of this invention and the comparative method on the Tanks and Temples dataset for 3D mesh reconstruction;

[0086] Figure 8 This is a qualitative comparison of the 3D mesh reconstruction of the framework of this invention and the comparison method on the self-built Geo-Wild dataset. Detailed Implementation

[0087] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0088] like Figure 1 As shown, a preferred embodiment of the present invention, the GeoSemantic-GS framework for scene surface reconstruction, includes:

[0089] A semantic prior generation module is used to receive multi-source perceptual data streams, convert the data streams into a prior set, and output multimodal semantic priors and physical semantic priors;

[0090] This module aims to overcome the limitation of traditional 3D visual models in digital twin construction, which can only extract geometric features. It proposes an end-to-end structured prior generation engine. This engine transforms multi-source perceptual data streams into a prior set rich in spatial topology, physical properties, and domain-specific semantics, providing strong constraint guidance for subsequent asymmetric geometric reconstruction.

[0091] The semantic prior generation module includes:

[0092] The multi-granularity scene deconstruction unit based on hierarchical association tree is used to convert multi-source perception data streams into tree-like data structures with spatial nesting relationships, including multi-granularity mask generation sub-units, which extract mask sets across granularities;

[0093] Specifically, while existing zero-shot segmentation paradigms (such as Semantic-SAM) possess excellent pixel-level perception capabilities, their output discrete masks lack representation of the physical relationships and spatial topology of building components. To address this, this application deploys Semantic-SAM as a front-end multi-granularity mask generation subunit, extracting a mask set spanning six granularity levels (Level 1-6) from macroscopic to microscopic perspectives. .

[0094] The spatial association tree is constructed by building sub-units and determining the parent node relationship between masks based on the area inclusion function. ;

[0095] Specifically, in order to explicitly model the spatial nesting relationships between components (e.g., window lattices embedded in window frames, and window frames attached to the main wall), this application introduces a multi-granularity spatial association tree based on graph theory. Among them, vertex set Corresponding mask set ;

[0096] For any two masks and Directed edge The weights are the area coverage function. :

[0097] ;

[0098] In the formula, The area coverage between any two masks; A measure of the pixel area of ​​the mask; For mask and The intersection region; , For any two masks; The preset area coverage threshold is determined if and only if and The level to which it belongs is higher than At that time, establish for The direct parent node. This association tree reconstructs the discrete two-dimensional mask into a tree-like data structure with global topological consistency, providing crucial spatial context constraints for subsequent multimodal inference.

[0099] The multimodal physical attribute reasoning unit based on Qwen3.5 is connected to the multi-granularity scene deconstruction unit and is used to receive the association tree. It also generates structured priors for each node. Specifically, given the severe semantic deficiencies of general vision models in architectural and industrial scenarios, this application introduces the Qwen3.5 multimodal large model with a parameter scale of 9B to perform deep physical attribute inference units. Compared to earlier visual language models, Qwen3.5 possesses lossless text-vision fusion and advanced spatial perception capabilities, making it an ideal bridge connecting two-dimensional visual masks and three-dimensional geometric attributes.

[0100] This unit includes a joint input construction subunit, which generates a hint for each node containing probability distribution features of the global 3D space, density field parameters of the local spatial neighborhood, and pose metadata.

[0101] Specifically, for the association tree any node in Construct a joint input suggestion that includes global references and local details. ;in, Provides a global spatial reference in three-dimensional space. For based on Extracted local spatial feature set with background set to zero. This refers to the area ratio and parent node identifier extracted from the association tree;

[0102] The attribute reasoning decoding subunit uses the Qwen3.5 multimodal large model to output the structured attribute data of the Gaussian surface element. The structured attribute data of the Gaussian surface element includes spatial location, material, Atlanta orientation and geometric complexity.

[0103] Specifically, this attribute reasoning and decoding process is modeled as a conditional probability maximization problem, through the joint input prompts. Inputting the data into a Qwen3.5 multimodal large model with a parameter size of 9B, conditional probability maximization is performed to obtain a structured attribute feature set. :

[0104] ;

[0105] In the formula, The optimal structured attribute feature set obtained is denoted as the corresponding output structured attribute feature set. ; For candidate structured attribute feature sets; Provide a prompt for the combined input; These are the pre-trained parameters for Qwen3.5; It is the conditional probability distribution function; , , , These are spatial location, material, Atlanta orientation, and geometric complexity;

[0106] The semantic propagation subunit performs top-down high-confidence prior inheritance along the parent-child edges of the association tree to correct the inference results of the bottom-level nodes;

[0107] Specifically, when processing micro-components at Level 5-6, the extremely local receptive field often leads to observational uncertainty in MLLM (Multimodal Large Language Model), such as misclassifying a shadow on a flat wall as a complex geometric shape. To address this, this application utilizes a relational tree... The topological connectivity was used to design semantic propagation subunits. For child nodes... If its parent node satisfy And confidence level This forces the execution of Markov-style property overriding, making direct inheritance The physical flatness prior is used. This sub-unit not only theoretically filters out classification noise caused by missing underlying visual features, but also avoids redundant autoregressive inference calculations in engineering implementation, providing a robust guarantee for efficient and high-precision labeling in large-scale digital twin scenarios.

[0108] This reasoning process also heavily relies on Qwen3.5's core enhancements, including upgraded visual recognition and causal logic reasoning. Thanks to its massive pre-trained data, Qwen3.5 can overcome the limitations of traditional models in recognizing architectural terminology, accurately identifying microscopic building materials (such as microporous concrete and weathered stone cracks). Furthermore, its enhanced multimodal logic reasoning capabilities can penetrate complex visual interference, accurately performing causal analysis to determine whether the texture is visual noise or genuine geometric undulations. .

[0109] Advanced Spatial Perception and 3D Grounding: Qwen3.5 possesses superior viewpoint and occlusion detection capabilities. This enables the model to infer the absolute spatial orientation of components within the Atlanta-world model using zero-shot analysis of 2D images. This provides a decisive three-dimensional spatial anchor for subsequent directional geometric regularization.

[0110] Therefore, the multi-granularity scene deconstruction unit provides a spatial skeleton and topological constraints for the multimodal physical attribute reasoning unit, and the multimodal physical attribute reasoning unit injects semantic priors into the multi-granularity scene deconstruction unit. The two work together to output a structured prior, which is indexed by the association tree, and each node in the association tree carries a set of multimodal attributes.

[0111] The spatiotemporal filtering density control module has its input end connected to the output end of the semantic prior generation module. It is used to receive the multimodal semantic prior and physical semantic prior, and combine semantic masking and multi-view consistency evaluation to shield and remove dynamic interference data. At the same time, it adaptively controls the primitive density to curb the growth of redundant primitives and outputs the characterization data after dynamic interference removal and density optimization.

[0112] Specifically, in real-world general scene reconstruction, the presence of dynamic objects (such as pedestrians and vehicles) severely undermines the multi-view geometric consistency assumption of 3DGS optimization, leading to floating artifacts and geometric distortions in the reconstruction results. Furthermore, traditional primitive density control strategies tend to induce redundant computation in weakly textured regions. Therefore, this application proposes an adaptive optimization strategy combining semantic masking and multi-view consistency evaluation to achieve high-fidelity scene reconstruction and refined allocation of computational resources.

[0113] To address the topology disruption problem caused by dynamic interference, this application employs an efficient pixel-level gradient truncation mechanism, namely a semantic mask and gradient truncation unit for dynamic interference. This unit is configured to: based on temporal sampling data obtained from multimodal semantic priors, use the Qwen3.5 model to determine transient interference in the feature space of data points, generating a spatiotemporally consistent binary mask. ,Should ;

[0114] In the differentiable rendering and backpropagation stages of 3D Gaussian Splash (3DGS), the mask is directly applied to the photometric loss function. ,calculate :

[0115] ;

[0116] In the formula, A spacetime consistent binary mask; This is the original photometric loss function; This represents the Hadamard product; by forcibly cutting off the gradient backpropagation of pixels containing dynamic interference in the image space, the optimization engine fundamentally eliminates the driving force for generating Gaussian units at these spatial locations. This mechanism not only has extremely low computational overhead, but also implicitly suppresses erroneous geometric generation of occluded areas by relying on the multi-view co-view information of the surrounding static background, effectively ensuring the purity of the static background.

[0117] Traditional 3DGS relies on view-space positional gradients from a single perspective for splitting and cloning Gaussian primitives. In edge regions after removing dynamic interference or in large, weakly textured areas of architecture, this local heuristic strategy easily leads to the disorderly proliferation of redundant primitives. To address this, this framework proposes a primitive addition and deletion mechanism based on "multi-view rendering contribution," namely a semantically aware multi-view consistency density control unit, which is deeply coupled with the physical semantic prior generated by the semantic prior generation module.

[0118] The input of this unit is connected to the output of the semantic mask and gradient truncation unit of the dynamic interference, and is used to receive the clean rendered image after masking and the corresponding photometric residual information. It is configured to perform multi-view photometric residual accumulation and densification to avoid the blind growth of primitives and to perform primitive pruning that is aware of rendering contribution, so as to maintain the extreme lightweight of the model.

[0119] Performing multi-view photometric residual accumulation densification includes: for any candidate Gaussian element in space Random sampling during training Spatial observation pose parameter points Calculate the primitive's projected footprint. Multi-view average photometric residual contribution :

[0120]

[0121] In the formula, The number of randomly sampled spatial observation pose parameter points; For spatial observation pose parameter point index; For projected footprints Projection data sampling points within; For the candidate Gaussian element Projected footprints; It is an indicator function; For the first Sampling points of projection data under spatial observation pose parameter points Normalized high-error distribution mask; This is the error sensitivity threshold; For the first Sampling points of projection data under spatial observation pose parameter points of Photometric reconstruction error; The density threshold is set only when the cumulative residual... Strictly greater than the denser threshold At that time, it triggers primitive splitting;

[0122] More importantly, the physical semantic prior is introduced as an asymmetric threshold regulator: a punitive threshold increase is applied to regions judged as FLAT_SURFACE. To suppress primitive growth in planar regions; while maintaining the original threshold for COMPLEX_GEOMETRY regions. To capture high-frequency geometric details.

[0123] Performing rendering contribution-aware primitive pruning includes defining a comprehensive impact factor for each Gaussian primitive on the global multi-view reconstruction quality. :

[0124] ;

[0125] In the formula, It is a nonlinear normalization operator; The number of randomly sampled spatial observation pose parameter points; For spatial observation pose parameter point index; For projected footprints Projection data sampling points within; The projected footprint of the candidate Gaussian element; It is an indicator function; For the first Sampling points of projection data under spatial observation pose parameter points Normalized high-error distribution mask; This is the error sensitivity threshold; For the first Sampling points of projection data under spatial observation pose parameter points of Photometric reconstruction error; The preset pruning threshold is used; when the contribution of the primitive... Below the preset pruning threshold At that time, forced removal is performed, thereby eliminating the geometric noise remaining after the masking operation through the combined effect of multi-view consistency checks and semantic prior constraints, and concentrating computational resources on high-complexity topological regions. This combination of multi-view consistency checks and semantic prior constraints not only automatically eliminates the scattered geometric noise that may remain after the masking operation, but also precisely allocates core computational resources to high-complexity topological regions, achieving bidirectional optimization of reconstruction fidelity and model storage overhead.

[0126] An asymmetric Gaussian generation module, whose input is connected to the output of the semantic prior generation module and the output of the spatiotemporal filter density control module, is used to receive the multimodal semantic prior, physical semantic prior and the optimized representation data, and perform asymmetric Gaussian surface parameter optimization driven by multimodal semantics based on the two-dimensional Gaussian surface element representation, and output the generated three-dimensional spatial data.

[0127] Specifically, traditional 3DGS uses volumetric anisotropic ellipsoidal representation. When processing structured environments (such as interiors and large building facades), due to the lack of explicit surface definitions and geometric constraints, it is prone to producing depth blur and "wavy" artifacts in areas lacking texture. To address this issue, this framework proposes an asymmetric reconstruction optimization mechanism driven by multimodal semantics, based on 2D Gaussian surface elements.

[0128] This module includes two-dimensional Gaussian surface element representation units. To achieve a high-precision explicit representation of building surface geometry, this application abandons the 3D Gaussian surface element with three-dimensional volume and instead adopts a dimension-reduced 2D Gaussian surface element. In three-dimensional space, the dimension-reduced 2D Gaussian surface element is used, causing each surface element to degenerate into an oriented planar disk; wherein, the first... The geometric state of a Gaussian surface element is determined by its center point. Normal vector Two-dimensional scale vector And the joint definition of a local coordinate system composed of two orthogonal tangent vectors;

[0129] For any point in space The impact is expressed as follows:

[0130] ;

[0131] In the formula, Let be any three-dimensional point in space; For the first The center point of a Gaussian surface element; For the first The rotation matrix of Gaussian surface elements, and Its third column strictly corresponds to the normal vector of the surface element. ; Flattened covariance matrix The inverse matrix, and ,in and Two-dimensional scale vector The amount, It is a very small positive number used to represent the surface element thickness to ensure the invertibility and numerical stability of the covariance matrix; this unit outputs the geometric state parameters, normal vector and radiation properties of each surface element.

[0132] Compared to a 3D ellipsoid, this representation utilizes normal vectors. The local tangent planes of the scene are explicitly modeled, eliminating the surface thickness expansion caused by volume rendering from the underlying data structure, and providing a precise algebraic interface for introducing higher-order plane regularization.

[0133] To overcome the geometric ambiguity problem in large-scale low-texture regions, the oriented plane regularization unit based on the Atlanta-world prior connects its input to the output of the two-dimensional Gaussian surface element representation unit to obtain the normal vector of each surface element. and semantic tags And configured to: introduce global gravity direction Construct directional hard-constraint operators based on semantic categories Apply orthogonal constraints to the wall to force the Gaussian surface element normal to be perpendicular to the direction of gravity:

[0134] ;

[0135] Apply parallel constraints to the ground or ceiling to force the Gaussian surface element normals to be collinear with the direction of gravity:

[0136] ;

[0137] Calculate the first Oriented plane regularization penalty function for Gaussian surface elements :

[0138] ;

[0139] In the formula, It is a set of Gaussian surface elements; Let be the index of the Gaussian surface element, and ; For the first Semantic mask weights of Gaussian surface elements; For the first Confidence level of physical flatness of a Gaussian surface element; For the directional hard-constraint operator constructed based on semantic categories, a square metric is used to ensure the smooth differentiability of the optimization;

[0140] Through this semantically weighted asymmetric penalty function, the optimization engine can apply a strong planar torsional force to the macroscopic building structure during the training phase, forcing the scattered Gaussian disks to spontaneously align and adhere to the ideal building physical plane.

[0141] While achieving strong geometric constraints in macroscopically flat regions, preventing excessive smoothing of high-frequency geometric details is a challenge in high-fidelity digital twin reconstruction. To address this, this application proposes an asymmetric optimization mechanism for constraint relaxation, namely an asymmetric optimization constraint relaxation and topology detail preservation unit. Its first input is connected to the output of the two-dimensional Gaussian surface element representation unit to obtain geometric state parameters and radiation properties, and its second input is connected to the output of the oriented plane regularization unit to obtain the oriented plane regularization loss function. Confidence of flatness Its third input terminal receives multi-view photometric error signals. Structural Similarity (SSIM) Loss Function and the local normal consistency loss function This unit is configured to construct the joint objective function. :

[0142] ;

[0143] In the formula, The directional plane regularization loss function; For use in balancing Preset weighting coefficients for photometric error loss and structural similarity loss; The loss weights are used to control the strength of local normal consistency constraints. This is the loss weight coefficient used to adjust the strength of the plane regularization constraint.

[0144] For component areas classified as COMPLEX_GEOMETRY by the multimodal large model (such as carved or openwork window frames with three-dimensional undulations), their flatness is a priori. At this point, in the equation The term is spontaneously blocked, and the Gaussian elements in this region are completely freed from the rigid orthogonal / parallel constraints. They are fully controlled by the multi-view photometric error and can freely rotate and scale evolve in the local three-dimensional space.

[0145] This "semantic-geometric" feedback loop establishes an asymmetric scene reconstruction paradigm: the model precisely tilts the limited parameter degrees of freedom to the high-frequency topological region, thereby achieving smooth noise reduction of large-area backgrounds at the macro level and ensuring topological fidelity of complex artistic features at the micro level.

[0146] The laser vision deep fusion module has its input end connected to the output end of the semantic prior generation module, and its output end connected to the input end of the asymmetric Gaussian generation module, for providing physical-level geometric anchoring to the asymmetric Gaussian generation module.

[0147] Specifically, although purely visual multimodal priors can provide excellent reconstruction guidance, the presence of extremely weak texture regions in large-scale industrial mapping scenarios still poses significant challenges to geometric initialization and denoising. To address this, this framework designs a plug-and-play multimodal data fusion extension module for digital twin datasets equipped with multiple sensors such as LiDAR. This module introduces absolute physical measures from LiDAR at the initialization starting point and primitive pruning stages, significantly improving the geometric accuracy and representational compactness of large-scale scenes with extremely low computational cost.

[0148] Traditional 3D Gaussian splashing relies heavily on sparse point clouds generated by Structure-from-Motion (SfM) for initialization. This approach is prone to geometric blind spots when dealing with weakly textured building surfaces, causing the engine to blindly proliferate primitives during the "cold start" phase. To address this, this module first aligns the visually sparse point cloud estimated by COLMAP with the LiDAR dense point cloud to an absolute metric coordinate system, constructing a fused point cloud set. .

[0149] More importantly, unlike conventional initialization strategies that only utilize point cloud spatial coordinates, this framework proposes a high-order geometric attribute joint initialization mechanism, namely, a multi-source point cloud registration and geometric attribute joint initialization unit. For two-dimensional Gaussian surface elements, normals and scale priors are directly injected into them during the initialization phase. For any point in the set... Utilizing its local spatial neighborhood Constructing the covariance matrix :

[0150] ;

[0151] In the formula, For point The local spatial neighborhood; | | represents the number of neighboring points in the local spatial neighborhood; These are the neighboring points within the local spatial neighborhood; Represents the transpose operation of a matrix or vector;

[0152] Principal component analysis (PCA) was used to select... The eigenvector corresponding to the smallest eigenvalue is used as the initial normal vector. And construct the rotation matrix to make its local The axis is strictly parallel to the normal. Simultaneously, the initial two-dimensional scale vector of the Gaussian surface element is... The constraints are:

[0153] ;

[0154] In the formula, and The initial two-dimensional scale vector Scale components; For the local spatial neighborhood The number of neighboring points in the array, and satisfying ; For any point in the fused point cloud set; These are the neighboring points within the local spatial neighborhood; express Norm;

[0155] This unit outputs the initialized geometric properties of the Gaussian surface element (center point, normal, rotation matrix, and 2D scale), which serve as the input for the subsequent asymmetric Gaussian generation module.

[0156] This joint initialization strategy ensures that Gaussian surface elements are "fitted" to the physical surface with the correct spatial orientation and adaptive scale from the very beginning of training. Combined with the asymmetric Gaussian generation module, the optimization engine completely bypasses the long and error-prone morphological evolution stage, achieving extremely rapid convergence to the real building boundary.

[0157] Besides the enhancements in the initialization phase, ensuring the ultimate purity and lightweight nature of the final 3D model is another core requirement for industrial-grade deployment. Traditional 3DGS pruning strategies mainly rely on the opacity characteristics of primitives, making it difficult to effectively remove persistent floating artifacts in free space. To address this issue, this framework introduces a mandatory spatial pruning rule based on LiDAR absolute physical boundaries. This mandatory pruning unit, based on absolute physical boundaries, has its input connected to the output of the multi-source point cloud registration and geometric attribute joint initialization unit. Since LiDAR scanning provides a rigorous truth value of the scene's physical shell, this application uses it as the ultimate spatial verifier for Gaussian primitive survival. During the periodic Adaptive Density Control phase, the center of each 2D Gaussian primitive in the system's computational space... To LiDAR physical surface point cloud Shortest absolute Euclidean distance :

[0158] ;

[0159] In the formula, For the current number The center point position vector of each Gaussian element; The reference point position vector in the LiDAR physical surface point cloud; Point cloud of the physical surface of the LiDAR; express Norm;

[0160] Based on this shortest absolute Euclidean distance, a hard space elimination indicator function is defined. :

[0161] ;

[0162] In the formula, It is an indicator function; The preset physical tolerance threshold; once The framework ignores the contribution of the primitive to the multi-view photometric residual and performs a forced deletion operation. This mechanism based on physical rigid boundaries achieves a "hard fallback" noise reduction method with extremely simple mathematical form and extremely low computational overhead, further establishing the extreme lightweight advantage of this framework in the reconstruction of large-scale digital twin assets.

[0163] The multi-source point cloud registration and geometric attribute joint initialization unit provides an absolute physical boundary reference point cloud for the forced pruning unit. The output of the forced pruning unit is fed back to the Gaussian reconstruction module to remove geometric noise that is detached from the real topology.

[0164] 1. Experiment setup.

[0165] To quantitatively evaluate the performance of the proposed GeoSemantic-GS framework in engineering applications such as digital twins and structural mapping, this application conducted a series of experiments on several representative real-world benchmark datasets.

[0166] 1.1 Dataset.

[0167] To rigorously and comprehensively evaluate the effectiveness and generalization ability of the proposed GeoSemantic-GS framework, this application conducted detailed experiments on three widely used public 3D reconstruction benchmark datasets and a self-built natural scene dataset.

[0168] For quantitative evaluation of basic geometric accuracy, this application uses the DTU dataset. This multi-view stereo vision (MVS) dataset contains a series of object-centered scanned scenes and provides extremely high-precision camera poses and ground truth 3D point clouds. This application selects 15 widely used scenes, initializes them with sparse point clouds generated using Colmap, and uniformly downsamples the image resolution to 800×600 to improve computational efficiency. This dataset is specifically used to rigorously test the error level of microscopic high-precision geometric surface reconstruction. For evaluation of large-scale macroscopic structures, this application uses the Tanks and Temples dataset. This dataset covers complex real-world scenes such as large vehicles, buildings, and sculptures, and is equipped with ground truth point clouds acquired by high-precision laser scanners. Due to its large physical scale and highly challenging topological structures, this dataset is primarily used to verify the robustness of this framework in capturing large-scale spatial topology and macroscopic physical boundaries. Furthermore, for evaluation of rendering quality, this application introduces the Mip-NeRF360 dataset. This dataset contains 360-degree panoramic surround complex scenes captured in an unconstrained environment. Due to the lack of surface geometry ground truth, this application mainly uses it to qualitatively and quantitatively evaluate the performance and photometric fidelity of the framework in complex multi-view environments for novel view synthesis.

[0169] In addition to the aforementioned public benchmarks, to further verify the engineering application potential of this framework in real-world, unconstrained environments, this application introduces a novel self-built dataset—the Geo-Wild dataset. For example... Figure 2 As shown, Figure 2 This paper presents sparse point clouds, camera spatial pose parameters, and corresponding measured reference images of some typical test scenes (scene numbers: 18, 19, 20, 22, 23, 24) in this self-built dataset, obtained through the COLMAP reconstruction algorithm during the preprocessing stage of this application. Figure 1The camera parameter distribution clearly shows that this dataset contains multiple sequences of natural scenes captured in real-world conditions, comprehensively covering scenes with dynamic interference (such as pedestrians and vehicles) and purely static scenes. Unlike traditional photogrammetry, which often relies on strict gridded path planning to ensure extremely high image overlap, the Geo-Wild dataset employs more flexible and free unstructured trajectories for image acquisition. This unconventional acquisition method is closer to the actual operating habits of ordinary users, but it also naturally leads to sparse viewpoints and insufficient image overlap in some areas of the scene. By introducing this self-built dataset, this application focuses on testing the multimodal semantic priors and spatial constraint mechanisms upon which this framework relies, examining whether it can still exhibit strong system robustness and reliably reconstruct high-quality 3D surfaces with topological integrity and visual purity when faced with non-ideal data acquisition conditions (i.e., the dual interference of irregular low overlap and dynamic noise).

[0170] 1.2 Evaluation Indicators.

[0171] To comprehensively evaluate the competitive advantages of the proposed method, this application establishes an evaluation system based on three dimensions: rendering fidelity, geometric accuracy, and model compactness. Regarding rendering fidelity, Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Perceptual Patch Similarity (LPIPS) are used to objectively assess the visual quality of the synthesized view.

[0172] In terms of geometric accuracy, the F1 score and Chamfer Distance were used to rigorously measure the spatial error between the reconstructed geometric surface and the real data.

[0173] In addition, to fully demonstrate the feasibility of this framework in industrial-grade lightweight deployment, this application focuses on introducing indicators to represent compactness and engineering efficiency, specifically recording the total number of the final generated 3D Gaussian primitives, the number of triangular faces and vertices in the derived mesh model, and the physical storage size (MB) of the overall scene model.

[0174] 1.3 Implementation and Training Details.

[0175] All experiments in this application were implemented using the PyTorch deep learning framework and run on an NVIDIA RTX 4090 GPU equipped with 24GB of VRAM. The hyperparameter configuration throughout the optimization process largely followed the basic settings of the original 3DGS, with the total number of training iterations for all scenes uniformly set to 30,000. In the geometry reconstruction stage, the system first renders depth maps for each training viewpoint from the optimized Gaussian representation, and then integrates these depth priors into a global voxel mesh using the truncated signed distance fusion (TSDF Fusion) algorithm. During this process, the voxel resolution was finely set to 0.004, the truncation distance was defined according to the absolute scale of different scenes, and the moving cubes algorithm was finally used to extract the structured mesh. Furthermore, for stages involving multimodal large model parsing, the system uniformly calls the Qwen3.5-9B model, setting its visual inference temperature parameter to 0.2 to ensure deterministic output. Simultaneously, the spatial association tree is pre-computed before training to ensure spatiotemporal consistency during the semantic-geometric feature propagation process.

[0176] 2. Reconstruction results.

[0177] 2.1 Rendering quality assessment.

[0178] As shown in Table 1, the proposed method outperforms the original 3DGS and other methods on the Mip-NeRF360 dataset in both SSIM and PSNR, demonstrating that this framework achieves a significant improvement in rendering quality and geometric accuracy. Simultaneously, this framework achieves a lower LPIPS value than most methods, proving that the asymmetric Gaussian reconstruction strategy can better preserve the global perceptual consistency of the scene and effectively reduce distortion during the reconstruction process.

[0179] Table 1. Quantitative results of rendering quality of synthesized new perspectives on the Mip-NeRF360 dataset.

[0180]

[0181] Note: "Red," "orange," and "yellow" represent the best, second-best, and third-best results, respectively. The rendering quality of this invention's framework is close to that of 3DGS and superior to most mainstream methods.

[0182] In terms of visual qualitative comparisons using publicly available datasets (Mip-NeRF360 and Tanks and Temples), Figure 3 The rendering differences between the various methods are visually demonstrated. Figure 3From left to right: (a) Ground Truth; (b) 2DGS; (c) GS2Mesh; (d) GOF; (e) PGSR; (f) GeoSemantic-GS (framework of this application); The figure shows the potted plant and kitchen scenes from the Mip-NeRF360 dataset and the statue and barn scenes from the Tanks and Temples dataset from top to bottom.

[0183] In terms of visual qualitative comparisons using the self-built Geo-Wild dataset, Figure 4 The rendering differences between the various methods are visually demonstrated. Figure 4 This is a qualitative comparison chart of the present invention on the Geo-Wild dataset. Figure 4 From left to right: (a) Ground Truth; (b) 2DGS; (c) GS2Mesh; (d) GOF; (e) PGSR; (f) GeoSemantic-GS (the framework of this application). The figures show, from top to bottom, the reconstruction results and magnified comparisons of local area details for scenes numbered 18, 20, 22, 23, and 24. Compared to the original 3DGS and other variants, the framework of this application successfully and thoroughly reduces visual artifacts caused by unobserved viewpoints or redundant primitives by introducing multi-view photometric residual accumulation and adaptive density control mechanisms. Furthermore, for textureless flat areas commonly found in large-scale architectural scenes, this framework effectively solves the problem of frequent floating artifacts on the model surface by relying on the physical flatness prior extracted by MLLM. Without significantly increasing the computational cost of the rendering stage, the framework of this application not only ensures the extreme smoothness of macroscopic architectural structures but also perfectly preserves the high-fidelity textures of complex geometric areas.

[0184] 2.2 Geometric accuracy assessment.

[0185] In industrial surveying and digital twin applications, the physical fidelity of geometric structures is crucial. This application rigorously evaluates the geometric accuracy of various methods on the DTU and Tanks and Temples datasets using Chamfer Distance (CD) and F1 score.

[0186] Table 2 shows the quantitative comparison results of the reconstruction accuracy (average chamfer distance / mm, with lower values ​​indicating higher accuracy) and training time of the proposed method and various comparative methods on the publicly available benchmark dataset DTU. The quantitative analysis data in Table 2 shows that while some traditional implicit geometric reconstruction methods (such as NeRF, VolSDF, NeuS, and Neuralangelo) can achieve low average geometric errors, the massive query overhead of their multilayer perceptrons (MLPs) results in extremely long training times, generally exceeding 12 hours (e.g., NeuS takes more than 12 hours), severely limiting their practical engineering application value. In the explicit Gaussian surface element reconstruction method, the framework of this application achieves an average geometric error of 0.59 mm while maintaining an extremely short optimization time of only 0.5 hours. This result represents a several-fold improvement in reconstruction efficiency compared to traditional implicit reconstruction methods; in terms of reconstruction accuracy, the framework of this application significantly outperforms the GS2Mesh and GOF methods, which are also explicit surface reconstruction methods, and achieves leading reconstruction accuracy in several specific scanning scenarios. This strongly demonstrates that the framework proposed in this application can achieve a synergy between ultra-high reconstruction accuracy and extremely fast optimization efficiency in the geometric reconstruction of micro-objects, thus reaching a better balance between performance and efficiency.

[0187] Table 2 Quantitative comparison based on DTU dataset

[0188]

[0189] Table 3 shows the quantitative comparison results of the reconstruction geometric fidelity (F1 score, with higher values ​​indicating more complete reconstruction topology) and training time of the proposed framework and various comparative methods on the large-scale outdoor scene dataset Tanks and Temples. The quantitative data in Table 3 shows that in the task of reconstructing large-scale complex macroscopic scenes, the proposed framework exhibits extremely strong geometric reconstruction robustness on six typical terrain scenes. The average F1 score of the proposed framework reaches 0.49, significantly outperforming the classic SuGaR, GS2Mesh, 2DGS, and GOF methods among explicit reconstruction methods. Furthermore, compared to the implicit state-of-the-art method Neuralangelo, the proposed framework, while maintaining almost equivalent reconstruction accuracy, reduces the training optimization time from over 128 hours to only 2 hours, achieving a computational efficiency leap of over 98%. This strongly demonstrates that the proposed framework achieves an optimal balance between "macroscopic smoothness" and "microscopic fidelity" in mesh reconstruction of large-scale real complex topological spaces, and possesses excellent engineering feasibility.

[0190] Table 3 shows the quantitative results of the reconstructed F1 Score on the Tanks and Temples dataset.

[0191]

[0192] This application ranks among the top methods.

[0193] To visually demonstrate the physical realism and versatility of this application's framework in microscopic topology reconstruction, such as Figure 5 As shown, Figure 5 This paper showcases the overall performance of the proposed framework in 3D mesh reconstruction across 15 typical scanning scenarios on the DTU dataset. The 15 typical scenarios are, in order: scan 24, scan 37, scan 40, scan 55, scan 63, scan 65, scan 69, scan 83, scan 97, scan 105, scan 106, scan 110, scan 114, scan 118, and scan 122. Figure 5 The overall 3D mesh surface extraction results show that, for statue models with complex micro-irregular surfaces and undulating textures (such as scans 114 and 122) and fine-edged components (such as the scissor blades and holes in scan 37), the framework of this application can reconstruct their smooth mesh surfaces, sharp geometric edges, and complete physical topology with extremely high accuracy. This proves that, by getting rid of the one-size-fits-all mechanism of global smoothing, this framework can achieve high-fidelity micro-reconstruction of real-world scenes with multiple scales and materials.

[0194] In terms of qualitative evaluation of geometric accuracy for indoor object-level and local scene reconstruction, such as Figure 6 As shown, Figure 6 This paper presents a qualitative comparison of the 3D mesh reconstruction quality of different methods on the Mip-NeRF360 dataset. Figure 6 From left to right: (a) original reference image; (b) 2DGS method; (c) GS2Mesh method; (d) GOF method; (e) PGSR method; (f) framework of this application. The figure shows the reconstructed contrast meshes of four scenes: "room", "kitchen", "potted plant" and "counter" from top to bottom. Compared with the contrast method, which is prone to large-scale wavy distortion or mesh holes on flat structures (such as the walls in the "room" scene and the smooth tray in the "counter" scene) due to the ambiguity of pure visual light consistency (such as the jagged edges and severe holes produced by 2DGS (b) and GS2Mesh (c) in the white wall and tray parts), the framework of this application (f) deeply couples the physical flatness and plane orientation hard constraints of Atlanta-world extracted from the large model and propagated down the hierarchical tree during the optimization process, so that large-area primitives such as walls and tabletops are forced to be aligned and adhered to the real building physical plane. The reconstructed mesh plane in this application exhibits an excellent manifold structure that is extremely smooth, continuous, and free of voids and noise, significantly improving the geometric fidelity of the artificial plane.

[0195] To further test the geometric boundary reproduction accuracy and detail preservation capabilities of this framework in large-scale macro scenes, such as Figure 7 As shown, Figure 7 This paper presents a comparison of different methods for 3D mesh reconstruction on the Tanks and Temples datasets. Figure 7 From left to right: (a) original reference image; (b) 2DGS method; (c) GS2Mesh method; (d) GOF method; (e) PGSR method; (f) GeoSemantic-GS (framework of this application). The image shows five classic large-scale macroscopic test scenes from top to bottom: "Truck," "Conference Room," "Statue," "Courtroom," and "Barn." Through magnified detail comparisons of the areas within the red boxes, it can be seen that when dealing with the narrow shadow area under the wheels of the "Truck," the rigid beams and columns at the top of the "Conference Room," the Roman column capitals in front of the "Courtroom," and the sloping eaves on the side of the "Barn," traditional comparison methods (b, c, d, e) suffer from large areas of scattered, suspended impurity meshes under the vehicle frame, or mesh breakage and severe jagged edges on small columns and the edges of carvings (such as the portico structure in the "Courtroom" scene). This is due to the simplistic primitive density management methods or the global blindness of the geometric regularization strategy. In contrast, the framework (f) of this application spontaneously releases local geometric constraints in non-complex planar regions, allowing core computing resources and parameter degrees of freedom to be precisely tilted toward these complex topological regions with high-frequency details. Thus, while completely smoothing out the macroscopic large-area background, it extremely sharply and completely reconstructs the column head patterns and edge details, achieving accurate and true local micro-topology.

[0196] To verify the accuracy of geometric reconstruction in non-ideal data sources such as irregular, low-overlap acquisition trajectories in the field and those with strong dynamic noise, such as... Figure 8 As shown, Figure 8 The paper presents a qualitative comparison of the various methods in 3D mesh reconstruction on a self-built Geo-Wild dataset. Figure 8From left to right: (a) original reference image; (b) 2DGS method; (c) GS2Mesh method; (d) GOF method; (e) PGSR method; (f) GeoSemantic-GS (framework of this application). The figure shows four sets of measured outdoor natural and ancient building scenes, numbered 18, 22, 23, and 24, from top to bottom. Under extreme challenges including complex leaf reflections, stone step textures, moving vehicles on the roadside, and pedestrians, traditional contrast methods (b, c, d, e) lack effective dynamic disturbance truncation strategies and absolute scale anchoring, resulting in severe scale geometric drift, fractures, and fragmented floating mesh blocks in the reconstructed 3D mesh (e.g., on the complex relief stone pillar surface of scene 24, the surface mesh of the contrast method exhibits severe holes and wave distortion). The framework (f) of this application integrates LiDAR absolute physical boundary reference point cloud for front-end registration and performs hard pruning, and uses zero-sample large model attributes as a priori causal judgment guidance during optimization, completely eliminating and removing various geometric noise defects caused by occlusion and dynamic targets, and reconstructing a highly smooth, highly closed, and clearly defined three-dimensional mesh model, which fully demonstrates the practical value of this invention in the construction of industrial-grade digital twins.

[0197] 2.3 Evaluation of model compactness and computational efficiency.

[0198] To meet the practical deployment needs of industrial surveying and digital twins on edge devices, the lightweight nature and optimization efficiency of 3D representation are core standards for evaluating the practical value of algorithms. Therefore, this application jointly evaluates model compactness (total number of Gaussian elements, grid size, and physical storage size) and computational cost (training time), with detailed quantitative comparison results shown in Table 4.

[0199] Experimental data shows that traditional explicit or implicit reconstruction frameworks (such as GOF and PGSR) are prone to causing disordered proliferation of redundant primitives in weakly textured or flat regions when dealing with large scenes due to a lack of prior knowledge of macroscopic topology. For example, in the Mip-NeRF360 dataset, the total number of Gaussian primitives in PGSR ( The number of Gaussian primitives surged to approximately 3.68 million, resulting in a mesh patch count (NF) of 22.22 million, leading to a combined storage overhead of over 1.4 GB for both Gaussian representation and mesh. In contrast, the GeoSemantic-GS framework exhibits exceptionally superior representation compactness. Thanks to a semantically guided asymmetric optimization strategy, this framework implements stringent primitive proliferation suppression and directional alignment in regions deemed flat, forcing a massive number of chaotic primitives to collapse into a very small number of regular surface primitives. Under the same Mip-NeRF360 scenario, this framework drastically reduces the number of Gaussian primitives to around 400,000, corresponding to a Gaussian representation storage (…). The size is only 94.8MB (compressed by approximately 89% compared to PGSR); at the same time, the number of extracted mesh faces is significantly reduced to 5.34 million, and the mesh storage size is only 138.5MB. This extreme compression capability has been stably maintained in datasets covering large-scale spaces (Tanks and Temples) and dynamic disturbances (Geo-Wild), fully demonstrating that this framework not only performs well in terms of vision and geometry, but also perfectly adapts to the stringent storage constraints of cloud distribution and edge rendering.

[0200] Table 4 Comparison of model compactness and training efficiency of different methods on Mip-NeRF360, DTU, Tanks and Temples, and Geo-Wild datasets.

[0201]

[0202] In terms of computational cost and training efficiency, the GeoSemantic-GS framework also overcomes the time bottleneck that may be caused by the introduction of large multimodal models. Intuitively, introducing complex semantic parsing and large model inference processes will inevitably increase system latency; however, due to the pre-computation strategy and the "heuristic attribute inheritance mechanism" based on multi-granularity spatial trees, the inference and decoding overhead of multimodal features is significantly reduced. More importantly, combined with an adaptive density control mechanism based on multi-view consistency, the system automatically removes dynamic interference through masking blinding in the early stages of iteration and strictly eliminates a large number of invalid primitives that do not contribute to rendering, which fundamentally reduces the computational load of rasterization and backpropagation. As shown in Table 4, on the challenging self-built Geo-Wild dataset, the overall training time of this framework is only 35.2 minutes, which is far lower than that of GOF (179.8 minutes).

[0203] To address the engineering bottlenecks of traditional 3D Gaussian splashing (3DGS) in the reconstruction of large-scale structured scenes, including primitive redundancy, geometric distortion, and extreme sensitivity to dynamic disturbances, this application proposes a novel semantic-driven asymmetric Gaussian reconstruction framework—GeoSemantic-GS. This application overcomes the inherent limitations of pure low-level geometric optimization by innovatively integrating the physical common sense and spatial causal reasoning capabilities of a multimodal large language model (Qwen3.5-VL) into a differentiable rendering pipeline. By constructing a multi-granularity spatial association tree, efficient propagation from 2D zero-sample visual features to 3D physical flatness and structural orientation priors is achieved. Utilizing spatiotemporal consistency masks and multi-view residual accumulation mechanisms, transient disturbance artifacts are eliminated at the low level, and the disorderly proliferation of redundant primitives is curbed. Finally, based on 2DGS surface element representation, a semantically guided asymmetric orientation regularization is constructed, achieving absolute anchoring of building planes and adaptive protection of complex topologies.

[0204] Extensive experiments on multiple real-world benchmark datasets demonstrate that the GeoSemantic-GS framework successfully achieves a dynamic balance between extremely smooth macroscopic structures and exceptionally sharp microscopic details without requiring intensive manual annotation. Compared to state-of-the-art surface reconstruction methods, this framework not only maintains superior competitiveness in rendering fidelity and geometric accuracy but also reduces the number of Gaussian elements and model storage overhead by several orders of magnitude, resulting in a significant leap in rendering frame rate. This application provides a novel cross-paradigm for intelligent radiation field optimization based on large models, and its high-fidelity, lightweight digital assets pave the way for efficient cloud deployment of industrial surveying, smart cities, and digital twins of cultural heritage.

[0205] Without causing conflict, those skilled in the art can freely combine and use the above-mentioned additional technical features.

[0206] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A GeoSemantic-GS framework for scene surface reconstruction, characterized in that, include: A semantic prior generation module is used to receive multi-source perceptual data streams, convert the data streams into a prior set, and output multimodal semantic priors and physical semantic priors; The spatiotemporal filtering density control module has its input end connected to the output end of the semantic prior generation module. It is used to receive the multimodal semantic prior and physical semantic prior, and combine semantic masking and multi-view consistency evaluation to shield and remove dynamic interference data. At the same time, it adaptively controls the primitive density to curb the growth of redundant primitives and outputs the characterization data after dynamic interference removal and density optimization. An asymmetric Gaussian generation module, whose input is connected to the output of the semantic prior generation module and the output of the spatiotemporal filter density control module, is used to receive the multimodal semantic prior, physical semantic prior and the optimized representation data, and perform asymmetric Gaussian surface parameter optimization driven by multimodal semantics based on the two-dimensional Gaussian surface element representation, and output the generated three-dimensional spatial data. The laser vision deep fusion module has its input end connected to the output end of the semantic prior generation module, and its output end connected to the input end of the asymmetric Gaussian generation module, for providing physical-level geometric anchoring to the asymmetric Gaussian generation module.

2. The GeoSemantic-GS framework for scene surface reconstruction according to claim 1, characterized in that, The semantic prior generation module includes: A multi-granularity scene deconstruction unit is used to convert multi-source sensing data streams into a tree-like data structure with spatial nesting relationships. It includes a multi-granularity mask generation subunit and an association tree construction subunit. The multi-granularity mask generation subunit is used to extract mask sets across granularities. The association tree construction subunit is used to construct a spatial association tree based on the area inclusion function to determine the parent node relationship between masks. ; The multimodal physical attribute reasoning unit, connected to the multi-granularity scene deconstruction unit, is used to receive the association tree. It generates a structured prior for each node, which includes a joint input construct subunit for generating a hint for each node containing probability distribution features of the global 3D space, density field parameters of the local spatial neighborhood, and pose metadata. The attribute reasoning decoding subunit uses the Qwen3.5 multimodal large model to output the structured attribute data of Gaussian surface elements; The semantic propagation subunit performs top-down high-confidence prior inheritance along the parent-child edges of the association tree to correct the inference results of the bottom-level nodes; Furthermore, the multi-granularity scene deconstruction unit provides a spatial skeleton and topological constraints for the multimodal physical attribute reasoning unit, and the multimodal physical attribute reasoning unit injects semantic priors into the multi-granularity scene deconstruction unit. The two work together to output a structured prior, which is indexed by the association tree, and each node in the association tree carries a set of multimodal attributes.

3. The GeoSemantic-GS framework for scene surface reconstruction according to claim 2, characterized in that, The multi-granularity mask generation subunit is configured to: employ a Semantic-SAM zero-shot segmentation model to extract a mask set spanning six granularity levels from macroscopic to microscopic, denoted as . ; The association tree construction subunit is configured to: construct a multi-granularity spatial association tree. Among them, vertex set Corresponding mask set ; For any two masks and Directed edge The weights are the area coverage function. : ； In the formula, A measure of the pixel area of ​​a mask; For mask and The intersection region; The preset area coverage threshold is determined if and only if and The level to which it belongs is higher than At that time, establish for The direct parent node.

4. The GeoSemantic-GS framework for scene surface reconstruction according to claim 3, characterized in that, The joint input construct subunit is configured to: for association trees any node in Construct a joint input suggestion that includes global references and local details. ;in, Provides a global spatial reference in three-dimensional space. Based on Extracted local spatial feature set with background set to zero. This refers to the area ratio and parent node identifier extracted from the association tree; The attribute reasoning decoding subunit is configured to: provide the joint input hints Inputting the data into a Qwen3.5 multimodal large model with a parameter size of 9B, conditional probability maximization is performed to obtain a structured attribute feature set. : ； In the formula, The optimal structured attribute feature set obtained is denoted as the corresponding output structured attribute feature set. ; For candidate structured attribute feature sets; Provide a prompt for the combined input; These are the pre-trained parameters for Qwen3.5; It is the conditional probability distribution function; , , , The structured property data of Gaussian surface elements; The semantic propagation subunit is configured to utilize the association tree. The topological connectivity is used to perform Markov property coverage on micro-level nodes, where: if child nodes parent node satisfy And confidence level Then let the child nodes Directly inherit from the parent node The physical flatness prior.

5. The GeoSemantic-GS framework for scene surface reconstruction according to claim 1, characterized in that, The spatiotemporal filter density control module includes: The semantic masking and gradient truncation unit is configured to: based on temporal sampling data obtained from multimodal semantic priors, use the Qwen3.5 model to determine transient interference of data points in the feature space, and generate a spatiotemporally consistent binary masking mask. ,Should ; In the differentiable rendering and backpropagation stages of 3D Gaussian splashing, the mask is directly applied to the photometric loss function. Calculate the photometric loss function after shielding. : ； In the formula, A spacetime consistent binary mask; This is the original photometric loss function; Represents the Hadama product; The density control unit, whose input is connected to the output of the semantic mask and gradient truncation unit, is used to receive the clean rendering features after masking and the corresponding photometric residual information, and is configured to perform multi-view photometric residual accumulation densification and perform primitive pruning with rendering contribution awareness.

6. The GeoSemantic-GS framework for scene surface reconstruction according to claim 5, characterized in that, The multi-view photometric residual accumulation and densification includes: for any candidate Gaussian element in space... Random sampling during training Spatial observation pose parameter points ; Calculate the primitive's projected footprint Multi-view average photometric residual contribution : In the formula, The number of randomly sampled spatial observation pose parameter points; For spatial observation pose parameter point index; For projected footprints Projection data sampling points within; For the candidate Gaussian element Projected footprints; It is an indicator function; For the first Pixels from each viewpoint Normalized high-error distribution mask; This is the error sensitivity threshold; For the first Sampling points of projection data under spatial observation pose parameter points of Photometric reconstruction error; The density threshold is set only when the cumulative residual... Strictly greater than the denser threshold At that time, it triggers primitive splitting; Introducing physical semantic priors as an asymmetric threshold modulator: punitive threshold increases are applied to regions judged as FLAT_SURFACE. To suppress primitive growth in planar regions; while maintaining the original threshold for COMPLEX_GEOMETRY regions. To capture high-frequency geometric details.

7. The GeoSemantic-GS framework for scene surface reconstruction according to claim 5, characterized in that, The primitive pruning that is aware of the rendering contribution includes: defining a comprehensive influence factor for each Gaussian primitive on the global multi-view reconstruction quality. : ； In the formula, It is a nonlinear normalization operator; The number of randomly sampled spatial observation pose parameter points; For spatial observation pose parameter point index; For projected footprints Projection data sampling points within; The projected footprint of the candidate Gaussian element; It is an indicator function; For the first Sampling points of projection data under spatial observation pose parameter points Normalized high-error distribution mask; This is the error sensitivity threshold; For the first Sampling points of projection data under spatial observation pose parameter points of Photometric reconstruction error; The preset pruning threshold; when the contribution of the primitive... Below the preset pruning threshold At that time, a forced removal will be performed.

8. The GeoSemantic-GS framework for scene surface reconstruction according to claim 1, characterized in that, The asymmetric Gaussian generation module includes: a two-dimensional Gaussian surface element representation unit, configured to: employ dimensionality-reduced 2D Gaussian surface elements in three-dimensional space, such that each surface element degenerates into an oriented planar disk; wherein, the first... The geometric state of a Gaussian surface element is determined by its center point. Normal vector Two-dimensional scale vector And the joint definition of a local coordinate system composed of two orthogonal tangent vectors; For any point in space The impact is expressed as follows: ； In the formula, For the first The center point of a Gaussian surface element; For the first The rotation matrix of Gaussian surface elements, and Its third column strictly corresponds to the normal vector of the surface element. ; Flattened covariance matrix The inverse matrix, and ,in and Two-dimensional scale vector The amount, It is a very small positive number used to represent the surface element thickness to ensure the invertibility and numerical stability of the covariance matrix; The oriented plane regularization unit, whose input is connected to the output of the two-dimensional Gaussian surface element representation unit, is used to obtain the normal vector of each surface element. and semantic tags And it is configured to: introduce a global gravity direction. Construct directional hard-constraint operators based on semantic categories Apply orthogonal constraints to vertical surface elements Apply parallel constraints to horizontal surface elements ; Calculate the directional plane regularization loss function : ； In the formula, It is a set of Gaussian surface elements; Let be the index of the Gaussian surface element, and ; For the first Semantic mask weights of Gaussian surface elements; For the first Confidence level of physical flatness of a Gaussian surface element; A directional hard constraint operator constructed based on semantic categories; The constraint relaxation and topology detail preservation unit has its first input connected to the output of the two-dimensional Gaussian surface element representation unit to obtain geometric state parameters and radiation properties, and its second input connected to the output of the oriented plane regularization unit to obtain the oriented plane regularization loss function. Confidence of flatness Its third input terminal receives multi-view photometric error signals. , and the local normal consistency loss function This unit is configured to construct the joint objective function. : ； In the formula, The joint objective function is constructed; for Photometric error loss function; The structural similarity (SSIM) loss function is used. This is the local normal consistency loss function; Let be the directional plane regularization loss function; For use in balancing Preset weighting coefficients for photometric error loss and structural similarity loss; The loss weights are used to control the strength of local normal consistency constraints. These are the loss weight coefficients used to adjust the strength of the plane regularization constraint; For component regions identified as COMPLEX_GEOMETRY by the multimodal large model, their corresponding flatness confidence scores are used. Spontaneous blocking The effect of the term on this region frees the Gaussian elements in this region from orthogonality / parallelism constraints, making them completely driven by multi-view photometric errors; for non-complex geometric regions, it maintains complete planar regularization constraints.

9. The GeoSemantic-GS framework for scene surface reconstruction according to claim 1, characterized in that: The laser vision deep fusion module includes a multi-source point cloud registration and geometric attribute joint initialization unit, configured to: uniformly align the visual sparse point cloud estimated by COLMAP and the LiDAR dense point cloud to an absolute metric coordinate system, and construct a fused point cloud set. For any point in the set Utilizing its local spatial neighborhood Constructing the covariance matrix : ； In the formula, For any point in the merged point cloud set; For point The local spatial neighborhood; | | represents the number of neighboring points in the local spatial neighborhood; These are the neighboring points within the local spatial neighborhood; Represents the transpose operation of a matrix or vector; Principal component analysis (PCA) was used to select... The eigenvector corresponding to the smallest eigenvalue is used as the initial normal vector. And construct the rotation matrix to make its local The axis is strictly parallel to the normal. Simultaneously, the initial two-dimensional scale vector of the Gaussian surface element is... The constraints are: ； In the formula, and The initial two-dimensional scale vector Scale components; For the local spatial neighborhood The number of neighboring points in the array, and satisfying ; For any point in the fused point cloud set; These are the neighboring points within the local spatial neighborhood; express Norm; The forced pruning unit has its input connected to the output of the multi-source point cloud registration and geometric attribute joint initialization unit, and periodically receives the center point positions of all current Gaussian units during training. It is configured as: the center of each primitive in the computation space To LiDAR physical surface point cloud Shortest absolute Euclidean distance : ； In the formula, For the current number The center point position vector of each Gaussian element; This is the reference point position vector in the LiDAR physical surface point cloud; Point cloud of the physical surface of the LiDAR; express Norm; Based on this shortest absolute Euclidean distance, a hard space elimination indicator function is defined. : ； In the formula, It is an indicator function; The preset physical tolerance threshold; once The framework ignores the primitive's contribution to the multi-view photometric residuals and performs a forced deletion operation.

10. The application of the GeoSemantic-GS framework for scene surface reconstruction according to any one of claims 1 to 9, characterized in that: The GeoSemantic-GS framework is applied to the structured reconstruction of complex real-world scenes to generate structured 3D Gaussian scene models.

Images