Deep learning based method and system for intelligent reconstruction of building three-dimensional model

Abstract

This invention relates to the field of intelligent reconstruction technology for 3D models, specifically a method and system for intelligent reconstruction of 3D building models based on deep learning. The method includes acquiring multi-source data of the target ancient building; generating a dense point cloud through motion reconstruction and multi-view stereo matching; identifying structural and non-structural components through semantic segmentation and reconstructing an initial mesh model with semantically labeled information; using a graph neural network to analyze the spatial topological relationships of structural components, and combining this with a historical construction knowledge base to generate hierarchical combination methods and connection constraint rules; adaptively adjusting the pose of mesh nodes according to the connection constraint rules to drive the precise assembly of parameterized components, and fusing them to obtain a structured 3D model of the target ancient building. This invention can restore the structural component connection relationships in hidden areas of ancient buildings under non-contact acquisition conditions, significantly improving the structural rationality and geometric accuracy of the 3D model.

Description

Technical Field

[0001] This invention relates to the field of intelligent reconstruction technology of 3D models, specifically to a method and system for intelligent reconstruction of 3D building models based on deep learning. Background Technology

[0002] Currently, 3D modeling of buildings mainly relies on technologies such as photogrammetry and LiDAR scanning. These technologies reconstruct the external geometry by collecting images or point cloud data, and are widely used in modern architecture. However, data acquisition and modeling of ancient buildings face unique challenges. Because ancient buildings are often protected cultural relics, close contact or structural dismantling is usually prohibited, making it difficult to obtain crucial structural data such as the internal structure of beams and frames, the gaps between bracket sets, and hidden mortise and tenon joints. Furthermore, the complex structures of ancient buildings make it difficult to reconstruct hidden structural relationships such as bracket sets and mortise and tenon joints using conventional modeling methods.

[0003] Therefore, existing technologies can often only construct surface geometric models of ancient buildings, making it difficult to effectively reconstruct internal structural relationships, component styles, and their combination logic. Especially when faced with weathering erosion, component incompleteness, or other issues, the generated models often suffer from low morphological accuracy, poor component integrity, and unclear structural representation. This not only affects the accuracy of the digital archives but also interferes with subsequent restoration design and structural safety assessments.

[0004] Therefore, there is an urgent need for intelligent reconstruction methods and systems for 3D building models based on deep learning, to deeply analyze the structural intentions of ancient buildings, and to construct intelligent 3D models with high realism and interpretability of construction. Summary of the Invention

[0005] (1) Technical problems to be solved

[0006] The purpose of this invention is to provide a method and system for intelligent reconstruction of three-dimensional building models based on deep learning, so as to solve the technical problem that the structural logic of the three-dimensional model is broken and the assembly integrity is insufficient due to the lack of connection relationship of hidden components or interference from weathered and damaged components.

[0007] (2) Technical solution

[0008] To achieve the above objectives, on the one hand, the present invention provides a method for intelligent reconstruction of 3D building models based on deep learning, the method comprising:

[0009] Step S1: Acquire multi-source data of the target ancient building, including visible light image sequences, laser point cloud data, and depth image data; obtain sparse point clouds from the multi-source data using a motion reconstruction algorithm, and obtain dense point clouds from the sparse point clouds using multi-view stereo matching; obtain semantic label information from the dense point clouds through semantic segmentation, including first information corresponding to structural components and second information corresponding to non-structural components; obtain an initial mesh model from the dense point clouds carrying the semantic label information through surface reconstruction.

[0010] Step S2: The first information is analyzed through a neural network to resolve the connection relationships between structural components, thereby obtaining the hierarchical combination method and connection constraint rules of the structural components; the first information and the hierarchical combination method are matched with the corresponding component parameter templates through a preset construction knowledge base.

[0011] Step S3: Obtain the mesh nodes corresponding to the structural components in the initial mesh model, adjust the spatial position of the mesh nodes according to the connection constraint rules to obtain the final mesh model; assemble the assembly structure according to the component parameter template and the hierarchical combination method; merge the structural component mesh area in the final mesh model with the assembly structure according to the connection constraint rules to obtain the structured three-dimensional model of the target ancient building.

[0012] Furthermore, the method for obtaining semantic label information from the dense point cloud through semantic segmentation, wherein the semantic label information includes first information corresponding to structural components and second information corresponding to unstructured components, includes:

[0013] The dense point cloud is divided into multiple sub-blocks. The local geometry, normal vector distribution, color texture information and spatial adjacency of each sub-block are extracted to form a multimodal feature vector. The multimodal feature vector is then used to obtain the semantic label information corresponding to each point in the dense point cloud through a pre-trained point cloud semantic segmentation model.

[0014] The first information includes the category labels and confidence scores for columns, beams, and brackets; the second information includes the category labels and confidence scores for painted decorations, roof tiles, and corbels. When the confidence score is lower than the preset confidence threshold, the category labels are corrected.

[0015] Furthermore, the method for analyzing the connection relationships between structural components using a neural network to obtain the hierarchical combination method and connection constraint rules of the structural components includes:

[0016] Based on the category labels of structural components in the first information and the spatial distribution of dense point clouds, dense point cloud fragments with the same category labels and spatially adjacent are clustered into candidate regions.

[0017] A topology graph is constructed using the candidate regions as nodes, and the connecting edges of the topology graph represent the spatial pose relationships between nodes. The connection weights of the connecting edges between nodes are analyzed using a graph neural network. Based on the connection weights, a hierarchical combination method and connection constraint rules for structural components are established.

[0018] Furthermore, the method for establishing the hierarchical combination method and connection constraint rules of structural components based on the connection weights includes:

[0019] Obtain the geometric centroid height of the structural component corresponding to the first information in the dense point cloud, sort the structural components according to the direction of gravity to obtain the sorting result; obtain the optimized hierarchical combination method and connection constraint rules based on the sorting result; and perform historical connection verification on the optimized hierarchical combination method and connection constraint rules.

[0020] Furthermore, the method for historical connection verification of the optimized hierarchical combination method and connection constraint rules includes:

[0021] Obtain the frequency statistics matrix of historical connection instances from the constructed knowledge base; calculate the occurrence probability of the optimized hierarchical combination method in the frequency statistics matrix; when the occurrence probability is lower than a preset occurrence probability threshold, replace the optimized hierarchical combination method with the historical connection order with the highest occurrence probability in the frequency statistics matrix.

[0022] Furthermore, the method for obtaining the mesh nodes corresponding to structural components in the initial mesh model and adjusting the spatial positions of the mesh nodes according to connection constraint rules to obtain the final mesh model includes:

[0023] Based on the hierarchical combination method and connection constraint rules of structural components, identify the combination of structural components with interface cooperation relationship, obtain the corresponding set of nodes to be aligned based on the combination of structural components, and calculate the spatial pose deviation of the set of nodes to be aligned through the initial mesh model.

[0024] An optimization function is constructed with the goal of minimizing spatial pose deviation, and a curvature constraint term is added. The optimization function is then used to update the spatial coordinates of the node set to be aligned through an iterative optimization algorithm. The final mesh model is obtained when the norm of the spatial coordinate update vector in the adjacent iteration step is less than a preset convergence threshold.

[0025] Furthermore, the method for obtaining an assembly structure by spatial assembly based on the component parameter template and hierarchical combination method includes:

[0026] The parameterized component model is obtained by retrieving the component parameter template and hierarchical combination method from the pre-built component model library; the interface region and the interface matching method defined in the connection constraint rules in the final mesh model are obtained; the spatial transformation matrix is ​​obtained according to the interface region and interface matching method; the interface node position of the parameterized component model is adjusted according to the spatial transformation matrix and aligned with the mesh node to obtain the spatial positioning model of the structural component; the spatial positioning model is organized according to the hierarchical combination method to generate structural component assembly units.

[0027] The attachment model is calculated using the normal projection method by the attachment conditions defined in the category label, geometry and preset decorative base surface rules of the non-structural component; the assembly unit of the structural component is spatially combined with the attachment model to obtain the assembly structure.

[0028] Furthermore, the method for adjusting the interface node positions of the parameterized component model according to the spatial transformation matrix and aligning them with the mesh nodes to obtain the spatial positioning model of the structural component includes:

[0029] The boundary contour of the parameterized component is registered with the adjacent region in the final mesh model, and the joint deviation is obtained through geometric consistency check. When the joint deviation exceeds the preset assembly tolerance threshold, the spatial pose of the anchor point is iteratively adjusted within the preset degree of freedom constraint range using a nonlinear optimization algorithm. The joint deviation is recalculated after each iteration adjustment. When the joint deviation meets the assembly tolerance threshold, the spatial positioning model of the structural component is obtained.

[0030] Furthermore, the method for fusing the structural component mesh regions in the final mesh model with the assembled structure according to connection constraint rules to obtain a structured three-dimensional model of the target ancient building includes:

[0031] Obtain the mesh regions of structural components in the final mesh model, and establish mapping relationships based on the spatial pose of the assembly structure; process the mesh regions according to the mapping relationships and the interface cooperation methods defined in the connection constraint rules to obtain the geometric representation of the structural components; generate an adjacency graph according to the connection constraint rules of the structural components; combine the geometric representation with the adjacency graph to generate a structured 3D model.

[0032] Based on the same inventive concept, this invention also provides a deep learning-based intelligent reconstruction system for 3D building models, the system comprising:

[0033] The data acquisition and preprocessing module is used to acquire multi-source data of the target ancient building, including visible light image sequences, laser point cloud data, and depth image data; the multi-source data is processed using a motion reconstruction algorithm to obtain a sparse point cloud, and a dense point cloud is obtained based on the sparse point cloud through multi-view stereo matching; the dense point cloud is processed through semantic segmentation to obtain semantic label information, the semantic label information including first information corresponding to structural components and second information corresponding to non-structural components; the dense point cloud carrying the semantic label information is processed through surface reconstruction to obtain an initial mesh model.

[0034] The structural analysis and template matching module is used to analyze the connection relationship between structural components through a neural network to obtain the hierarchical combination method and connection constraint rules of the structural components; and to match the first information and the hierarchical combination method with the corresponding component parameter template through a preset construction knowledge base.

[0035] The model optimization and assembly output module is used to obtain the mesh nodes corresponding to the structural components in the initial mesh model, adjust the spatial position of the mesh nodes according to the connection constraint rules to obtain the final mesh model, perform spatial assembly according to the component parameter template and hierarchical combination method to obtain the assembly structure, and fuse the structural component mesh area in the final mesh model with the assembly structure according to the connection constraint rules to obtain the structured three-dimensional model of the target ancient building.

[0036] (3) Beneficial effects

[0037] Compared with the prior art, the beneficial effects of the present invention are:

[0038] 1. This invention uses graph neural networks combined with a historical construction knowledge base to analyze and optimize the connection relationships of structural components in hidden areas, enabling the restoration of force transmission paths in ancient buildings that conform to traditional techniques under non-contact acquisition conditions.

[0039] 2. This invention uses an assembly mechanism that combines parametric template-driven and connection constraint rules to perform high-precision geometric fusion of structural and non-structural components, generate a unified geometric expression and establish an adjacency graph, significantly improving the level of 3D model in terms of structural rationality, geometric accuracy and historical authenticity. Attached Figure Description

[0040] Figure 1 This is a flowchart of the intelligent reconstruction method for 3D building models based on deep learning according to Embodiment 1 of the present invention;

[0041] Figure 2 This is a schematic diagram of the module composition of the intelligent reconstruction system for three-dimensional building models based on deep learning according to Embodiment 2 of the present invention. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] Before providing examples, it is necessary to explain the application scenarios of this invention. Due to the complex structure and diverse components of ancient buildings, and the fact that most are under cultural relic protection, close contact or disassembly and measurement are impossible. This has led to a long-standing reliance on manual experience and structural inference for 3D modeling, resulting in high error rates, low modeling efficiency, and unclear component connection relationships. Especially in situations with multiple layers of brackets and densely distributed mortise and tenon joints causing significant interference, traditional methods based on a single data source or pure geometric reconstruction are insufficient to effectively reconstruct the structural logic. This invention addresses these application challenges by proposing a 3D reconstruction method for buildings that integrates multi-source perception, deep learning structural analysis, and parametric model assembly. It is suitable for scenarios such as digital restoration, structural analysis, and BIM integration of cultural relic protection buildings, and has advantages such as being inaccessible, non-destructive, and producing reusable results with a reasonable structure.

[0044] Example 1: As Figure 1 As shown, this embodiment provides a method for intelligent reconstruction of 3D building models based on deep learning. The method includes:

[0045] Step S1: Acquire multi-source data of the target ancient building, including visible light image sequences, laser point cloud data, and depth image data; obtain sparse point clouds from the multi-source data using a motion reconstruction algorithm, and obtain dense point clouds from the sparse point clouds using multi-view stereo matching; obtain semantic label information from the dense point clouds through semantic segmentation, including first information corresponding to structural components and second information corresponding to non-structural components; obtain an initial mesh model from the dense point clouds carrying the semantic label information through surface reconstruction.

[0046] Step S2: The first information is analyzed through a neural network to resolve the connection relationships between structural components, thereby obtaining the hierarchical combination method and connection constraint rules of the structural components; the first information and the hierarchical combination method are matched with the corresponding component parameter templates through a preset construction knowledge base.

[0047] Step S3: Obtain the mesh nodes corresponding to the structural components in the initial mesh model, and adjust the spatial position of the mesh nodes according to the connection constraint rules to obtain the final mesh model; retrieve the matching parametric component model from the pre-built component model library according to the component parameter template and hierarchical combination method; assemble the assembly structure according to the component parameter template and hierarchical combination method; merge the structural component mesh area in the final mesh model with the assembly structure according to the connection constraint rules to obtain the structured three-dimensional model of the target ancient building.

[0048] For example, digital modeling of the ancient building A utilizes a quadcopter drone equipped with an industrial camera and LiDAR for aerial photography. Depth images penetrate the physical gaps in the bracket set to reconstruct the mortise and tenon joints; the TOF depth camera has a minimum penetration capability of 15cm. Over 800 frames of visible light images are acquired along the flight path, and the LiDAR point cloud accuracy is 0.5cm, meeting national standards. A structure-of-motion (SOG) algorithm is used to process the image sequence, generating a sparse point cloud by constraining the pose with the LiDAR point cloud. This sparse point cloud is then used to generate a dense point cloud with a density of 12,000 points per square meter through multi-view stereo matching, and divided into multiple sub-blocks of fixed size for subsequent feature extraction.

[0049] Dense point clouds with semantically labeled information are reconstructed using a Poisson surface and converted into an initial mesh model. A topology graph is constructed by analyzing the spatial poses of column and beam nodes. The connection edges between nodes are defined by their spatial pose relationships, and the connection weights of these edges are analyzed using a graph neural network to generate hierarchical combination relationships from columns to beams to brackets. Parametric templates (such as the tenon and mortise dimensions of hip-roofed main beams) are matched with parameters from a knowledge base and combined with preset hierarchical rules and interface parameters from this knowledge base to form connection constraint rules. These rules are then used to adjust and optimize the nodes of the initial mesh model, completing the assembly of structural components.

[0050] Specifically, the construction of the knowledge base includes: converting the mortise and tenon types of brackets and the dimensions of beam-column interfaces in the "Engineering Practice Rules" and "Building Standards" into a parameterized template library. For example, the opening depth of the dovetail tenon is 0.3 times the width of the component and the slope is 12°. The depth of the groove at the top of the raised beam column is not less than 1 / 5 of the beam height. The component combination probability matrix is ​​statistically analyzed based on 27,000 sets of official architectural survey data, and regional parameters are stored according to regional types such as Jin School, Su School, and Beijing School. For example, the height-to-width ratio of the camel hump in Jin School is 0.8, and the curvature radius of the corbel in Su School is 0.5m. At the same time, the hierarchical rules and interface matching parameters of structural components are stored to provide a basis for connection relationship analysis and assembly constraints.

[0051] The method for obtaining semantic label information from the dense point cloud through semantic segmentation, wherein the semantic label information includes first information corresponding to structural components and second information corresponding to unstructured components, includes:

[0052] The dense point cloud is divided into multiple sub-blocks. The local geometry, normal vector distribution, color texture information and spatial adjacency of each sub-block are extracted to form a multimodal feature vector. The multimodal feature vector is then used to obtain the semantic label information corresponding to each point in the dense point cloud through a pre-trained point cloud semantic segmentation model.

[0053] The first information includes the category labels and confidence scores for columns, beams, and brackets; the second information includes the category labels and confidence scores for painted decorations, roof tiles, and corbels. When the confidence score is lower than the preset confidence threshold, the category labels are corrected.

[0054] For example, this technology improves the semantic segmentation accuracy of point clouds through multimodal feature fusion, playing a crucial role in complex structural areas of ancient buildings. Taking the double-layered bracket area of ​​ancient building A as an example, the point cloud is divided into cubic sub-blocks with a side length of 2.5 meters. This size is derived from the physical size statistics of typical bracket groups and can completely cover common components. Each sub-block extracts four types of features: local geometry, normal vector distribution, color and texture information, and spatial adjacency. Among them, the principal curvature value at the bracket turning point is greater than 0.8, the normal variation coefficient of the column base connection area is significantly higher than the average level, and the color saturation distribution of the painted area shows a concentrated trend. These features form multimodal vectors and are input into the residual PointNet++ model for processing.

[0055] When the model's output confidence score falls below 0.6, the system initiates a two-stage correction mechanism. The confidence threshold of 0.6 is determined by plotting the precision-recall curve on the validation set and identifying its inflection point. The first stage involves neighborhood label voting, with the neighborhood set to 15 neighboring points, calculated from the average density of the dense point cloud. The second stage performs geometric consistency checks, including verification of normal angle constraints and gravity direction deviations. After correction, the boundary positioning error of structural components is controlled within 2 cm, meeting the requirements for digital archiving in the "Code for Surveying and Mapping Ancient Buildings."

[0056] This embodiment employs existing point cloud semantic segmentation networks for processing, with preferred structures such as PointNet++ or KPConv. It takes 3D point cloud coordinates and normal vectors as input and outputs a component category label for each point. The model can be trained using existing ancient building scan data or small-sample manually labeled data, and possesses strong structural component recognition capabilities.

[0057] The method for analyzing the connection relationships between structural components using a neural network to obtain the hierarchical combination method and connection constraint rules of the structural components includes:

[0058] Based on the category labels of structural components in the first information and the spatial distribution of dense point clouds, dense point cloud fragments with the same category labels and spatially adjacent are clustered into candidate regions.

[0059] A topology graph is constructed using the candidate regions as nodes, and the connecting edges of the topology graph represent the spatial pose relationships between nodes. The connection weights of the connecting edges between nodes are analyzed using a graph neural network. Based on the connection weights, a hierarchical combination method and connection constraint rules for structural components are established.

[0060] For example, in the process of parsing the connection relationships of structural components in ancient buildings, the system first clusters the point clouds of columns, beams, and brackets obtained from semantic segmentation based on the consistency of category labels and spatial proximity to form a set of candidate regions. The clustering process sets a spatial radius constraint to maintain the integrity of the point cloud fragments; for example, the clustering radius of the columns and beams in the front hall of Wanshou Palace is set to 0.6 meters.

[0061] Subsequently, a topological graph of the spatial pose relationships between nodes is constructed, using candidate regions as nodes. The spatial pose relationships are comprehensively defined by the component spacing, the angle between interface normal vectors, and the height difference of the geometric center of gravity. For example, the measured column-beam spacing is 1.2 meters (fluctuation ±0.3 meters), the cosine value of the angle between interface normal vectors is ≥0.98, and the height difference of the beam-column center of gravity meets the traditional frame benchmark requirements.

[0062] The constructed topology graph is input into a Graph Attention Network (GAT) to analyze the connection weights of the edges connecting nodes. Valid connection edges are filtered using a weight threshold of 0.85 to obtain an initial sequence of component connections; for example, the column-beam connection weight is 0.91, and the beam-arch connection weight is 0.87. Specifically, the weight threshold is set to 0.85, a value determined based on the lower limit of valid connection edges in the distribution of connection weights from a large number of historical component connection samples, effectively distinguishing between real structural connections and noisy connections.

[0063] Based on the connection weights obtained from the analysis, hierarchical combination methods and connection constraint rules are generated. When dealing with dense double-layer bracket areas, the system identifies the axial fit relationship of the tenon and mortise through the normal vector angle constraint. For example, the deviation of the normal vector angle between the bracket's angzui and the huagong interface is ≤3°, and the axis alignment error is ≤1.8cm. This ensures that the generated connection constraint rules clearly define the horizontal displacement and rotation limits, providing precise boundary conditions for subsequent assembly.

[0064] The method for establishing the hierarchical combination method and connection constraint rules of structural components based on the connection weights includes:

[0065] Obtain the geometric centroid height of the structural component corresponding to the first information in the dense point cloud, sort the structural components according to the direction of gravity to obtain the sorting result; obtain the optimized hierarchical combination method and connection constraint rules based on the sorting result; and perform historical connection verification on the optimized hierarchical combination method and connection constraint rules.

[0066] For example, the construction of ancient buildings has clear hierarchical rules, which are stored in a construction knowledge base. These rules are formed by historical norms such as the "Yingzao Fashi" and "Gongcheng Zuozao Zeli" and statistical data from 27,000 sets of official building survey data. They include typical component connection sequences (such as column-beam-dougong-rafter-roof components) and allowed combinations of interface types.

[0067] During the connection relationship resolution process, if the component type combination corresponding to a connection edge in the topology graph is not in the above-mentioned hierarchical rules, the connection edge is marked as a candidate edge for incorrect connection. For example, if a column node directly connects to a roof component node, but this combination does not exist in the construction knowledge base, then the connection edge is a candidate edge for incorrect connection. The system performs a rejection operation on candidate edges for incorrect connection to remove connection paths that do not conform to the historical construction logic.

[0068] To optimize the rationality of the remaining connection paths, the system extracts the geometric centroid height of each structural component corresponding to the first information in the dense point cloud. The geometric centroid height refers to the Z-axis coordinate of the centroid of the component mesh model in the three-dimensional coordinate system. The components are sorted according to the direction of gravity (from bottom to top), and the height difference threshold is set to be no less than 0.5 meters. This threshold is derived from the minimum vertical spacing standard between load-bearing components of traditional wooden frames.

[0069] Taking the gate tower of an ancient building as an example, after removing erroneously connected edges where columns directly connect to roof components, the system sorts the components by their geometric center of gravity height as follows: column (0 meters), beam (3.2 meters), bracket set (4.1 meters), rafter (5.7 meters), and roof component (6.3 meters). This order conforms to the traditional bottom-up gravity transfer logic. The system further combines the sorting results with connection constraint rules to form an optimized hierarchical combination method and connection constraint rules. These rules include the allowed connection types between components, interface directionality, and relative height difference constraints.

[0070] Finally, the system updates the topology map based on the optimized rules to obtain the complete force transmission path from the load-bearing base to the roof components. For example, the height difference between the bracket set and the rafter is 1.6 meters, with an allowable fluctuation of ±0.2 meters. This tolerance is derived from the standard deviation statistics of historical survey data. Comparison with the actual measured structure shows that the optimized path achieves 97.3% consistency with the actual structure.

[0071] The method for historical connection verification of the optimized hierarchical combination method and connection constraint rules includes:

[0072] Obtain the frequency statistics matrix of historical connection instances from the constructed knowledge base; calculate the occurrence probability of the optimized hierarchical combination method in the frequency statistics matrix; when the occurrence probability is lower than a preset occurrence probability threshold, replace the optimized hierarchical combination method with the historical connection order with the highest occurrence probability in the frequency statistics matrix.

[0073] For example, for the optimized hierarchical combination method and connection constraint rules, the system performs historical connection verification to ensure the consistency of the path with the traditional construction rules.

[0074] The historical connection verification is based on a pre-built historical connection instance frequency statistics matrix in the knowledge base. This matrix is ​​generated by statistically analyzing 27,000 sets of official architectural survey data, recording the connection order between different component types and their frequency of occurrence in historical instances, and storing them separately according to regional styles (such as Shanxi style, Suzhou style, and Beijing style) to reflect regional differences.

[0075] The system calculates the probability of the optimized hierarchical combination appearing in the frequency statistics matrix. When the probability of occurrence is lower than a preset threshold of 5%, the system replaces the combination with the historical connection order with the highest probability in the matrix, while retaining the interface cooperation method and relative height difference constraint defined in the connection constraint rules. The preset threshold of 5% is set based on a statistical confidence level to ensure that combinations below this value are rare in history and are likely to be erroneous connections.

[0076] For example, in the gable wall area of ​​ancient buildings, the optimized hierarchical combination is "dougong (bracket set) → roof components → decorative components," with a geometric center height sequence of 4.1 meters for dougong → 6.3 meters for roof components → 5.2 meters for decorative components. A search of the knowledge base shows that this combination occurs with a probability of only 0.8%, below the 5% threshold. The historical frequency matrix shows that the high-frequency path in this area is "dougong → rafters → roof components," with a probability of 41.3%, corresponding to a height sequence of 4.1 meters for dougong → 5.7 meters for rafters → 6.3 meters for roof components.

[0077] Based on this, the system replaced the original combination with a high-frequency path and adjusted the connection direction in the topology diagram to conform to the gravity transmission logic of the historical structure. After the replacement, the connection weight between the brackets and rafters increased from 0.82 to 0.91. Actual measurements verified that the spatial consistency between the updated path and the actual structure reached 98.2%, and the interface direction angle deviation was reduced to 1.5°.

[0078] The method for obtaining the mesh nodes corresponding to structural components in the initial mesh model and adjusting the spatial positions of the mesh nodes according to connection constraint rules to obtain the final mesh model includes:

[0079] Based on the hierarchical combination method and connection constraint rules of structural components, identify the combination of structural components with interface cooperation relationship, obtain the corresponding set of nodes to be aligned based on the combination of structural components, and calculate the spatial pose deviation of the set of nodes to be aligned through the initial mesh model.

[0080] An optimization function is constructed with the goal of minimizing spatial pose deviation, and a curvature constraint term is added. The optimization function is then used to update the spatial coordinates of the node set to be aligned through an iterative optimization algorithm. The final mesh model is obtained when the norm of the spatial coordinate update vector in the adjacent iteration step is less than a preset convergence threshold.

[0081] For example, in the column-beam connection area of ​​the main hall of an ancient building, the initial mesh has a positive Y-axis offset of 3.8cm at the beam end. The coordinates of the column top center are [2.1, 5.3, 0], and the center of the beam bottom surface is [2.1, 5.66, 0]. The system identifies the top contact fit type based on the connection relationship diagram and extracts 12 grid points at the column top and 8 grid points at the beam bottom as an alignment node set. The principal component of the spatial pose deviation ΔY = 3.8cm is calculated.

[0082] Construct the optimization function:

[0083] .

[0084] Where N=12 is the number of nodes to be aligned, corresponding to the 8 nodes at the bottom of the beam being mapped through coordinate transformation; The coordinates of the top node of the column; The coordinates of the bottom node of the beam; The average curvature value of the triangular facet is calculated by the difference of the normal vectors of adjacent nodes. The node normal vector is calculated by weighting the normal vectors of adjacent triangular faces. The facet normal vector is determined by the cross product of the position vectors of its three vertices. and Triangular facets The average curvature values ​​of the initial mesh and the optimized mesh; It is a rigid transformation. ;in, Let represent the 3D rotation group, which is the set of all possible 3D rotation matrices, satisfying ... and It is used to characterize the rotational attitude of a component in three-dimensional space; A three-dimensional coordinate vector representing a point in a dense point cloud or mesh model (usually represented as...) In this optimization function, the coordinates of the top node of the column are specifically referred to. Or the coordinates of the bottom node of the beam ; A three-dimensional translation vector, and a rotation matrix Together they constitute a rigid transformation This is used to adjust the spatial orientation of the components; the curvature constraint term weight λ=0.3, calculated based on the ±0.5cm tolerance of the dovetail joint as specified in the "Engineering Practice Rules," ensuring that the deformation optimization conforms to the tolerance of traditional woodworking techniques. An iterative optimization algorithm is used to update the node coordinates, terminating when the displacement vector norm is less than 0.001mm. The convergence threshold of 0.001mm is equivalent to the accuracy of traditional dovetail joints. After optimization, the maximum distance between the boundary points of the beam bottom and column top is 1.5mm, meeting the load-bearing requirements of the raised beam frame and conforming to the 2mm assembly tolerance threshold.

[0085] The method for obtaining an assembly structure by spatial assembly based on the component parameter template and hierarchical combination method includes:

[0086] The parameterized component model is obtained by retrieving the component parameter template and hierarchical combination method from the pre-built component model library; the interface region and the interface matching method defined in the connection constraint rules in the final mesh model are obtained; the spatial transformation matrix is ​​obtained according to the interface region and interface matching method; the interface node position of the parameterized component model is adjusted according to the spatial transformation matrix and aligned with the mesh node to obtain the spatial positioning model of the structural component; the spatial positioning model is organized according to the hierarchical combination method to generate structural component assembly units.

[0087] The attachment model is calculated using the normal projection method by the attachment conditions defined in the category label, geometry and preset decorative base surface rules of the non-structural component; the assembly unit of the structural component is spatially combined with the attachment model to obtain the assembly structure.

[0088] For example, in the construction of the roof of the main hall in ancient architecture, based on the category label information of structural components, a 4.8-meter span hip-and-gable roof main beam needs to be assembled above the column-bracket composite structure. The interface regions corresponding to the main beam in the final mesh model (i.e., the beam end node set and the column top node set) are used as alignment objects. According to the interface matching method defined in the connection constraint rules, a spatial transformation matrix for interface alignment is calculated. This matrix includes a 5.2° rotation compensation parameter on the Y-axis (measured from the normal direction of the column top node set) and a 3.6cm translation vector (calculated from the difference between the column top center coordinates (2.1, 5.3, 0) and the theoretical centroid of the beam (2.1, 5.66, 0)). This is used to correct the interface node positions of the parametric main beam model, aligning it spatially with the mesh nodes, thus generating a spatial positioning model of the structural component.

[0089] Subsequently, the spatial positioning model was organized according to the hierarchical combination method, and assembly constraints were applied between beams and columns-brackets according to the mortise and tenon fitting angle of 12° and the dovetail depth tolerance of ±0.5mm defined in the connection constraint rules, so as to generate stable structural component assembly units.

[0090] For non-structural components, based on their category label information and geometry, and combined with the attachment conditions defined in the decorative base surface rules, the attachment position is calculated using the normal projection method to obtain the attachment model of the non-structural component. This model is then spatially combined with the structural component assembly unit to generate a complete assembly structure.

[0091] The measured mortise and tenon joint clearance meets the tolerance requirements of the "Engineering Practice Rules and Regulations", and the deviation of the included angle between the attached non-structural components and the normal of the decorative base surface does not exceed 1.5°.

[0092] The method for adjusting the interface node positions of the parameterized component model according to the spatial transformation matrix and aligning them with the mesh nodes to obtain a spatial positioning model of the structural component includes:

[0093] The boundary contour of the parameterized component is registered with the adjacent region in the final mesh model, and the joint deviation is obtained through geometric consistency check. When the joint deviation exceeds the preset assembly tolerance threshold, the spatial pose of the anchor point is iteratively adjusted within the preset degree of freedom constraint range using a nonlinear optimization algorithm. The joint deviation is recalculated after each iteration adjustment. When the joint deviation meets the assembly tolerance threshold, the spatial positioning model of the structural component is obtained.

[0094] For example, the spatial registration mechanism provided by this invention ensures accuracy through a dual tolerance system: the structural component assembly tolerance threshold of 2mm is derived from the upper limit of the mortise and tenon fit tolerance in the "Engineering Practice Rules", and the non-structural component pattern continuity threshold of 0.5 pixels corresponds to the 4K texture mapping standard; the degree of freedom constraint range of the component is translation ±10 mm and rotation ±3°. This range is calculated based on the thermal expansion and contraction tolerance of 0.8% per annum of wood deformation rate, ensuring that the model maintains structural stability and historical appearance consistency in an environment of −20 ℃ to 40 ℃.

[0095] For example, at the interface between the ridge beam and the roof tile of ancient building A, the geometric consistency inspection detected a misalignment of 3.5 mm in the X-axis direction, which is higher than the 2 mm assembly tolerance threshold. The system calls the anchor point degree of freedom constraint of the parameterized beam model to construct an objective function that minimizes the projection error of the joint profile, and uses the Levenberg-Marquardt algorithm for iterative optimization: in the 6th iteration, a 1.2° Z-axis rotation compensation reduces the misalignment to 2.5 mm; in the 12th iteration, the displacement vector norm converges to 0.0003 mm, corresponding to a misalignment of 1.2 mm. The corrected ridge beam meets the assembly tolerance requirements, and finite element analysis verifies that the uniformity of the contact pressure distribution between it and the roof tile is improved by 82% compared to before the correction.

[0096] The method for fusing the structural component mesh regions in the final mesh model with the assembled structure according to connection constraint rules to obtain a structured three-dimensional model of the target ancient building includes:

[0097] Obtain the mesh regions of structural components in the final mesh model, and establish mapping relationships based on the spatial pose of the assembly structure; process the mesh regions according to the mapping relationships and the interface cooperation methods defined in the connection constraint rules to obtain the geometric representation of the structural components; generate an adjacency graph according to the connection constraint rules of the structural components; combine the geometric representation with the adjacency graph to generate a structured 3D model.

[0098] For example, after completing node refinement and component assembly, the system establishes a vertex-to-surface projection mapping relationship between the structural component mesh region in the final mesh model and the already positioned parametric component model, based on the spatial pose of the assembly structure. The threshold for determining the mapping residual is 2mm, which is derived from the upper limit of the mortise and tenon fit tolerance specified in the "Engineering Practice Rules". Combining the interface fit method defined in the connection constraint rules, geometric replacement or fusion is performed within the interface neighborhood: when the bidirectional Hausdorff distance does not exceed 1.5mm and the normal angle does not exceed 5°, the original mesh triangular facet is replaced with the parametric component surface; otherwise, within a neighborhood radius of 40mm, the vertex coordinates are weighted and fused according to the mapping correspondence to obtain the geometric expression of the structural component. The neighborhood radius of 40mm is derived from the statistics of typical mortise and tenon lengths and shoulder widths in the construction knowledge base.

[0099] In this embodiment, the six 0.5m columns with a diameter error of no more than 0.02m and three 4.8m span main beams in the front porch area of ​​the main hall are replaced / fused to form a continuous and seamless geometric representation. Subsequently, an adjacency graph is constructed based on the connection constraints between components, retaining valid connection edges with a weight threshold of 0.85 or higher, and recording the connection type and force transmission direction in the edge attributes. The weight threshold of 0.85 comes from the effective lower limit of the historical sample distribution. The force transmission direction is determined from low to high according to the geometric centroid height, for example, column → beam → bracket → rafter → roof component. In this area, the system generates a total of 18 edges, including typical connections such as "column-beam (mortise and tenon joint, upward)" and "beam-bracket joint (mortise and tenon joint, upward)". Finally, the geometric representation is associated with the adjacency graph to obtain a structured 3D model that can be rendered and contains component-level connection type and force transmission direction information, used for subsequent assembly verification and structural analysis.

[0100] Example 2: Based on the same inventive concept, such as Figure 2 As shown, this embodiment also provides a deep learning-based intelligent reconstruction system for 3D building models, the system comprising:

[0101] The data acquisition and preprocessing module is used to acquire multi-source data of the target ancient building, including visible light image sequences, laser point cloud data, and depth image data; the multi-source data is processed using a motion reconstruction algorithm to obtain a sparse point cloud, and a dense point cloud is obtained based on the sparse point cloud through multi-view stereo matching; the dense point cloud is processed through semantic segmentation to obtain semantic label information, the semantic label information including first information corresponding to structural components and second information corresponding to non-structural components; the dense point cloud carrying the semantic label information is processed through surface reconstruction to obtain an initial mesh model.

[0102] The structural analysis and template matching module is used to analyze the connection relationship between structural components through a neural network to obtain the hierarchical combination method and connection constraint rules of the structural components; and to match the first information and the hierarchical combination method with the corresponding component parameter template through a preset construction knowledge base.

[0103] The model optimization and assembly output module is used to obtain the mesh nodes corresponding to the structural components in the initial mesh model, adjust the spatial position of the mesh nodes according to the connection constraint rules to obtain the final mesh model, perform spatial assembly according to the component parameter template and hierarchical combination method to obtain the assembly structure, and fuse the structural component mesh area in the final mesh model with the assembly structure according to the connection constraint rules to obtain the structured three-dimensional model of the target ancient building.

[0104] It should be noted that the specific ways in which each module operates in the system described in the above embodiments have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0105] Finally, it should be noted that although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent reconstruction of 3D building models based on deep learning, characterized in that, The method includes: The process involves acquiring multi-source data of the target ancient building, including visible light image sequences, laser point cloud data, and depth image data; using a motion-based structure reconstruction algorithm to obtain a sparse point cloud, and then using multi-view stereo matching to obtain a dense point cloud; performing semantic segmentation on the dense point cloud to obtain semantic label information, which includes first information corresponding to structural components and second information corresponding to non-structural components; and finally, using surface reconstruction to obtain an initial mesh model from the dense point cloud carrying the semantic label information. The first information is analyzed through a neural network to parse the connection relationships between structural components, thereby obtaining the hierarchical combination method and connection constraint rules of the structural components; the first information and the hierarchical combination method are matched with the corresponding component parameter templates through a preset construction knowledge base; Obtain the mesh nodes corresponding to the structural components in the initial mesh model, and adjust the spatial position of the mesh nodes according to the connection constraint rules to obtain the final mesh model; assemble the assembly structure according to the component parameter template and the hierarchical combination method; merge the structural component mesh area in the final mesh model with the assembly structure according to the connection constraint rules to obtain the structured three-dimensional model of the target ancient building.

2. The intelligent reconstruction method for 3D building models based on deep learning according to claim 1, characterized in that, The method for obtaining semantic label information from the dense point cloud through semantic segmentation, wherein the semantic label information includes first information corresponding to structural components and second information corresponding to unstructured components, includes: The dense point cloud is divided into multiple sub-blocks. The local geometry, normal vector distribution, color texture information and spatial adjacency of each sub-block are extracted to form a multimodal feature vector. The multimodal feature vector is then used to obtain the semantic label information corresponding to each point in the dense point cloud through a pre-trained point cloud semantic segmentation model. The first information includes the category labels and confidence scores for columns, beams, and brackets; the second information includes the category labels and confidence scores for painted decorations, roof tiles, and corbels. When the confidence score is lower than the preset confidence threshold, the category labels are corrected.

3. The intelligent reconstruction method for 3D building models based on deep learning according to claim 2, characterized in that, The method for analyzing the connection relationships between structural components using a neural network to obtain the hierarchical combination method and connection constraint rules of the structural components includes: Based on the category labels of structural components in the first information and the spatial distribution of dense point clouds, dense point cloud fragments with the same category labels and spatially adjacent are clustered into candidate regions. A topology graph is constructed using the candidate regions as nodes, and the connecting edges of the topology graph represent the spatial pose relationships between nodes. The connection weights of the connecting edges between nodes are analyzed using a graph neural network. Based on the connection weights, a hierarchical combination method and connection constraint rules for structural components are established.

4. The intelligent reconstruction method for 3D building models based on deep learning according to claim 3, characterized in that, The method for establishing the hierarchical combination method and connection constraint rules of structural components based on the connection weights includes: Obtain the geometric centroid height of the structural component corresponding to the first information in the dense point cloud, sort the structural components according to the direction of gravity to obtain the sorting result; obtain the optimized hierarchical combination method and connection constraint rules based on the sorting result; and perform historical connection verification on the optimized hierarchical combination method and connection constraint rules.

5. The intelligent reconstruction method for 3D building models based on deep learning according to claim 4, characterized in that, The method for historical connection verification of the optimized hierarchical combination method and connection constraint rules includes: Obtain the frequency statistics matrix of historical connection instances from the constructed knowledge base; calculate the occurrence probability of the optimized hierarchical combination method in the frequency statistics matrix; when the occurrence probability is lower than a preset occurrence probability threshold, replace the optimized hierarchical combination method with the historical connection order with the highest occurrence probability in the frequency statistics matrix.

6. The intelligent reconstruction method for 3D building models based on deep learning according to claim 3, characterized in that, The method for obtaining the mesh nodes corresponding to structural components in the initial mesh model and adjusting the spatial positions of the mesh nodes according to connection constraint rules to obtain the final mesh model includes: Based on the hierarchical combination method and connection constraint rules of structural components, identify the combination of structural components with interface cooperation relationship, obtain the corresponding set of nodes to be aligned based on the combination of structural components, and calculate the spatial pose deviation of the set of nodes to be aligned through the initial mesh model; An optimization function is constructed with the goal of minimizing spatial pose deviation, and a curvature constraint term is added. The optimization function is then used to update the spatial coordinates of the node set to be aligned through an iterative optimization algorithm. The final mesh model is obtained when the norm of the spatial coordinate update vector in the adjacent iteration step is less than a preset convergence threshold.

7. The intelligent reconstruction method for 3D building models based on deep learning according to claim 6, characterized in that, The method for obtaining an assembly structure by spatial assembly based on the component parameter template and hierarchical combination method includes: The component parameter template and hierarchical combination method are retrieved from the pre-built component model library to obtain a parameterized component model; the interface region and the interface matching method defined in the connection constraint rules in the final mesh model are obtained; a spatial transformation matrix is ​​obtained according to the interface region and interface matching method; the interface node positions of the parameterized component model are adjusted according to the spatial transformation matrix and aligned with the mesh nodes to obtain a spatial positioning model of the structural component; the spatial positioning model is organized according to the hierarchical combination method to generate structural component assembly units; The attachment model is calculated using the normal projection method by the attachment conditions defined in the category label, geometry and preset decorative base surface rules of the non-structural component; the assembly unit of the structural component is spatially combined with the attachment model to obtain the assembly structure.

8. The intelligent reconstruction method for 3D building models based on deep learning according to claim 7, characterized in that, The method for adjusting the interface node positions of the parameterized component model according to the spatial transformation matrix and aligning them with the mesh nodes to obtain a spatial positioning model of the structural component includes: The boundary contour of the parameterized component is registered with the adjacent region in the final mesh model, and the joint deviation is obtained through geometric consistency check. When the joint deviation exceeds the preset assembly tolerance threshold, the spatial pose of the anchor point is iteratively adjusted within the preset degree of freedom constraint range using a nonlinear optimization algorithm. The joint deviation is recalculated after each iteration adjustment. When the joint deviation meets the assembly tolerance threshold, the spatial positioning model of the structural component is obtained.

9. The intelligent reconstruction method for 3D building models based on deep learning according to claim 7, characterized in that, The method for fusing the structural component mesh regions in the final mesh model with the assembled structure according to connection constraint rules to obtain a structured three-dimensional model of the target ancient building includes: Obtain the mesh regions of structural components in the final mesh model, and establish mapping relationships based on the spatial pose of the assembly structure; process the mesh regions according to the mapping relationships and the interface cooperation methods defined in the connection constraint rules to obtain the geometric representation of the structural components; generate an adjacency graph according to the connection constraint rules of the structural components; combine the geometric representation with the adjacency graph to generate a structured 3D model.

10. A deep learning-based intelligent reconstruction system for 3D building models, characterized in that, The system includes: The data acquisition and preprocessing module is used to acquire multi-source data of the target ancient building, including visible light image sequences, laser point cloud data, and depth image data; the multi-source data is processed using a motion reconstruction algorithm to obtain a sparse point cloud, and a dense point cloud is obtained based on the sparse point cloud through multi-view stereo matching; the dense point cloud is processed through semantic segmentation to obtain semantic label information, the semantic label information including first information corresponding to structural components and second information corresponding to non-structural components; the dense point cloud carrying the semantic label information is processed through surface reconstruction to obtain an initial mesh model; The structural analysis and template matching module is used to analyze the connection relationship between structural components through a neural network to obtain the hierarchical combination method and connection constraint rules of the structural components; and to match the first information and the hierarchical combination method with the corresponding component parameter template through a preset construction knowledge base. The model optimization and assembly output module is used to obtain the mesh nodes corresponding to the structural components in the initial mesh model, adjust the spatial position of the mesh nodes according to the connection constraint rules to obtain the final mesh model, perform spatial assembly according to the component parameter template and hierarchical combination method to obtain the assembly structure, and fuse the structural component mesh area in the final mesh model with the assembly structure according to the connection constraint rules to obtain the structured three-dimensional model of the target ancient building.

Images