A multi-dimensional ancient building maintenance and restoration method based on 3D laser scanning

Abstract

This invention is a multi-dimensional ancient building maintenance and restoration method based on 3D laser scanning, relating to the field of computer processing technology. The method comprises the following steps: S01, using a terrestrial laser scanner to acquire basic 3D point cloud data of the ancient building. This basic 3D point cloud data includes geometric information of the building facade, ground structure, and near-ground decorative components; S02, using an unmanned aerial vehicle equipped with a multi-line laser scanner to scan the building roof and high-altitude structures to generate supplementary point cloud data. This supplementary point cloud data covers the millimeter-level precision 3D coordinates of the sloping roof, tower eaves, and eaves brackets; S03, using a multispectral imaging device to simultaneously collect surface texture data of the ancient building. This invention achieves millimeter-level accuracy in 3D reconstruction of ancient buildings through multi-source data fusion and a hierarchical optimization algorithm. Combined with automated defect identification and multimodal analysis techniques, this method improves the efficiency of maintenance decision-making.

Description

Technical Field

[0001] The present invention relates to the field of computer processing technology, and in particular to a multi-dimensional ancient building maintenance and restoration method based on three-dimensional laser scanning. Background Art

[0002] Traditional 3D scanning technology for ancient buildings primarily relies on terrestrial laser scanners, which are limited by the instrument's mounting height and viewing angle, making it difficult to capture detailed data on the tops of ancient buildings (such as sloping roofs and tower eaves) and complex structures (such as flying eaves and brackets). Although drone oblique photography can supplement aerial imagery, the resulting models are relatively inaccurate (typically with errors down to the centimeter level), and are prone to geometric misalignment and texture distortion when fused with ground point cloud data. Summary of the Invention

[0003] In response to the above technical problems, the technical solution adopted by the present invention is a multi-dimensional ancient building maintenance and restoration method based on three-dimensional laser scanning, which includes the following steps:

[0004] S01. Using a terrestrial laser scanner to obtain basic three-dimensional point cloud data of the ancient building, the basic three-dimensional point cloud data includes geometric information of the building facade, ground structure, and near-ground decorative components;

[0005] S02. Scan the building top and high-altitude structures using a multi-line laser scanning device mounted on an unmanned aerial vehicle to generate supplementary point cloud data. The supplementary point cloud data covers the millimeter-level precision three-dimensional coordinates of the sloping roof, tower eaves, and eaves brackets;

[0006] S03. Using multispectral imaging equipment to synchronously collect surface texture data of ancient buildings, wherein the texture data includes material spectral reflectance, color degradation gradient, and structural damage characteristics;

[0007] S04. Construct a multimodal deep learning network including a spatial registration module, a feature fusion module, a semantic enhancement module, and a restoration decision module;

[0008] S05. Performing multi-scale coarse registration on the basic three-dimensional point cloud data and the supplementary point cloud data based on an improved iterative closest point algorithm to generate an initial fused point cloud;

[0009] S06, extracting a spectral joint descriptor of the local geometry of the initial fused point cloud and texture data using an adaptive scale-invariant feature transformation algorithm;

[0010] S07. Performing non-rigid registration optimization on the spectral joint descriptor of the local geometry through a hierarchical graph convolutional network to eliminate data splicing dislocation and generate a geometrically continuous multimodal fusion model;

[0011] S08. Using the semantic enhancement module to perform component-level semantic segmentation on the multimodal fusion model to identify fractures in mortise and tenon joints, peeling of painted layers, and traces of historical repairs;

[0012] S09. Based on the structural integrity assessment result output by the restoration decision module, the structural integrity assessment result includes the three-dimensional coordinates of the damaged component, traditional process parameters, and digital maintenance solution results of material compatibility.

[0013] Preferably, in step S02, the drone equipped with a multi-line laser scanning device scans the building top and high-altitude structure, including:

[0014] S21. Calculate the required number of scan lines according to formula (1) based on the target building height H and the safe flight height h of the UAV.

[0015] (1),

[0016] Where θ is the maximum elevation angle of the cornice, d res The resolution required by the design.

[0017] S22, the dynamic posture compensation algorithm of the supplementary point cloud data includes:

[0018] S221, establish the calibration conversion matrix T from the IMU coordinate system to the laser scanner calib ;

[0019] S222, for each laser point’s timestamp t, interpolate and obtain the drone’s pose T at the corresponding moment t ;

[0020] S223, obtain the corrected global coordinate system point cloud coordinates: P global =T t ·T calib ·P raw , where P raw Correct the original point cloud coordinates;

[0021] S23. Extract the step height measurement value extracted by the point cloud model based on the acquired global coordinate system point cloud coordinates, and calculate the relative error between the step height measurement value extracted by the point cloud model and the true value based on a step block of known size fixed at a typical position of the eaves.

[0022] Preferably, the processing step of synchronously collecting the surface texture data of the ancient building by the multispectral imaging device in step S03 includes:

[0023] S31. Use the reflectance normalization algorithm to extract the paint layer aging characteristics from the UV band data. The normalization formula is:

[0024] ,

[0025] Among them, R UV is the original reflectivity, R min is the minimum reflectance threshold of a healthy paint layer, R max is the maximum reflectance threshold of the healthy paint layer, is the standard reflectance threshold of a healthy paint layer;

[0026] S32. Use principal component analysis to separate the decayed areas inside the wood using near-infrared band data. The principal component contribution rate calculation satisfies:

[0027] ,

[0028] Among them, λ k is the eigenvalue of the covariance matrix, n is the total number of bands, and k is the number of covariance matrices;

[0029] S33, the superpixel level alignment includes:

[0030] S331, generate a Voronoi diagram on the point cloud surface, define the super pixel block center point set {c i} to segment a multispectral image into geometrically related superpixel blocks:

[0031] ,

[0032] Among them, V i is the spatial area covered by the i-th superpixel block, α is the spatial weight coefficient, which is 0.5, c i is the center point coordinate of the i-th superpixel block, d 2 (p,c i ) is from point p to center point c i The Euclidean distance of

[0033] S332, calculating a joint histogram of the HSV color space and the near-infrared reflectivity for each pixel block;

[0034] S333: Establish a mapping relationship between the superpixel block and the point cloud triangle patch using a thin plate spline interpolation algorithm. The interpolation function is:

[0035] ,

[0036] Among them, (x, y) is the target pixel coordinate, (x i ,y i ): coordinates of the i-th control point, a0, a1, a2 are affine transformation coefficients, w i is the radial basis function weight.

[0037] S334. Encode the material spectral reflectance and the structural damage characteristics into additional point cloud attribute fields.

[0038] Preferably, the improved iterative closest point algorithm in step S05 includes:

[0039] S51. Introduce curvature constraints to filter initial matching point pairs and eliminate those with curvature differences exceeding the threshold. point;

[0040] S52, using a bidirectional KD tree to accelerate the nearest neighbor search, and optimizing the rigid body transformation matrix based on the Gauss-Newton method;

[0041] S53, the coarse registration error is verified by the following method:

[0042] S531. Arrange an array of control points on the standard specimen of the ancient building and obtain the reference coordinates using a total station.

[0043] S532. Calculate the root mean square error of the Euclidean distance between the point cloud and the control point coordinates after alignment using the iterative closest point algorithm.

[0044] Preferably, the non-rigid registration optimization of the local geometric spectral joint descriptor by the hierarchical graph convolutional network in step S07 includes:

[0045] S71. Construct an attribute graph with point cloud normal vectors, curvature and spectral features as nodes, with a neighborhood radius of r = 5 cm;

[0046] S72, the first level uses the EdgeConv operator, and the feature aggregation function is:

[0047] ,

[0048] Among them, Θ is the learnable parameter matrix, N(i) is the r neighborhood of point i, For the The feature vector of the i-th node in the layer, For the The feature vector of the jth node in the layer, For the The feature vector of the i+1th node in the layer;

[0049] S73. The second level introduces the channel attention mechanism, and the feature weighting formula is:

[0050] ,

[0051] Among them, g c is the global average pooling feature of the cth channel, m cis the global maximum pooling feature of the c-th channel, σ is the sigmoid function, W1 and W2 are the basis function weights of the first node and the second node.

[0052] Preferably, the step S08 of performing component-level semantic segmentation on the multimodal fusion model using the semantic enhancement module includes the following steps:

[0053] S81. Use three-dimensional morphological opening operation to extract the connection gap of mortise and tenon joints. The structural element is the radius r. morph =3mm sphere;

[0054] S82. Determine the fracture grade based on the point cloud density gradient, and mark it as a severe fracture when the density drop rate η is ≥ 30%;

[0055] S83. For the peeling area of ​​the painted layer, compare the spectral reflectance difference matrix with the historical data. If the difference value ΔR ≥ 0.2, it is determined to be man-made damage;

[0056] S84. For historical repair traces, use graph neural network to analyze the similarity of process characteristics and extract the curvature radius r of tool marks. tool and the angle θ between the wood fibers fiber .

[0057] Preferably, the digital maintenance solution result in step S09 further includes:

[0058] S91. Calculation of stress concentration factor K of damaged components based on finite element mechanics simulation model t , when K t When the value is ≥2.5, it is marked as a high-risk area, otherwise, it is a low-risk area;

[0059] S92, combined with the traditional craftsman database to match the mortise and tenon types, according to the formula Select the historical process parameters with the highest adaptability, where S match is the best historical process parameter set after matching, D current is the digital feature vector of the current damaged component, D history is the process parameter feature vector stored in the historical process database;

[0060] S93. Generate a NC machining path for repairing the slope roof surface. The machining step length is adaptively adjusted according to the curvature. The adjustment formula is:

[0061] ,

[0062] in, is the NC machining step size, and κ is the Gaussian curvature of the surface.

[0063] The present invention has at least the following beneficial effects:

[0064] 1. Through the collaborative optimization of an improved iterative closest point algorithm and a hierarchical graph convolutional network, millimeter-level registration of ground and aerial point clouds is achieved. This can effectively solve the geometric misalignment problem at the connection between cornices and columns, and ensure the topological accuracy of the 3D reconstruction of complex wooden structures.

[0065] 2. Based on the component-level semantic segmentation algorithm, it can automatically identify structural defects such as mortise and tenon fractures and peeling of painted patterns, and can also trace the craftsmanship characteristics of historical repair traces.

[0066] 3. By integrating multispectral texture and geometric point cloud data, the correlation analysis between material aging and structural deformation is achieved through super-pixel alignment technology (HSV-near-infrared joint histogram), which improves the accuracy of detecting decayed areas of wood and quantifies the degree of aging of the paint layer. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0068] Figure 1 A flowchart of a method for maintaining and restoring multiple ancient buildings based on three-dimensional laser scanning provided in Example 1 of the present invention;

[0069] Figure 2 This is a schematic diagram of the implementation structure of S01 and S02 provided in the first embodiment of the present invention. DETAILED DESCRIPTION

[0070] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0071] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way are interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0072] Example 1:

[0073] This embodiment provides a method for maintaining and restoring multiple ancient buildings based on three-dimensional laser scanning, which includes the following steps: Figure 1 As shown:

[0074] S01. Use a terrestrial laser scanner to obtain basic 3D point cloud data of the ancient building. The basic 3D point cloud data includes geometric information of the building facade, ground structure, and near-ground decorative components.

[0075] Specifically, such as Figure 2 As shown in the figure, the purpose of using terrestrial laser scanning to acquire a basic point cloud is to establish the geometric benchmark for the main structure of the ancient building, capturing millimeter-level precision 3D data of near-ground components such as facades, column bases, and steps. This provides a spatial reference framework for subsequent high-altitude point cloud registration, ensuring a unified coordinate system for multi-source data. The so-called geometric information is the 3D coordinates, that is, the coordinates of each feature in the 3D point cloud data.

[0076] This method uses a phase-shifted terrestrial laser scanner (such as the Faro Focus S350) with a scanning rate of ≥976,000 points / second and a ranging accuracy of ±1mm. Basic 3D point cloud data acquisition is performed by deploying scanning stations around the building perimeter, with spacing ≤15m between stations and ensuring a point cloud overlap of ≥30%. Target spheres (10cm diameter) are then used as common control points to aid in cloud stitching across multiple stations.

[0077] Furthermore, the acquired basic 3D point cloud data is processed, and the detailed processing steps include:

[0078] S11, remove outliers through statistical filtering (threshold: 2σ outside the point cloud from the mean);

[0079] S12. Based on the target sphere coordinates, the least squares method is used to calculate the rigid body transformation matrix between the sites;

[0080] S13. Use voxel filtering (resolution 5mm) to compress the data volume and retain the geometric features.

[0081] S02. Scan the building top and high-altitude structures using a multi-line laser scanning device mounted on a drone to generate supplementary point cloud data. The supplementary point cloud data covers the millimeter-level precision three-dimensional coordinates of the sloping roof, tower eaves, and flying eaves and brackets;

[0082] Specifically, drones equipped with multi-line laser scanners scan building roofs and high-altitude structures to generate supplemental point cloud data. This helps cover blind spots in ground-based scanning (such as sloping roofs and overhanging eaves), addressing the gaps and low precision of traditional drone photogrammetry models. Dynamic pose compensation eliminates the impact of drone flight jitter on point cloud quality.

[0083] The above-mentioned drone scanning acquisition method includes the following steps:

[0084] S21. Calculate the required number of scan lines according to formula (1) based on the target building height H and the safe flight height h of the UAV.

[0085] (1),

[0086] Where θ is the maximum elevation angle of the cornice, d res The resolution required by the design.

[0087] S22. The dynamic pose compensation algorithm for supplementing point cloud data includes:

[0088] S221, establish the calibration conversion matrix T from the IMU coordinate system to the laser scanner calib ;

[0089] S222, for each laser point’s timestamp t, interpolate and obtain the drone’s pose T at the corresponding moment t ;

[0090] S223, obtain the corrected global coordinate system point cloud coordinates: P global =T t ·T calib ·P raw , where P raw Correct the original point cloud coordinates;

[0091] S23. Extract the step height measurement value extracted by the point cloud model based on the acquired global coordinate system point cloud coordinates, and calculate the relative error between the step height measurement value extracted by the point cloud model and the true value based on a step block of known size fixed at a typical position of the eaves.

[0092] Specifically, such as Figure 2As shown, the building height H and the drone's flight altitude h jointly determine the relative spatial position of the scanning system and the target component. When the building height increases or the drone's flight altitude needs to be increased due to safety restrictions, a formula automatically increases the number of scanning lines to ensure vertical data acquisition density. Maximum eave elevation angle θ: Given the unique eaves and bracket structures of traditional Chinese architecture, the elevation parameter is directly related to the scanning device's field of view coverage. For example, when θ > 45°, the formula automatically increases the number of lines to compensate for point cloud loss on steeply curved surfaces. Resolution dres: Based on the minimum feature recognition size set for maintenance requirements (such as tile patterns or crack width), when dres ≤ 2mm, the system increases the number of lines to meet high-precision requirements. Through mathematical constraints between parameters, empirically driven parameter settings are transformed into quantitative decisions based on building physical characteristics, addressing the scanning blind spots caused by improper manual settings in traditional methods.

[0093] Furthermore, the transformation matrix T from the IMU coordinate system to the laser scanner is obtained through laboratory pre-calibration. calib , establish the spatial pose mapping relationship of six degrees of freedom (6-DoF). The calibration process uses a checkerboard target multi-point acquisition to minimize the system error caused by installation deviation. Secondly, the discrete pose data T output by the IMU is calibrated using the B-spline curve. t Continuous processing is performed to ensure that the timestamp t of each laser point can obtain the corresponding real-time position of the drone. The interpolation frequency is synchronized with the laser scanning rate (usually 10-20Hz) to avoid phase delay of motion information. Furthermore, the motion compensation formula P is applied global =T t ·T calib ·P raw , reconstructing the coordinates of the original point cloud. The physical meaning of this formula is to inversely compensate the drone's pose (including position offset and attitude angle change) at the moment the laser pulse is emitted into the point cloud data. For example, when the drone experiences pitch oscillation, the algorithm eliminates the resulting stretching deformation of the point cloud by inversely rotating the matrix.

[0094] S03. Use multispectral imaging equipment to synchronously collect surface texture data of ancient buildings. The texture data includes material spectral reflectance, color degradation gradient, and structural damage characteristics.

[0095] Specifically, multispectral imaging captures surface textures to obtain the material's spectral fingerprint, identifying hidden defects such as paint aging and wood decay. It also provides multimodal data support for subsequent semantic segmentation, enhancing the reliability of structural damage interpretation.

[0096] During the actual implementation, the multispectral imaging equipment used was a HySpex VNIR-1800 imaging spectrometer (180 wavelengths, spectral range 400-1000 nm). The laser scanner was triggered synchronously during the data acquisition process to ensure spatiotemporal alignment of the spectral image and the point cloud. Data acquisition was performed under overcast conditions (light intensity 5000-10000 lux) to avoid interference from specular reflections. The same area was captured from multiple angles (angle intervals ≤ 15°) to eliminate shadows.

[0097] The processing steps for synchronously collecting surface texture data of ancient buildings by the multispectral imaging device include:

[0098] S31. Use the reflectance normalization algorithm to extract the paint layer aging characteristics from the UV band data. The normalization formula is:

[0099] ,

[0100] Among them, R UV is the original reflectivity, R min is the minimum reflectance threshold of a healthy paint layer, R max is the maximum reflectance threshold of the healthy paint layer, is the standard reflectance threshold of a healthy paint layer;

[0101] S32. Use principal component analysis to separate the decayed areas inside the wood using near-infrared band data. The principal component contribution rate calculation satisfies:

[0102] ,

[0103] Among them, λ k is the eigenvalue of the covariance matrix, n is the total number of bands;

[0104] S33, superpixel alignment includes:

[0105] S331, generate a Voronoi diagram on the point cloud surface, define the super pixel block center point set {c i} to segment a multispectral image into geometrically related superpixel blocks:

[0106] ,

[0107] Among them, V i is the spatial area covered by the i-th superpixel block, is the spatial weight coefficient, which is 0.5, c i is the center point coordinate of the i-th superpixel block, d 2 (p,c i ) is from point p to center point c i The Euclidean distance of

[0108] S332, calculating a joint histogram of the HSV color space and the near-infrared reflectivity for each superpixel block;

[0109] S333. A mapping relationship between superpixel blocks and point cloud triangles is established using a thin plate spline interpolation algorithm. The interpolation function is:

[0110] ,

[0111] Among them, (x, y) is the target pixel coordinate, (x i ,y i ): coordinates of the i-th control point, a0, a1, a2 are affine transformation coefficients, w i is the radial basis function weight.

[0112] S334. Encode the material spectral reflectance and structural damage characteristics as additional point cloud attribute fields.

[0113] This method, through a UV reflectance normalization algorithm, enables the first nondestructive quantitative detection of the degree of aging in traditional paint coatings, overcoming the subjective limitations of traditional visual inspection. Secondly, using near-infrared principal component analysis, a mapping relationship between internal decay characteristics and spectral response is established, enabling detection of internal defects up to 15 cm deep. Furthermore, a geometric-spectral joint modeling approach based on superpixel alignment accurately maps texture attributes to a 3D point cloud, addressing issues such as texture stretching and seam misalignment that plague traditional mapping methods.

[0114] Furthermore, normalization processing can eliminate the influence of different lighting conditions, where: R min Taken from the laboratory calibration value of the unaged paint layer (such as the benchmark reflectance of 0.32 of the red lead paint commonly used in Qing Dynasty paintings), R max The reflectivity threshold corresponding to the completely carbonized paint layer is 0.08. The normalized value range [0,1] directly reflects the degree of aging. When <0.4, a maintenance warning is triggered.

[0115] PCA dimensionality reduction was performed on 128 near-infrared bands (900-1700 nm) in the near-infrared band data, retaining the first three principal components. The first principal component reflects the density distribution of wood, the second principal component represents the moisture content, and the third principal component is associated with microbial metabolites. By establishing the cluster center of the decay area in the feature space (cluster center = [-1.2, 0.6, 0.3]), the Mahalanobis distance was used to determine the decay probability D M , when D M When the value is >2.5, it is judged as internal decay.

[0116] Next, a Voronoi diagram is generated on the point cloud surface, and superpixel segmentation is performed using the triangle patch centroid as the seed point. The objective function balances geometric proximity and spectral consistency. The dimensions of the HSV-NIR joint histogram for each superpixel block (approximately 5 cm × 5 cm) are (18 × 12 × 8). Sub-pixel mapping is established using a thin plate spline interpolation function, with an interpolation error of less than 0.5 pixels.

[0117] S04. Construct a multimodal deep learning network including a spatial registration module, a feature fusion module, a semantic enhancement module, and a restoration decision module;

[0118] Specifically, the significance of building a multimodal deep learning network lies in designing an end-to-end network architecture to achieve geometric and spectral data fusion analysis and automated decision-making. This also replaces traditional manual interpretation and solves the problem of low efficiency in processing complex structures of ancient buildings. The aforementioned multimodal deep learning network architecture is as follows:

[0119] Spatial registration module: Achieve rough alignment of point clouds based on an improved iterative closest point algorithm (see step S05).

[0120] Feature fusion module: adopts a two-stream network (geometric stream + spectral stream) to fuse features through a cross-attention mechanism.

[0121] Semantic enhancement module: 3DU-Net structure, supports point cloud semantic segmentation (see step S08).

[0122] Restoration decision module: Combines reinforcement learning (PPO algorithm) to optimize maintenance plan generation.

[0123] S05. Perform multi-scale coarse registration of the basic 3D point cloud data and the supplementary point cloud data based on the improved iterative closest point algorithm to generate an initial fused point cloud;

[0124] Specifically, the above solution solves the initial pose deviation of large-scale point clouds, providing high-quality initial values ​​for fine registration. Curvature constraints are also used to improve registration robustness and reduce noise interference.

[0125] The improved iterative closest point algorithm mentioned above includes:

[0126] S51. Introduce curvature constraints to filter initial matching point pairs and eliminate those with curvature differences exceeding the threshold. point;

[0127] S52, using a bidirectional KD tree to accelerate the nearest neighbor search, and optimizing the rigid body transformation matrix based on the Gauss-Newton method;

[0128] S53. The coarse registration error is verified by the following method:

[0129] S531. Arrange an array of control points on the standard specimen of the ancient building and obtain the reference coordinates using a total station.

[0130] S532. Calculate the root mean square error of the Euclidean distance between the point cloud and the control point coordinates after alignment using the iterative closest point algorithm.

[0131] Specifically, the original point cloud is fitted with the moving least squares (MLS) surface method, and the maximum principal curvature k1 and minimum principal curvature k2 of each point are calculated. The curvature difference threshold is , to remove points that do not meet the conditions.

[0132] When constructing a bidirectional KD tree structure, the ground point cloud tree depth is set to 12, and the drone point cloud tree depth is set to 10. The Best-Bin-First (BBF) algorithm is used for search, and the maximum number of backtracking is limited to 200 times.

[0133] Furthermore, the control point array arranged on the ancient building standard specimen is 24 spherical control points (2 cm in diameter) arranged on the surface of the ancient building specimen, and the feature points are screened through curvature constraints to reduce the erroneous correspondence between the eaves surface and the column plane.

[0134] S06, using an adaptive scale-invariant feature transformation algorithm to extract a spectral joint descriptor of the local geometry of the initial fused point cloud and texture data;

[0135] Specifically, the significance of the spectral joint descriptor of the local geometry of the initial fusion point cloud and texture data is to build cross-modal feature associations, solve the problem of heterogeneous matching of geometric and spectral data, and enhance the feature discrimination of complex structures such as flying eaves and brackets.

[0136] In the actual implementation process,

[0137] Extract ISS (Intrinsic Shape Signatures) key points on the point cloud surface, with a response value threshold > 0.8.

[0138] Extract SURF (Speeded-Up Robust Features) key points from multispectral images.

[0139] Descriptor construction:

[0140] Geometric descriptor: Calculate the normal vector histogram (SHOT descriptor) of the key point neighborhood (radius 10cm).

[0141] Spectral descriptor: Extracts multi-band reflectance statistics (mean, variance) of a 5×5 pixel window around the key point.

[0142] Joint descriptor: The geometric and spectral features are concatenated into a 256-dimensional vector, and PCA is used to reduce the dimensionality to 64 dimensions.

[0143] S07. Optimize the non-rigid registration of the spectral joint descriptor of local geometry through a hierarchical graph convolutional network to eliminate data splicing misalignment and generate a geometrically continuous multimodal fusion model.

[0144] Specifically, the above solves the local registration error caused by non-rigid deformation (such as wood deformation) and adaptively fuses multi-source data features through the attention mechanism.

[0145] A merchant's node is each point in the point cloud, with attributes including coordinates, normal vector, curvature, and spectral characteristics. An edge is a neighborhood point within a 5cm radius, with edge weights determined by both Euclidean distance and feature similarity.

[0146] Furthermore, the non-rigid registration optimization of the local geometric spectral joint descriptor through the hierarchical graph convolutional network includes:

[0147] S71. Construct an attribute graph with point cloud normal vectors, curvature and spectral features as nodes, with a neighborhood radius of r = 5 cm;

[0148] S72, the first level uses the EdgeConv operator, and the feature aggregation function is:

[0149] ,

[0150] Among them, Θ is the learnable parameter matrix, N(i) is the r neighborhood of point i, For the The feature vector of the i-th node in the layer, For the The feature vector of the jth node in the layer, For the The feature vector of the i+1th node in the layer;

[0151] S73. The second level introduces the channel attention mechanism, and the feature weighting formula is:

[0152] ,

[0153] Among them, g c is the global average pooling feature of the cth channel, m c is the global maximum pooling feature of the c-th channel, σ is the sigmoid function, W1 and W2 are the basis function weights of the first node and the second node.

[0154] Specifically, the first level (geometric consistency constraint): when constructing the attribute graph, the node features include normal vector, curvature (k) and near-infrared reflectivity (R NIR), the neighborhood radius of the EdgeConv operator is r = 5 cm, and the parameter matrix Θ in the feature aggregation function is 64 × 128. Local geometric variation patterns are captured through maximum pooling. This layer outputs a 128-dimensional feature vector that represents the local surface geometry.

[0155] Second level (attention feature fusion): the channel attention module receives the global average pooling feature g c And the maximum pooling feature m c Through the weight matrix W1∈R 128×64 , W2∈R 64×1 , calculate the channel importance coefficient , where σ is the sigmoid function. Apply 1.2 times weight compensation to the drone point cloud features, and the final fusion feature is .

[0156] S08. Use the semantic enhancement module to perform component-level semantic segmentation on the multimodal fusion model to identify fractured mortise and tenon joints, peeling of painted layers, and traces of historical repairs;

[0157] The above-mentioned component-level semantic segmentation of the multimodal fusion model using the semantic enhancement module includes the following steps:

[0158] S81. Use three-dimensional morphological opening operation to extract the connection gap of mortise and tenon joints. The structural element is the radius r. morph =3mm sphere;

[0159] S82. Determine the fracture grade based on the point cloud density gradient. A density drop rate η ≥ 30% is marked as a severe fracture.

[0160] S83. For the peeling area of ​​the painted layer, compare the spectral reflectance difference matrix with the historical data. If the difference value ΔR ≥ 0.2, it is determined to be man-made damage;

[0161] S84. For historical repair traces, use graph neural network to analyze the similarity of process characteristics and extract the curvature radius r of tool marks. tool and the angle θ between the wood fibers fiber .

[0162] Specifically, a 3D opening operation is performed using a spherical structural element with a radius of 3 mm. open =(IB)+B, where I is the point cloud density field and B is the structural element. Calculate the volume change rate before and after the opening operation When η≥30%, it is judged as structural fracture.

[0163] Furthermore, the spectral difference matrix is ​​constructed Combined with the fluorescence characteristics of the ultraviolet band: natural aging shows a gradual difference (ΔR ≤ 0.15), while human damage shows a step change (ΔR ≥ 0.2). Markov random fields (MRF) are used to optimize spatial consistency in disputed areas to eliminate light interference.

[0164] Furthermore, the cross-sectional curve of the tool mark is generated through the 3D point cloud, and the curvature radius is calculated. , a traditional chisel from the Qing Dynasty tool The typical value range is 2.5-3.5mm, and modern power tools are 0.8-1.2mm, which can be used to determine the age of the repair. fiber The angle between the wood fiber direction and the tool mark is used to determine the processing technology. The formula is:

[0165] ,

[0166] in, is the direction vector, is the first-order derivative of the tool mark, is the second-order derivative of the tool mark, These are hand-carved marks from the Qing Dynasty. For now power tool chisel marks.

[0167] S09. Based on the structural integrity assessment results output by the restoration decision module, the structural integrity assessment results include the three-dimensional coordinates of the damaged components, traditional process parameters, and digital maintenance plan results of material compatibility.

[0168] The results of the aforementioned digital maintenance solution also include:

[0169] S91. Calculation of stress concentration factor K of damaged components based on finite element mechanics simulation model t , when K t When the value is ≥2.5, it is marked as a high-risk area, otherwise, it is a low-risk area;

[0170] S92, combined with the traditional craftsman database to match the mortise and tenon types, according to the formula Select the historical process parameters with the highest adaptability, where S match is the best historical process parameter set after matching, D current is the digital feature vector of the current damaged component, D history is the process parameter feature vector stored in the historical process database;

[0171] S93. Generate a NC machining path for repairing the slope roof surface. The machining step length is adaptively adjusted according to the curvature. The adjustment formula is:

[0172] ,

[0173] in, is the CNC machining step length, and k is the Gaussian curvature of the surface.

[0174] Specifically, the point cloud model was converted into a finite element mesh (element size ≤ 5 mm), and loaded with deadweight and wind load (0.45 kN / m 2 ) and snow load (0.3kN / m 2 ). Calculate the Mises stress distribution, when the stress concentration factor K t =σ max / σ nom ≥2.5K, it is determined that the component is at risk of fracture.

[0175] More than 2000 sets of historical process parameters (such as mortise and tenon types, tenon size tolerances) are stored in the craftsman database. match Find the best match. The system automatically matches the "dovetail tenon" craftsmanship of the Qing Dynasty (tenon length error ±0.3mm).

[0176] The machining step size (Δs) (in mm) is adaptively adjusted based on the surface's Gaussian curvature, k. A 5mm step size is used to improve efficiency in low-curvature areas (k < 0.1), while a 0.2mm step size is used to ensure accuracy in high-curvature cornices (k > 0.5). The generated tool compensation parameters ensure that the machining error of the wood components is ≤ 0.15mm.

[0177] In summary, the above-mentioned embodiment 1 realizes millimeter-level registration of ground and high-altitude point clouds through the collaborative optimization of the improved iterative nearest point algorithm and the hierarchical graph convolutional network, which can effectively solve the problem of geometric misalignment at the connection between the eaves and the columns, and ensure the topological accuracy of the three-dimensional reconstruction of complex wooden structures. Secondly, based on the component-level semantic segmentation algorithm, it can automatically identify structural defect characteristics such as mortise and tenon fractures and peeling of painted materials, and can trace the process characteristics of historical repair traces. Furthermore, by fusing multispectral texture and geometric point cloud data, the correlation analysis of material aging and structural deformation is realized through superpixel-level alignment technology (HSV-near-infrared joint histogram), which improves the detection accuracy of wood decay areas and quantifies the degree of paint layer aging.

[0178] Example 2:

[0179] An embodiment of the present invention provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by a processor to implement the steps:

[0180] Use a terrestrial laser scanner to obtain three-dimensional point cloud data of the ancient building foundation;

[0181] Use a drone equipped with a multi-line laser scanner to scan building tops and high-altitude structures to generate supplementary point cloud data;

[0182] Use multispectral imaging equipment to synchronously collect surface texture data of ancient buildings;

[0183] Build a multimodal deep learning network including spatial registration module, feature fusion module, semantic enhancement module and restoration decision module;

[0184] Based on the improved iterative closest point algorithm, the basic 3D point cloud data and the supplementary point cloud data are coarsely registered at multiple scales to generate the initial fused point cloud.

[0185] Adaptive scale-invariant feature transformation algorithm is used to extract the spectral joint descriptor of the local geometry of the initial fused point cloud and texture data;

[0186] A hierarchical graph convolutional network is used to perform non-rigid registration optimization on the spectral joint descriptor of local geometry, eliminating data splicing misalignment and generating a geometrically continuous multimodal fusion model.

[0187] The semantic enhancement module is used to perform component-level semantic segmentation on the multimodal fusion model to identify broken mortise and tenon joints, peeling of painted layers, and historical repair traces.

[0188] Structural integrity assessment results output by the restoration decision module.

[0189] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware using a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the various embodiments provided herein may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0190] Those skilled in the art will clearly understand that for the sake of convenience and brevity in description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0191] Example 3:

[0192] An embodiment of the present invention provides an electronic device, including a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the following steps:

[0193] Use a terrestrial laser scanner to obtain three-dimensional point cloud data of the ancient building foundation;

[0194] Use a drone equipped with a multi-line laser scanner to scan building tops and high-altitude structures to generate supplementary point cloud data;

[0195] Use multispectral imaging equipment to synchronously collect surface texture data of ancient buildings;

[0196] Build a multimodal deep learning network including spatial registration module, feature fusion module, semantic enhancement module and restoration decision module;

[0197] Based on the improved iterative closest point algorithm, the basic 3D point cloud data and the supplementary point cloud data are coarsely registered at multiple scales to generate the initial fused point cloud.

[0198] Adaptive scale-invariant feature transformation algorithm is used to extract the spectral joint descriptor of the local geometry of the initial fused point cloud and texture data;

[0199] A hierarchical graph convolutional network is used to perform non-rigid registration optimization on the spectral joint descriptor of local geometry, eliminating data splicing misalignment and generating a geometrically continuous multimodal fusion model.

[0200] The semantic enhancement module is used to perform component-level semantic segmentation on the multimodal fusion model to identify broken mortise and tenon joints, peeling of painted layers, and historical repair traces.

[0201] Structural integrity assessment results output by the restoration decision module.

[0202] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as a preferred embodiment as above, it is not intended to limit the present invention. Any technician familiar with the present profession can make some changes or modifications to equivalent embodiments of the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A multi-dimensional ancient building maintenance and restoration method based on 3D laser scanning, characterized in that: The method comprises the following steps: S01. Using a terrestrial laser scanner to obtain basic three-dimensional point cloud data of the ancient building, the basic three-dimensional point cloud data includes geometric information of the building facade, ground structure, and near-ground decorative components; S02. Scan the building top and high-altitude structures using a multi-line laser scanning device mounted on an unmanned aerial vehicle to generate supplementary point cloud data. The supplementary point cloud data covers the millimeter-level precision three-dimensional coordinates of the sloping roof, tower eaves, and eaves brackets; S03. Using multispectral imaging equipment to synchronously collect surface texture data of ancient buildings, wherein the texture data includes material spectral reflectance, color degradation gradient, and structural damage characteristics; S04. Construct a multimodal deep learning network including a spatial registration module, a feature fusion module, a semantic enhancement module, and a restoration decision module; S05. Performing multi-scale coarse registration on the basic three-dimensional point cloud data and the supplementary point cloud data based on an improved iterative closest point algorithm to generate an initial fused point cloud; S06, extracting a spectral joint descriptor of the local geometry of the initial fused point cloud and texture data using an adaptive scale-invariant feature transformation algorithm; S07. Performing non-rigid registration optimization on the spectral joint descriptor of the local geometry through a hierarchical graph convolutional network to eliminate data splicing dislocation and generate a geometrically continuous multimodal fusion model; S08. Using the semantic enhancement module to perform component-level semantic segmentation on the multimodal fusion model to identify fractures in mortise and tenon joints, peeling of painted layers, and traces of historical repairs; S09. Based on the structural integrity assessment result output by the restoration decision module, the structural integrity assessment result includes the three-dimensional coordinates of the damaged component, traditional process parameters, and digital maintenance solution results of material compatibility.

2. The method for maintaining and restoring multiple ancient buildings based on three-dimensional laser scanning according to claim 1 is characterized in that: The step S02 wherein the drone is equipped with a multi-line laser scanning device to scan the building top and high-altitude structure includes: S21. Calculate the required number of scan lines according to formula (1) based on the target building height H and the safe flight height h of the UAV. （1）， Where θ is the maximum elevation angle of the cornice, d res The resolution required by the design; S22, the dynamic posture compensation algorithm of the supplementary point cloud data includes: S221, establish the calibration conversion matrix T from the IMU coordinate system to the laser scanner calib ; S222, for each laser point’s timestamp t, interpolate and obtain the drone’s pose T at the corresponding moment t ; S223, obtain the corrected global coordinate system point cloud coordinates: P global =T t ·T calib ·P raw , where P raw Correct the original point cloud coordinates; S23. Extract the step height measurement value extracted by the point cloud model based on the acquired global coordinate system point cloud coordinates, and calculate the relative error between the step height measurement value extracted by the point cloud model and the true value based on a step block of known size fixed at a typical position of the eaves.

3. The method for maintaining and restoring multiple ancient buildings based on three-dimensional laser scanning according to claim 1 is characterized in that: The processing steps of synchronously collecting the surface texture data of the ancient building by the multispectral imaging device in step S03 include: S31. Use reflectance normalization algorithm to extract paint layer aging characteristics from UV band data. The normalization formula is: ， Among them, R UV is the original reflectivity, R min is the minimum reflectance threshold of a healthy paint layer, R max is the maximum reflectance threshold of the healthy paint layer, is the standard reflectance threshold of a healthy paint layer; S32. Use principal component analysis to separate the decayed areas inside the wood using near-infrared band data. The principal component contribution rate calculation satisfies: ， Among them, λ k is the eigenvalue of the covariance matrix, n is the total number of bands; S33, superpixel alignment includes: S331, generate a Voronoi diagram on the point cloud surface, define the super pixel block center point set {c i } to segment a multispectral image into geometrically related superpixel blocks: ， Among them, V i is the spatial area covered by the i-th superpixel block, is the spatial weight coefficient, which is 0.5, c i is the center point coordinate of the i-th superpixel block, d 2 (p,c i ) is from point p to center point c i The Euclidean distance of S332. Calculate a joint histogram of the HSV color space and the near-infrared reflectivity for each superpixel block; S333: Establish a mapping relationship between the superpixel block and the point cloud triangle patch using a thin plate spline interpolation algorithm. The interpolation function is: ， Among them, (x, y) is the target pixel coordinate, (x i ,y i ): coordinates of the i-th control point, a0, a1, a2 are affine transformation coefficients, w i is the radial basis function weight; S334. Encode the material spectral reflectance and the structural damage characteristics into additional point cloud attribute fields.

4. The method for maintaining and restoring multiple ancient buildings based on three-dimensional laser scanning according to claim 1 is characterized in that: The improved iterative closest point algorithm in step S05 includes: S51. Introduce curvature constraints to filter initial matching point pairs and eliminate those with curvature differences exceeding the threshold. point; S52, using a bidirectional KD tree to accelerate the nearest neighbor search, and optimizing the rigid body transformation matrix based on the Gauss-Newton method; S53, the coarse registration error is verified by the following method: S531. Arrange an array of control points on the standard specimen of the ancient building and obtain the reference coordinates using a total station. S532. Calculate the root mean square error of the Euclidean distance between the point cloud and the control point coordinates after alignment using the iterative closest point algorithm.

5. The method for maintaining and restoring multiple ancient buildings based on three-dimensional laser scanning according to claim 1 is characterized in that: The non-rigid registration optimization of the spectral joint descriptor of the local geometry by the hierarchical graph convolutional network in step S07 includes: S71. Construct an attribute graph with point cloud normal vectors, curvature and spectral features as nodes, with a neighborhood radius of r = 5 cm; S72, the first level uses the EdgeConv operator, and the feature aggregation function is: ， Among them, Θ is the learnable parameter matrix, N(i) is the r neighborhood of point i, For the The feature vector of the i-th node in the layer, For the The feature vector of the jth node in the layer, For the The feature vector of the i+1th node in the layer; S73. The second level introduces the channel attention mechanism, and the feature weighting formula is: ， Among them, g c is the global average pooling feature of the cth channel, m c is the global maximum pooling feature of the c-th channel, σ is the sigmoid function, W1 and W2 are the basis function weights of the first node and the second node.

6. The method for maintaining and restoring multiple ancient buildings based on three-dimensional laser scanning according to claim 1 is characterized in that: The step S08 of using the semantic enhancement module to perform component-level semantic segmentation on the multimodal fusion model includes the following steps: S81. Use three-dimensional morphological opening operation to extract the connection gap of mortise and tenon joints. The structural element is the radius r. morph =3mm sphere; S82: Determine the fracture grade based on the point cloud density gradient. A fracture is marked as severe when the density drop rate η is ≥ 30%. S83. For the peeling area of ​​the painted layer, compare the spectral reflectance difference matrix with the historical data. If the difference value ΔR ≥ 0.2, it is determined to be man-made damage; S84. For historical repair traces, use graph neural network to analyze the similarity of process characteristics and extract the curvature radius r of tool marks. tool and the angle θ between the wood fibers fiber .

7. The method for maintaining and restoring multiple ancient buildings based on three-dimensional laser scanning according to claim 1 is characterized in that: The digital maintenance solution result in step S09 also includes: S91. Calculation of stress concentration factor K of damaged components based on finite element mechanics simulation model t , when K t When the value is ≥2.5, it is marked as a high-risk area, otherwise, it is a low-risk area; S92, combined with the traditional craftsman database to match the mortise and tenon types, according to the formula Select the historical process parameters with the highest adaptability, where S match is the best historical process parameter set after matching, D current is the digital feature vector of the current damaged component, D history is the process parameter feature vector stored in the historical process database; S93. Generate a NC machining path for repairing the slope roof surface. The machining step length is adaptively adjusted according to the curvature. The adjustment formula is: ， in, is the NC machining step size, and κ is the Gaussian curvature of the surface.

8. A non-transitory computer-readable storage medium, wherein at least one instruction or at least one program is stored in the non-transitory computer-readable storage medium, characterized in that: The at least one instruction or the at least one program is loaded and executed by the processor to implement the steps of the multi-dimensional ancient building maintenance and restoration method based on three-dimensional laser scanning as described in any one of claims 1-7.

9. An electronic device, characterized in that: It includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the steps of the multi-dimensional ancient building maintenance and restoration method based on three-dimensional laser scanning as described in any one of claims 1 to 7.

Images