Multi-element historic building maintenance and restoration method based on three-dimensional laser scanning

Abstract

The invention discloses a multi-element historic building maintenance and restoration method based on three-dimensional laser scanning, and relates to the technical field of computer processing, and the method comprises the following steps: S01, obtaining the three-dimensional point cloud data of a historic building foundation through a ground laser scanner, the basic three-dimensional point cloud data comprises geometric information of a building facade, a ground structure and a near-ground decoration component; s02, an unmanned aerial vehicle carries a multi-line laser scanning device to scan the top and the high-altitude structure of the building, supplementary point cloud data is generated, and the millimeter-level precision three-dimensional coordinates of the pitched roof, the tower eave and the cornice bracket are covered with the supplementary point cloud data; and S03, synchronously acquiring surface texture data of the historic building by adopting multispectral imaging equipment. According to the method, the millimeter-level precision of three-dimensional reconstruction of the historic building is realized through multi-source data fusion and a hierarchical optimization algorithm, and the maintenance decision efficiency is improved by combining automatic disease recognition and a multi-modal analysis technology.

Description

Technical Field

[0001] The present invention relates to the technical field of computer processing, and particularly to a method for maintaining and restoring multiple ancient buildings based on three-dimensional laser scanning. Background Art

[0002] Traditional three-dimensional scanning technologies for ancient buildings mainly rely on terrestrial laser scanners, which are limited by the instrument erection height and viewing angle, and it is difficult to cover the detailed data of the top of ancient buildings (such as pitched roofs, tower eaves) and complex structures (such as flying eaves and brackets). Although the drone oblique photography technology can supplement high-altitude images, the generated model has low accuracy (usually with an error of centimeter level), and problems such as geometric misalignment and texture distortion are prone to occur when fusing with ground point cloud data. Summary of the Invention

[0003] In view of the above technical problems, the technical solution adopted by the present invention is a method for maintaining and restoring multiple ancient buildings based on three-dimensional laser scanning, and the method includes the following steps:

[0004] S01. Use a terrestrial laser scanner to obtain the basic three-dimensional point cloud data of the ancient building, and the basic three-dimensional point cloud data includes the geometric information of the building facade, ground structure and near-ground decorative components;

[0005] S02. Scan the top and high-altitude structures of the building by a multi-line laser scanning device carried by a drone to generate supplementary point cloud data, and the supplementary point cloud data covers the three-dimensional coordinates with millimeter-level accuracy of pitched roofs, tower eaves, and flying eaves and brackets;

[0006] S03. Synchronously collect the surface texture data of the ancient building by a multi-spectral imaging device, and the texture data includes the spectral reflectivity of the material, the color degradation gradient and the structural disease characteristics;

[0007] S04. Construct a multi-modal deep learning network including a spatial registration module, a feature fusion module, a semantic enhancement module and a restoration decision module;

[0008] S05. Perform multi-scale coarse registration on the basic three-dimensional point cloud data and the supplementary point cloud data based on an improved iterative closest point algorithm to generate an initial fused point cloud;

[0009] S06. Extract the local geometric spectral joint descriptors of the initial fused point cloud and the texture data by using an adaptive scale-invariant feature transform algorithm;

[0010] S07. Perform non-rigid registration optimization on the local geometric spectral joint descriptors through a hierarchical graph convolutional network to eliminate data stitching misalignment and generate a geometrically continuous multi-modal fusion model;

[0011] S08. Use the semantic enhancement module to perform component-level semantic segmentation on the multi-modal fusion model to identify the fractures of mortise and tenon joints, the peeling of painted layers, and the traces of historical repairs;

[0012] S09. According to the structural integrity assessment result output by the restoration decision module, the structural integrity assessment result includes the three-dimensional coordinates of damaged components, the digital maintenance plan results of traditional process parameters, and material suitability.

[0013] Preferably, the step of using the multi-line laser scanning device carried by the unmanned aerial vehicle to scan the top and high-altitude structures of the building in step S02 includes:

[0014] S21. Calculate the required number of scan lines according to the target building height H and the safe flight height h of the unmanned aerial vehicle using formula (1),

[0015] (1),

[0016] where θ is the maximum elevation angle of the cornice, and d res is the resolution required by the design.

[0017] S22. The dynamic pose compensation algorithm for supplementing point cloud data includes:

[0018] S221. Establish the calibration transformation matrix T from the IMU coordinate system to the laser scanner calib ;

[0019] S222. For the timestamp t of each laser point, interpolate to obtain the pose T of the unmanned aerial vehicle at the corresponding moment t ;

[0020] S223. Obtain the corrected point cloud coordinates in the global coordinate system: P global = T t · T calib · P raw , where P raw corrects the original point cloud coordinates;

[0021] S23. Extract the step height measurement value extracted from the point cloud model based on the obtained point cloud coordinates in the global coordinate system, and calculate the relative error value between the step height measurement value extracted from the point cloud model and the true value according to the step gauge with a known size fixed at a typical position of the cornice.

[0022] Preferably, the processing steps of the multi-spectral imaging device synchronously collecting the surface texture data of the ancient building in step S03 include:

[0023] S31. Use the reflectance normalization algorithm to extract the aging characteristics of the paint layer from the ultraviolet band data, and the normalization formula is:

[0024] ,

[0025] Among them, R UV is the original reflectivity, R min is the minimum reflectivity threshold of the healthy paint layer, R max is the maximum reflectivity threshold of the healthy paint layer, is the standard reflectivity threshold of the healthy paint layer;

[0026] S32. Separate the internal decay area of the wood from the near-infrared band data through principal component analysis. The calculation of the principal component contribution rate satisfies:

[0027] ,

[0028] Among them, λ k is the eigenvalue of the covariance matrix, n is the total number of bands, and k is the number of covariance matrices;

[0029] S33. The superpixel-level alignment includes:

[0030] S331. Generate a Voronoi diagram on the point cloud surface and define the set of superpixel block center points {c i} to segment the multispectral image into geometrically associated superpixel blocks:

[0031] ,

[0032] Among them, V i is the spatial region covered by the i-th superpixel block, α is the spatial weight coefficient, taking 0.5, c i is the center point coordinate of the i-th superpixel block, d 2 (p, c i ) is the Euclidean distance from point p to the center point c i ;

[0033] S332. Calculate the joint histogram of the HSV color space and the near-infrared reflectivity for each superpixel block;

[0034] S333. Establish the mapping relationship between the superpixel block and the point cloud triangular facet through the thin plate spline interpolation algorithm. The interpolation function is:

[0035] ,

[0036] Among them, (x, y) is the target pixel coordinate, (x i , y i ) is the coordinate of the i-th control point, a0, a1, a2 are the affine transformation coefficients, and w i is the radial basis function weight.

[0037] S334. Encode the material reflectivity and the disease characteristics into additional point cloud attribute fields.

[0038] Preferably, the improved iterative closest point algorithm in step S05 includes:

[0039] S51. Introduce curvature constraint conditions to screen initial matching point pairs and eliminate points with curvature differences exceeding the threshold .

[0040] S52. Use a bidirectional KD tree to accelerate the nearest neighbor search and optimize the rigid body transformation matrix based on the Gauss-Newton method;

[0041] S53. The coarse registration error is verified in the following way:

[0042] S531. Layout a control point array on the ancient building standard specimen and obtain the reference coordinates by using a total station for measurement;

[0043] S532. Calculate the root mean square error of the Euclidean distance between the point cloud after registration by the iterative closest point algorithm and the control point coordinates.

[0044] Preferably, the non-rigid registration optimization of the spectral joint descriptor of the local geometry by the hierarchical graph convolutional network in step S07 includes:

[0045] S71. Construct an attribute graph with point cloud normal vectors, curvatures, and spectral features as nodes, and the neighborhood radius r = 5 cm;

[0046] S72. In the first layer, use the EdgeConv operator, and the feature aggregation function is:

[0047] ,

[0048] where Θ is a learnable parameter matrix, N(i) is the r-neighborhood of point i, is the feature vector of the i-th node in the -th layer, is the feature vector of the j-th node in the -th layer, is the feature vector of the (i + 1)-th node in the -th layer;

[0049] S73. In the second layer, introduce a channel attention mechanism, and the feature weighting formula is:

[0050] ,

[0051] where g c is the global average pooling feature of the c-th channel, m cis the global maximum pooling feature of the c-th channel, σ is the sigmoid function, and W1 and W2 are the radial basis function weights of the first and second nodes.

[0052] Preferably, the step of using the semantic enhancement module to perform component-level semantic segmentation on the multi-modal fusion model in step S08 includes the following steps:

[0053] S81. Perform three-dimensional morphological opening operation on the mortise and tenon joints to extract the connection gap, and the structural element is a sphere with a radius r morph = 3mm;

[0054] S82. Determine the fracture grade based on the point cloud density gradient. When the density decrease rate η ≥ 30%, it is marked as a severe fracture;

[0055] S83. For the peeling area of the painted layer, compare it with the historical data through the spectral reflectance difference matrix. When the difference value ΔR ≥ 0.2, it is determined as human damage;

[0056] S84. For the historical repair traces, use the graph neural network to analyze the similarity of process features, and extract the curvature radius r of the tool marks tool and the included angle θ between the wood fiber directions fiber .

[0057] Preferably, the result of the digital maintenance plan in step S09 further includes:

[0058] S91. Calculate the stress concentration coefficient K of the damaged component based on the finite element mechanical simulation model t , when K t ≥ 2.5, it is marked as a high-risk area, otherwise, it is a low-risk area;

[0059] S92. Combine the traditional craftsmanship database to match the mortise and tenon types, and select the historical process parameters with the highest adaptability according to the formula , where S match is the set of the best historical process parameters after matching, D current is the digital feature vector of the current damaged component, and D history is the process parameter feature vector stored in the historical process database;

[0060] S93. Generate the numerical control machining path for the repair of the pitched roof surface, and the machining step size is adaptively adjusted according to the curvature. The adjustment formula is:

[0061] ,

[0062] where is the numerical control machining step size, and κ is the Gaussian curvature of the surface.

[0063] The present invention has at least the following beneficial effects:

[0064] 1. Through the collaborative optimization of the improved iterative closest point algorithm and the hierarchical graph convolutional network, millimeter-level registration of ground and aerial point clouds is achieved, which can effectively solve the geometric misalignment problem at the connection between the cornice and the column, ensuring the topological accuracy of the 3D reconstruction of complex wooden frameworks.

[0065] 2. Based on the component-level semantic segmentation algorithm, disease characteristics such as mortise-tenon fracture and painted peeling can be automatically identified, and the technological characteristics of historical repair traces can be traced.

[0066] 3. By fusing multi-spectral texture and geometric point cloud data, the correlation analysis of material aging and structural deformation is realized through the superpixel-level alignment technology (HSV-near-infrared joint histogram), which improves the detection accuracy of wood decay areas and can quantify the aging degree of the paint layer. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0068] Figure 1 It is a flowchart of a multi-element ancient building maintenance and restoration method based on 3D laser scanning provided in Embodiment 1 of the present invention;

[0069] Figure 2 It is a schematic structural diagram of the implementation of S01 and S02 provided in Embodiment 1 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0070] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present invention.

[0071] It should be noted that the terms "first", "second", etc. in the description, claims and above-mentioned drawings of the present invention are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or server comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0072] Embodiment 1:

[0073] This embodiment provides a method for maintaining and restoring multiple ancient buildings based on three-dimensional laser scanning. The method includes the following steps, as Figure 1 shown:

[0074] S01. Use a terrestrial laser scanner to obtain the basic three-dimensional point cloud data of the ancient building. The basic three-dimensional point cloud data includes the geometric information of the building facade, ground structure and near-ground decorative components;

[0075] Specifically, as Figure 2 shown, the purpose of using terrestrial laser scanning to obtain the basic point cloud is to establish the geometric reference of the main structure of the ancient building and capture the three-dimensional data with millimeter-level accuracy of near-ground components such as facades, column bases, and steps. It provides a spatial reference framework for subsequent aerial point cloud registration to ensure the unified coordinate system of multi-source data. And the so-called geometric information is the three-dimensional coordinates, that is, the coordinates of each feature of the three-dimensional point cloud data.

[0076] The above-mentioned phase-type terrestrial laser scanner (such as Faro Focus S350) is used, with a scanning rate ≥ 976,000 points / second and a ranging accuracy of ±1 mm. The acquisition of the basic three-dimensional point cloud data is carried out by arranging scanning stations around the building, with the station spacing ≤ 15 m to ensure that the point cloud overlap rate ≥ 30%. Then, a target ball (diameter 10 cm) is used as a common control point to assist in the stitching of multi-station point clouds.

[0077] Furthermore, the obtained basic three-dimensional point cloud data is processed. The detailed processing steps include:

[0078] S11. Remove outliers through statistical filtering (threshold: points outside 2σ of the distance mean);

[0079] S12. Based on the target ball coordinates, use the least squares method to calculate the rigid body transformation matrix between stations;

[0080] S13. Compress the data volume using voxel grid filtering (resolution 5 mm) and retain geometric features.

[0081] S02. Use a multi-line laser scanning device carried by a drone to scan the building top and high-altitude structures, generate supplementary point cloud data, and the supplementary point cloud data covers the millimeter-level precision three-dimensional coordinates of the pitched roof, tower eaves, and flying eaves brackets.

[0082] Specifically, using a multi-line laser scanning device carried by a drone to scan the building top and high-altitude structures, the significance of generating supplementary point cloud data is to cover the ground scanning blind areas (such as pitched roofs and flying eaves), and solve the problems of holes and low precision in traditional drone photogrammetry models. Through dynamic pose compensation, eliminate the influence of drone flight jitter on the point cloud quality.

[0083] And the method for obtaining the above-mentioned drone scanning includes the following steps:

[0084] S21. According to the target building height H and the safe flight height h of the drone, calculate the required number of scanning lines according to formula (1).

[0085] (1).

[0086] Where, θ is the maximum elevation angle of the flying eaves, and d res is the resolution required by the design.

[0087] S22. The dynamic pose compensation algorithm for supplementary point cloud data includes:

[0088] S221. Establish the calibration transformation matrix T from the IMU coordinate system to the laser scanner. calib ;

[0089] S222. For the timestamp t of each laser point, interpolate to obtain the drone pose T at the corresponding moment. t ;

[0090] S223. Obtain the corrected point cloud coordinates in the global coordinate system: P global = T t · T calib · P raw where P raw corrects the original point cloud coordinates.

[0091] S23. Extract the step height measurement value extracted from the point cloud model based on the obtained point cloud coordinates in the global coordinate system, and calculate the relative error value between the step height measurement value extracted from the point cloud model and the true value according to the known-size step gauge fixed at the typical position of the flying eaves.

[0092] Specifically, as Figure 2As shown, the building height H and the UAV flight height h jointly determine the spatial relative position between the scanning system and the target component. When the building height increases or the UAV needs to raise its flight height due to safety restrictions, the number of scanning lines is automatically increased through a formula to ensure the data acquisition density in the vertical direction. The maximum eave elevation angle θ: For the unique eave bracket structure of traditional Chinese architecture, the elevation angle parameter is directly related to the field of view coverage of the scanning device. For example, when θ > 45°, the formula will automatically increase the number of lines to compensate for the lack of point clouds on steep surfaces. Resolution dres: The minimum feature recognition size (such as the pattern of eaves tiles or the width of cracks) set according to maintenance requirements. When dres ≤ 2 mm, the system will increase the line number configuration to match the high-precision requirements. Through the mathematical constraint relationship between parameters, the experience-driven parameter setting is transformed into a quantitative decision based on the physical characteristics of the building, solving the problem of scanning blind spots caused by improper manual setting in traditional methods.

[0093] Furthermore, the transformation matrix T from the IMU coordinate system to the laser scanner is obtained through laboratory pre-calibration calib , and a six-degree-of-freedom (6-DoF) spatial pose mapping relationship is established. The calibration process uses a checkerboard target for multi-point collection to minimize the systematic error caused by installation deviation. Secondly, spline curves are used to continuousize the discrete pose data T output by the IMU t to ensure that the corresponding real-time UAV pose can be obtained for each time stamp t of the laser point. The interpolation frequency is synchronized with the laser scanning rate (usually 10 - 20 Hz) to avoid phase delay of motion information. Moreover, the motion compensation formula P global =T t ·T calib ·P raw is applied to reconstruct the coordinates of the original point cloud. The physical meaning of this formula is: reverse-compensate the UAV pose (including position offset and attitude angle change) at the moment when the laser pulse is emitted into the point cloud data. For example, when the UAV has a pitch angle oscillation, the algorithm eliminates the resulting point cloud stretching deformation through the reverse rotation matrix.

[0094] S03. A multi-spectral imaging device is used to synchronously collect the surface texture data of ancient buildings. The texture data includes material spectral reflectance, color degradation gradient, and structural disease characteristics;

[0095] Specifically, the significance of multi-spectral imaging for collecting surface texture is to obtain the material spectral fingerprint to identify hidden diseases such as paint layer aging and wood decay. At the same time, it provides multi-modal data support for subsequent semantic segmentation and enhances the reliability of structural damage interpretation.

[0096] In the actual implementation process, a hyperspectral imaging device uses a HySpex VNIR-1800 imaging spectrometer (with 180 bands and a spectral range of 400-1000 nm). During the operation of scanning and acquiring data, a laser scanner is synchronously triggered to ensure the spatio-temporal alignment of the spectral image and the point cloud. Moreover, during the actual acquisition process, the data is collected under cloudy sky conditions (light intensity 5000-10000 lux) to avoid the interference of specular reflection. The same area is photographed from multiple angles (angle interval ≤ 15°) to eliminate the influence of shadows.

[0097] The processing steps for the above hyperspectral imaging device to synchronously acquire the surface texture data of ancient buildings include:

[0098] S31. For the ultraviolet band data, a reflectance normalization algorithm is used to extract the aging characteristics of the paint layer. The normalization formula is:

[0099] ,

[0100] where, R UV is the original reflectance, R min is the minimum reflectance threshold of the healthy paint layer, R max is the maximum reflectance threshold of the healthy paint layer, is the standard reflectance threshold of the healthy paint layer;

[0101] S32. For the near-infrared band data, the internal decay area of the wood is separated by principal component analysis. The calculation of the principal component contribution rate satisfies:

[0102] ,

[0103] where, λ k is the eigenvalue of the covariance matrix, and n is the total number of bands;

[0104] S33. The superpixel-level alignment includes:

[0105] S331. Generate a Voronoi diagram on the surface of the point cloud, and define the set of center points {c i} of the superpixel blocks to segment the hyperspectral image into geometrically related superpixel blocks:

[0106] ,

[0107] where, V i is the spatial region covered by the i-th superpixel block, is the spatial weight coefficient, taking 0.5, c i is the center point coordinate of the i-th superpixel block, and d 2 (p, c i ) is the Euclidean distance from point p to the center point c i ;

[0108] S332. Calculate the joint histogram of the HSV color space and the near-infrared reflectance for each superpixel block;

[0109] S333. Establish the mapping relationship between the superpixel block and the point cloud triangular facet through the thin plate spline interpolation algorithm. The interpolation function is:

[0110] ,

[0111] where (x, y) is the target pixel coordinate, (x i , y i ): the coordinate of the i-th control point, a0, a1, a2 are the affine transformation coefficients, and w i is the radial basis function weight.

[0112] S334. Encode the material reflectivity and disease characteristics as additional point cloud attribute fields.

[0113] The above-mentioned ultraviolet band reflectance normalization algorithm realizes the non-destructive quantitative detection of the aging degree of traditional paint layers for the first time, breaking through the subjectivity limitation of traditional visual inspection. Secondly, using the near-infrared principal component analysis technology, the mapping relationship between the internal decay characteristics of wood and the spectral response is established, and internal defects up to 15 cm deep can be detected. Furthermore, based on the geometric-spectral joint modeling method of superpixel-level alignment, the texture attributes are accurately mapped to the three-dimensional point cloud, solving problems such as texture stretching and seam misalignment existing in the traditional texture mapping method.

[0114] Furthermore, normalization processing can eliminate the influence of illumination condition differences, where: R min is taken from the laboratory calibration value of the unaged paint layer (such as the reference reflectivity of 0.32 for the lead red paint commonly used in Qing Dynasty colored paintings), and R max corresponds to the reflectivity threshold (0.08) of the completely carbonized paint layer. The normalized value range [0, 1] directly reflects the aging degree, and when <0.4, a maintenance warning is triggered.

[0115] Perform PCA dimensionality reduction on 128 near-infrared bands (900 - 1700 nm) in the near-infrared band data, and retain the first 3 principal components. The first principal component reflects the wood density distribution, the second principal component characterizes the moisture content, and the third principal component is related to microbial metabolites. By establishing the clustering center of the decay area in the feature space (clustering center = [-1.2, 0.6, 0.3]), the Mahalanobis distance is used to determine the decay probability D M , and when D M > 2.5, it is determined as internal decay.

[0116] Furthermore, a Voronoi diagram is generated on the surface of the point cloud, and superpixel segmentation is performed with the centroid of the triangular patches as the seed points. The objective function balances geometric proximity and spectral consistency. The HSV-NIR joint histogram of each superpixel block (about 5 cm × 5 cm) has a dimension of (18×12×8). A sub-pixel level mapping relationship is established through a thin plate spline interpolation function, and the interpolation error is controlled within 0.5 pixels.

[0117] S04. Construct a multi-modal deep learning network including a spatial registration module, a feature fusion module, a semantic enhancement module, and a restoration decision module;

[0118] Specifically, the significance of constructing the multi-modal deep learning network lies in designing an end-to-end network architecture to achieve geometric-spectral data fusion analysis and automated decision-making. And it replaces traditional manual interpretation to solve the problem of low processing efficiency of the complex structure of ancient buildings. The above-mentioned multi-modal deep learning network architecture is as follows:

[0119] Spatial registration module: Coarse alignment of the point cloud is achieved based on an improved iterative closest point algorithm (see step S05).

[0120] Feature fusion module: A two-stream network (geometry stream + spectral stream) is adopted, and features are fused through a cross-attention mechanism.

[0121] Semantic enhancement module: A 3D U-Net structure, which supports point cloud semantic segmentation (see step S08).

[0122] Restoration decision module: Combining reinforcement learning (PPO algorithm) to optimize the generation of maintenance plans.

[0123] S05. Perform multi-scale coarse registration on the basic 3D point cloud data and the supplementary point cloud data based on the improved iterative closest point algorithm to generate the initial fused point cloud;

[0124] Specifically, the above method solves the initial pose deviation of the large-scale point cloud and provides a high-quality initial value for fine registration. And the registration robustness is improved through curvature constraints to reduce noise interference.

[0125] The above-mentioned improved iterative closest point algorithm includes:

[0126] S51. Introduce curvature constraint conditions to screen the initial matching point pairs and eliminate the points with curvature differences exceeding the threshold of;

[0127] S52. Use a bidirectional KD tree to accelerate the nearest neighbor search and optimize the rigid body transformation matrix based on the Gauss-Newton method;

[0128] S53. The coarse registration error is verified in the following way:

[0129] S531. Layout an array of control points on the standard specimen of ancient architecture, and measure the reference coordinates using a total station.

[0130] S532. Calculate the root mean square error of the Euclidean distance between the point cloud after registration by the iterative closest point algorithm and the coordinates of the control points.

[0131] Specifically, perform moving least squares (MLS) surface fitting on the original point cloud, calculate the maximum principal curvature k1 and the minimum principal curvature k2 of each point, and eliminate points with a curvature difference threshold of , so as to eliminate points that do not meet the conditions.

[0132] Build a two-way KD tree structure, set the depth of the ground point cloud tree to 12, and the depth of the UAV point cloud tree to 10. When searching, use the Best-Bin-First (BBF) algorithm, and limit the maximum number of backtracking to 200 times.

[0133] Furthermore, the above-mentioned layout of the control point array on the standard specimen of ancient architecture is to arrange 24 spherical control points (diameter 2 cm) on the surface of the ancient architecture specimen, and screen feature points through curvature constraints to reduce the mis-correspondence between the eaves surface and the column plane.

[0134] S06. Use the adaptive scale-invariant feature transform algorithm to extract the spectral joint descriptors of the local geometry of the initial fused point cloud and texture data;

[0135] Specifically, the significance of the above-mentioned spectral joint descriptors of the local geometry of the initial fused point cloud and texture data: construct cross-modal feature associations, solve the problem of geometric-spectral data heterogeneous matching, and enhance the feature distinctiveness of complex structures such as eaves and brackets.

[0136] During the actual execution process,

[0137] Extract ISS (Intrinsic Shape Signatures) key points on the surface of the point cloud, and the response value threshold > 0.8.

[0138] Extract SURF (Speeded-Up Robust Features) key points from the multi-spectral image.

[0139] Descriptor construction:

[0140] Geometric descriptor: Calculate the normal vector histogram (SHOT descriptor) of the key point neighborhood (radius 10 cm).

[0141] Spectral descriptor: Extract the multi-band reflectance statistics (mean, variance) of the 5×5 pixel window around the key point.

[0142] Joint descriptor: Concatenate the geometric and spectral features into a 256-dimensional vector, and reduce the dimension to 64 dimensions by PCA.

[0143] S07. Non-rigid registration optimization is performed on the spectral joint descriptor of local geometry through a hierarchical graph convolutional network to eliminate data stitching misalignment and generate a geometrically continuous multi-modal fusion model;

[0144] Specifically, the above-mentioned method solves the local registration error caused by non-rigid deformation (such as wood deformation), and adaptively fuses the multi-source data features through an attention mechanism.

[0145] Nodes of merchants: Each point in the point cloud, with attributes including coordinates, normal vectors, curvatures, and spectral features. And edges: Neighborhood points within a radius of 5 cm, and the edge weights are jointly determined by the Euclidean distance and feature similarity.

[0146] Furthermore, the non-rigid registration optimization of the spectral joint descriptor of local geometry through a hierarchical graph convolutional network includes:

[0147] S71. Construct an attribute graph with the normal vector, curvature, and spectral features of the point cloud as nodes, and the neighborhood radius r = 5 cm;

[0148] S72. The EdgeConv operator is used in the first layer, and the feature aggregation function is:

[0149] ,

[0150] where Θ is a learnable parameter matrix, N(i) is the r-neighborhood of point i, is the feature vector of the ith node in the th layer, is the feature vector of the jth node in the th layer, and

[0151] is the feature vector of the

[0152] ,

[0153] where g c is the global average pooling feature of the cth channel, m c is the global maximum pooling feature of the cth channel, σ is the sigmoid function, and W1 and W2 are the radial basis function weights of the first node and the second node path.

[0154] Specifically, the first layer (geometric consistency constraint): When constructing the attribute graph, the node features include the normal vector, curvature (k), and near-infrared reflectance (R NIR), the neighborhood radius r of the EdgeConv operator is 5 cm, the dimension of the parameter matrix Θ in the feature aggregation function is 64×128, and the local geometric change pattern is captured by max pooling. This layer outputs a 128-dimensional feature vector, representing the local surface geometric characteristics.

[0155] The second layer (attention feature fusion): The channel attention module receives the global average pooling feature g c and the max pooling feature m c and calculates the channel importance coefficient 128×64 through the weight matrices W1∈R 64×1 , W2∈R , where σ is the sigmoid function. Apply a 1.2-fold weight compensation to the UAV point cloud features, and the final fused feature is F fusion =α·F aerial +(1−α) ·F ground .

[0156] S08. Use the semantic enhancement module to perform component-level semantic segmentation on the multi-modal fusion model to identify mortise-tenon joint fractures, painted layer peeling, and historical repair marks;

[0157] The above-mentioned use of the semantic enhancement module to perform component-level semantic segmentation on the multi-modal fusion model includes the following steps:

[0158] S81. Use three-dimensional morphological opening operation on the mortise-tenon joint to extract the connection gap, and the structural element is a sphere with a radius r morph = 3 mm;

[0159] S82. Determine the fracture grade based on the point cloud density gradient. When the density decrease rate η≥30%, it is marked as a severe fracture;

[0160] S83. For the painted layer peeling area, compare the spectral reflectance difference matrix with historical data. When the difference value ΔR≥0.2, it is determined as human damage;

[0161] S84. For historical repair marks, use the graph neural network to analyze the similarity of process features, and extract the curvature radius r tool of the tool mark and the included angle θ fiber between the wood fiber directions.

[0162] Specifically, perform three-dimensional opening operation with a spherical structural element with a radius of 3 mm, I open =(I - B)+B, where I is the point cloud density field and B is the structural element. Calculate the volume change rate η=(V original −V open ) / V original . When η≥30%, it is determined as a structural fracture.

[0163] Furthermore, construct a spectral difference matrix ΔR = ∣R current − R historical ∣, and combine it with the fluorescence characteristics in the ultraviolet band: natural aging shows a gradual difference (ΔR ≤ 0.15), and man-made damage shows a step change (ΔR ≥ 0.2). Use the Markov random field (MRF) to optimize the spatial consistency of the disputed area and eliminate the influence of light interference.

[0164] Even further, generate a cross-sectional curve of the tool mark from the 3D point cloud and calculate the radius of curvature , the typical value range of r tool of traditional chisels in the Qing Dynasty is 2.5 - 3.5 mm, and that of modern power tools is 0.8 - 1.2 mm, and thus judge the repair age. θ fiber is the angle between the wood fiber direction and the tool mark, and judge the processing technology. The formula is:

[0165] ,

[0166] where is the direction vector, is the first derivative of the tool mark, is the second derivative of the tool mark, is the manual chiseling mark in the Qing Dynasty, is the chiseling mark of modern power tools.

[0167] S09. According to the structural integrity assessment results output by the restoration decision module, the structural integrity assessment results include the three-dimensional coordinates of damaged components, digital maintenance plan results of traditional process parameters and material adaptability.

[0168] The above digital maintenance plan results also include:

[0169] S91. Calculate the stress concentration coefficient K t of the damaged component based on the finite element mechanics simulation model. When K t ≥ 2.5, it is marked as a high-risk area; otherwise, it is a low-risk area;

[0170] S92. Combine the traditional craftsmanship database to match the tenon and mortise types, and select the historical process parameters with the highest adaptability according to the formula . Among them, S match is the set of the best historical process parameters after matching, D current is the digital feature vector of the current damaged component, and D history is the process parameter feature vector stored in the historical process database;

[0171] S93. Generate the numerical control machining path for the repair of the pitched roof surface, and the machining step size is adaptively adjusted according to the curvature. The adjustment formula is:

[0172] ,

[0173] Among them, is the CNC machining step size, and k is the Gaussian curvature of the surface.

[0174] Specifically, convert the point cloud model into a finite element mesh (element size ≤ 5 mm), and load the self-weight, wind load (0.45 kN / m 2 ), and snow load (0.3 kN / m 2 ). Calculate the von Mises stress distribution. When the stress concentration factor K t = σ max / σ nom ≥ 2.5K, it is determined that the component has a fracture risk.

[0175] Store more than 2000 groups of historical process parameters (such as mortise and tenon types, tenon size tolerances) in the craftsmanship database, and find the best match through the optimization function S match . The system automatically matches processes such as the "swallowtail tenon" process in the Qing Dynasty (tenon length error ± 0.3 mm).

[0176] Adaptive adjustment of the machining step size according to the Gaussian curvature k of the surface: Δs (unit: mm). A 5 mm step size is adopted in the low curvature area (k < 0.1) to improve efficiency, and it is switched to a 0.2 mm step size in the high curvature cornice area (k > 0.5) to ensure accuracy. The generated tool compensation parameters ensure that the machining error of the wooden component ≤ 0.15 mm.

[0177] In summary, in the above-mentioned first embodiment, through the collaborative optimization of the improved iterative closest point algorithm and the hierarchical graph convolutional network, millimeter-level registration of ground and aerial point clouds is achieved, which can effectively solve the geometric misalignment problem at the connection between the cornice and the column, and ensure the topological accuracy of the 3D reconstruction of complex wooden frameworks. Secondly, based on the component-level semantic segmentation algorithm, disease characteristics such as mortise and tenon fractures and painted peeling can be automatically identified, and the process characteristics of historical repair traces can be traced. Furthermore, by fusing multi-spectral texture and geometric point cloud data, the correlation analysis of material aging and structural deformation is realized through the superpixel-level alignment technology (HSV-near infrared joint histogram), which improves the detection accuracy of wood decay areas and can quantify the aging degree of the paint layer.

[0178] Embodiment 2:

[0179] The embodiment of the present invention provides a non-transitory computer-readable storage medium, in which at least one instruction or at least one program segment is stored, and at least one instruction or at least one program segment is loaded and executed by a processor to implement the steps:

[0180] Use a ground laser scanner to obtain the three-dimensional point cloud data of the ancient building foundation;

[0181] Use a multi-line laser scanning device carried by a drone to scan the top of the building and high-altitude structures, generating supplementary point cloud data;

[0182] Use a multi-spectral imaging device to synchronously collect the surface texture data of ancient buildings;

[0183] Build a multi-modal deep learning network including a spatial registration module, a feature fusion module, a semantic enhancement module, and a restoration decision module;

[0184] Based on an improved iterative closest point algorithm, perform multi-scale rough registration on the basic three-dimensional point cloud data and the supplementary point cloud data to generate an initial fused point cloud;

[0185] Use an adaptive scale-invariant feature transform algorithm to extract the local geometric spectral joint descriptors of the initial fused point cloud and the texture data;

[0186] Through a hierarchical graph convolutional network, perform non-rigid registration optimization on the local geometric spectral joint descriptors, eliminate data stitching misalignment, and generate a geometrically continuous multi-modal fusion model;

[0187] Use the semantic enhancement module to perform component-level semantic segmentation on the multi-modal fusion model to identify tenon-mortise joint fractures, painted layer peeling, and historical renovation traces;

[0188] According to the structural integrity evaluation result output by the restoration decision module.

[0189] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0190] Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the above-mentioned division of each functional unit and module is used as an example for illustration. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0191] Embodiment 3:

[0192] The embodiment of the present invention provides an electronic device, including a processor and a memory. At least one instruction or at least one program segment is stored in the memory, and the at least one instruction or the at least one program segment is loaded and executed by the processor to implement the steps:

[0193] Use a terrestrial laser scanner to obtain three-dimensional point cloud data of the ancient building foundation;

[0194] Use a multi-line laser scanning device carried by a drone to scan the top and high-altitude structures of the building to generate supplementary point cloud data;

[0195] Use a multi-spectral imaging device to synchronously collect the surface texture data of the ancient building;

[0196] Build a multi-modal deep learning network including a spatial registration module, a feature fusion module, a semantic enhancement module, and a restoration decision-making module;

[0197] Based on the improved iterative closest point algorithm, perform multi-scale rough registration on the basic three-dimensional point cloud data and the supplementary point cloud data to generate an initial fusion point cloud;

[0198] Use the adaptive scale-invariant feature transform algorithm to extract the local geometric spectral joint descriptors of the initial fusion point cloud and the texture data;

[0199] Through a hierarchical graph convolutional network, perform non-rigid registration optimization on the local geometric spectral joint descriptors to eliminate data stitching misalignment and generate a geometrically continuous multi-modal fusion model;

[0200] Use the semantic enhancement module to perform component-level semantic segmentation on the multi-modal fusion model to identify tenon and mortise joint fractures, peeling of the painted layer, and historical repair traces;

[0201] According to the structural integrity evaluation result output by the restoration decision-making module.

[0202] The above are only the preferred embodiments of the present invention, and do not impose any form of limitation on the present invention. Although the present invention has been disclosed above with the preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some changes or modifications to equivalent embodiments by using the above-disclosed technical content within the scope of the technical solution of the present invention. However, as long as the content does not depart from the technical solution of the present invention, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A method for maintaining and restoring multiple ancient buildings based on three-dimensional laser scanning, characterized in that, The method includes the following steps: S01. Use a terrestrial laser scanner to obtain three-dimensional point cloud data of the ancient building foundation, where the foundation three-dimensional point cloud data includes geometric information of the building facade, ground structure, and near-ground decorative components; S02. Use a multi-line laser scanning device carried by a drone to scan the building top and high-altitude structure to generate supplementary point cloud data, where the supplementary point cloud data covers the millimeter-level precision three-dimensional coordinates of the pitched roof, tower eaves, and flying eaves brackets; S03. Use a multi-spectral imaging device to synchronously collect surface texture data of the ancient building, where the texture data includes material spectral reflectance, color degradation gradient, and structural disease characteristics; S04. Construct a multi-modal deep learning network including a spatial registration module, a feature fusion module, a semantic enhancement module, and a restoration decision module; S05. Based on an improved iterative closest point algorithm, perform multi-scale rough registration on the foundation three-dimensional point cloud data and the supplementary point cloud data to generate an initial fused point cloud; S06. Use an adaptive scale-invariant feature transform algorithm to extract the local geometric spectral joint descriptor of the initial fused point cloud and the texture data; S07. Through a hierarchical graph convolutional network, perform non-rigid registration optimization on the local geometric spectral joint descriptor to eliminate data stitching misalignment and generate a geometrically continuous multi-modal fusion model; S08. Use the semantic enhancement module to perform component-level semantic segmentation on the multi-modal fusion model to identify broken tenon and mortise joints, peeling of painted layers, and historical repair traces; S09. According to the structural integrity evaluation result output by the restoration decision module, the structural integrity evaluation result includes the digital maintenance plan result of the three-dimensional coordinates of damaged components, traditional process parameters, and material adaptability.

2. The method for maintaining and restoring multiple ancient buildings based on 3D laser scanning according to claim 1, characterized in that, In step S02, the process of using the multi-line laser scanning device carried by the drone to scan the building top and high-altitude structure includes: S21. According to the target building height H and the safe flight height h of the drone, calculate the required number of scan lines according to formula (1); （1）， where θ is the maximum elevation angle of the cornice, and d res is the resolution required by the design; S22. The dynamic pose compensation algorithm for the supplementary point cloud data includes: S221. Establish the calibration transformation matrix T from the IMU coordinate system to the lidar scanner calib ; S222. For the timestamp t of each laser point, interpolate to obtain the UAV pose T at the corresponding moment t ; S223. Obtain the point cloud coordinates in the corrected global coordinate system: P global =T t ·T calib ·P raw , where P raw corrects the original point cloud coordinates; S23. Based on the step height measurement value extracted from the point cloud model by the point cloud coordinates in the obtained global coordinate system, and according to the known size step gauge fixed at the typical position of the flying eaves, calculate the relative error value between the step height measurement value extracted from the point cloud model and the true value.

3. A method for maintaining and restoring multiple ancient buildings based on three-dimensional laser scanning according to claim 1, characterized in that, In step S03, the processing steps of using the multi-spectral imaging device to synchronously collect surface texture data of the ancient building include: S31. Use a reflectance normalization algorithm for the ultraviolet band data to extract the aging characteristics of the paint layer, and the normalization formula is: ， Among them, R UV is the original reflectance, R min is the minimum reflectance threshold of the healthy paint layer, R max is the maximum reflectance threshold of the healthy paint layer, is the standard reflectance threshold of the healthy paint layer; S32. Use principal component analysis for the near-infrared band data to separate the rotten areas inside the wood, and the principal component contribution rate calculation satisfies: ， where λ k is the eigenvalue of the covariance matrix, and n is the total number of bands; S33. The superpixel-level alignment includes: S331. Generate a Voronoi diagram on the point cloud surface, and define the central point set {c i} of the superpixel blocks to segment the multispectral image into geometrically related superpixel blocks: ， Among them, V i is the spatial region covered by the i-th superpixel block, is the spatial weight coefficient, taking 0.5, c i is the center point coordinate of the i-th superpixel block, d 2 (p, c i ) is the Euclidean distance from point p to the center point c i ; S332. Calculate the joint histogram of the HSV color space and the near-infrared reflectance for each super pixel block; S333. Establish the mapping relationship between the super pixel block and the point cloud triangular patch through a thin plate spline interpolation algorithm, and the interpolation function is: ， where (x, y) are the coordinates of the target pixel, and (x i , y i ) are the coordinates of the i-th control point, a0, a1, and a2 are the affine transformation coefficients, and w i is the weight of the radial basis function; S334. Encode the material reflectance and the disease characteristics as additional point cloud attribute fields.

4. A method for maintaining and restoring multiple ancient buildings based on 3D laser scanning according to claim 1, characterized in that, In step S05, the improved iterative closest point algorithm includes: S51. Introduce curvature constraint conditions to screen initial matching point pairs and eliminate points whose curvature difference exceeds the threshold ; S52. Accelerate the nearest neighbor search using a bidirectional KD tree and optimize the rigid body transformation matrix based on the Gauss-Newton method; S53. The coarse registration error is verified in the following manner: S531. Layout a control point array on the ancient building standard specimen and obtain the reference coordinates by using a total station for measurement; S532. Calculate the root mean square error of the Euclidean distance between the point cloud after registration by the iterative closest point algorithm and the control point coordinates.

5. A method for maintaining and restoring multiple ancient buildings based on 3D laser scanning according to claim 1, characterized in that In step S07, the non-rigid registration optimization of the spectral joint descriptor of the local geometry by the hierarchical graph convolutional network includes: S71. Construct an attribute graph with the point cloud normal vector, curvature, and spectral features as nodes, and the neighborhood radius r = 5 cm; S72. In the first layer, use the EdgeConv operator, and the feature aggregation function is: ， where Θ is a learnable parameter matrix, and N(i) is the r-neighborhood of point i, is the feature vector of the i-th node in the -th layer, is the feature vector of the j-th node in the -th layer, and is the feature vector of the (i + 1)-th node in the S73. In the second layer, introduce a channel attention mechanism, and the feature weighting formula is: ， where, g c is the global average pooling feature of the c-th channel, m c is the global maximum pooling feature of the c-th channel, σ is the sigmoid function, and W1 and W2 are the radial basis function weights of the first node and the second node path.

6. A method for maintaining and restoring multiple ancient buildings based on 3D laser scanning according to claim 1, characterized in that, In step S08, the component-level semantic segmentation of the multimodal fusion model by using the semantic enhancement module includes the following steps: S81. Extract the connection gap of the mortise-tenon joint by using three-dimensional morphological opening operation, and the structural element is a sphere with a radius r morph = 3 mm; S82. Determine the fracture grade based on the point cloud density gradient. When the density decrease rate η ≥ 30%, it is marked as a severe fracture; S83. For the peeling area of the painted layer, compare the spectral reflectance difference matrix with the historical data. When the difference value ΔR ≥ 0.2, it is determined as human damage; S84. For historical renovation traces, use a graph neural network to analyze the similarity of process features and extract the curvature radius r of the tool marks tool and the included angle θ between the wood fiber directions fiber .

7. A method for maintaining and restoring multiple ancient buildings based on 3D laser scanning according to claim 1, characterized in that In step S09, the results of the digital maintenance plan also include: S91. Calculate the stress concentration coefficient K of the damaged component based on the finite element mechanics simulation model t , when K t ≥ 2.5, mark it as a high-risk area; otherwise, it is a low-risk area. S92. Match the mortise and tenon type by combining with the traditional craftsmanship database, and select the historical process parameters with the highest adaptability according to the formula where S match is the set of the best historical process parameters after matching, D current is the digital feature vector of the current damaged component, and D history is the process parameter feature vector stored in the historical process database; S93. Generate the numerical control machining path for the repair of the pitched roof surface. The machining step size is adaptively adjusted according to the curvature, and the adjustment formula is: ， wherein, is the CNC machining step size, and κ is the Gaussian curvature of the surface.

8. A non-transitory computer-readable storage medium storing at least one instruction or at least one program segment, characterized in that, The at least one instruction or the at least one program segment is loaded and executed by a processor to implement the steps of the three-dimensional laser scanning-based multi-element ancient building maintenance and restoration method according to any one of claims 1-7.

9. An electronic device, characterized in that, Comprising a processor and a memory, wherein at least one instruction or at least one program segment is stored in the memory, and the at least one instruction or the at least one program segment is loaded and executed by a processor to implement the steps of the three-dimensional laser scanning-based multi-element ancient building maintenance and restoration method according to any one of claims 1-7.

Images