Non-contact multi-scale deformation monitoring system and method for ancient building

Abstract

The invention discloses a non-contact multi-scale deformation monitoring system and method for an ancient building, and belongs to the technical field of ancient building monitoring protection, and the method comprises the following steps: S1, collecting three-dimensional laser point cloud data, depth image data and color image data of the ancient building, and carrying out the preprocessing of the three-dimensional laser point cloud data, depth image data and color image data; constructing a fusion image based on the preprocessed depth image data and color image data; s2, monitoring planar deformation data of the ancient building; s3, monitoring local micro-change data of the ancient building; s4, monitoring overall deformation data of the ancient building; s5, constructing a visual three-dimensional model of the historic building based on the three-dimensional laser point cloud data of the historic building and the fused image, associating the monitored planar deformation data, local micro-change data and overall deformation data of the historic building, and performing early warning when the data exceed a preset threshold value; according to the invention, non-contact multi-scale deformation monitoring of the ancient building is realized on the basis of monitoring the planar deformation data, the local micro-change data and the overall deformation data of the ancient building, and the monitoring precision is high.

Description

Technical Field

[0001] The present invention belongs to the technical field of ancient building monitoring and protection, and in particular relates to a non-contact multi-scale deformation monitoring system and method for ancient buildings. Background Art

[0002] Ancient buildings refer to buildings and structures left over from ancient humans that have important historical, cultural, artistic and scientific value.

[0003] Ancient buildings will deform due to long-term influence of natural erosion and human destruction. If the deformation of ancient buildings is not monitored in time, they are prone to collapse and cause safety accidents. It is also not conducive to the protection of ancient buildings. Therefore, deformation monitoring of ancient buildings is particularly important.

[0004] Current deformation monitoring of ancient buildings generally relies on traditional monitoring technologies, including strain gauges and displacement sensors. The working principle is to sense tiny deformations of building structures through physical contact. Although it has a reliable monitoring effect within a certain range, its installation process is complicated and may cause potential damage to the original appearance of the ancient buildings. At the same time, different sensors are susceptible to environmental factors such as temperature changes, humidity fluctuations and corrosion during long-term use, resulting in reduced accuracy and stability of monitoring data.

[0005] In view of this, a non-contact multi-scale deformation monitoring system and method for ancient buildings are designed to solve the above problems. Summary of the invention

[0006] To solve the problems raised in the above background technology, the present invention provides a non-contact multi-scale deformation monitoring system and method for ancient buildings, which can realize non-contact multi-scale deformation monitoring of ancient buildings based on the surface deformation data, local micro-deformation data and overall deformation data of the ancient buildings, and has the characteristics of high monitoring accuracy.

[0007] Another object of the present invention is to provide a non-contact multi-scale deformation monitoring system for ancient buildings.

[0008] To achieve the above object, the present invention provides the following technical solution: a non-contact multi-scale deformation monitoring method for ancient buildings, comprising the following steps:

[0009] S1: The SLAM data acquisition system equipped with LiDAR and Kinect collects and preprocesses the 3D laser point cloud data, depth image data and color image data of the ancient building, and constructs a fused image based on the preprocessed depth image data and color image data;

[0010] S2: monitor the three-dimensional laser point cloud data after the previous and current preprocessing to obtain the single deformation amount of the ancient building, monitor the three-dimensional laser point cloud data after the initial and current preprocessing to obtain the cumulative deformation amount of the ancient building, that is, monitor the surface deformation data of the ancient building;

[0011] S3: Based on the similarity criterion, the pixels of the fused image are clustered according to the color and spatial information to form superpixels through the SLIC algorithm, segmentation is performed based on the superpixels, the superpixel features of the previous and current times are monitored, and the local single micro-change data of the ancient building are obtained. The superpixel features of the first and current times are monitored to obtain the local cumulative micro-change data of the ancient building. The local single micro-change data and the local cumulative micro-change data of the ancient building are mapped to the fused image, the superpixel feature changes are counted, and the local micro-variables of the ancient building are obtained, that is, the local micro-change data of the ancient building are monitored;

[0012] S4: construct an ancient building model through multi-phase three-dimensional laser point cloud data, monitor the initial and current ancient building models, and obtain the overall deformation of the ancient building, that is, monitor the overall deformation data of the ancient building;

[0013] S5: Construct a visualized 3D model of the ancient building based on the 3D laser point cloud data and fused images of the ancient building, and simultaneously associate the surface deformation data, local micro-deformation data and overall deformation data of the ancient building monitoring, and issue an alert when the data exceeds the preset threshold.

[0014] Furthermore, in step S1, the preprocessing step of the three-dimensional laser point cloud data includes:

[0015] The collected 3D laser point cloud data of ancient buildings are filtered and denoised by using Gaussian filtering algorithm to reduce the noise and abnormal values ​​caused by the external environment and instruments during data collection;

[0016] The collected 3D laser point cloud data of ancient buildings are synchronized through time stamps to maintain the consistency of multi-sensor data;

[0017] Through the adaptive local feature extraction algorithm, feature points are extracted according to the local feature relationship of 3D laser point cloud data;

[0018] The extracted feature points are comprehensively described by the FPFH algorithm;

[0019] Roughly align 3D laser point cloud data by random sampling consistency;

[0020] The ICP algorithm is used to accurately align the 3D laser point cloud data.

[0021] Furthermore, in step S1, the preprocessing step of the depth image data includes:

[0022] The collected depth image data is filtered and denoised through the bilateral filtering image processing algorithm to improve the smoothness of the depth image and retain the edge features, that is, to improve the clarity and detail feature expression of the depth image;

[0023] Fill the data-free area of ​​the depth image by spatial interpolation;

[0024] The depth image error caused by camera lens distortion is corrected by Zhang’s calibration method;

[0025] Adjust the resolution of the depth image by resolution to keep the resolution of the depth image consistent.

[0026] Furthermore, in step S1, the preprocessing step of the color image data includes:

[0027] Performing color correction on color image data by using a template matching image color correction algorithm;

[0028] Image enhancement is performed on color image data using histogram equalization algorithm.

[0029] Furthermore, in step S1, the specific steps of constructing a fused image from the depth image data and the color image data include:

[0030] Generating a depth image based on the preprocessed depth image data, including:

[0031] Filter the depth image data of adjacent frames based on the Sobel operator to generate a Sobel feature map;

[0032] Based on Sobel feature matching, high-confidence matching points are extracted as support points;

[0033] Construct a Delaunay triangulated mesh for the extracted support points;

[0034] Construct a probabilistic generative model for stereo matching;

[0035] The probability generation model performs dense stereo matching based on maximum a posteriori probability density estimation to obtain the initial disparity map;

[0036] The obtained initial disparity map is processed to obtain a depth image, wherein the processing includes invalid hole interpolation, adaptive median filtering, connected domain noise removal and left-right consistency check;

[0037] The depth image and the color image are fused to generate a fused image, including:

[0038] The depth image and color image are converted to the camera coordinate system by using the coordinate reference conversion method to maintain the consistency of the coordinate references of the depth image and the color image;

[0039] The SIFT feature detection algorithm is used to extract local feature point pairs that match the depth image and the color image;

[0040] Match local feature point pairs of depth image and color image through FLANN;

[0041] Use the RANSAC algorithm to remove mismatched points in depth images and color images;

[0042] The homography matrix of the depth image and color image is estimated by the DLT algorithm, the color image is transformed to the coordinate system of the depth image, and a perspective transformation is performed;

[0043] The depth image and color image are fused and registered using the ICP algorithm to generate a fused image.

[0044] Furthermore, in step S3, the specific steps of clustering the fused image pixels according to color and spatial information to form superpixels based on the similarity criterion by using the SLIC algorithm include:

[0045] Determine the number of superpixels K;

[0046] Initialize clusters uniformly in the fused image;

[0047] Search within the 3×3 area of ​​each cluster and find the point with the smallest gradient as the cluster center k;

[0048] Calculate the comprehensive distance metric between each pixel i and each cluster center k in the fused image, expressed as:

[0049]

[0050] Where: d ik It is expressed as an intensity heterogeneity measure, usually using color distance, m is represented as a weight factor, which is used to balance the importance of color distance and spatial distance, and its value range is [1,20]. sk It is expressed as a spatial distance metric, which is used to calculate the Euclidean distance between pixel i and cluster center k;

[0051] Among them, the expression of intensity heterogeneity measure is:

[0052]

[0053] Where: l i 、a i and b i Represented as the color value of pixel i, l k 、a k and b k Represented as the color value of cluster center k;

[0054] The expression of spatial distance metric is:

[0055]

[0056] Where: x i and i Expressed as the coordinates of pixel i, x k and k Represented as the coordinates of cluster center k;

[0057] Assign pixel i in the fused image to the cluster with the smallest comprehensive distance metric, cluster center k;

[0058] Recalculate the average color value and position of all pixels i in each cluster as the new cluster center k, and repeat the above comprehensive distance measurement and allocation until the iteration stops when the cluster center changes minimally, thus forming a superpixel.

[0059] Furthermore, the specific steps of step S4 include:

[0060] The multi-temporal 3D laser point cloud is registered to a unified coordinate reference through the coordinate projection registration method;

[0061] The three-dimensional laser point cloud data is reconstructed into a triangulated network through encapsulation to obtain a surface reconstructed ancient building model;

[0062] Taking the initial ancient building model as a reference, the deviation of the overall fused point cloud in different phases is calculated;

[0063] By setting the threshold value to conduct multi-temporal point cloud deviation statistics and generate a three-dimensional point cloud deviation distribution map, the overall deformation monitoring of the ancient building can be realized.

[0064] Furthermore, in step S5, the specific steps of constructing a visualized three-dimensional model of the ancient building based on the three-dimensional laser point cloud data and the fused image of the ancient building include:

[0065] The 3D point cloud data and fused image are converted to the camera coordinate system through the coordinate reference conversion method to maintain the consistency of the coordinate reference of the 3D point cloud data and the fused image;

[0066] Convert 3D point cloud data into triangular mesh models;

[0067] Mapping the fused image onto the triangular mesh model, a three-dimensional model of the ancient building is constructed;

[0068] The constructed three-dimensional model of the ancient building is put onto the visualization platform for visualization.

[0069] The non-contact multi-scale deformation monitoring system for ancient buildings comprises:

[0070] The data acquisition module collects the 3D laser point cloud data, depth image data and color image data of the ancient buildings through the SLAM data acquisition system equipped with LiDAR and Kinect;

[0071] The data preprocessing module preprocesses the collected 3D laser point cloud data, depth image data and color image data of the ancient buildings;

[0072] A fused image generation module constructs a fused image based on the preprocessed depth image data and color image data;

[0073] The surface deformation data monitoring module monitors the three-dimensional laser point cloud data after the previous and current preprocessing to obtain the single deformation amount of the ancient building, monitors the three-dimensional laser point cloud data after the initial and current preprocessing to obtain the cumulative deformation amount of the ancient building, that is, monitors the surface deformation data of the ancient building;

[0074] The local micro-change data monitoring module uses the SLIC algorithm to cluster the fused image pixels according to color and spatial information based on the similarity criterion to form superpixels, performs segmentation based on superpixels, monitors the previous and current superpixel features, obtains the local single micro-change data of the ancient building, monitors the initial and current superpixel features, obtains the local cumulative micro-change data of the ancient building, maps the local single micro-change data and local cumulative micro-change data of the ancient building to the fused image, counts the superpixel feature changes, and obtains the local micro-variables of the ancient building, that is, monitors the local micro-change data of the ancient building;

[0075] The overall deformation data monitoring module builds the ancient building model through multi-phase three-dimensional laser point cloud data, monitors the initial and current ancient building models, and obtains the overall deformation of the ancient building, that is, monitors the overall deformation data of the ancient building;

[0076] Model visualization module, which constructs a visualized 3D model of the ancient building based on the 3D laser point cloud data and fused images of the ancient building, and simultaneously associates the surface deformation data, local micro-deformation data and overall deformation data of the ancient building monitoring;

[0077] The early warning module issues an early warning when the deformation data exceeds the preset threshold.

[0078] Compared with the prior art, the present invention has the following beneficial effects:

[0079] 1. The present invention collects three-dimensional laser point cloud data, depth image data and color image data of the ancient building through a SLAM data acquisition system equipped with LiDAR and Kinect and performs preprocessing, compares and monitors the surface deformation data of the ancient building based on the preprocessed three-dimensional laser point cloud data, generates a fused image based on the preprocessed depth image data and color image data, clusters the fused image pixels according to color and spatial information through the SLIC algorithm to form superpixels for segmentation, compares and monitors the local micro-change data of the ancient building based on the superpixels, constructs an ancient building model through multi-phase three-dimensional laser point cloud data, monitors the overall deformation data of the ancient building based on the comparison of the ancient building model, and realizes non-contact multi-scale deformation monitoring of the ancient building based on the monitoring of the surface deformation data, local micro-change data and overall deformation data of the ancient building, with high monitoring accuracy.

[0080] 2. The present invention constructs a three-dimensional model of the ancient building based on the pre-processed three-dimensional laser point cloud data, depth image data and color image data of the ancient building and performs visualization processing, and at the same time associates it with the multi-scale deformation monitoring data of the ancient building, which can realize the visualization of the deformation monitoring of the ancient building, and at the same time can issue an early warning when the deformation exceeds the preset value, thereby improving the deformation monitoring effect of the ancient building. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 is a flow chart of the method of the present invention;

[0082] Figure 2 It is a system framework diagram of the present invention;

[0083] In the figure: 1. Data acquisition module; 2. Data preprocessing module; 3. Fusion image generation module; 4. Surface deformation data monitoring module; 5. Local micro-change data monitoring module; 6. Overall deformation data monitoring module; 7. Model visualization module; 8. Early warning module. DETAILED DESCRIPTION

[0084] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0085] Embodiment 1

[0086] See attached Figure 1 The present invention provides the following technical solution: a non-contact multi-scale deformation monitoring method for ancient buildings, comprising the following steps:

[0087] S1: The SLAM data acquisition system equipped with LiDAR and Kinect collects and preprocesses the 3D laser point cloud data, depth image data and color image data of the ancient building, and constructs a fused image based on the preprocessed depth image data and color image data;

[0088] LiDAR, or light detection and ranging, is a spatial measurement system that integrates three technologies: laser, global positioning system, and inertial navigation system. It calculates the distance to the target object by emitting a laser beam and measuring its echo time, and can obtain ultra-high-precision three-dimensional point clouds;

[0089] Kinect, a 3D somatosensory camera, can obtain high-precision depth maps and color images;

[0090] SLAM, or simultaneous localization and mapping, usually relies on sensor data including lidar, cameras, and inertial sensors to obtain environmental information. The data is used to estimate the robot's position and movement and build a map of the environment;

[0091] The construction of the environmental map is based on the EKF framework, using point features to estimate the position and posture of the current device, and establishing observation models of points, lines, and planes. At the same time, a data association and state update model for line segments is formulated, and plane constraints are added. The unknown environment is described through line segment and plane feature maps, so as to achieve baseline data acquisition as the data benchmark for subsequent monitoring;

[0092] During the system data collection process, in order to maintain the matching degree of multi-phase monitoring data, a camera repositioning method based on point-line feature fusion matching is adopted to realize camera repositioning in multi-phase data collection, maximize the accuracy and reliability of multi-phase data collection, and avoid monitoring errors caused by position differences;

[0093] S2: monitor the three-dimensional laser point cloud data after the previous and current preprocessing to obtain the single deformation amount of the ancient building, monitor the three-dimensional laser point cloud data after the initial and current preprocessing to obtain the cumulative deformation amount of the ancient building, that is, monitor the surface deformation data of the ancient building;

[0094] S3: Based on the similarity criterion, the pixels of the fused image are clustered according to the color and spatial information to form superpixels through the SLIC algorithm, segmentation is performed based on the superpixels, the superpixel features of the previous and current times are monitored, and the local single micro-change data of the ancient building are obtained. The superpixel features of the first and current times are monitored to obtain the local cumulative micro-change data of the ancient building. The local single micro-change data and the local cumulative micro-change data of the ancient building are mapped to the fused image, the superpixel feature changes are counted, and the local micro-variables of the ancient building are obtained, that is, the local micro-change data of the ancient building are monitored;

[0095] S4: construct an ancient building model through multi-phase three-dimensional laser point cloud data, monitor the initial and current ancient building models, and obtain the overall deformation of the ancient building, that is, monitor the overall deformation data of the ancient building;

[0096] S5: Construct a visualized 3D model of the ancient building based on the 3D laser point cloud data and fused images of the ancient building, and simultaneously associate the surface deformation data, local micro-deformation data and overall deformation data of the ancient building monitoring, and issue an alert when the data exceeds the preset threshold.

[0097] Specifically, in step S1, the preprocessing step of the 3D laser point cloud data includes:

[0098] The collected 3D laser point cloud data of ancient buildings are filtered and denoised by using Gaussian filtering algorithm to reduce the noise and abnormal values ​​caused by the external environment and instruments during data collection;

[0099] The collected 3D laser point cloud data of ancient buildings are synchronized through time stamps to maintain the consistency of multi-sensor data;

[0100] Through the adaptive local feature extraction algorithm, feature points are extracted according to the local feature relationship of 3D laser point cloud data;

[0101] The extracted feature points are comprehensively described by the FPFH algorithm;

[0102] Roughly align 3D laser point cloud data by random sampling consistency;

[0103] The ICP algorithm is used to accurately align the 3D laser point cloud data.

[0104] Specifically, in step S1, the preprocessing step of the depth image data includes:

[0105] The collected depth image data is filtered and denoised through the bilateral filtering image processing algorithm to improve the smoothness of the depth image and retain the edge features, that is, to improve the clarity and detail feature expression of the depth image;

[0106] Fill the data-free area of ​​the depth image by spatial interpolation;

[0107] The depth image error caused by camera lens distortion is corrected by Zhang’s calibration method;

[0108] Adjust the resolution of the depth image by resolution to keep the resolution of the depth image consistent.

[0109] Specifically, in step S1, the preprocessing step of the color image data includes:

[0110] Performing color correction on color image data by using a template matching image color correction algorithm;

[0111] Image enhancement is performed on color image data using histogram equalization algorithm.

[0112] Specifically, in step S1, the specific steps of constructing a fused image from the depth image data and the color image data include:

[0113] Generating a depth image based on the preprocessed depth image data, including:

[0114] Filter the depth image data of adjacent frames based on the Sobel operator to generate a Sobel feature map;

[0115] Based on Sobel feature matching, high-confidence matching points are extracted as support points;

[0116] Construct a Delaunay triangulated mesh for the extracted support points;

[0117] Construct a probabilistic generative model for stereo matching;

[0118] The probability generation model performs dense stereo matching based on maximum a posteriori probability density estimation to obtain the initial disparity map;

[0119] The obtained initial disparity map is processed to obtain a depth image, wherein the processing includes invalid hole interpolation, adaptive median filtering, connected domain noise removal and left-right consistency check;

[0120] The depth image and the color image are fused to generate a fused image, including:

[0121] The depth image and color image are converted to the camera coordinate system by using the coordinate reference conversion method to maintain the consistency of the coordinate references of the depth image and the color image;

[0122] The SIFT feature detection algorithm is used to extract local feature point pairs that match the depth image and the color image;

[0123] Match local feature point pairs of depth image and color image through FLANN;

[0124] Use the RANSAC algorithm to remove mismatched points in depth images and color images;

[0125] The homography matrix of the depth image and color image is estimated by the DLT algorithm, the color image is transformed to the coordinate system of the depth image, and a perspective transformation is performed;

[0126] The depth image and color image are fused and registered using the ICP algorithm to generate a fused image.

[0127] Specifically, in step S3, the specific steps of clustering the fused image pixels according to color and spatial information to form superpixels based on the similarity criterion by using the SLIC algorithm include:

[0128] Determine the number of superpixels K;

[0129] Initialize clusters uniformly in the fused image;

[0130] Search within the 3×3 area of ​​each cluster and find the point with the smallest gradient as the cluster center k;

[0131] Calculate the comprehensive distance metric between each pixel i and each cluster center k in the fused image, expressed as:

[0132]

[0133] Where: d ik It is expressed as an intensity heterogeneity measure, usually using color distance, m is represented as a weight factor, which is used to balance the importance of color distance and spatial distance, and its value range is [1,20]. sk It is expressed as a spatial distance metric, which is used to calculate the Euclidean distance between pixel i and cluster center k;

[0134] Among them, the expression of intensity heterogeneity measure is:

[0135]

[0136] Where: l i 、a i and b i Represented as the color value of pixel i, l k 、a k and b k Represented as the color value of cluster center k;

[0137] The expression of spatial distance metric is:

[0138]

[0139] Where: x i and i Expressed as the coordinates of pixel i, x k and k Represented as the coordinates of cluster center k;

[0140] Assign pixel i in the fused image to the cluster with the smallest comprehensive distance metric, cluster center k;

[0141] Recalculate the average color value and position of all pixels i in each cluster as the new cluster center k, and repeat the above comprehensive distance measurement and allocation until the iteration stops when the cluster center changes minimally, thus forming a superpixel.

[0142] Specifically, the specific steps of step S4 include:

[0143] The multi-temporal 3D laser point cloud is registered to a unified coordinate reference through the coordinate projection registration method;

[0144] The three-dimensional laser point cloud data is reconstructed into a triangulated network through encapsulation to obtain a surface reconstructed ancient building model;

[0145] Taking the initial ancient building model as a reference, the deviation of the overall fused point cloud in different phases is calculated;

[0146] By setting the threshold value to conduct multi-temporal point cloud deviation statistics and generate a three-dimensional point cloud deviation distribution map, the overall deformation monitoring of the ancient building can be realized.

[0147] Specifically, in step S5, the specific steps of constructing a visualized three-dimensional model of the ancient building based on the three-dimensional laser point cloud data and the fused image of the ancient building include:

[0148] The 3D point cloud data and fused image are converted to the camera coordinate system through the coordinate reference conversion method to maintain the consistency of the coordinate reference of the 3D point cloud data and the fused image;

[0149] Convert 3D point cloud data into triangular mesh models;

[0150] Mapping the fused image onto the triangular mesh model, a three-dimensional model of the ancient building is constructed;

[0151] The constructed three-dimensional model of the ancient building is put onto the visualization platform for visualization.

[0152] Embodiment 2

[0153] See attached Figure 2 , a non-contact multi-scale deformation monitoring system for ancient buildings, including:

[0154] Data acquisition module 1, collects three-dimensional laser point cloud data, depth image data and color image data of ancient buildings through a SLAM data acquisition system equipped with LiDAR and Kinect;

[0155] Data preprocessing module 2, preprocessing the collected three-dimensional laser point cloud data, depth image data and color image data of the ancient building;

[0156] A fused image generation module 3 constructs a fused image based on the preprocessed depth image data and color image data;

[0157] The surface deformation data monitoring module 4 monitors the three-dimensional laser point cloud data after the previous and current preprocessing to obtain the single deformation amount of the ancient building, monitors the three-dimensional laser point cloud data after the initial and current preprocessing to obtain the cumulative deformation amount of the ancient building, that is, monitors the surface deformation data of the ancient building;

[0158] The local micro-change data monitoring module 5 clusters the fused image pixels according to the color and spatial information to form superpixels based on the similarity criterion through the SLIC algorithm, performs segmentation based on the superpixels, monitors the superpixel features of the previous and current times, obtains the local single micro-change data of the ancient building, monitors the superpixel features of the first and current times, obtains the local cumulative micro-change data of the ancient building, maps the local single micro-change data and the local cumulative micro-change data of the ancient building to the fused image, counts the superpixel feature changes, obtains the local micro-variables of the ancient building, that is, monitors the local micro-change data of the ancient building;

[0159] The overall deformation data monitoring module 6 constructs an ancient building model through multi-phase three-dimensional laser point cloud data, monitors the initial and current ancient building models, and obtains the overall deformation of the ancient building, that is, monitors the overall deformation data of the ancient building;

[0160] Model visualization module 7, constructs a visualized 3D model of the ancient building based on the 3D laser point cloud data and fused images of the ancient building, and simultaneously associates the surface deformation data, local micro-deformation data and overall deformation data of the ancient building monitoring;

[0161] The early warning module 8 issues an early warning when the deformation data exceeds a preset threshold.

[0162] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A non-contact multi-scale deformation monitoring method for ancient buildings, characterized in that: The following steps are involved: S1: The SLAM data acquisition system equipped with LiDAR and Kinect collects and preprocesses the 3D laser point cloud data, depth image data and color image data of the ancient building, and constructs a fused image based on the preprocessed depth image data and color image data; S2: monitor the three-dimensional laser point cloud data after the previous and current preprocessing to obtain the single deformation amount of the ancient building, monitor the three-dimensional laser point cloud data after the initial and current preprocessing to obtain the cumulative deformation amount of the ancient building, that is, monitor the surface deformation data of the ancient building; S3: Based on the similarity criterion, the pixels of the fused image are clustered according to the color and spatial information to form superpixels through the SLIC algorithm, segmentation is performed based on the superpixels, the superpixel features of the previous and current times are monitored, and the local single micro-change data of the ancient building are obtained. The superpixel features of the first and current times are monitored to obtain the local cumulative micro-change data of the ancient building. The local single micro-change data and the local cumulative micro-change data of the ancient building are mapped to the fused image, the superpixel feature changes are counted, and the local micro-variables of the ancient building are obtained, that is, the local micro-change data of the ancient building are monitored; S4: construct an ancient building model through multi-phase three-dimensional laser point cloud data, monitor the initial and current ancient building models, and obtain the overall deformation of the ancient building, that is, monitor the overall deformation data of the ancient building; S5: Construct a visualized 3D model of the ancient building based on the 3D laser point cloud data and fused images of the ancient building, and simultaneously associate the surface deformation data, local micro-deformation data and overall deformation data of the ancient building monitoring, and issue an alert when the data exceeds the preset threshold.

2. The non-contact multi-scale deformation monitoring method for ancient buildings according to claim 1 is characterized in that: In step S1, the preprocessing step of the 3D laser point cloud data includes: The collected 3D laser point cloud data of ancient buildings are filtered and denoised by using Gaussian filtering algorithm to reduce the noise and abnormal values ​​caused by the external environment and instruments during data collection; The collected 3D laser point cloud data of ancient buildings are synchronized through time stamps to maintain the consistency of multi-sensor data; Through the adaptive local feature extraction algorithm, feature points are extracted according to the local feature relationship of 3D laser point cloud data; The extracted feature points are comprehensively described by the FPFH algorithm; Roughly align 3D laser point cloud data by random sampling consistency; The ICP algorithm is used to accurately align the 3D laser point cloud data.

3. The non-contact multi-scale deformation monitoring method for ancient buildings according to claim 1 is characterized in that: In step S1, the preprocessing step of the depth image data includes: The collected depth image data is filtered and denoised through the bilateral filtering image processing algorithm to improve the smoothness of the depth image and retain the edge features, that is, to improve the clarity and detail feature expression of the depth image; Fill the data-free area of ​​the depth image by spatial interpolation; The depth image error caused by camera lens distortion is corrected by Zhang’s calibration method; Adjust the resolution of the depth image by resolution to keep the resolution of the depth image consistent.

4. The non-contact multi-scale deformation monitoring method for ancient buildings according to claim 1 is characterized in that: In step S1, the preprocessing step of the color image data includes: Performing color correction on color image data by using a template matching image color correction algorithm; Image enhancement is performed on color image data using histogram equalization algorithm.

5. The non-contact multi-scale deformation monitoring method for ancient buildings according to claim 1 is characterized in that: In step S1, the specific steps of constructing a fused image with the depth image data and the color image data include: Generating a depth image based on the preprocessed depth image data, including: Filter the depth image data of adjacent frames based on the Sobel operator to generate a Sobel feature map; Based on Sobel feature matching, high-confidence matching points are extracted as support points; Construct a Delaunay triangulated mesh for the extracted support points; Construct a probabilistic generative model for stereo matching; The probability generation model performs dense stereo matching based on maximum a posteriori probability density estimation to obtain the initial disparity map; The obtained initial disparity map is processed to obtain a depth image, wherein the processing includes invalid hole interpolation, adaptive median filtering, connected domain noise removal and left-right consistency check; The depth image and the color image are fused to generate a fused image, including: The depth image and color image are converted to the camera coordinate system by using the coordinate reference conversion method to maintain the consistency of the coordinate references of the depth image and the color image; The SIFT feature detection algorithm is used to extract local feature point pairs that match the depth image and the color image; Match local feature point pairs of depth image and color image through FLANN; Use the RANSAC algorithm to remove mismatched points in depth images and color images; The homography matrix of the depth image and color image is estimated by the DLT algorithm, the color image is transformed to the coordinate system of the depth image, and a perspective transformation is performed; The depth image and color image are fused and registered using the ICP algorithm to generate a fused image.

6. The non-contact multi-scale deformation monitoring method for ancient buildings according to claim 1, characterized in that: In step S3, the specific steps of clustering the fused image pixels according to color and spatial information to form superpixels based on the similarity criterion by using the SLIC algorithm include: Determine the number of superpixels K; Initialize clusters uniformly in the fused image; Search within the 3×3 area of ​​each cluster and find the point with the smallest gradient as the cluster center k; Calculate the comprehensive distance metric between each pixel i and each cluster center k in the fused image, expressed as: Where: d ik It is expressed as an intensity heterogeneity measure, usually using color distance, m is represented as a weight factor, which is used to balance the importance of color distance and spatial distance, and its value range is [1,20]. sk It is expressed as a spatial distance metric, which is used to calculate the Euclidean distance between pixel i and cluster center k; Among them, the expression of intensity heterogeneity measure is: Where: l i 、a i and b i Represented as the color value of pixel i, l k 、a k and b k Represented as the color value of cluster center k; The expression of spatial distance metric is: Where: x i and i Expressed as the coordinates of pixel i, x k and k Represented as the coordinates of cluster center k; Assign pixel i in the fused image to the cluster with the smallest comprehensive distance metric, cluster center k; Recalculate the average color value and position of all pixels i in each cluster as the new cluster center k, and repeat the above comprehensive distance measurement and allocation until the iteration stops when the cluster center changes minimally, thus forming a superpixel.

7. The non-contact multi-scale deformation monitoring method for ancient buildings according to claim 1 is characterized by: The specific steps of step S4 include: The multi-temporal 3D laser point cloud is registered to a unified coordinate reference through the coordinate projection registration method; The three-dimensional laser point cloud data is reconstructed into a triangulated network through encapsulation to obtain a surface reconstructed ancient building model; Taking the initial ancient building model as a reference, the deviation of the overall fused point cloud in different phases is calculated; By setting the threshold value to conduct multi-temporal point cloud deviation statistics and generate a three-dimensional point cloud deviation distribution map, the overall deformation monitoring of the ancient building can be realized.

8. The non-contact multi-scale deformation monitoring method for ancient buildings according to claim 1, characterized in that: In step S5, the specific steps of constructing a visualized three-dimensional model of the ancient building based on the three-dimensional laser point cloud data and the fused image of the ancient building include: The 3D point cloud data and fused image are converted to the camera coordinate system through the coordinate reference conversion method to maintain the consistency of the coordinate reference of the 3D point cloud data and the fused image; Convert 3D point cloud data into triangular mesh models; Mapping the fused image onto the triangular mesh model, a three-dimensional model of the ancient building is constructed; The constructed three-dimensional model of the ancient building is put onto the visualization platform for visualization.

9. A non-contact multi-scale deformation monitoring system for ancient buildings according to any one of claims 1 to 8, characterized in that: include: The data acquisition module (1) collects three-dimensional laser point cloud data, depth image data and color image data of the ancient building through a SLAM data acquisition system equipped with LiDAR and Kinect; A data preprocessing module (2) preprocesses the collected three-dimensional laser point cloud data, depth image data and color image data of the ancient building; A fused image generation module (3) constructs a fused image based on the preprocessed depth image data and color image data; The surface deformation data monitoring module (4) monitors the three-dimensional laser point cloud data after the previous and current preprocessing to obtain the single deformation amount of the ancient building, and monitors the three-dimensional laser point cloud data after the initial and current preprocessing to obtain the cumulative deformation amount of the ancient building, that is, monitors the surface deformation data of the ancient building; The local micro-change data monitoring module (5) clusters the fused image pixels according to the color and spatial information to form superpixels based on the similarity criterion through the SLIC algorithm, performs segmentation based on the superpixels, monitors the superpixel features of the previous and current times, obtains the local single micro-change data of the ancient building, monitors the superpixel features of the first and current times, obtains the local cumulative micro-change data of the ancient building, maps the local single micro-change data and the local cumulative micro-change data of the ancient building to the fused image, counts the superpixel feature changes, obtains the local micro-variables of the ancient building, that is, monitors the local micro-change data of the ancient building; The overall deformation data monitoring module (6) constructs an ancient building model through multi-phase three-dimensional laser point cloud data, monitors the initial and current ancient building models, and obtains the overall deformation of the ancient building, that is, monitors the overall deformation data of the ancient building; A model visualization module (7) constructs a visualized three-dimensional model of the ancient building based on the three-dimensional laser point cloud data and fused images of the ancient building, and simultaneously associates the surface deformation data, local micro-deformation data and overall deformation data of the ancient building monitoring; The early warning module (8) issues an early warning when the deformation data exceeds a preset threshold.

Images