Dam displacement and settlement monitoring method and system

Abstract

The invention provides a dam displacement and settlement monitoring method and system, and the method comprises the steps: obtaining monitoring region images of a first stage and a second stage of a target dam, recording the monitoring region images as a source image and a target image, and recording monitoring region point clouds of the first stage and the second stage as a source point cloud and a target point cloud; converting the target point cloud to the same position as the source point cloud to obtain a registration point cloud; performing registration fusion on the source image and the source point cloud to obtain first three-dimensional point cloud data, and performing registration fusion on the registration point cloud and the target image to obtain second three-dimensional point cloud data; marking the two kinds of three-dimensional point cloud data to form a data set; training the point cloud semantic segmentation network by using the data set, and extracting two-stage monitoring results of the dam by using the trained point cloud semantic segmentation network; and obtaining dam displacement and settlement monitoring data by differentiating the monitoring results of the two stages. Based on the method, the invention further provides a dam displacement and settlement monitoring system. The displacement and settlement conditions of the dam can be accurately monitored.

Description

Technical Field

[0001] The present invention belongs to the technical field of engineering monitoring, and particularly relates to a method and system for monitoring the displacement and settlement of a dam. Background Art

[0002] The safety of a dam is crucial for protecting the lives and property of the downstream areas. Dam safety monitoring is the measurement and observation of the main structure, foundation, both banks of the slope, related facilities and the surrounding environment of a water conservancy and hydropower project through instrument observation and inspection tours; "monitoring" includes both instrument observation of fixed measuring points on the building at certain frequencies and regular or irregular visual inspections and instrument explorations of a large range of objects on the surface and inside of the building.

[0003] The safety of a dam is crucial for protecting the lives and property of the downstream areas. Although traditional dam monitoring methods such as GNSS observation and leveling are effective, they have limitations such as long data acquisition time, limited coverage, and limited accuracy. With the development of unmanned aerial vehicle (UAV) technology and light detection and ranging (LiDAR) technology, using a UAV equipped with a laser scanning device for dam monitoring has become an effective means. The method based on DSM difference is a method for change detection based on point cloud data, but this method depends on elevation and requires manual adjustment of thresholds to extract ground objects and their change information, with low automation. Summary of the Invention

[0004] To solve the above technical problems, the present invention proposes a method and system for monitoring the displacement and settlement of a dam. By multi-source data fusion and change detection technology combined with classification and detection, the displacement and settlement of the dam are accurately monitored, providing a scientific basis for dam safety management.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A method for monitoring the displacement and settlement of a dam, comprising the following steps:

[0007] Obtain the monitoring area image of the target dam in the first phase as the source image, and the monitoring area point cloud of the target dam in the first phase as the source point cloud; obtain the monitoring area image of the target dam in the second phase as the target image, and the monitoring area point cloud of the target dam in the second phase as the target point cloud;

[0008] Convert the target point cloud to the same position as the source point cloud to complete point cloud registration to obtain the registered point cloud;

[0009] Register and fuse the source image and the source point cloud to obtain the first three-dimensional point cloud data, and register and fuse the registered point cloud and the target image to obtain the second three-dimensional point cloud data;

[0010] Construct a first dataset for point cloud semantic segmentation after annotating the first three-dimensional point cloud data; use the first dataset to train a point cloud semantic segmentation network, and use the trained point cloud semantic segmentation network to extract the first monitoring results of the dam; construct a first dataset for point cloud semantic segmentation after annotating the second three-dimensional point cloud data; use the second dataset to train a point cloud semantic segmentation network, and use the trained point cloud semantic segmentation network to extract the second monitoring results of the dam; use the difference between the first monitoring results and the second monitoring results to obtain the final data of dam displacement and settlement monitoring.

[0011] Further, the process of obtaining the dam settlement by using the difference between the first monitoring results and the second monitoring results includes: calculating the first elevation mean matrix by using the first monitoring results; calculating the first elevation mean matrix by using the second monitoring results; judging the settlement conditions of each position of the dam based on the difference between the first elevation mean matrix and the second elevation mean matrix.

[0012] Further, the process of obtaining the dam displacement by using the difference between the first monitoring results and the second monitoring results includes: calculating the first horizontal displacement of the point cloud change by using the first monitoring results; calculating the second horizontal displacement of the point cloud change by using the second monitoring results; when either the first horizontal displacement or the second horizontal displacement is greater than a preset displacement threshold, it is judged that the dam has displaced in this area.

[0013] Further, the specific process of registering the target point cloud to the same position as the source point cloud to obtain the registered point cloud includes:

[0014] Determine the corresponding homologous point cloud pairs in the source point cloud and the target point cloud;

[0015] Construct a source point cloud dataset {t i}, for each point in the source point cloud dataset {t i}, determine the point {s i} in the target point cloud that is closest to it through a spatial search algorithm; when the error function between the source point cloud dataset {t i} and the closest point {s i} is minimized, determine the optimal first rotation matrix R1 and first translation vector T1 between the source point cloud and the target point cloud, and calculate the registered coordinates {s' i} of the target point cloud by using the first rotation matrix R1 and the first translation vector T1;

[0016] Use the registered coordinates {s' i} of the target point cloud as the registered point cloud set to compare with the source point cloud dataset {t i}, and calculate {s i} and {t iThe average distance d between the two points is used as the error metric, and the first rotation matrix R1 and the first translation vector T1 are updated iteratively. When the error metric d is less than the threshold or reaches the predetermined maximum number of iterations, the final first rotation matrix and the first translation vector are obtained, and the registration between the target point cloud and the source point cloud is completed to obtain the registered point cloud.

[0017] Furthermore, the target point cloud registration coordinates {s′ i The calculation method of} is:

[0018] s′ i =R1s i +T1;

[0019] Among them, i represents the point cloud number.

[0020] Furthermore, the process of registering and fusing the source image and the source point cloud to obtain the first three-dimensional point cloud data includes:

[0021] After filtering the source point cloud, extracting the first image feature points from the source image and extracting the first point cloud feature points from the source point cloud respectively;

[0022] Among them, the three-dimensional coordinates of the first point cloud feature point are P1 = [x1, y1, z1] T ; The pixel coordinates of the first image feature point are Q1 = [u1, v1] T ; The second rotation matrix is expressed as Where γ2 is the rotation angle from the first point cloud feature point to the first image feature point along the z-axis; the second translation vector is expressed as: Among them, t 2x is the x-axis translation of the first point cloud feature point in the image coordinate system; t 2y is the y-axis translation of the first image feature point in the image coordinate system;

[0023] The formula Q′1=R2P1+T2 is used to complete the registration of the first image feature point and the first point cloud feature point; wherein Q1′ is the pixel coordinate after the first image feature point and the first point cloud feature point are registered;

[0024] The registered pixel coordinates are assigned to the corresponding source point cloud data to obtain the expression of each point cloud: (X1, Y1, Z1, Intensity1, R1, G1, B1); X1, Y1, Z1 represent the Cartesian space x-coordinate, y-coordinate and z-coordinate of each point in the source point cloud; Intensity1 represents the first reflection intensity of the precision radar source point cloud; R1, G1, B1 represent the spectral information red, green and blue given to the source point cloud by the image, and the fusion between the source image and the source point cloud is completed to obtain the first three-dimensional point cloud data.

[0025] Further, the process of registering and fusing the registered point cloud and the target image to obtain the second 3D point cloud data includes:

[0026] After filtering the registered point cloud to remove outliers, extract second image feature points from the target image and second point cloud feature points from the registered target point cloud respectively;

[0027] The three-dimensional coordinates of the second point cloud feature point are P2 = [x2, y2, z2] T ; The pixel coordinates of the second image feature point are Q2 = [u2, v2] T ; The third rotation matrix is expressed as where γ3 is the rotation angle of the second point cloud feature point to the second image feature point along the z-axis; The third translation vector is expressed as: where, t 3x is the x-axis translation amount of the second point cloud feature point in the image coordinate system; t 3y is the y-axis translation amount of the second point cloud feature point in the image coordinate system;

[0028] Use the formula Q'2 = R3P2 + T3 to complete the registration of the second image feature point and the second point cloud feature point; where Q'2 is the pixel coordinates after registration of the second image feature point and the second point cloud feature point;

[0029] Assign the registered pixel coordinates to the corresponding registered point cloud to obtain the expression of each registered point cloud as: (X2, Y2, Z2, Intensity2, R2, G2, B2); where X2, Y2, Z2 represent the Cartesian space x coordinate, y coordinate and z coordinate of each point of the registered point cloud; Intensity2 represents the second reflection intensity of the lidar registered point cloud; R2, G2, B2 represent the spectral information red, green and blue given by the image to the registered point cloud, and complete the fusion between the registered target point cloud and the target image to obtain the second 3D point cloud data.

[0030] Further, after annotating the first 3D point cloud data, construct a first dataset for point cloud semantic segmentation; use the first dataset to train the point cloud semantic segmentation network, and the specific process of using the trained point cloud semantic segmentation network to extract the first monitoring result of the dam includes:

[0031] Annotate the first 3D point cloud data with the dam surface, dam embankment, dam slope and horse path to form a first dataset for point cloud semantic segmentation; divide the first dataset into a first training set and a first validation set according to a preset ratio;

[0032] The trained point cloud semantic segmentation network is obtained by training the point cloud semantic segmentation network using the first training set. The trained point cloud semantic segmentation network is used to extract the first monitoring target of the dam, and the first verification set is used to verify the accuracy of the first monitoring target.

[0033] Furthermore, after annotating the second 3D point cloud data, the first dataset for point cloud semantic segmentation is constructed; the process of using the second dataset to train the point cloud semantic segmentation network and using the trained point cloud semantic segmentation network to extract the second monitoring result of the dam specifically includes:

[0034] After annotating the second 3D point cloud data with the dam surface, dam embankment, dam slope, and berm, the second dataset for point cloud semantic segmentation is formed; the second dataset is divided into a second training set and a second verification set according to a preset ratio;

[0035] The trained point cloud semantic segmentation network is obtained by training the point cloud semantic segmentation network using the second training set. The trained point cloud semantic segmentation network is used to extract the second monitoring target of the dam, and the second verification set is used to verify the accuracy of the second monitoring target.

[0036] The present invention also proposes a monitoring system for dam displacement and settlement, including a data acquisition module, a first registration module, a registration fusion module, and a monitoring module;

[0037] The data acquisition module is used to obtain the monitoring area image of the first phase of the target dam, denoted as the source image, and the monitoring area point cloud of the first phase of the target dam, denoted as the source point cloud; obtain the monitoring area image of the second phase of the target dam, denoted as the target image, and the monitoring area point cloud of the second phase of the target dam, denoted as the target point cloud;

[0038] The first registration module is used to transform the target point cloud to the same position as the source point cloud to complete point cloud registration and obtain the registered point cloud;

[0039] The registration fusion module is used to register and fuse the source image and the source point cloud to obtain the first 3D point cloud data, and register and fuse the registered point cloud and the target image to obtain the second 3D point cloud data;

[0040] The monitoring module is used to annotate the first 3D point cloud data to construct the first dataset for point cloud semantic segmentation; use the first dataset to train the point cloud semantic segmentation network, and use the trained point cloud semantic segmentation network to extract the first monitoring result of the dam; annotate the second 3D point cloud data to construct the first dataset for point cloud semantic segmentation; use the second dataset to train the point cloud semantic segmentation network, and use the trained point cloud semantic segmentation network to extract the second monitoring result of the dam; obtain the final data of dam displacement and settlement monitoring by taking the difference between the first monitoring result and the second monitoring result.

[0041] The effects provided in the invention content are only the effects of the embodiments, rather than all the effects of the invention. One of the technical solutions in the above technical solutions has the following advantages or beneficial effects:

[0042] The present invention proposes a method and system for monitoring the displacement and settlement of a dam. The method includes the following steps: obtaining the monitoring area image of the first phase of the target dam as the source image, and the monitoring area point cloud of the first phase of the target dam as the source point cloud; obtaining the monitoring area image of the second phase of the target dam as the target image, and the monitoring area point cloud of the second phase of the target dam as the target point cloud; converting the target point cloud to the same position as the source point cloud to complete point cloud registration to obtain the registered point cloud; registering and fusing the source image and the source point cloud to obtain the first three-dimensional point cloud data, and registering and fusing the registered point cloud and the target image to obtain the second three-dimensional point cloud data; annotating the first three-dimensional point cloud data to construct the first data set for point cloud semantic segmentation; using the first data set to train the point cloud semantic segmentation network, and using the trained point cloud semantic segmentation network to extract the first monitoring result of the dam; annotating the second three-dimensional point cloud data to construct the first data set for point cloud semantic segmentation; using the second data set to train the point cloud semantic segmentation network, and using the trained point cloud semantic segmentation network to extract the second monitoring result of the dam; using the difference between the first monitoring result and the second monitoring result to obtain the final data for dam displacement and settlement monitoring. Based on a method for monitoring the displacement and settlement of a dam, a system for monitoring the displacement and settlement of a dam is also proposed. The present invention combines the point cloud semantic segmentation algorithm based on change detection to improve the automation level, and can monitor the changes in the horizontal and vertical positions of different categories and different positions according to the preset target.

[0043] Through multi-source data fusion and change detection technology, the present invention can efficiently and accurately monitor the displacement and settlement conditions of different areas of the dam, providing a scientific basis for dam safety management. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flowchart of a method for monitoring the displacement and settlement of a dam proposed in Embodiment 1 of the present invention;

[0045] Figure 2 It is a simulated top-down source image of the dam proposed in Embodiment 1 of the present invention;

[0046] Figure 3 It is a schematic diagram of the original point cloud of the dam simulation proposed in Embodiment 1 of the present invention;

[0047] Figure 4 It is the registered point cloud data after registration and fusion of the source image and the original point cloud proposed in Embodiment 1 of the present invention;

[0048] Figure 5 It is a schematic diagram of a system for monitoring the displacement and settlement of a dam proposed in Embodiment 2 of the present invention. Specific Embodiments

[0049] To clearly illustrate the technical features of this solution, the present invention will be described in detail below through specific embodiments and in conjunction with its accompanying drawings. The following disclosure provides many different embodiments or examples for implementing different structures of the present invention. To simplify the disclosure of the present invention, the components and settings of specific examples are described below. In addition, the present invention may repeat reference numerals and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed. It should be noted that the components illustrated in the accompanying drawings are not necessarily drawn to scale. The present invention omits the description of well-known components and processing technologies and processes to avoid unnecessarily limiting the present invention.

[0050] Embodiment 1

[0051] Embodiment 1 of the present invention proposes a method for monitoring the displacement and settlement of a dam. This method is based on the acquired UAV remote sensing images and airborne LIDAR point cloud data to monitor the displacement and settlement of the dam.

[0052] In step S1, the monitoring area image of the first phase of the target dam is obtained and recorded as the source image, and the monitoring area point cloud of the first phase of the target dam is recorded as the source point cloud; the monitoring area image of the second phase of the target dam is obtained and recorded as the target image, and the monitoring area point cloud of the second phase of the target dam is recorded as the target point cloud.

[0053] In this application, a UAV is used to carry an RGB camera and a laser scanning device to obtain UAV images and laser point cloud data in the dam monitoring area for two phases.

[0054] Fly multiple flight lines in the dam monitoring area to obtain high-resolution RGB images (resolution 3840×2160). The UAV flight path design should cover the entire dam area, and a certain overlap (30%-50%) should be ensured between adjacent flight lines. The monitoring area image of the first phase obtained is recorded as the source image, and the monitoring area image of the second phase is recorded as the target image. The multiple strip images are stitched using the SIFT feature detection and matching algorithm to generate a complete image of the monitoring area. Figure 2 This is the simulated top-down source image of the dam proposed in Embodiment 1 of the present invention.

[0055] Use an airborne LIDAR device to simultaneously obtain high-precision laser point clouds. The monitoring area point cloud of the first phase obtained is recorded as the source point cloud, and the monitoring area point cloud of the second phase is recorded as the target point cloud. Figure 3 This is the schematic diagram of the original point cloud of the dam simulation proposed in Embodiment 1 of the present invention. The obtained source point cloud Spc is an array of 135468*3, and the target point cloud Tpc is an array of 1873692*3.

[0056] In step S2, the target point cloud is transformed to the same position as the source point cloud to complete point cloud registration and obtain the registered point cloud; that is, the target point cloud is transformed to the same position as the source point cloud through a rigid transformation to complete point cloud registration and obtain the registered point cloud.

[0057] Using a rough registration method, such as initial alignment based on feature point matching, to find the preliminary correspondence between the source point cloud and the target point cloud. Specifically: Use feature extraction methods such as FPFH (Fast Point Feature Histogram) to extract point cloud feature points, and perform preliminary registration through the RANSAC (Random Sample Consensus) algorithm to obtain corresponding homologous point cloud pairs in the source point cloud and the target point cloud;

[0058] Use the ICP (Iterative Closest Point) algorithm for fine registration, and construct the source point cloud data set {t i}, for each point in the source point cloud data set {t i}, determine the point {s i} in the target point cloud that is closest to it through a spatial search algorithm; when the error function between the source point cloud data set {t i} and the closest point {s i} is minimized, determine the optimal first rotation matrix R1 and first translation vector T1 between the source point cloud and the target point cloud, and calculate the registered coordinates {s' i} of the target point cloud using the first rotation matrix R1 and the first translation vector T1;

[0059] Take the registered coordinates {s' i} of the target point cloud as the registered point cloud set and compare it with the source point cloud data set {t i}, calculate the average distance d between {s i} and {t i} as the error metric, and by continuously iteratively updating the first rotation matrix R1 and the first translation vector T1, when the error metric d is less than the threshold or reaches the predetermined maximum number of iterations, obtain the final first rotation matrix and first translation vector, and complete the registration between the target point cloud and the source point cloud to obtain the registered point cloud Tpc_c.

[0060] The calculation method of the registered coordinates {s' i} of the punctuation cloud is:

[0061] s' i = R1s i + T1;

[0062] Where i represents the point cloud serial number.

[0063]

[0064] In step S3, the source image and the source point cloud are registered and fused to obtain the first three-dimensional point cloud data, and the registered point cloud and the target image are registered and fused to obtain the second three-dimensional point cloud data;

[0065] The process of registering and fusing the source image and the source point cloud to obtain the first three-dimensional point cloud data includes:

[0066] Filter the source point cloud to remove outliers, extract the first image feature points from the source image respectively, and extract the first point cloud feature points from the source point cloud; For image feature point extraction, algorithms such as SIFT and SURF (Speeded Up Robust Features) can be used, and for point cloud feature point extraction, algorithms such as ISS (Intrinsic Shape Signature) can be used.

[0067] Among them, the three-dimensional coordinates of the first point cloud feature point are P1 = [x1, y1, z1] T ; The pixel coordinates of the first image feature point are Q1 = [u1, v1] T ;

[0068] The second rotation matrix is expressed as where γ2 is the rotation angle along the z-axis from the first point cloud feature point to the first image feature point;

[0069] The second translation vector is expressed as: where, t 2x is the translation amount of the first point cloud feature point along the x-axis in the image coordinate system; t 2y is the translation amount of the first image feature point along the y-axis in the image coordinate system;

[0070] Use the formula Q′1 = R2P1 + T2 to complete the registration of the first image feature point and the first point cloud feature point; where Q1' is the pixel coordinates after the registration of the first image feature point and the first point cloud feature point;

[0071]

[0072] Assign the registered pixel coordinates to the corresponding source point cloud data to obtain the expression of each point cloud as: (X1, Y1, Z1, Intensity1, R1, G1, B1); where X1, Y1, Z1 represent the Cartesian space x coordinate, y coordinate and z coordinate of each point of the source point cloud; Intensity1 represents the first reflection intensity of the lidar source point cloud; R1, G1, B1 represent the spectral information red, green and blue assigned by the image to the source point cloud, and complete the fusion between the source image and the source point cloud to obtain the first three-dimensional point cloud data. Figure 4 This is a schematic diagram of the registered point cloud data after the registration and fusion of the source image and the original point cloud proposed in Embodiment 1 of the present invention.

[0073] In step S4, after annotating the first three-dimensional point cloud data, a first dataset for point cloud semantic segmentation is constructed; the point cloud semantic segmentation network is trained using the first dataset, and the first monitoring result of the dam is extracted using the trained point cloud semantic segmentation network;

[0074] The specific process includes: based on the Terrascan software of MicroStation, the first three-dimensional point cloud data is annotated with the dam surface, dam embankment, dam slope and berm to form a first dataset for point cloud semantic segmentation; the first dataset is divided into a first training set and a first validation set according to a preset ratio; the first dataset is randomly assigned to the first training set (1,607,328 points) and the first validation set (401,832 points) in a ratio of 8:2;

[0075] The RandLA-Net point cloud semantic segmentation network is trained using the first training set to obtain a trained point cloud semantic segmentation network. The trained point cloud semantic segmentation network is used to extract the first monitoring target of the dam, and the first validation set is used to verify the accuracy of the first monitoring target. The iou of the first validation set is 88.7%.

[0076] After annotating the second three-dimensional point cloud data, a first dataset for point cloud semantic segmentation is constructed; the point cloud semantic segmentation network is trained using the second dataset, and the second monitoring result of the dam is extracted using the trained point cloud semantic segmentation network;

[0077] The specific process includes: based on the Terrascan software of MicroStation, the second three-dimensional point cloud data is annotated with the dam surface, dam embankment, dam slope and berm to form a second dataset for point cloud semantic segmentation; the second dataset is divided into a second training set and a second validation set according to a preset ratio; the second dataset is randomly assigned to the second training set (1,607,328 points) and the second validation set (401,832 points) in a ratio of 8:2;

[0078] The RandLA-Net point cloud semantic segmentation network is trained using the second training set to obtain a trained point cloud semantic segmentation network. The trained point cloud semantic segmentation network is used to extract the second monitoring target of the dam, and the second validation set is used to verify the accuracy of the second monitoring target. The iou of the second validation set is 88.7%.

[0079] In step S5, the final data of the dam displacement and settlement monitoring is obtained by differencing the first monitoring result and the second monitoring result.

[0080] The process of obtaining the dam settlement by differencing the first monitoring result and the second monitoring result includes: calculating the first elevation mean matrix using the first monitoring result; calculating the first elevation mean matrix using the second monitoring result; judging the settlement situation of each position of the dam based on the difference between the first elevation mean matrix and the second elevation mean matrix.

[0081] Due to the possible uneven density of point clouds under different weather conditions, there may be no point clouds at the same position, uneven point cloud density, and incomplete alignment of point cloud coordinates. Therefore, direct elevation calculation cannot be performed, and the grid method is used for dam settlement monitoring. In this embodiment, a 5m×5m grid is adopted. For the point clouds after semantic segmentation of the first monitoring result, the minimum and maximum coordinates of the point clouds are calculated according to the category to determine the grid range, and then the grid is divided according to the set resolution; for each grid, the point cloud data falling into the grid is found, and the mean value of the Z coordinate is calculated as the elevation value of the grid. The elevation mean matrices of the two periods of point clouds are calculated respectively to obtain the difference between the two periods of grid elevation data based on different categories, and then the settlement conditions of each position are judged. Assume that the elevation value of a certain position on the dam surface is A, and the elevation value in the later period is B. According to the difference in the projection results of the previous and later periods, settlement judgment is carried out, and the specific rules are as follows: If the later height B is greater than the previous height A and A is 0, a new dam surface is added at this position; if the later height B is greater than the previous height A and A is not 0, the dam surface bulges at this position; if the later height B is less than the previous height A and B is 0, it means that the dam surface at this position is demolished; if the later height B is less than the previous height A and B is 0, it means that the dam surface at this position has settled.

[0082] The process of obtaining the dam displacement by using the difference between the first monitoring result and the second monitoring result includes: calculating the first horizontal displacement of the point cloud change by using the first monitoring result; calculating the second horizontal displacement of the point cloud change by using the second monitoring result; when either the first horizontal displacement or the second horizontal displacement is greater than the preset displacement threshold, it is judged that the dam has displaced in this area.

[0083] The dam area is divided into several sub-areas according to the longitudinal section, and local analysis is carried out at intervals of 100m; for the point clouds in each category (dam surface, dam embankment, dam slope, etc.) in each area, the minimum circumscribed rectangle is calculated. When calculating the geometric center, the weighted average of the point cloud is considered, that is, for each point {x i ,y i ,z i}, its weight is the number of point clouds per square decimeter centered on this point. The specific formula is:

[0084]

[0085] where w i is the weight of point cloud i, C x and C y are the X and Y coordinates of the geometric center respectively; for each category area, the geometric centers C x (A), C y (A) and C x (B), Cy (B), and calculate its horizontal displacement:

[0086] ΔC x = C x (B) - C x (A); ΔC y = C y (B) - C y (A)

[0087] ΔC x is the first horizontal displacement of the point cloud change calculated using the first monitoring result; ΔC y is the second horizontal displacement of the point cloud change calculated using the second monitoring result;

[0088] According to the calculated displacement amounts ΔC x , ΔC y and the set displacement thresholds, determine whether there is a horizontal displacement at this position: If |ΔC x | > displacement threshold 1 or |ΔC y | > displacement threshold 2, then there is a horizontal displacement in this area, otherwise, it means that there is no significant horizontal displacement in this area, where displacement threshold 1 and displacement threshold 2 are adjusted according to service requirements.

[0089] A monitoring method for dam displacement and settlement proposed in Embodiment 1 of the present invention improves the automation level by combining the point cloud semantic segmentation algorithm based on change detection, and can monitor the changes in horizontal and vertical positions of different categories and different positions according to preset targets.

[0090] A monitoring method for dam displacement and settlement proposed in Embodiment 1 of the present invention can efficiently and accurately monitor the displacement and settlement conditions of different areas of the dam through multi-source data fusion and change detection technology, providing a scientific basis for dam safety management.

[0091] Embodiment 2

[0092] Based on the monitoring method for dam displacement and settlement proposed in Embodiment 1 of the present invention, Embodiment 2 of the present invention also proposes a monitoring system for dam displacement and settlement, Figure 5 is a schematic diagram of a monitoring system for dam displacement and settlement proposed in Embodiment 2 of the present invention. The system includes: a data acquisition module, a first registration module, a registration fusion module, and a monitoring module;

[0093] The data acquisition module is used to obtain the monitoring area image of the target dam in the first period, denoted as the source image, and the monitoring area point cloud of the target dam in the first period, denoted as the source point cloud; obtain the monitoring area image of the target dam in the second period, denoted as the target image, and the monitoring area point cloud of the target dam in the second period, denoted as the target point cloud;

[0094] The first registration module is used to transform the target point cloud to the same position as the source point cloud to complete point cloud registration and obtain the registered point cloud;

[0095] The registration and fusion module is used to register and fuse the source image and the source point cloud to obtain the first 3D point cloud data, and to register and fuse the registered point cloud and the target image to obtain the second 3D point cloud data;

[0096] The monitoring module is used to construct the first dataset for point cloud semantic segmentation after annotating the first 3D point cloud data; use the first dataset to train the point cloud semantic segmentation network, and use the trained point cloud semantic segmentation network to extract the first monitoring result of the dam; construct the first dataset for point cloud semantic segmentation after annotating the second 3D point cloud data; use the second dataset to train the point cloud semantic segmentation network, and use the trained point cloud semantic segmentation network to extract the second monitoring result of the dam; use the difference between the first monitoring result and the second monitoring result to obtain the final data of dam displacement and settlement monitoring.

[0097] In the data acquisition module, a UAV is used to carry an RGB camera and a laser scanning device to obtain UAV images and laser point cloud data in the dam monitoring area for two periods.

[0098] In the first registration module, the specific process of transforming the target point cloud to the same position as the source point cloud to complete the registration of the target point cloud and obtain the registered point cloud includes:

[0099] Determine the corresponding homologous point cloud pairs in the source point cloud and the target point cloud;

[0100] Construct the source point cloud dataset {t i}, for each point in the source point cloud dataset {t i}, determine the point {s i} in the target point cloud that is closest to it through a spatial search algorithm; when the error function between the source point cloud dataset {t i} and the closest point {s i} is minimized, determine the optimal first rotation matrix R1 and the first translation vector T1 between the source point cloud and the target point cloud, and use the first rotation matrix R1 and the first translation vector T1 to calculate the registered coordinate {s' i} of the target point cloud;

[0101] Use the registered coordinate {s' i} of the target point cloud as the registered point cloud set to compare with the source point cloud dataset {t i}, and calculate {s i} and {t iThe average distance d between them is used as an error metric. By continuously iteratively updating the first rotation matrix R1 and the first translation vector T1, when the error metric d is less than the threshold or the predetermined maximum number of iterations is reached, the final first rotation matrix and the first translation vector are obtained, and the registration between the target point cloud and the source point cloud is completed to obtain the registered point cloud.

[0102] The calculation method of the registered coordinates {s′} of the target point cloud is: i}

[0103] s′ i = R1s i + T1;

[0104] Where i represents the point cloud serial number.

[0105] In the registration and fusion module, the process of registering and fusing the source image and the source point cloud to obtain the first three-dimensional point cloud data includes: after filtering the source point cloud, extracting the first image feature points from the source image and the first point cloud feature points from the source point cloud respectively;

[0106] Among them, the three-dimensional coordinates of the first point cloud feature point are P1 = [x1, y1, z1] T ; The pixel coordinates of the first image feature point are Q1 = [u1, v1] T ; The second rotation matrix is expressed as Where γ2 is the rotation angle of the first point cloud feature point to the first image feature point along the z-axis; The second translation vector is expressed as: Among them, t 2x is the translation amount of the first point cloud feature point on the x-axis in the image coordinate system; t 2y is the translation amount of the first image feature point on the y-axis in the image coordinate system;

[0107] The registration of the first image feature point and the first point cloud feature point is completed using the formula Q1' = R2P1 + T2; Where Q1' is the pixel coordinates after registration of the first image feature point and the first point cloud feature point;

[0108] Assigning the registered pixel coordinates to the corresponding source point cloud data, the expression of each point cloud is: (X1, Y1, Z1, Intensity1, R1, G1, B1); Where X1, Y1, Z1 represent the Cartesian space x coordinate, y coordinate and z coordinate of each point of the source point cloud; Intensity1 represents the first reflection intensity of the lidar source point cloud; R1, G1, B1 represent the spectral information red, green and blue given to the source point cloud by the image, and the fusion between the source image and the source point cloud is completed to obtain the first three-dimensional point cloud data.

[0109] The process of registering and fusing the registration point cloud and the target image to obtain the second three-dimensional point cloud data includes: after filtering the registration point cloud to remove outliers, extracting the second image feature points from the target image, and extracting the second point cloud feature points from the registered target point cloud;

[0110] The three-dimensional coordinates of the second point cloud feature point are P2 = [x2, y2, z2] T ; The pixel coordinates of the second image feature point are Q2 = [u2, v2] T ; The third rotation matrix is expressed as Where γ3 is the rotation angle from the second point cloud feature point to the second image feature point along the z-axis; the third translation vector is expressed as: Among them, t 3x is the x-axis translation of the second point cloud feature point in the image coordinate system; t 3y is the y-axis translation of the second point cloud feature point in the image coordinate system;

[0111] The formula Q'2=R3P2+T3 is used to complete the registration of the second image feature point and the second point cloud feature point; wherein Q'2 is the pixel coordinate after the second image feature point and the second point cloud feature point are registered;

[0112] The registered pixel coordinates are assigned to the corresponding registered point cloud to obtain the expression of each registered point cloud: (X2, Y2, Z2, Intensity2, R2, G2, B2); X2, Y2, Z2 represent the Cartesian space x-coordinate, y-coordinate and z-coordinate of each point in the registered point cloud; Intensity2 represents the second reflection intensity of the precision light radar registered point cloud; R2, G2, B2 represent the spectral information red, green and blue given to the registered point cloud by the image. After the registration is completed, the fusion between the target point cloud and the target image is obtained to obtain the second three-dimensional point cloud data.

[0113] In the monitoring module: after annotating the first three-dimensional point cloud data, a first data set for point cloud semantic segmentation is constructed; the point cloud semantic segmentation network is trained using the first data set, and the specific process of extracting the first monitoring result of the dam using the trained point cloud semantic segmentation network includes: after annotating the first three-dimensional point cloud data with the upper dam surface, dam embankment, dam slope and horse path, the first data set is formed for point cloud semantic segmentation; the first data set is divided into a first training set and a first verification set according to a preset ratio; the point cloud semantic segmentation network is trained using the first training set to obtain a trained point cloud semantic segmentation network, the trained point cloud semantic segmentation network is used to extract the first monitoring target of the dam, and the accuracy of the first monitoring target is verified using the first verification set.

[0114] After annotating the second three-dimensional point cloud data, a first dataset for point cloud semantic segmentation is constructed; the process of using the second dataset to train a point cloud semantic segmentation network and using the trained point cloud semantic segmentation network to extract the second monitoring results of the dam specifically includes: annotating the second three-dimensional point cloud data with the dam surface, dam embankment, dam slope and horse path to form a second dataset for point cloud semantic segmentation; dividing the second dataset into a first training set and a second validation set according to a preset ratio; using the first training set to train the point cloud semantic segmentation network to obtain a trained point cloud semantic segmentation network, using the trained point cloud semantic segmentation network to extract the second monitoring target of the dam, and using the second validation set to verify the accuracy of the second monitoring target.

[0115] The process of obtaining the dam settlement by differencing the first monitoring result and the second monitoring result includes: calculating a first elevation mean matrix using the first monitoring result; calculating a second elevation mean matrix using the second monitoring result; judging the settlement situation of each position of the dam based on the difference between the first elevation mean matrix and the second elevation mean matrix.

[0116] The process of obtaining the dam displacement by differencing the first monitoring result and the second monitoring result includes: calculating a first horizontal displacement of the point cloud change using the first monitoring result; calculating a second horizontal displacement of the point cloud change using the second monitoring result; when either the first horizontal displacement or the second horizontal displacement is greater than a preset displacement threshold, judging that displacement has occurred in this area of the dam.

[0117] A monitoring system for dam displacement and settlement proposed in Embodiment 2 of the present invention improves the automation level by combining a point cloud semantic segmentation algorithm based on change detection, and can monitor the changes in horizontal and vertical positions of different categories and different positions according to preset targets.

[0118] Embodiment 2 of the present invention proposes a monitoring system for dam displacement and settlement. Through multi-source data fusion and change detection technology, it can efficiently and accurately monitor the displacement and settlement conditions of different areas of the dam, providing a scientific basis for dam safety management.

[0119] For the description of the relevant parts in the monitoring system for dam displacement and settlement provided in Embodiment 2 of this application, reference can be made to the detailed description of the corresponding parts in the monitoring method for dam displacement and settlement provided in Embodiment 1 of this application, which will not be elaborated here.

[0120] It should be noted that in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that the elements inherent in a process, method, article or device comprising a series of elements are included. Without further limitation, an element defined by the statement "comprising an..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the said element. In addition, the parts of the above technical solutions provided by the embodiments of the present application that are consistent with the corresponding technical solutions in the prior art are not described in detail to avoid excessive repetition.

[0121] Although the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, it is not a limitation on the protection scope of the present invention. For those skilled in the art, other different forms of modification or variation can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Various modifications or variations that can be made by those skilled in the art without creative efforts on the basis of the technical solutions of the present invention are still within the protection scope of the present invention.

Claims

1. A monitoring method for dam displacement and settlement, characterized in that, The following steps are involved: The image of the monitoring area of the first phase of the target dam is obtained and recorded as the source image, and the point cloud of the monitoring area of the first phase of the target dam is recorded as the source point cloud; the image of the monitoring area of the second phase of the target dam is obtained and recorded as the target image, and the point cloud of the monitoring area of the second phase of the target dam is recorded as the target point cloud; The target point cloud is converted to the same position as the source point cloud to complete the point cloud registration and obtain the registered point cloud; Registering and fusing the source image with the source point cloud to obtain first three-dimensional point cloud data, and registering and fusing the registered point cloud with the target image to obtain second three-dimensional point cloud data; After annotating the first three-dimensional point cloud data, a first data set for point cloud semantic segmentation is constructed; the point cloud semantic segmentation network is trained using the first data set, and the first monitoring result of the dam is extracted using the trained point cloud semantic segmentation network; After annotating the second three-dimensional point cloud data, construct a first data set for point cloud semantic segmentation; The point cloud semantic segmentation network is trained using the second data set, and the second monitoring result of the dam is extracted using the trained point cloud semantic segmentation network; the final data of the dam displacement and settlement monitoring is obtained by differentially analyzing the first monitoring result and the second monitoring result.

2. The monitoring method for dam displacement and settlement according to claim 1, characterized in that, The process of obtaining the dam settlement by using the difference between the first monitoring result and the second monitoring result includes: calculating the first elevation mean matrix using the first monitoring result; calculating the first elevation mean matrix using the second monitoring result; and judging the settlement of each position of the dam based on the difference between the first elevation mean matrix and the second elevation mean matrix.

3. The monitoring method for dam displacement and settlement according to claim 1, characterized in that, The process of obtaining the dam displacement by using the difference of the first monitoring result and the second monitoring result includes: using the first monitoring result to calculate the first horizontal displacement of the point cloud change; using the second monitoring result to calculate the second horizontal displacement of the point cloud change; when either the first horizontal displacement or the second horizontal displacement is greater than a preset displacement threshold, it is determined that displacement has occurred in the area of the dam.

4. A monitoring method for dam displacement and settlement according to claim 1, characterized in that, The specific process of converting the target point cloud to the same position as the source point cloud to complete the target point cloud registration to obtain the registered point cloud includes: Determine the corresponding point cloud pairs with the same name in the source point cloud and the target point cloud; Construct the source point cloud dataset {t i}, for each point in the source point cloud dataset {t i}, determine the point {s i} in the target point cloud that is closest to it through a spatial search algorithm; when the error function between the source point cloud dataset {t i} and the closest point {s i} is minimized, determine the optimal first rotation matrix R1 and first translation vector T1 between the source point cloud and the target point cloud, and calculate the registered coordinates {s i '} of the target point cloud using the first rotation matrix R1 and the first translation vector T1. Take the registered coordinates {s of the target point cloud i} as the registered point cloud set and the source point cloud data set {t i} for comparison. Calculate the average distance d between {s i} and {t i} as the error metric. By continuously iteratively updating the first rotation matrix R1 and the first translation vector T1, when the error metric d is less than the threshold or reaches the predetermined maximum number of iterations, obtain the final first rotation matrix and the first translation vector, and complete the registration of the target point cloud to the source point cloud to obtain the registered point cloud.

5. A monitoring method for dam displacement and settlement according to claim 4, characterized in that The calculation method of the target point cloud registration coordinates {s i '} is as follows: s i ' = R1s i + T1; Among them, i represents the point cloud number.

6. The monitoring method for dam displacement and settlement according to claim 1, characterized in that, The process of registering and fusing the source image and the source point cloud to obtain the first three-dimensional point cloud data includes: After filtering the source point cloud, extracting the first image feature points from the source image and extracting the first point cloud feature points from the source point cloud respectively; Among them, the three-dimensional coordinates of the first point cloud feature point are P1 = [x1, y1, z1] T ; the pixel coordinates of the first image feature point are Q1 = [u1, v1] T ; the second rotation matrix is expressed as where γ2 is the rotation angle of the first point cloud feature point to the first image feature point along the z-axis; the second translation vector is expressed as: where, t 2x is the x-axis translation amount of the first point cloud feature point in the image coordinate system; t 2y is the y-axis translation amount of the first image feature point in the image coordinate system; The formula Q1'=R2P1+T2 is used to complete the registration of the first image feature point and the first point cloud feature point; wherein Q1' is the pixel coordinate after the first image feature point and the first point cloud feature point are registered; The registered pixel coordinates are assigned to the corresponding source point cloud data to obtain the expression of each point cloud: (X1, Y1, Z1, Intensity1, R1, G1, B1); X1, Y1, Z1 represent the Cartesian space x-coordinate, y-coordinate and z-coordinate of each point in the source point cloud; Intensity1 represents the first reflection intensity of the precision radar source point cloud; R1, G1, B1 represent the spectral information red, green and blue given to the source point cloud by the image, and the fusion between the source image and the source point cloud is completed to obtain the first three-dimensional point cloud data.

7. A monitoring method for dam displacement and settlement according to claim 1, characterized in that, The process of registering and fusing the registered point cloud and the target image to obtain the second three-dimensional point cloud data includes: After filtering the registered point cloud to remove outliers, second image feature points are extracted from the target image, and second point cloud feature points are extracted from the registered target point cloud; The three-dimensional coordinates of the second point cloud feature point are P2 = [x2, y2, z2] T ; The pixel coordinates of the second image feature point are Q2 = [u2, v2] T ; The third rotation matrix is expressed as where γ3 is the rotation angle of the second point cloud feature point to the second image feature point along the z-axis; The third translation vector is expressed as: where, t 3x is the x-axis translation amount of the second point cloud feature point in the image coordinate system; t 3y is the y-axis translation amount of the second point cloud feature point in the image coordinate system; The registration of the second image feature points and the second point cloud feature points is completed using the formula Q'2 = R3P2 + T3; where Q'2 is the pixel coordinates after the registration of the second image feature points and the second point cloud feature points; Assign the registered pixel coordinates to the corresponding registered point cloud, and the expression of each registered point cloud is: (X2, Y2, Z2, Intensity2, R2, G2, B2); where X2, Y2, Z2 represent the Cartesian space x coordinate, y coordinate, and z coordinate of each point of the registered point cloud; Intensity2 represents the second reflection intensity of the lidar registered point cloud; R2, G2, B2 represent the spectral information red, green, and blue given by the image to the registered point cloud, and the fusion between the registered target point cloud and the target image is completed to obtain the second three-dimensional point cloud data.

8. A monitoring method for dam displacement and settlement according to claim 1, characterized in that After annotating the first three-dimensional point cloud data, a first dataset for point cloud semantic segmentation is constructed; The process of using the first dataset to train the point cloud semantic segmentation network and using the trained point cloud semantic segmentation network to extract the first monitoring result of the dam includes: Annotate the first three-dimensional point cloud data with the dam surface, dam embankment, dam slope, and horse path to form a first dataset for point cloud semantic segmentation; divide the first dataset into a first training set and a first validation set according to a preset ratio; Use the first training set to train the point cloud semantic segmentation network to obtain a trained point cloud semantic segmentation network, use the trained point cloud semantic segmentation network to extract the first monitoring target of the dam, and use the first validation set to verify the accuracy of the first monitoring target.

9. A method for monitoring the displacement and settlement of a dam according to claim 1, characterized in that, After annotating the second three-dimensional point cloud data, a first dataset for point cloud semantic segmentation is constructed; The process of using the second dataset to train the point cloud semantic segmentation network and using the trained point cloud semantic segmentation network to extract the second monitoring result of the dam includes: Annotate the second three-dimensional point cloud data with the dam surface, dam embankment, dam slope, and horse path to form a second dataset for point cloud semantic segmentation; divide the second dataset into a second training set and a second validation set according to a preset ratio; Use the second training set to train the point cloud semantic segmentation network to obtain a trained point cloud semantic segmentation network, use the trained point cloud semantic segmentation network to extract the second monitoring target of the dam, and use the second validation set to verify the accuracy of the second monitoring target.

10. A monitoring system for dam displacement and settlement, characterized in that, It includes a data acquisition module, a first registration module, a registration fusion module, and a monitoring module; The data acquisition module is used to obtain the monitoring area image of the first phase of the target dam, denoted as the source image, and the monitoring area point cloud of the first phase of the target dam, denoted as the source point cloud; obtain the monitoring area image of the second phase of the target dam, denoted as the target image, and the monitoring area point cloud of the second phase of the target dam, denoted as the target point cloud; The first registration module is used to transform the target point cloud to the same position as the source point cloud to complete point cloud registration and obtain the registered point cloud; The registration and fusion module is used to register and fuse the source image and the source point cloud to obtain the first three-dimensional point cloud data, and register and fuse the registered point cloud and the target image to obtain the second three-dimensional point cloud data; The monitoring module is used to construct a first dataset for point cloud semantic segmentation after annotating the first three-dimensional point cloud data; use the first dataset to train the point cloud semantic segmentation network, and use the trained point cloud semantic segmentation network to extract the first monitoring result of the dam; Construct a first dataset for point cloud semantic segmentation after annotating the second three-dimensional point cloud data; Use the second dataset to train the point cloud semantic segmentation network, and use the trained point cloud semantic segmentation network to extract the second monitoring result of the dam; obtain the final data of the dam displacement and settlement monitoring by differentiating the first monitoring result and the second monitoring result.

Images