High-speed construction monitoring method and system based on unmanned aerial vehicle laser and vision fusion

Abstract

The application discloses a high-speed construction monitoring method and system based on unmanned aerial vehicle laser and vision fusion, which comprises the following steps: step 1, collecting images of different time phases by using an unmanned aerial vehicle; step 2, matching image point clouds by using a multi-measure semi-global matching algorithm on the unmanned aerial vehicle images; step 3, collecting LiDAR point clouds of different time phases by using the unmanned aerial vehicle; step 4, removing noises from the LiDAR point clouds; step 5, registering two kinds of point cloud data by using an iterative closest point algorithm; step 6, establishing a digital surface model by using a local partition box algorithm; and step 7, calculating the difference between digital surface models of different time phases and performing change detection. The application has the advantages that unmanned aerial vehicle laser and vision can be fused and high-speed construction monitoring can be completed automatically at a relatively low cost.

Description

Technical Field

[0001] The present invention relates to the field of engineering monitoring, and more specifically to a high-speed construction monitoring method based on unmanned aerial vehicle laser and vision fusion. Background Art

[0002] The construction of expressways has significantly boosted the rapid development of the transportation and logistics industries, regional economies, and other sectors. It can effectively reduce operating costs, shorten transportation times, and accelerate industrial development, significantly impacting production, life, investment, and daily transportation. Consequently, higher standards are being placed on the quality and reliability of expressway construction, making accurate and effective monitoring of the construction process crucial.

[0003] Traditional high-speed construction monitoring methods primarily rely on measurement equipment such as levels and total stations. These methods require significant manpower and resources, are inefficient, and pose certain risks. The rapid development of remote sensing technology has significantly improved the efficiency of construction monitoring. Remote sensing imagery, with its wide coverage, rich semantic information, and high geometric accuracy, is currently the most commonly used data source for change detection. However, due to issues such as complex internal structures, mixed pixels, and low image resolution, the accuracy of change detection based on remote sensing imagery is difficult to guarantee. Furthermore, remote sensing imagery primarily captures two-dimensional information about objects, making it difficult to effectively detect changes in their three-dimensional attributes. Light Detection and Ranging (LiDAR) technology can rapidly acquire three-dimensional spatial information about objects, offering advantages such as high precision, high efficiency, and high density. It offers unparalleled advantages in representing the three-dimensional topological relationships of complex structures. However, the difficulty, high cost, and processing of LiDAR data have limited its application in change detection research. To address the shortcomings of the above methods, drones can be used to capture aerial photographs and generate 3D point cloud data through dense matching of multiple views to represent the 3D structure of the ground objects. However, the 3D point cloud data obtained through image vision is less accurate than the point cloud data collected by LiDAR and is easily affected by the dense matching algorithm.

[0004] Therefore, it is necessary to integrate UAV laser and image data, fully utilize the advantages of both, and quickly and efficiently obtain high-precision change detection results for high-speed construction monitoring in a relatively low-cost manner. Summary of the Invention

[0005] The purpose of the present invention is to provide a high-speed construction monitoring method based on the fusion of drone laser and vision, which can automatically fuse drone laser and vision at a relatively low cost and complete high-speed construction monitoring.

[0006] In order to achieve the above object, the technical scheme of the present application is as follows: a high-speed construction monitoring method based on unmanned aerial vehicle laser and vision fusion, characterized by comprising the following steps,

[0007] Step 1: collecting unmanned aerial vehicle image data at different time phases;

[0008] Operating the unmanned aerial vehicle, multiple times of conventional shooting in the high-speed construction section to obtain unmanned aerial vehicle images of the high-speed construction section at different time phases;

[0009] Step 2: using the unmanned aerial vehicle image data at different time phases obtained in step 1, using a multi-measure semi-global matching algorithm for matching, and further processing to obtain image point clouds;

[0010] Step 3: collecting LiDAR point cloud data, simultaneously performing step 1, and collecting LiDAR point cloud data at different time phases in the high-speed construction section;

[0011] Step 4: LiDAR point cloud denoising, denoising the point cloud data obtained in step 3;

[0012] Step 5: point cloud data registration, using an iterative closest point algorithm to register the point cloud data obtained in steps 2 and 4;

[0013] Step 6: establishing a digital surface model, according to the registered point cloud data obtained in step 5, using a local partitioning algorithm to establish a digital surface model;

[0014] Step 7: construction change monitoring, repeating steps 1-6 at different times to obtain digital surface models at different time phases, calculating the difference between the digital surface models at different time phases, comparing with a set threshold, and obtaining a change detection result.

[0015] Preferably, in step 2, the specific process of the multi-measure semi-global matching algorithm is as follows:

[0016] (1) using an automatic color scale method to enhance the original image, then generating a core line image and establishing an image pyramid, and defining the image levels from bottom to top as r0, 1, … r n .

[0017] (2) in the highest level r n of the image pyramid, using a Census similarity measure, traversing each pixel p, and calculating the matching cost C(p, d) when the disparity d is in the disparity search range d range .

[0018] (3) calculating the path cost of pixel p in different directions in turn, and calculating the cumulative cost S(p, d), and selecting the disparity corresponding to the minimum cumulative cost as the disparity result of the pixel.

[0019] (4) The cumulative cost at the parallax d, d-1 and d+1 is fitted with a quadratic parabola to calculate the optimal sub-pixel level parallax to improve the matching accuracy.

[0020] (5) The parallax map is segmented according to the principle that the change value of the parallax of adjacent pixels is less than a certain threshold, and if the segmented area is less than a certain threshold, it is considered that there is an error and it is removed.

[0021] (6) Dynamically adjust the parallax search range d of the pixels in the next level image range , starting from the r n-1 th level, using mutual information as the similarity measure, and the initial value of the parallax is the parallax result of the previous level, repeating the processes (2) to (5), until matching to the bottom layer pyramid image ends.

[0022] (7) After obtaining the parallax map by the multi-measure semi-global matching algorithm, each image is processed in turn, and the parallax map is fused to obtain discrete three-dimensional point cloud data.

[0023] Preferably, in step 5, the specific process of the iterative closest point algorithm is as follows:

[0024] (1) For each point in the point cloud to be registered P = {p0, p1, … p m}, find the point in the reference point cloud Q = {q0, q1, … q n} that is closest to it in Euclidean distance to form a point pair set C k = {(p i k , q i k ) | i = 1, 2, …, N}, N is the number of point pairs, and k is the iteration number.

[0025] (2) Take equation (1) as the objective function, and calculate the new rotation matrix R k and translation matrix T k to make the error value minimum.

[0026]

[0027] (3) Use the calculated transformation matrix parameters R k and T k to transform the point cloud to be registered to obtain a new point cloud to be registered p i k+1 as follows:

[0028] p i k+1 = R k p i k+T k (2)

[0029] (4)Calculate the average distance d between the new point cloud to be registered and the corresponding point pairs of the reference point cloud k+1 As follows:

[0030]

[0031] (5) If the number of iterations k reaches the specified maximum number of iterations, d k+1 -dk<ε or d k <ξ, stop iteration, otherwise repeat the above process, ε is the difference threshold value of the average distance of the point pairs of the adjacent two iterations, ξ is the average distance threshold value of the point pairs of a single iteration.

[0032] Preferably, in step 6, the process of the local box algorithm is:

[0033] Project the registered point cloud onto the XY plane to establish a virtual grid, each grid is a square, and the side length is Res. The number of rows m and the number of columns n of the grid can be calculated as:

[0034]

[0035] In the formula: X max , X min , Y max , Y min are the maximum and minimum values of the X and Y coordinates of the point cloud, and ceil(*) represents the upward rounding operation. Thus, the coordinates (X i , Y j ) of each grid corner point can be obtained as follows:

[0036]

[0037] In the formula: i=0,1,2,…n, j=0,1,2,…m represent the column number and row number of each grid corner point, respectively.

[0038] Let P={p k , k=1,2,…,τ} be the point cloud set, τ be the total number of point clouds, and for each grid corner point V i,j , define a circular neighborhood with a radius r, then the point cloud set within the neighborhood can be represented as:

[0039] R(V i,j , r)={p k |d(V i,j , p k )≤r, p k ∈P} (6)

[0040] In the formula: is the grid corner point V i,j With point p k (x k ,y k , z k ) on the XY plane. Let the point cloud set R(V i,j ,r) is the elevation set of each point Then the value Z of the grid corner point i,j It can be calculated as follows:

[0041]

[0042] Where: NaN represents a null value, Represents an empty set. When there is no point cloud within the r neighborhood of a grid corner point, the value of that grid corner point is null. By traversing all grid corner points, the digital surface model of the target point cloud can be calculated.

[0043] Preferably, in step 7, the deformation monitoring process is: using the local binning algorithm to calculate the digital surface models of the point cloud data of two different phases, comparing the elevation differences of the corresponding grids in the digital surface models, and setting the elevation difference threshold D th , if the elevation difference is greater than D th , then the grid is an elevation increase area. If the elevation difference is less than - th , then the grid is an elevation decrease area. By traversing and calculating the elevation difference of all grids in the digital surface model and comparing it with the elevation difference threshold, all change areas can be identified.

[0044] The present invention provides a high-speed construction monitoring system based on UAV laser and vision fusion, which includes the following modules:

[0045] Image data acquisition module, used for collecting drone image data at different phases;

[0046] Operate the drone to conduct regular photography on the highway construction section multiple times to obtain drone images of the highway construction section at different times;

[0047] The matching module is used to use the obtained drone image data of different phases to match using a multi-measure semi-global matching algorithm, and further process it to obtain an image point cloud;

[0048] Point cloud data acquisition module, used for LiDAR point cloud data acquisition, collects LiDAR point cloud data simultaneously with image data acquisition, and obtains LiDAR point cloud data of high-speed construction sections at different phases;

[0049] Denoising module, used for LiDAR point cloud denoising, denoising the point cloud data obtained by the point cloud data acquisition module;

[0050] A registration module is configured to register the point cloud data obtained by the matching module and the denoising module by using an iterative closest point algorithm.

[0051] A digital surface model establishment module is configured to establish a digital surface model by using a local box algorithm based on the registered point cloud data obtained by the registration module.

[0052] A change monitoring module is configured to monitor construction changes by repeating all the above modules at different times to obtain digital surface models at different time phases, calculating the difference between the digital surface models at different time phases, comparing the difference with a set threshold, and obtaining a change detection result.

[0053] Preferably, in the matching module, the specific process of the multi-measure semi-global matching algorithm is as follows:

[0054] (21) The original image is enhanced by using an automatic color scale method, a core line image is generated, and an image pyramid is established, and the image levels are defined as r0, 1, … r n from bottom to top.

[0055] (22) In the highest level r n of the image pyramid, a Census similarity measure is used to traverse each pixel p, and the matching cost C(p, d) of the pixel p when the disparity d is in the disparity search range d range is calculated.

[0056] (23) The path cost of the pixel p in different directions is calculated in sequence, and the cumulative cost S(p, d) is calculated, and the disparity corresponding to the minimum cumulative cost is selected as the disparity result of the pixel.

[0057] (24) The cumulative costs at the disparities d, d-1 and d+1 are fitted by using a quadratic parabola, and the optimal sub-pixel level disparity is calculated to improve the matching accuracy.

[0058] (25) The disparity map is segmented according to the principle that the change value of the disparity of adjacent pixels is less than a certain threshold, and if the segmented area is less than a certain threshold, it is considered that there is an error and the error is removed.

[0059] (26) The disparity search range d range of the pixels in the next level image is dynamically adjusted, and the process starts from the r n-1 th level, the mutual information is used as the similarity measure, the initial value of the disparity is the disparity result of the previous level, and the processes (22) to (25) are repeated until the matching to the bottom pyramid image is completed.

[0060] (27) After the disparity map is obtained by the multi-measure semi-global matching algorithm, each image is processed in sequence, the disparity map is fused, and then discrete three-dimensional point cloud data is obtained.

[0061] Preferably, in the registration module, the specific process of the iterative closest point algorithm is as follows:

[0062] (51) For each point in the point cloud to be registered P = {p0,1, … p m}, find the point in the reference point cloud Q = {q0,1, … q n} with the closest Euclidean distance to it to form a point pair set C k = {(p i k , i k )| = 1, 2, …, N}, N is the number of point pairs, and k is the iteration number;

[0063] (52) Take the function in equation (51) as the objective function, and minimize the error value to obtain a new rotation matrix R k and a translation matrix T k ;

[0064]

[0065] (53) Use the calculated transformation matrix parameters R k and T k to transform the point cloud to be registered to obtain a new point cloud to be registered p i k+1 as follows:

[0066] p i k+1 = R k p i k + T k (2)

[0067] (54) Calculate the average distance d k+1 between the new point cloud to be registered and the corresponding point pairs in the reference point cloud as follows:

[0068]

[0069] (55) If the iteration number k reaches the specified maximum iteration number, d k+1 -d k <ε or d k <ξ, stop iteration, otherwise repeat the above process, ε is the difference threshold value of the average distance of point pairs in adjacent two iterations, and ξ is the average distance threshold value of point pairs in a single iteration.

[0070] Preferably, in the digital surface model establishment module, the process of the local box algorithm is as follows:

[0071] The registered point cloud is projected onto the XY plane to establish a virtual grid. Each grid is a square with a side length of Res. The number of rows m and columns n of the grid can be calculated as follows:

[0072]

[0073] Where: X max 、X min 、Y max 、Y min They are the maximum and minimum values ​​of the X and Y coordinates of the point cloud, respectively. Ceil(*) means rounding up. The coordinates of each grid corner point (X i ,Y j )as follows:

[0074]

[0075] Where: i = 0, 1, 2, ... n, j = 0, 1, 2, ... m represents the column number and row number of each grid corner point respectively;

[0076] Let P = {p k ,k=1,2,…,τ} is the point cloud set, τ is the total number of point clouds, for each grid corner point V i,j , define a circular neighborhood with a radius of r, then the point cloud set within the neighborhood can be expressed as:

[0077] R(V i,j ,r)={p k |d(V i,j ,p k )≤r,p k ∈P} (6)

[0078] Where: is the grid corner point V i,j With point p k (x k ,y k ,z k ) on the XY plane; let the point cloud set R(V i,j ,r) is the elevation set of each point Then the value Z of the grid corner point i,j It can be calculated as follows:

[0079]

[0080] Where: NaN represents a null value, Represents an empty set. When there is no point cloud within the r neighborhood of a grid corner point, the value of the grid corner point is null. By traversing all grid corner points, the digital surface model of the target point cloud can be calculated.

[0081] Preferably, in the change monitoring module, the process of deformation monitoring is:

[0082] The digital surface model of the point cloud data of two different time points is calculated by using the local binning algorithm, the elevation difference of the corresponding grid in the digital surface model is compared, and the elevation difference threshold D is set th If the elevation difference is greater than D th , the grid is an elevation increasing area, and if the elevation difference is less than- th , the grid is an elevation decreasing area. By traversing and calculating the elevation difference of all grids in the digital surface model and comparing it with the elevation difference threshold, all change areas can be identified.

[0083] Compared with the prior art, the present application has the following advantages:

[0084] 1) The present application has low cost, does not need to consume a lot of manpower and material resources, has high work efficiency, and has no danger.

[0085] 2) The data acquisition difficulty of the present application is low, and the processing is simple.

[0086] 3) The present application introduces LiDAR data for registration, which improves the overall accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0087] Figure 1 is the flowchart of the present application.

[0088] Figure 2 is a schematic diagram of establishing a digital surface model by using the local binning algorithm of the present application.

[0089] Figure 3 is an example diagram of collecting unmanned aerial vehicle images of the present application.

[0090] Figure 4 is an example diagram of point cloud generated by using the collected unmanned aerial vehicle image dense matching of the present application.

[0091] Figure 5 is an example diagram of collecting LiDAR point cloud of the present application.

[0092] Figure 6 is an example diagram of digital surface model generated by using the present application on a certain expressway construction section in a certain province on August 26, 2022.

[0093] Figure 7 is an example diagram of digital surface model generated by using the present application on a certain expressway construction section in a certain province on October 8, 2022.

[0094] Figure 8 is a difference value diagram of the digital surface model of a certain expressway construction section in a certain province.

[0095] Figure 9 This is a graph showing the results of change detection on a highway construction section in a certain city in a certain province. DETAILED DESCRIPTION

[0096] The following detailed description of the embodiments of the present invention is given in conjunction with the accompanying drawings, which do not limit the present invention but are merely examples. The description makes the advantages of the present invention clearer and easier to understand.

[0097] The present invention is now described in detail with its application to a certain highway construction section in a certain city in a certain province as a research area, which also has a guiding role in the application of the present invention to the monitoring of other highway construction sections.

[0098] In this embodiment, a method for monitoring a highway construction section in a certain city in a certain province includes the following steps:

[0099] Step 1: Collect drone image data at different times;

[0100] Operate the drone to take regular photos of the highway construction section multiple times to obtain drone images of the highway construction section at different times. For example, Figure 3 As shown;

[0101] Step 2: Using the drone image data of different time phases obtained in step 1, we use the multi-measure semi-global matching algorithm to match and further process it to obtain the image point cloud. The result is as follows: Figure 4 As shown;

[0102] In step 2, the specific process of the multi-measure semi-global matching algorithm is:

[0103] (1) The original image is enhanced using the automatic color scale method, and then the epipolar image is generated and an image pyramid is established. The image levels are defined as r0, 1, ... r from bottom to top. n .

[0104] (2) At the highest level of the image pyramid r n Using the Census similarity measure, traverse each pixel p and calculate its disparity search range d range The matching cost C(p,d) when the internal disparity is d.

[0105] (3) Calculate the path cost of pixel p in different directions in turn, and calculate the cumulative cost S(p,d). Select the disparity corresponding to the minimum cumulative cost as the disparity result of the pixel.

[0106] (4) A quadratic parabola is used to fit the cumulative costs at disparities d, d-1, and d+1 to calculate the optimal sub-pixel level disparity to improve matching accuracy.

[0107] (5) According to the principle that the change value of the parallax of adjacent pixels is less than a certain threshold, the parallax map is segmented, and if the segmented area is less than a certain threshold, it is considered that there is an error and it is removed.

[0108] (6) Dynamically adjusting the parallax search range d of the pixels in the next level image range , starting from the r n-1 th level, using mutual information as the similarity measure, and the initial parallax value is the parallax result of the previous level, repeating processes (2) to (5), until matching to the bottom layer pyramid image.

[0109] (7) After obtaining the parallax map by the multi-measure semi-global matching algorithm, each image is processed in turn, and the parallax map is fused to obtain discrete three-dimensional point cloud data.

[0110] Step 3: LiDAR point cloud data acquisition, while performing step 1, collect LiDAR point cloud data to obtain LiDAR point cloud data of high-speed construction road sections at different time phases, as shown in the example Figure 5 .

[0111] Step 4: LiDAR point cloud denoising, denoising the point cloud data obtained in step 3;

[0112] Importing the open-pit track information obtained in step 2 into the flight control system of the unmanned aerial vehicle, taking off the unmanned aerial vehicle to automatically take pictures, and obtaining high-resolution open-pit target images;

[0113] Step 4: Fine point cloud result acquisition;

[0114] Step 5: Point cloud data registration, using the iterative closest point algorithm to register the point cloud data obtained in steps 2 and 4;

[0115] In step 5, the specific process of the iterative closest point algorithm is as follows:

[0116] (1) For each point in the point cloud to be registered P = {p0,1,…p m} in the reference point cloud Q = {q0,1,…q n}, find the point with the closest Euclidean distance to form a point pair set C k = {(p i k , i k )| = 1,2,…,N}, N is the number of point pairs, and k is the iteration number.

[0117] (2) Taking equation (1) as the objective function, let the error value be minimized, and calculate the new rotation matrix R k and translation matrix T k .

[0118]

[0119] (3) Using the calculated transformation matrix parameter R k and T k Transform the to-be-registered point cloud to obtain a new to-be-registered point cloud p i k+1 As follows:

[0120] p i k+1 = R k p i k + T k (2)

[0121] (4) Calculate the average distance d between the new to-be-registered point cloud and the corresponding point pair of the reference point cloud k+1 As follows:

[0122]

[0123] (5) If the iteration number k reaches the specified maximum iteration number, d k+1 -k< or d k <, stop iteration, otherwise repeat the above process, ε is the difference threshold value of the average distance of the point pair of the adjacent two iterations, and ξ is the average distance threshold value of the point pair of a single iteration.

[0124] Step 6: Establishment of digital surface model, according to the registered point cloud data obtained in step 5, a local part box algorithm is used to establish a digital surface model, and the result is as shown in Figure 6 Figure 7 ;

[0125] In step 6, the process of the local part box algorithm is:

[0126] Project the registered point cloud onto the XY plane to establish a virtual grid, each grid is a square, and the side length is Res. The number of rows m and the number of columns n of the grid can be calculated as follows:

[0127]

[0128] In the formula: X max , X min , Y max , Y min are the maximum and minimum values of the X and Y coordinates of the point cloud, and ceil(*) represents the upward rounding operation. The coordinates (X i , Y j ) of each grid corner point can be obtained as follows:

[0129]

[0130] where i = 0, 1, 2, … n, j = 0, 1, 2, … m represent the column number and row number of each grid corner point respectively.

[0131] Let P = {p k , k = 1, 2, …, τ} be the point cloud set, τ be the total number of point clouds, for each grid corner point V i,j , define a circular neighborhood with radius r, then the point cloud set within the neighborhood can be expressed as:

[0132] R(V i,j , r) = {p k | d(V i,j , p k ) ≤ r, p k ∈ P} (6)

[0133] where: is the geometric distance between the grid corner point V i,j and the point p k (x k , y k , z k ) in the XY plane. Let the elevation set of each point in the point cloud set R(V i,j , r) be then the value Z i,j of the grid corner point can be calculated as follows:

[0134]

[0135] where NaN represents null value, and empty set. When there is no point cloud within the r-neighborhood of the grid corner point, the value of the grid corner point is null value. By traversing all grid corner points, the digital surface model of the target point cloud can be calculated.

[0136] Step 7: Construction change monitoring, repeat steps 1-6 at different times to obtain digital surface models at different time phases, calculate the difference between the digital surface models at different time phases, compare with the set threshold value, obtain the change detection result, the result is shown in Figure 8 Figure 9 .

[0137] In step 7, the process of deformation monitoring is: the local box algorithm is used to calculate the digital surface models of the point cloud data at two different time phases respectively, the elevation difference of the corresponding grids in the digital surface models is compared, the elevation difference threshold D th is set, if the elevation difference is greater than D th , the grid is an area with increased elevation, if the elevation difference is less than D th, then the grid is an elevation decrease area. By traversing and calculating the elevation difference of all grids in the digital surface model and comparing it with the elevation difference threshold, all change areas can be identified.

[0138] Conclusion: This embodiment can automatically fuse UAV laser and vision at a relatively low cost and complete high-speed construction monitoring.

[0139] The present invention also provides a high-speed construction monitoring system based on UAV laser and vision fusion, which includes the following modules:

[0140] Image data acquisition module, used for collecting drone image data at different phases;

[0141] Operate the drone to conduct regular photography on the highway construction section multiple times to obtain drone images of the highway construction section at different times;

[0142] The matching module is used to use the obtained drone image data of different phases to match using a multi-measure semi-global matching algorithm, and further process it to obtain an image point cloud;

[0143] Point cloud data acquisition module, used for LiDAR point cloud data acquisition, collects LiDAR point cloud data simultaneously with image data acquisition, and obtains LiDAR point cloud data of high-speed construction sections at different phases;

[0144] Denoising module, used for LiDAR point cloud denoising, denoising the point cloud data obtained by the point cloud data acquisition module;

[0145] The registration module is used for point cloud data registration, and uses the iterative closest point algorithm to register the point cloud data obtained by the matching module and the denoising module;

[0146] The digital surface model building module is used to build the digital surface model. The digital surface model is built using the local binning algorithm based on the registered point cloud data obtained by the registration module.

[0147] The change monitoring module is used for construction change monitoring. All the above modules are repeated at different times to obtain digital surface models of different phases. The difference between the DSMs of different phases is calculated and compared with the set threshold to obtain the change detection results.

[0148] The specific implementation of each module corresponds to each step and is not described in detail in this invention.

[0149] The specific embodiments described herein are merely illustrative of the spirit of the present invention. Persons skilled in the art may make various modifications, additions, or substitutions to the described specific embodiments without departing from the spirit of the present invention or exceeding the scope of the appended claims.

Claims

1. A high-speed construction monitoring method based on UAV laser and vision fusion, characterized by: The following steps are included: Step 1: UAV image data collection at different time phases; Operate the drone to conduct regular photography on the highway construction section multiple times to obtain drone images of the highway construction section at different times; Step 2: Using the drone image data of different time phases obtained in step 1, a multi-measure semi-global matching algorithm is used for matching, and further processing is performed to obtain an image point cloud; Step 3: LiDAR point cloud data collection: while executing step 1, collect LiDAR point cloud data to obtain LiDAR point cloud data of the highway construction section at different time phases; Step 4: LiDAR point cloud denoising, denoising the point cloud data obtained in step 3; Step 5: Point cloud data registration: the point cloud data obtained in step 2 and step 4 are registered using the iterative closest point algorithm; Step 6: Building a digital surface model. Based on the registered point cloud data obtained in step 5, a local binning algorithm is used to build a digital surface model. In step 6, the process of the local binning algorithm is: The registered point cloud is projected onto the XY plane to establish a virtual grid. Each grid is a square with a side length of Res. The number of rows m and columns n of the grid can be calculated as follows: Where: X max 、X min 、Y max 、Y min They are the maximum and minimum values ​​of the X and Y coordinates of the point cloud, respectively. Ceil(*) represents the rounding operation. From this, the coordinates of each grid corner point (X i ,Y j )as follows: Where: i = 0, 1, 2, ... n, j = 0, 1, 2, ... m represents the column number and row number of each grid corner point respectively; Let P = {p k ,k=1,2,…,τ} is the point cloud set, τ is the total number of point clouds, for each grid corner point V i,j , define a circular neighborhood with a radius of r, then the point cloud set within the neighborhood can be expressed as: R(V i,j ,r)={p k |d(V i,j ,p k )≤r,p k ∈P} (6) Where: is the grid corner point V i,j With point p k (x k ,y k ,z k ) on the XY plane; let the point cloud set R(V i,j ,r) is the elevation set of each point Then the value Z of the grid corner point i,j It can be calculated as follows: Where: NaN represents a null value, Represents an empty set. When there is no point cloud in the r neighborhood of a grid corner point, the value of the grid corner point is null. By traversing all grid corner points, the digital surface model of the target point cloud can be calculated. Step 7, construction change monitoring, repeat steps 1 to 6 at different times to obtain digital surface models of different phases, calculate the difference between DSMs between different phases, compare it with the set threshold, and obtain the change detection result.

2. The high-speed construction monitoring method based on UAV laser and vision fusion according to claim 1 is characterized by: In step 2, the specific process of the multi-measure semi-global matching algorithm is: (21) The original image is enhanced using the automatic color scale method, and then the epipolar image is generated and an image pyramid is established. The image levels are defined as r0, r1, ... r from bottom to top. n ; (22) At the highest level of the image pyramid, r n Using the Census similarity measure, traverse each pixel p and calculate its disparity search range d range The matching cost C(p,d) when the internal disparity is d; (23) Calculate the path cost of pixel p in different directions in turn, and calculate the cumulative cost S(p,d). Select the disparity corresponding to the minimum cumulative cost as the disparity result of the pixel; (24) The cumulative cost at disparity d, d-1, and d+1 is fitted with a quadratic parabola to calculate the optimal sub-pixel level disparity to improve matching accuracy; (25) Segment the disparity map based on the principle that the change in disparity between adjacent pixels is less than a certain threshold. If the segmented area is less than a certain threshold, it is considered that there is an error and the image is removed. (26) Dynamically adjust the disparity search range d of pixels in the next level image range , from the r n-1 At the beginning of each level, mutual information is used as the similarity measure, and the initial disparity value is the disparity result of the previous level. The process (22) to (25) is repeated until the matching is completed when the bottom pyramid image is matched. (27) After obtaining the disparity map through the multi-measure semi-global matching algorithm, each image is processed in turn, and its disparity map is fused to obtain discrete three-dimensional point cloud data.

3. The high-speed construction monitoring method based on UAV laser and vision fusion according to claim 1 is characterized by: In step 5, the specific process of the iterative closest point algorithm is: (51) For the point cloud to be registered P = {p0, p1, ... p m }, in the reference point cloud Q = {q0,q1,…q n } to find the point with the closest Euclidean distance to form a point pair set C k ={(p i k ,q i k )|i=1,2,…,N}, N is the number of point pairs, k is the number of iterations; (52) Using formula (51) as the objective function, minimize its error value and calculate the new rotation matrix R k and the translation matrix T k ; (53) Using the calculated transformation matrix parameter R k and T k Transform the point cloud to be registered to obtain a new point cloud to be registered p i k+1 as follows: p i k+1 =R k p i k +T k (2) (54) Calculate the average distance d between the corresponding point pairs of the new point cloud to be registered and the reference point cloud k+1 as follows: (55) If the number of iterations k reaches the specified maximum number of iterations, d k+1 -d k <ε or d k <ξ, the iteration is stopped, otherwise the above process is repeated, ε is the difference threshold of the average value of the point-pair distances between two adjacent iterations, and ξ is the average value threshold of the point-pair distances in a single iteration.

4. The high-speed construction monitoring method based on UAV laser and vision fusion according to claim 1 is characterized by: In step 7, the deformation monitoring process is: The local binning algorithm is used to calculate the digital surface models of two point cloud data at different time phases, and the elevation difference of the corresponding grids in the digital surface model is compared to set the elevation difference threshold D. th , if the elevation difference is greater than D th , then the grid is an elevation increase area. If the elevation difference is less than -D th , then the grid is an elevation reduction area; by traversing and calculating the elevation differences of all grids in the digital surface model and comparing them with the elevation difference threshold, all change areas can be identified.

5. A high-speed construction monitoring system based on UAV laser and vision fusion, characterized by: Includes the following modules: Image data acquisition module, used for collecting drone image data at different phases; Operate the drone to conduct regular photography on the highway construction section multiple times to obtain drone images of the highway construction section at different times; The matching module is used to use the obtained drone image data of different phases to match using a multi-measure semi-global matching algorithm, and further process it to obtain an image point cloud; Point cloud data acquisition module, used for LiDAR point cloud data acquisition, collects LiDAR point cloud data simultaneously with image data acquisition, and obtains LiDAR point cloud data of high-speed construction sections at different phases; Denoising module, used for LiDAR point cloud denoising, denoising the point cloud data obtained by the point cloud data acquisition module; The registration module is used for point cloud data registration, and uses the iterative closest point algorithm to register the point cloud data obtained by the matching module and the denoising module; The digital surface model building module is used to build the digital surface model. The digital surface model is built using the local binning algorithm based on the registered point cloud data obtained by the registration module. The process of the local binning algorithm is: The registered point cloud is projected onto the XY plane to establish a virtual grid. Each grid is a square with a side length of Res. The number of rows m and columns n of the grid can be calculated as follows: Where: X max 、X min 、Y max 、Y min They are the maximum and minimum values ​​of the X and Y coordinates of the point cloud, respectively. Ceil(*) represents the rounding operation. From this, the coordinates of each grid corner point (X i ,Y j )as follows: Where: i = 0, 1, 2, ... n, j = 0, 1, 2, ... m represents the column number and row number of each grid corner point respectively; Let P = {p k ,k=1,2,…,τ} is the point cloud set, τ is the total number of point clouds, for each grid corner point V i,j , define a circular neighborhood with a radius of r, then the point cloud set within the neighborhood can be expressed as: R(V i,j ,r)={p k |d(V i,j ,p k )≤r,p k ∈P}(6) Where: is the grid corner point V i,j With point p k (x k ,y k ,z k ) on the XY plane; let the point cloud set R(V i,j ,r) is the elevation set of each point Then the value Z of the grid corner point i,j It can be calculated as follows: Where: NaN represents a null value, Represents an empty set. When there is no point cloud in the r neighborhood of a grid corner point, the value of the grid corner point is null. By traversing all grid corner points, the digital surface model of the target point cloud can be calculated. The change monitoring module is used for construction change monitoring. All the above modules are repeated at different times to obtain digital surface models of different phases. The difference between the DSMs of different phases is calculated and compared with the set threshold to obtain the change detection results.

6. The high-speed construction monitoring system based on drone laser and vision fusion as claimed in claim 5, characterized in that: In the matching module, the specific process of the multi-measure semi-global matching algorithm is: (21) The original image is enhanced using the automatic color scale method, and then the epipolar image is generated and an image pyramid is established. The image levels are defined as r0, r1, ... r from bottom to top. n ; (22) At the highest level of the image pyramid, r n Using the Census similarity measure, traverse each pixel p and calculate its disparity search range d range The matching cost C(p,d) when the internal disparity is d; (23) Calculate the path cost of pixel p in different directions in turn, and calculate the cumulative cost S(p,d). Select the disparity corresponding to the minimum cumulative cost as the disparity result of the pixel; (24) The cumulative cost at disparity d, d-1, and d+1 is fitted with a quadratic parabola to calculate the optimal sub-pixel level disparity to improve matching accuracy; (25) Segment the disparity map based on the principle that the change in disparity between adjacent pixels is less than a certain threshold. If the segmented area is less than a certain threshold, it is considered that there is an error and the image is removed. (26) Dynamically adjust the disparity search range d of pixels in the next level image range , from the r n-1 At the beginning of each level, mutual information is used as the similarity measure, and the initial disparity value is the disparity result of the previous level. The process (22) to (25) is repeated until the matching is completed when the bottom pyramid image is matched. (27) After obtaining the disparity map through the multi-measure semi-global matching algorithm, each image is processed in turn, and its disparity map is fused to obtain discrete three-dimensional point cloud data.

7. The high-speed construction monitoring system based on UAV laser and vision fusion according to claim 5 is characterized by: In the registration module, the specific process of the iterative closest point algorithm is: (51) For the point cloud to be registered P = {p0, p1, ... p m }, in the reference point cloud Q = {q0,q1,…q n } to find the point with the closest Euclidean distance to form a point pair set C k ={(p i k ,q i k )|i=1,2,…,N}, N is the number of point pairs, k is the number of iterations; (52) Using formula (51) as the objective function, minimize its error value and calculate the new rotation matrix R k and the translation matrix T k ; (53) Using the calculated transformation matrix parameter R k and T k Transform the point cloud to be registered to obtain a new point cloud to be registered p i k+1 as follows: p i k+1 =R k p i k +T k (2) (54) Calculate the average distance d between the corresponding point pairs of the new point cloud to be registered and the reference point cloud k+1 as follows: (55) If the number of iterations k reaches the specified maximum number of iterations, d k+1 -d k <ε or d k <ξ, the iteration is stopped, otherwise the above process is repeated, ε is the difference threshold of the average value of the point-pair distances between two adjacent iterations, and ξ is the average value threshold of the point-pair distances in a single iteration.

8. The high-speed construction monitoring system based on UAV laser and vision fusion according to claim 5 is characterized by: In the change monitoring module, the deformation monitoring process is: The local binning algorithm is used to calculate the digital surface models of two point cloud data at different time phases, and the elevation difference of the corresponding grids in the digital surface model is compared to set the elevation difference threshold D. th , if the elevation difference is greater than D th , then the grid is an elevation increase area. If the elevation difference is less than -D th , then the grid is an elevation reduction area; by traversing and calculating the elevation differences of all grids in the digital surface model and comparing them with the elevation difference threshold, all change areas can be identified.

Images