Non-contact shape detection system and program using UAV-LiDAR

Abstract

This invention provides a non-contact shape detection system and program using UAV-LiDAR that can detect the state of existing structures using three-dimensional point cloud data acquired by UAV-LiDAR. [Solution] The invention is characterized by comprising: a subsampling processing means for reducing the density of a point cloud by subsampling three-dimensional point cloud data; a noise processing means for removing low-density point clouds and retaining high-density point clouds based on the average distance and standard deviation to neighboring point clouds from the three-dimensional point cloud data; a primitive detection means for detecting a predetermined three-dimensional shape from the three-dimensional point cloud data after processing by the subsampling processing means and the noise processing means; and a deformation extraction means for extracting deformation of the constituent members of the existing structure based on the calculated distance between the predetermined three-dimensional shape and the three-dimensional point cloud data.

Description

Technical Field

[0001] The present invention relates to a non-contact shape detection system and program using UAV-LiDAR that can detect the state of existing structures based on three-dimensional point cloud data obtained by UAV-LiDAR.

Background Art

[0002] Conventionally, the state of structures has been detected using UAV-LiDAR. For example, in Patent Document 1, it is disclosed that high-rise structures such as iron towers can be surveyed with high precision based on the measurement data of LiDAR mounted on a UAV.

[0003] In addition, in Non-Patent Document 1, in the "Manual for Investigation and Diagnosis of Sump Wells" (reference material), in "8. New Investigation and Diagnosis Methods for Sump Wells", a method of opening the canopy, installing a scaffold inside the sump well, and installing a ground-based 3D scanner on the scaffold to investigate the deformation and damage of the sump well is disclosed.

[0004] Furthermore, conventionally, as shown in Fig. 25(a), a scaffold was installed on the canopy, and as shown in Fig. 25(b), the camera was moved from above to below to obtain a developed image as shown, and the state of the sump well was confirmed.

[0005] The above sump well is a facility aimed at preventing landslides by draining groundwater. However, damage to the sump well (for example, breakage or deformation of the wellbore or vertical stiffeners) may lead to loss of the function of collecting and draining groundwater, and it is particularly necessary to respond quickly to the emergency inspection of the sump well after an earthquake.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Non-Patent Documents

[0007] [Non-Patent Document 1] "(Reference Material) Manual for Inspection and Diagnosis of Water Collection Wells," [online], Ministry of Agriculture, Forestry and Fisheries, [Accessed October 11, 2024], Internet<https: / / www.maff.go.jp / j / nousin / sekkei / kanmin / attach / pdf / kanryou-147.pdf> [Overview of the Initiative] [Problems that the invention aims to solve]

[0008] However, water collection wells are scattered over a wide area (see Figure 3, etc.), and in many cases, there are no roads that allow vehicles to reach them. Due to this distribution characteristic of water collection wells, accessing them takes time, making it difficult to quickly inspect all of them with conventional technology. Furthermore, the inability to quickly acquire the three-dimensional shape of structures such as water collection wells and to detect the condition of structures including the wells remains a technical challenge.

[0009] Therefore, the present invention aims to provide a non-contact shape detection system and program using UAV-LiDAR that can detect the state of existing structures using three-dimensional point cloud data acquired by UAV-LiDAR. [Means for solving the problem]

[0010] The present invention relates to a non-contact shape detection system using UAV-LiDAR that detects the state of an existing structure using three-dimensional point cloud data acquired by UAV-LiDAR, and is characterized by comprising: a subsampling processing means for reducing the density of the point cloud by subsampling the three-dimensional point cloud data; a noise processing means for removing low-density point clouds and retaining high-density point clouds from the three-dimensional point cloud data based on the mean and standard deviation up to neighboring point clouds; a primitive detection means for detecting a predetermined three-dimensional shape from the three-dimensional point cloud data after processing by the subsampling processing means and the noise processing means; and a deformation extraction means for extracting deformation of the constituent members of the existing structure based on the calculated distance between the predetermined three-dimensional shape and the three-dimensional point cloud data.

[0011] Furthermore, according to one embodiment, the existing structure is an existing water collection well, and the predetermined three-dimensional shape is characterized in that it includes a cylindrical shape, and is a non-contact shape detection system using UAV-LiDAR.

[0012] Furthermore, according to one embodiment, the water collection well has a canopy on top, the primitive detection means is further capable of detecting the planar shape of the canopy, and the system is characterized by comprising an eccentricity estimation means that calculates the maximum inclination angle of the planar shape from the planar shape of the canopy and estimates the eccentricity of the water collection well.

[0013] Furthermore, according to one embodiment, in a non-contact shape detection program using UAV-LiDAR that detects the state of an existing structure using three-dimensional point cloud data acquired by UAV-LiDAR, the program is characterized by causing a computer to perform the following steps: a subsampling step of subsampling the three-dimensional point cloud data to reduce the density of the point cloud; a noise processing step of the three-dimensional point cloud data, which leaves high-density point clouds and deletes low-density point clouds based on the mean and standard deviation up to neighboring point clouds; a primitive detection step of detecting a predetermined three-dimensional shape from the three-dimensional point cloud data after processing by the subsampling step and the noise processing step; and a deformation extraction step of extracting deformation of the constituent members of the existing structure based on the calculated distance between the predetermined three-dimensional shape and the three-dimensional point cloud data.

[0014] According to the configuration of the present invention, it is possible to quickly and accurately detect the condition of existing structures located in hard-to-access locations. [Brief explanation of the drawing]

[0015] [Figure 1] This is an example of a detection flow in one embodiment of the present invention. [Figure 2] This is a map image showing the area of ​​concentrated water collection wells in one embodiment of the present invention. [Figure 3] This is a map image showing the location information of the water collection wells to be inspected. [Figure 4] This table shows the specifications of each water collection well being inspected. [Figure 5] These are photographic images showing the exterior and interior of the canopy of the water collection well (N-1). [Figure 6] These are photographic images showing the exterior and interior of the canopy of the water collection well (N-2). [Figure 7] These are photographic images showing the exterior and interior of the canopy of the water collection well (M-1). [Figure 8]A photographic image showing the appearance of the UAV-LiDAR and a diagram explaining the measurement mode of the catch basin. [Figure 9] A diagram explaining the mode of subsampling processing. [Figure 10] A diagram showing an example of a processed image when a cylinder is misdetected. [Figure 11] A diagram explaining the mode of noise processing of the canopy. [Figure 12] An image of the point cloud of the catch basin, the cylinder detected from the point cloud, and the image synthesized with the point cloud. [Figure 13] A diagram showing the definition of the eccentricity and the relationship between the maximum inclination angle and the eccentricity. [Figure 14] An RGB image of the point cloud in the catch basin (N-1) and a heat map showing the distribution patterns of the return number and the reflection intensity. [Figure 15] An RGB image of the point cloud in the catch basin (N-2) and a heat map showing the distribution patterns of the return number and the reflection intensity. [Figure 16] An RGB image of the point cloud in the catch basin (M-1) and a heat map showing the distribution patterns of the return number and the reflection intensity. [Figure 17] A heat map showing the density distribution of the point cloud in each catch basin. [Figure 18] A diagram showing the mode of the cylinders detected in each catch basin. [Figure 19] In the catch basin (N-1), a heat map showing the distance between the detected cylinder and the point cloud, and an image showing the classified point cloud. [Figure 20] In the catch basin (N-2), a heat map showing the distance between the detected cylinder and the point cloud, an image showing the classified point cloud, and an extraction image of the components. [Figure 21] In the catch basin (M-1), a heat map showing the distance between the detected cylinder and the point cloud, and an image showing the classified point cloud. [Figure 22] An image showing the distance between the cylinder and the point cloud in the components of the catch basin. [Figure 23]This image shows the input point cloud for the canopy of each water collection well, along with the detected plane and its maximum inclination angle and azimuth angle. [Figure 24] The images and diagrams shown here illustrate the conventional technology, specifically the camera-based photography conditions in a water collection well. [Modes for carrying out the invention]

[0016] The following describes an embodiment of the UAV-LiDAR-based non-contact shape detection system and program of the present invention, with reference to the drawings, using a non-destructive, non-contact method for detecting the state of an existing water collection well as an example of an existing structure.

[0017] Figure 1 shows an example of the detection flow of this embodiment. First, depending on the type of canopy 21 of the water collection well 2, it is determined whether the canopy 21 is made of expanded metal or other material that allows the laser light emitted from the UAV-LiDAR 1 to pass through, or whether it is made of corrugated sheet or other material that does not allow the laser light to pass through (S101).

[0018] If the canopy 21 of the water collection well 2 is one that allows laser light to pass through, measurements are taken using the UAV-LiDAR 1 (S103). On the other hand, if the canopy 21 is one that does not allow laser light to pass through, then images of the inside of the water collection well 2 are taken in advance (S102), and then measurements are taken using the UAV-LiDAR 1 (S103).

[0019] Next, the three-dimensional point cloud data obtained by the UAV-LiDAR1 measurement (S103) is input into a computer (not shown), where noise processing and subsampling (S104), described later, are performed. Based on the point cloud after these processes, the distance between the detected cylinder and the point cloud is calculated, and the deformation of the water collection well 2 and the deformation of each component are extracted (S105). Next, the maximum tilt angle of the canopy is estimated (S106).

[0020] The above outlines the general detection flow in this embodiment. Next, we will describe the inspection method for an existing water collection well 2.

[0021] Figure 2 is a map showing the location of water collection well 2, which is the subject of measurement. It is located very close to the epicenter of the Noto Peninsula Earthquake (to occur in 2024) shown in the figure, and is an area where damage to water collection well 2 is expected due to the earthquake.

[0022] The measurements using UAV-LiDAR1 are focused on three water collection wells 2 that were damaged in the Noto Peninsula Earthquake. Specifically, as shown in Figure 3, of the three water collection wells 2, two (N-1 and N-2) are located in Nawamata-cho, and one (M-1) is located in Monzen-cho.

[0023] Figure 4 shows the specifications of each of the above-mentioned water collection wells 2. The type of canopy 21 is expanded metal for N-1 and M-1, and corrugated sheet for N-2. The depth and distance to the water surface of each water collection well 2 are as shown in Figure 4.

[0024] Furthermore, Figure 5(a) shows the external view of the canopy of well 2 of N-1, and Figure 5(b) shows a photograph of the inside of the well casing of well 2 of N-1. Similarly, Figures 6 and 7 show photographs of the external view of the canopy and the inside of the well casings of well 2 of N-2 and M-1. As shown in Figure 5(b), in well 2 of N-1, sediment has flowed into the well due to shear failure, and deformation of the vertical stiffener and loss of the collection pipe can be confirmed. No visible deformation was observed in well 2 of N-2 and M-1.

[0025] Figure 8(a) shows a photographic example of the UAV-LiDAR1 in this embodiment. In the illustrated example, Livox's "Zenmuse" is mounted as LiDAR12, and DJI's "Matrice350 RTK" is used as UAV11. However, the equipment configuration is not necessarily limited to this example.

[0026] Furthermore, if the water collection well 2 to be measured has a diameter of 3.5m and a depth of 30.0m, point cloud data can be acquired down to the bottom of water collection well 2 by oblique flight, as shown in Figure 8(b), with the turning diameter a set to 8.2m, the flight altitude h set to 12.0m, and the LiDAR elevation angle θ set to 80°.

[0027] For computer analysis after measurements using UAV-LiDAR1, point cloud generation can be performed using software such as DJI Terra from DJI. For noise processing and primitive detection (detection of planes and cylinders) using RANSAC, open-source software such as CloudCompare can be used, which will be discussed in more detail later.

[0028] By detecting the cylindrical shape from the point cloud of the well casing of the water collection well 2 using the primitive detection described above, it is possible to detect the specifications and deformation of the water collection well 2. Furthermore, by detecting the plane of the canopy 21, it is possible to estimate the amount of eccentricity based on the maximum tilt angle.

[0029] Furthermore, in this embodiment, in order to detect deformation or damage to the water collection well 2 with high accuracy, subsampling is performed as a point cloud preprocessing before detecting the cylinder as described above. Specifically, Figure 10(a) shows an RGB image plotted with the acquired point cloud and an image showing the number of neighboring point clouds (r=0.05m) as a heatmap. In this state, as shown in Figure 11, there is a possibility that cylinders may be falsely detected in the high-density areas of the point cloud.

[0030] Therefore, in this embodiment, by performing subsampling before cylinder detection, it is possible to reduce the variation in point cloud density and to reduce false detection of cylinders in high-density areas of the point cloud. Figure 10(b) shows the RGB image after subsampling and an image showing the number of neighboring point clouds (r=0.05m) as a heat map.

[0031] Furthermore, in this embodiment, noise processing is performed before plane detection of the canopy 21. Specifically, the process involves retaining high-density point clouds and deleting low-density point clouds based on the average distance and standard deviation to neighboring point clouds.

[0032] For example, as shown in Figure 12(a), in the expanded metal canopy 21, a high-density and high-precision point cloud is obtained from the frame portion of the canopy 21, while a low-density and variable point cloud is obtained from the expanded metal portion of the canopy 21. In order to detect the canopy 21 as a plane with high precision, it is necessary to obtain the point cloud from the frame portion, and therefore the noise processing described above is performed. Figure 12(b) shows the RGB image after noise processing and an image showing the number of neighboring point clouds (r=0.05m) as a heatmap.

[0033] Next, we will explain the primitive detection (detection of planes and cylinders) in the embodiment described above. Generally, the point cloud acquired by UAV-LiDAR1 is unstructured. The algorithm for detecting basic shapes is called primitive detection, and in this embodiment, detection is performed by assuming that the well casing of the water collection well 2 is a cylinder and the canopy 21 of the water collection well 2 is a plane.

[0034] Since the detected cylinders are represented by their diameter and height, the diameter and depth of the water collection well 2 can be obtained by detecting the cylinders (right) from the point cloud (left), as shown in Figure 13. If the water level in water collection well 2 is high, the point cloud below the water surface cannot be acquired because the laser light is not reflected, so the height of the detected cylinder will be the distance to the water surface in water collection well 2.

[0035] In this embodiment, it is possible to detect the deformed well casing and vertical stiffener from the distance between the cylinder and the point cloud obtained as described above.

[0036] Furthermore, in this embodiment, the maximum inclination angle and azimuth angle are obtained from the inclination of the plane by detecting the plane of the point cloud of the canopy 21. As a result, as shown in Figures 14(a) and (b), it is possible to estimate the eccentricity of the water collection well 2 from the maximum inclination angle. The standard value for the construction management of water collection wells is set at an eccentricity of 150 mm. Assuming depths of 10 m, 20 m, and 30 m, and calculating based on Figure 14(b), the maximum inclination angles of the canopy 21 are 0.86°, 0.42°, and 0.29°, respectively, which are acceptable angles.

[0037] Figures 15-17 show the point cloud characteristics of the three water collection wells 2, N-1, N-2, and M-1. In each figure, the RGB image and the number of laser light returns and reflection intensity are shown as a heat map for the point cloud of each water collection well 2.

[0038] In Figure 15, the N-1 well 2 shows that the well casing is shallower than the other two due to shear failure, and significant deformation of the vertical stiffener can be observed. In Figure 16, the N-2 well 2 has a corrugated sheet roof 21. Therefore, the laser beam's path is limited to the opening in the roof 21, and the point cloud within the well 2 is acquired corresponding to the position of the opening. In Figure 17, the M-1 well 2 shows that the point cloud was acquired all the way to the bottom of the well 2, and it can be confirmed that even the spiral ladder has been reproduced.

[0039] In Figure 15(b), the number of laser beam returns for the N-1 well 2, and in Figure 17(b), the number of laser beam returns for the M-1 well 2, is 1 at the canopy 21 and 2 at the well casing. This is because a large amount of laser light is reflected and scattered by the expanded metal section. However, looking at the number of laser beam returns for the N-2 well 2 shown in Figure 16(b), where the canopy 21 is made of corrugated sheet, a return of 1 is confirmed even at the well casing, confirming that the laser light is entering the well 2 through the opening in the canopy 21 without any reduction in intensity.

[0040] Figures 18(a) to (c) show the density distribution of the point clouds for the three water collection wells 2, N-1, N-2, and M-1, as heatmaps. In all cases, the point cloud density is highest at the center of the canopy 21, and it can be seen that the point clouds become sparser in the deeper parts of the well casing. From these findings, it can be said that although the point cloud characteristics acquired by UAV-LiDAR1 depend on the depth of the water collection well 2 and the type of canopy 21, the point clouds of the water collection wells 2 can be acquired appropriately in all cases.

[0041] Figures 19(a) to (c) show the estimated dimensions of three water collection wells 2, N-1, N-2, and M-1, in the detection of cylinders in the embodiment described above. The design value for the diameter of the well casing is 3.5m in all cases, but the diameters of the detected cylinders are 3.216m (twice the radius shown) for water collection well 2 of N-1, 3.336m for water collection well 2 of N-2, and 3.516m for water collection well 2 of M-1.

[0042] In addition, while the measured distance to the water surface of well 2 in N-1 was 3.930m, the height of the cylinder was 4.075m; while the measured distance to the water surface of well 2 in N-2 was 14.209m, the height of the cylinder was 14.278m; and while the measured distance to the water surface of well 2 in M-1 was 28.888m, the height of the cylinder was 29.117m. This confirms that estimation is possible with extremely high accuracy, with an error of approximately 28cm in diameter and approximately 23cm in height (depth).

[0043] Figures 20 to 22 then show the deformation detection results for each of the two water collection wells, based on the distance difference between the cylinders and the point cloud detected by the primitive detection method described above. In other words, they show the results of classifying the distance difference between the cylinders and the point cloud detected in Figure 19 using the threshold method.

[0044] In the N-1 well 2 shown in Figure 20, as shown in Figure 20(a), the distance difference between the cylinder and the point cloud is -0.512 to -0.900 m at the deformed part of the vertical stiffener. Furthermore, at the end of the canopy 21, a positive value is observed because the diameter of the canopy 21 is larger than the diameter of the well casing. As shown in Figure 20(b), by using the distance difference between the cylinder and the point cloud as an indicator and setting a threshold of -0.300 m, it is possible to detect the deformation of the vertical stiffener with high accuracy.

[0045] To verify the validity of the above threshold, Figure 23 shows the distance between the cylinder and the point cloud in the constituent members of the water collection well 2. As shown in the figure, the liner plate forming the well casing is corrugated, and it can be confirmed that it is represented as a point cloud with thickness in the XY plane.

[0046] When examining the cases where the cylinder fits the point cloud outside the center of the liner plate and the case where it fits the center of the liner plate, it was shown that for ladders and undamaged vertical stiffeners, the point cloud fits at the center of the liner plate and is not detected when a threshold of -0.300m is used. In other words, when detecting a deformed vertical stiffener, if the cylinder is fitted to the center of the liner plate, the area inside the point between 0.425m and 0.444m, indicated by a threshold of -0.300m, is detected as the deformed area.

[0047] In the N-2 well 2 shown in Figure 21, as shown in Figure 21(a), it can be seen that the distance difference between the cylinder and the point cloud at the top of the well casing is 0.162m or more. Furthermore, as shown in Figure 21(b), it can be seen that it is possible to detect deformation of the well casing by setting the threshold for the distance difference between the cylinder and the point cloud to -0.200m and displaying the classified point cloud. Moreover, as shown in Figure 21(c), it is possible to accurately extract structural members such as reinforcing materials and ladders inside the well casing by setting the threshold for the distance difference between the cylinder and the point cloud to -0.06m and displaying the classified point cloud.

[0048] In the M-1 well 2 shown in Figure 22, as shown in Figure 22(a), a large deformation of 0.175m or more can be observed at the bottom of the well casing, where the distance difference between the cylinder and the point cloud is greater than 0.175m. Furthermore, as shown in Figure 22(b), it can be seen that it is possible to detect deformation of the well casing by setting the threshold for the distance difference between the cylinder and the point cloud to -0.200m and displaying the classified point cloud. In addition, it can be seen that it is possible to classify and extract the spiral staircase within well 2.

[0049] Figures 24(a) to (c) show the input point clusters for the canopy 21 in the three water collection wells 2, N-1, N-2, and M-1, as well as the maximum inclination angle and azimuth angle of the canopy 21 determined by the aforementioned plane detection. All of the water collection wells 2 do not meet the standard of eccentricity of 150 mm.

[0050] (Other embodiments) The non-contact shape detection system and program using UAV-LiDAR of the present invention have been described above, with an example of measurement in an existing water collection well 2. However, the present invention is not necessarily limited to the above-described configuration, and various modifications are possible as follows.

[0051] For example, the structures targeted for non-contact shape detection are not limited to the aforementioned water collection well 2, but can also include other structures such as sediment control dams. This enables rapid and highly accurate shape detection of various structures, especially those located in places where direct visual inspection is difficult, such as after an earthquake, or structures whose interiors are difficult to see.

[0052] Furthermore, the various analysis processes for shape and state detection described above can be executed as programs on a computer, such as a general-purpose PC, server, or cloud, and the present invention can be implemented by inputting the point cloud data acquired by UAV-LiDAR1.

[0053] Although embodiments of the present invention have been described above with reference to the drawings, the specific configurations are not limited to these embodiments. The scope of the present invention is indicated by the claims rather than the above-described embodiments, and furthermore, all modifications within the meaning and scope of equivalence to the claims are included. In addition, the specific materials, dimensions, shapes, etc., described in the above embodiments can be modified to the extent that they solve the problems of the present invention. [Explanation of Symbols]

[0054] 1 UAV-LiDAR 11 UAV 12 LiDAR 2 Water collection well 21 Canopy

Claims

1. In a non-contact shape detection system using UAV-LiDAR that detects the state of existing structures using three-dimensional point cloud data acquired by UAV-LiDAR, Subsampling processing means for reducing the density of the point cloud by subsampling the three-dimensional point cloud data, A noise processing means for the aforementioned three-dimensional point cloud data, which retains high-density point clouds and removes low-density point clouds based on the mean and standard deviation up to the nearest point cloud, After processing by the subsampling processing means and the noise processing means, primitive detection means for detecting a predetermined three-dimensional shape from the three-dimensional point cloud data, The system includes a deformation extraction means for extracting deformation of the constituent members of the existing structure based on the calculated distance between the predetermined three-dimensional shape and the three-dimensional point cloud data. A non-contact shape detection system using UAV-LiDAR, characterized by the following:

2. The aforementioned existing structure is an existing water collection well. The aforementioned predetermined three-dimensional shape includes a cylindrical shape. The non-contact shape detection system using UAV-LiDAR as described in claim 1.

3. The aforementioned water collection well has a canopy on top, The primitive detection means is also capable of detecting the planar shape of the canopy, The system includes an eccentricity estimation means for estimating the eccentricity of the water collection well by calculating the maximum inclination angle of the planar shape of the canopy. The non-contact shape detection system using UAV-LiDAR as described in claim 2.

4. In a non-contact shape detection program using UAV-LiDAR that detects the state of existing structures using three-dimensional point cloud data acquired by UAV-LiDAR, On the computer, A subsampling step of reducing the density of the point cloud by subsampling the three-dimensional point cloud data, The three-dimensional point cloud data is subjected to a noise processing step that removes low-density point clouds and retains high-density point clouds based on the mean and standard deviation up to the nearest point clouds. After processing by the subsampling step and the noise processing step, a primitive detection step is performed to detect a predetermined three-dimensional shape from the three-dimensional point cloud data, The system is configured to perform a deformation extraction step, which extracts the deformation of the constituent members of the existing structure based on the calculated distance between the predetermined three-dimensional shape and the three-dimensional point cloud data. A non-contact shape detection program using UAV-LiDAR, characterized by the following:

Images