Dent measurement device, dent measurement method, and dent measurement program

Abstract

To provide a hollow measuring device, a hollow measuring method, and a hollow measuring program capable of easily specifying and measuring a hollow formed on the surface of a feature.SOLUTION: A hollow measuring device 200 includes: a measurement data acquisition unit 201 that acquires point group data of a measurement space obtained by measuring a measurement space including at least part of a hollow formed on the surface of a feature with a three-dimensional sensor; a target region designating unit 221 that designates a target region including the hollow in the point group data in the measurement space; a hollow identification unit 222 that identifies the hollow based on the point group data of the target region; a volume calculation unit 223 that calculates the volume of the hollow based on the point group data of the identified hollow. The hollow identification unit 222 selects either first identification process of identifying the hollow from the leveled ground or second identification process of identifying the hollow from the uneven ground to identify the hollow.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a depression measurement device, a depression measurement method, and a depression measurement program. [Background technology]

[0002] In industries such as the electric power and gas industries, underground pipes must be inspected periodically, and inspections are carried out by digging the ground using excavators, etc. During excavation work, the amount of excavation is adjusted so that the pipes are exposed, but digging more than necessary risks damaging the underground pipes, so there is a need for technology that can accurately measure the amount of excavation.

[0003] Meanwhile, in the construction and civil engineering industries, there is a demand for technology that can measure the volume of excavated areas to determine the remaining amount of soil and sand needed for backfilling excavated areas caused by construction work, etc. Also, in the railway industry, there is a demand for technology that can detect areas of subsidence of tracks early on, as there is a risk of a sinkhole occurring if the tracks above ground sink due to underground cavities.

[0004] As a technique for measuring depressions caused by excavation, etc., a method for measuring the depth and volume of depressions based on point cloud data measured using a three-dimensional sensor is known. Patent Document 1 is an example of such a technique.

[0005] Patent Document 1 discloses the following data analysis device, data analysis method, and program. A neighborhood space setting means sets each element point as a focus point and sets an element point neighborhood space that includes the focus point and has a predetermined three-dimensional shape and size. A three-dimensional space filtering means extracts an element point as a feature point derived from a step or crack to be detected when a predetermined number of element points of a point cloud are included in each element point neighborhood space. A crack detection means detects the position of the detection target from the arrangement of the feature points. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-4588 Summary of the Invention [Problem to be solved by the invention]

[0007] The technology in Patent Document 1 is described as enabling automation of the work of detecting cracks and steps, which are deformations that have occurred on the surface of a feature, based on 3D coordinate data of a point cloud measured from the surface of the feature. However, the technology in Patent Document 1 detects depressions such as cracks and steps, but has the problem that it cannot identify depressions and measure their volume or dimensions.

[0008] In view of the above-mentioned problems, an object of the present invention is to provide a depression measurement device, a depression measurement method, and a depression measurement program that can easily identify and measure depressions formed on the surface of features. [Means for solving the problem]

[0009] A depression measurement device according to one embodiment includes a measurement data acquisition unit that acquires point cloud data of a measurement space obtained by measuring a measurement space including at least a portion of a depression formed on the surface of a feature using a three-dimensional sensor; a target area designation unit that designates a target area including a depression whose volume is to be determined using the point cloud data of the measurement space; a depression identification unit that identifies the depression based on the point cloud data of the target area; and a volume calculation unit that calculates the volume of the depression based on the point cloud data of the identified depression, and the depression identification unit selects either a first identification process that identifies depressions from level ground where the surface unevenness amount is less than a predetermined threshold, or a second identification process that identifies depressions from uneven ground where the surface unevenness amount is greater than or equal to the threshold, to identify the depression.

[0010] In addition, a depression measurement method in one embodiment includes the steps of acquiring point cloud data of a measurement space obtained by measuring a measurement space including at least a portion of a depression formed on the surface of a feature using a three-dimensional sensor, specifying a target area including the depression whose volume is to be determined in the point cloud data of the measurement space, identifying the depression based on the point cloud data of the target area, and calculating the volume of the depression based on the point cloud data of the identified depression, and in the step of identifying the depression, the depression is identified by selecting either a first identification method that identifies the depression from level ground whose surface unevenness is less than a predetermined threshold, or a second identification method that identifies the depression from uneven ground whose surface unevenness is greater than or equal to a threshold.

[0011] In addition, a depression measurement program in one embodiment is a depression measurement program executed by a computer, and executes the following processes: a process of acquiring point cloud data of a measurement space obtained by measuring a measurement space including at least a portion of a depression formed on the surface of a feature using a three-dimensional sensor; a process of specifying a target area including the depression whose volume is to be calculated in the point cloud data of the measurement space; a process of identifying the depression based on the point cloud data of the target area; and a process of calculating the volume of the depression based on the point cloud data of the identified depression; and in the process of identifying the depression, the program executes the process of identifying the depression by selecting either a first identification method that identifies the depression from a level ground whose surface unevenness is less than a predetermined threshold value, or a second identification method that identifies the depression from an uneven ground whose surface unevenness is greater than or equal to a threshold value. [Effects of the Invention]

[0012] A depression measuring device, a depression measuring method, and a depression measuring program are provided that can easily identify and measure depressions formed on the surface of features. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a block diagram showing a configuration of a depression measuring device according to a first embodiment. [Figure 2] FIG. 1 is a diagram showing how a measurement space is measured by a three-dimensional sensor. [Figure 3]FIG. 10 is a block diagram showing the configuration of a depression measuring device according to a second embodiment. [Figure 4] 4 is a flowchart showing the operation of the depression measuring device shown in FIG. 3. [Figure 5] 10 is a flowchart showing the flow of a first identification process in a recess identification unit. [Figure 6] This is an illustration of a feature with a depression on a leveled surface. [Figure 7] 10 is a flowchart showing the flow of a second identification process in a recess identification unit. [Figure 8] This is an illustration of a feature with a depression on uneven ground. [Figure 9] FIG. 10 is a conceptual diagram showing color data. [Figure 10] FIG. 10 is an image diagram showing projection data. [Figure 11] FIG. 10 is a diagram showing how a measurement space is measured by a three-dimensional sensor when there is an effect of occlusion. [Figure 12] 10 is a flowchart showing the flow of a first calculation process in a volume calculation unit. [Figure 13] FIG. 10 is an image diagram showing a missing portion before interpolation of a missing interpolation point group given by the lowest point P1. [Figure 14] FIG. 10 is an image diagram of the missing portion after interpolation of the missing interpolation points given by the lowest point P1. [Figure 15] FIG. 10 is an image diagram of a group of missing interpolation points given by the inclination angle θ of the depression in the missing portion before interpolation. [Figure 16] FIG. 10 is an image diagram of a defect portion after interpolation of a group of defect interpolation points given by the inclination angle θ of the depression. [Figure 17] FIG. 4 is an image diagram of the depression after measurement by the depression measurement device shown in FIG. 3. [Figure 18] FIG. 4 is a block diagram showing a hardware configuration of the depression measuring device shown in FIG. 3. DETAILED DESCRIPTION OF THE INVENTION

[0014] Embodiment 1 Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiments. For clarity of explanation, the following description and drawings have been simplified as appropriate. Furthermore, in the following description, identical or equivalent elements are designated by the same reference numerals, and redundant explanations will be omitted.

[0015] Fig. 1 is a block diagram showing the configuration of a depression measuring device according to embodiment 1. As shown in Fig. 1, depression measuring device 200 includes measurement data acquiring unit 201, target region designating unit 221, depression identifying unit 222, and volume calculating unit 223.

[0016] The measurement data acquisition unit 201 acquires measurement data 100, which is point cloud data of a measurement space obtained by measuring (photographing) a measurement space including at least a portion of a depression 1 formed on the surface of a feature 10 using a three-dimensional sensor 20. The target area designation unit 221 designates a target area 503 including the depression 1, the volume of which is to be calculated, for the measurement data 100. The depression identification unit 222 identifies the depression 1 based on the point cloud data of the target area 503. The depression identification unit 222 also selects either a first identification process that identifies the depression 1 (502) from a level ground 501 whose surface unevenness is less than a preset threshold, or a second identification process that identifies the depression 1 (702) from an uneven ground 701 whose surface unevenness is equal to or greater than the threshold, to identify the depression 1. The volume calculation unit 223 calculates the volume of the depression 1 based on the point cloud data of the identified depression 1.

[0017] The depression measuring device 200 according to this embodiment can easily identify and measure depressions formed on the surface of features.

[0018] Embodiment 2 Next, an embodiment of the present invention will be described with reference to Figures 2 to 18. In this embodiment, the recess measuring device 200 described in embodiment 1 will be described in more detail. First, Figure 2 is a diagram showing how a measurement space is measured by a three-dimensional sensor.

[0019] As shown in Figure 2, the depression measurement device 200 is a device that acquires measurement data 100, which is point cloud data of a measurement space obtained by measuring (photographing) the measurement space using a three-dimensional sensor 20, automatically identifies depression 1 (502, 702) formed in a feature 10 from the measurement data 100, and calculates the volume of the identified depression 1 (502, 702).

[0020] In this embodiment, the processing performed by the depression measurement device 200 will be described using as an example a feature 10 on which a depression 502 existing on level ground 501 with a flat surface and a depression 702 existing on irregular ground 701 with an irregular surface. In this embodiment, the level ground 501 will be described as a horizontal surface, but the level ground 501 may be a wall surface extending in the vertical direction, a slope, or a curved surface.

[0021] The three-dimensional sensor 20 is installed in a position where it can capture an image of a measurement space including at least a portion of the depressions 502, 702 formed on the surface of the feature 10. In the example shown in Fig. 2, the three-dimensional sensor 20 is installed diagonally above the depressions 502, 702. The three-dimensional sensor 20 is a sensor that measures distances in three-dimensional space, such as a ToF (Time-of-Flight) camera, a stereo camera, or a 3D-LiDAR (Light Detection And Ranging).

[0022] The three-dimensional sensor 20 generates measurement data 100 from the captured image. The three-dimensional sensor 20 is communicably connected to the depression measuring device 200 via a network, which may be a wired or wireless communication network, for example.

[0023] The measurement data 100 is three-dimensional point cloud data including the distance from the sensor to each measurement point. The three dimensions may be latitude, longitude, and altitude (height) information, or may be a three-dimensional Euclidean coordinate system or polar coordinate system with the origin at a specific position set by the user. In the following example, a three-dimensional Euclidean coordinate system (each direction is represented by an X, Y, or Z coordinate) with the origin set by the user is assumed. The units of each coordinate are expressed in meters (m), centimeters (cm), and millimeters (mm), but other units may also be used. The value of the Z coordinate represents height (depth) information.

[0024] A 3D point is a point to which information such as the time the point cloud data was captured, the laser reflection intensity, and color information such as red, blue, or green is assigned. There is no limit to the information assigned to a 3D point, but at least 3D coordinates (X, Y, and Z coordinates) that are position information are assigned, and a point cloud is a collection of two or more such 3D points. In this embodiment, the Z-axis direction of the Euclidean coordinate system represents the vertical direction, and the two-dimensional plane spanned by the X-axis and Y-axis represents the horizontal plane.

[0025] The depression measurement device 200 may acquire measurement data 100 generated by combining the three-dimensional sensor 20 with other sensors such as position and orientation sensors such as a global positioning system (GPS) or an inertial measurement unit (IMU), a red-green-blue (RGB) camera, or an environmental sensor. The depression measurement device 200 may also have a function to link with external systems (for example, external devices, inspections, construction work, etc.).

[0026] Any method can be used to acquire the measurement data 100. For example, the 3D sensor 20 may be mounted on a moving object such as a vehicle, such as an automobile, or a mobile robot, and the measurement space may be measured while the 3D sensor 20 is moved by the moving object. As one method for acquiring the measurement data 100, the 3D sensor 20 may be mounted on an aerial moving object such as a drone, allowing the measurement space to be photographed from the air.

[0027] Next, the configuration of the depression measuring device 200 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the depression measuring device according to the second embodiment. As shown in Fig. 3, the depression measuring device 200 is made up of a preprocessing unit 210, a measuring unit 220, and a user interface unit 230.

[0028] 1, and acquires measurement data 100 from three-dimensional sensor 20. Preprocessing unit 210 performs preprocessing on measurement data 100 to shape the data before processing in measurement unit 220, and outputs the preprocessed measurement data 100 to measurement unit 220. Preprocessing unit 210 has a format conversion unit 211, a noise removal unit 212, and an angle conversion unit 213.

[0029] The format conversion unit 211 has a function of converting the data format of the measurement data 100 into a predetermined format that can be used by the depression measurement device 200. The format conversion unit 211 outputs the measurement data 100 converted into the predetermined format to the noise removal unit 212. Note that if the measurement data 100 acquired from the three-dimensional sensor 20 is in the predetermined format, the processing of the format conversion unit 211 is omitted. Furthermore, the format conversion unit 211 may not only convert the data format, but also synthesize multiple pieces of data, or merge or integrate data acquired from other sensors or external systems.

[0030] The noise removal unit 212 has a function of removing unnecessary point cloud data such as outliers from the point cloud data included in the measurement data 100. Noise removal methods include, for example, smoothing processing, filtering, outlier removal processing, correction processing, etc. The noise removal unit 212 outputs the measurement data 100 from which noise has been removed to the angle conversion unit 213.

[0031] The angle conversion unit 213 has a function of performing angle conversion to rotate the measurement data 100 to a predetermined angle as preprocessing for the shortest distance measurement. The angle conversion unit 213 performs angle conversion on the measurement data 100 based on information from other sensors or external systems. For example, the angle conversion unit 213 can perform angle conversion on the measurement data 100 by acquiring a vertical direction that forms a predetermined angle using an IMU, or by referring to information such as topographical and geographical information from other sensors or external systems, CAD (Computer Aided Design), and as-built drawings. In this embodiment, angle conversion is performed on the measurement data 100 to a predetermined angle by rotating the depth direction of the depressions 502, 702 so that it is aligned vertically (Z-axis direction).

[0032] Furthermore, angle conversion unit 213 may perform angle conversion so that measurement data 100 has a predetermined angle by rotating the recessed surfaces of depressions 502 and 702 so that they are horizontal. Furthermore, angle conversion of measurement data 100 may be performed manually based on information input by a user operation received by operation unit 231, without using angle conversion unit 213.

[0033] Next, the measurement unit 220 has a function of identifying at least a portion of the depressions 502, 702 based on the measurement data 100, and calculating (estimating) the volume of the depressions 502, 702 from at least a portion of the identified depressions 502, 702. The measurement unit 220 is composed of a target area designation unit 221, a depression identification unit 222, and a volume calculation unit 223.

[0034] The target area designation unit 221 has a function of designating a target area 503 containing depressions 502, 702 whose volumes are to be calculated for the measurement data 100 acquired from the preprocessing unit 210, based on designated conditions. The target area designation unit 221 designates the target area 503 based on conditions (height, width, depth, etc.) designated by a user operation received by the operation unit 231, for example. The user can designate the target area 503 by excluding areas in the measurement space where no depressions 502, 702 exist. The target area designation unit 221 performs processing to store information about the target area 503 based on the conditions acquired via the operation unit 231 in the memory 120 of the depression measuring device 200, and outputs the information about the target area 503 to the depression identification unit 222. The information about the target area 503 includes the conditions acquired via the operation unit 231 and a target area point cloud, which is point cloud data of the target area 503.

[0035] If no conditions are specified, the target area designation unit 221 designates the entire measurement data 100 (the entire measurement space) as the target area 503. In other words, if no conditions are specified, the target area designation unit 221 outputs a target area point cloud corresponding to the measurement data 100 for which preprocessing has been completed to the depression identification unit 222. The target area designation unit 221 may store multiple pieces of information related to the designated target area 503, and may be able to switch the target area 503 by reading out the stored data.

[0036] The depression identifying unit 222 has a function of automatically identifying (clustering) depressions 502, 702 within the target area 503 based on the target area point cloud acquired from the target area designation unit 221. A specific method for identifying the depressions 502, 702 includes a method of extracting the difference obtained by comparing point cloud data before and after the depressions 502, 702 appear on the surface of the feature 10. Other methods for identifying the depressions 502, 702 include a method of setting thresholds for the dimensions and volume of the depressions 502, 702 and extracting locations that exceed the thresholds, and a method of displaying images that are color-coded according to the dimensions and volume of the depressions 502, 702 on the display unit 232. The depression identifying unit 222 may also identify the depressions 502, 702 in cooperation with other sensors such as the above-mentioned RGB camera.

[0037] The depression identification unit 222 selects either a first identification process or a second identification process depending on the shape (amount of unevenness) of the surface of the feature 10, and can identify depressions 502 that exist on level ground 501 or depressions 702 that exist on uneven ground 701. The depression identification unit 222 generates a three-dimensional map that is point cloud data of at least a portion of the identified depressions 502, 702, and outputs the map to the volume calculation unit 223. The first identification process and the second identification process in the depression identification unit 222 will be described in detail later.

[0038] The volume calculation unit 223 has a function of calculating (estimating) the volume of the depressions 502, 702 based on the three-dimensional map acquired from the depression identification unit 222. The volume calculation unit 223 can calculate (estimate) the volume of the depressions 502, 702 by selecting either the first calculation process or the second calculation process depending on whether or not there is an effect of occlusion that cannot be measured by the three-dimensional sensor 20.

[0039] Furthermore, the volume calculation unit 223 can calculate (estimate) not only the volume of the depressions 502 and 702 but also at least one of the height, width, and depth dimensions of the depressions 502 and 702. When calculating the dimensions of the depressions 502 and 702, the maximum and minimum values ​​of the X, Y, and Z coordinates are obtained from the point cloud data included in the three-dimensional map, and the dimensions are calculated from the difference between the maximum and minimum values ​​of each coordinate.

[0040] A method for checking whether or not there is an effect of occlusion is to specify a target area 503 using the target area designation unit 221 so that the depressions 502, 702 are contained as closely as possible, and to make a judgment based on the dimensions of the depressions 502, 702 identified based on the target area 503.

[0041] The volume calculation unit 223 outputs the volume calculation result to the display unit 232. The calculation result may include not only the volume but also the calculation result of the dimensions. Details of the first calculation process and the second calculation process in the volume calculation unit 223 will be described later.

[0042] The volume calculation unit 223 may calculate the volume of the depressions 502, 702 using information acquired from other sensors. For example, the volume calculation unit 223 can improve the accuracy of the volume calculation by correcting the position information and size (dimension) information of the depressions 502, 702 using information from an RGB camera, an environmental sensor, etc. Furthermore, the volume calculation unit 223 may calculate the volume using information acquired from an external system or information input by a user operation accepted by the operation unit 231. Here, information for calculating the volume of the depressions 502, 702 may include the type of the identified object (e.g., material), the situation (e.g., soil viscosity, soil moisture content), the environment (e.g., precipitation, temperature, humidity), etc.

[0043] The user interface unit 230 has a function of visualizing and operating the point cloud data handled by the depression measurement device 200, and displaying the measurement results measured by the measurement unit 220. The user interface unit 230 has an operation unit 231 and a display unit 232. The user interface unit 230 may include not only interface circuits to the operation unit 231 and the display unit 232, but also an interface circuit to an external system (for example, an alarm system, etc.).

[0044] The user interface unit 230 may also perform a process of storing input or calculated information in chronological order in the memory 120 of the recess measuring device 200, or a process of storing abstract data of the input or calculated information in the memory 120 of the recess measuring device 200. Furthermore, the user interface unit 230 may also perform a process of reading out the information and data stored in the memory 120 and transmitting it to a user or an external system.

[0045] The operation unit 231 accepts operations from the user. The operation unit 231 has an input device that allows the user to input instructions, information, and the like to the recess measuring device 200. The input device may be physical keys such as a keyboard, a mouse, a numeric keypad, or buttons, or may be configured as a touch panel integrated with a display. By operating the operation unit 231, the user can input conditions for specifying the target region 503, for example, to the recess measuring device 200.

[0046] The display unit 232 has a display device that displays the identification results of the depressions 502 and 702 identified by the depression identification unit 222, the calculation results of the volume calculated by the volume calculation unit 223, etc. The display device is, for example, a platform panel display such as a liquid crystal display, a plasma display, or an organic EL (Electro-Luminescence) display.

[0047] Next, the operation of the recess measuring device 200 according to this embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the operation of the recess measuring device shown in Fig. 3. As shown in Fig. 4, the recess measuring device 200 executes a recess measurement process of steps S301 to S307.

[0048] In step S301, the preprocessing unit 210 acquires the measurement data 100. In this step, the measurement data 100 is acquired by measuring the measurement space with the three-dimensional sensor 20. The format conversion unit 211 that has acquired the measurement data 100 converts the data format of the measurement data 100 into a predetermined format as necessary.

[0049] Subsequently, in step S302, the noise removal unit 212 removes unnecessary point cloud data such as outliers from the point cloud data included in the measurement data 100 acquired in step S301.

[0050] Next, in S303, angle conversion unit 213 performs angle conversion to rotate measurement data 100 that has been subjected to noise removal processing in step S302 so that it is at a predetermined angle. In this step, angle conversion is performed so that the depth direction from the top surface to the bottom surface of depressions 502, 702 in measurement data 100 is oriented vertically downward (downward along the Z axis). In this way, measurement data 100 for which preprocessing has been completed is generated.

[0051] Next, in step S304, the target area designation unit 221 designates a target area 503 containing the depressions 502, 702 whose volumes are to be calculated for the measurement data 100 for which preprocessing in step S303 has been completed. In this step, the target area 503 is designated for the measurement data 100 for which preprocessing has been completed based on the conditions accepted by the operation unit 231, and a target area point cloud is generated.

[0052] Next, in step S305, the depression identification unit 222 identifies (clusters) the depressions 502 and 702 based on the target area point cloud generated in step S304. The depression measurement device 200 identifies the depressions by selecting either a first identification process that identifies depressions 502 that exist on level ground 501, or a second identification process that identifies depressions 702 that exist on uneven ground 701. The depression measurement device 200 identifies the depressions 502 and 702 by automatically switching between the first identification process and the second identification process depending on the amount of unevenness on the surface of the feature 10.

[0053] Specifically, if the amount of unevenness, which is the difference in Z coordinate values ​​between the convex and concave portions on the surface of the feature 10, is smaller than a threshold value, the depression identification unit 222 determines that the ground is level 501 and executes the first identification process (step S400), and if the amount of unevenness on the surface of the feature 10 is greater than or equal to the threshold value, the depression identification unit 222 determines that the ground is uneven 701 and executes the second identification process (step S600).

[0054] The flow of the first identification process will now be described with reference to Figures 5 and 6. Figure 5 is a flowchart showing the flow of the first identification process in the depression identification unit. Figure 6 is an image diagram of a feature in which a depression exists on a leveled ground. The depression measurement device 200 executes the first identification process (step S400) shown in steps S401 to S403 in Figure 5, and identifies a depression 502 on a leveled ground 501 shown in Figure 6.

[0055] First, in step S401, the depression identification unit 222 generates a plane-removed point cloud by performing plane removal to remove plane portions (for example, flat ground, road surface, etc.) from the target area point cloud acquired from the target area designation unit 221. In this step, the depression identification unit 222 preferably acquires from the target area designation unit 221 a target area point cloud in which areas in the measurement space where no depressions 502 exist have been excluded.

[0056] Planar regions can be extracted using an algorithm such as RANSAC (Random Sample Consensus). By removing the planar regions, flat objects with the same shape, such as the ground or wall surfaces, are removed, allowing for accurate clustering and reducing misclassification. The noise removal unit 212 may be configured to perform noise removal processing on the plane-removed point cloud as needed.

[0057] Next, in step S402, clustering is performed on the plane-removed point cloud generated in step S401 to generate plane-removed clusters. Clustering is a method for dividing or classifying point cloud data (here, the plane-removed point cloud) into multiple clusters, and for example, k-means or the like can be adopted.

[0058] Furthermore, in step S403, a 3D map in which the top surface interpolation points are interpolated is generated by performing point cloud interpolation to interpolate the top surface interpolation points, which are point cloud data of the top surface portion 502a (vertically upper surface) of the depression 502 missing for each plane-removed cluster generated in step S402. The 3D map generated here is a 3D map representing the depressions 502 present on the leveled ground 501, and is generated for each identified depression 502. The 3D coordinates of the top surface interpolation points are given by the conditions of the target area 503 specified in step S304.

[0059] Here, "interpolation" refers to calculating values ​​for unmeasured areas based on values ​​for areas obtained by measurement. The depression identification unit 222 outputs the generated 3D map to the volume calculation unit 223. This completes the flow of the first identification process.

[0060] Meanwhile, the flow of the second identification process will be described with reference to Figures 7 and 8. Figure 7 is a flowchart showing the flow of the second identification process in the depression identification unit. Figure 8 is an image diagram of a feature where a depression exists on uneven ground. The depression measurement device 200 executes the second identification process shown in steps S601 to S610 shown in Figure 7, and identifies a depression 702 on uneven ground 701 shown in Figure 8.

[0061] First, in S601, the depression identification unit 222 samples 3D points using the Z coordinate value of the 3D coordinates (X, Y, Z coordinates) of each 3D point constituting the target area point cloud acquired from the target area designation unit 221 as a reference, and layers the target area point cloud. The number of samples at this time is set to n. In this step, the depression identification unit 222 preferably acquires a target area point cloud for which no conditions are specified from the target area designation unit 221. The Z axis indicates the depth direction of the depression 702 along the vertical direction. The X axis indicates the width direction of the depression 702, and the Y axis indicates the depth direction of the depression 702.

[0062] The depression identification unit 222 generates color data in which the sampled three-dimensional points are color-coded according to depth (Z coordinate value). Fig. 9 is an image diagram showing the color data. The color data shown in Fig. 9 represents the side of the feature 10 as viewed from the X-axis direction. In the example shown in Fig. 9, a darker color indicates a deeper depth, and a lighter color indicates a shallower depth.

[0063] Next, in S602, the three-dimensional coordinates (X, Y, Z coordinates) of the three-dimensional points sampled in step S601 are projected onto the XY plane and converted into two-dimensional coordinates (X, Y coordinates), generating two-dimensional projection data. FIG. 10 is an image diagram showing the projection data. The projection data shown in FIG. 10 is made up of two-dimensional point cloud data, which is a collection of two-dimensional points obtained by converting the three-dimensional coordinates into two-dimensional coordinates, and represents a plan view of the feature 10 as viewed from the Z-axis direction. In the example shown in FIG. 10, darker areas indicate deeper depths, and lighter areas indicate shallower depths. The XY plane refers to the depression surface of the depression 702, which is perpendicular to the Z-axis.

[0064] Next, in S603, a layer counter m is set to "1." The layer counter m is a counter used in the second identification process to perform a depression search by searching the layered target area point cloud from an upper layer (m=1) to a lower layer (m=n). The lower layer is a layer located vertically below the upper layer.

[0065] Next, in S604, a layer point cloud, which is point cloud data of the m-th layer from the layered target area point cloud, is acquired and added to a search cluster for grouping the searched layers.

[0066] Next, in S605, clustering is performed on the search clusters to generate layer clusters for each depression 702.

[0067] Next, in S606, for each layer cluster, a search is made to determine whether a lower layer exists on the XY plane within the region of each layer cluster. If the search result indicates that a lower layer exists on the XY plane (step S606; YES), it is determined that a depression 702 exists, and the process proceeds to step S607. Then, in step S607, the layer point cloud of the lower layer searched in step S606 is acquired, and the process proceeds to step S608. If a lower layer does not exist on the XY plane (step S606; NO), it is determined that a depression 702 does not exist, and the process proceeds to step S608.

[0068] Next, in S608, the layer counter is incremented by "1" (m=m+1). After that, the process returns to S604, and the series of depression search processes shown in steps S604 to S607 are repeated until the lowest layer (m=n) of the layered target area point cloud is searched. When depression search is completed up to the lowest layer where the layer counter becomes m=n, the process proceeds to step S609.

[0069] Next, in S609, projection restoration is performed on the layer point clouds of each layer m=1 to n obtained by the depression search in steps S604 to S608. In this step, the two-dimensional coordinates (X, Y coordinates) of the two-dimensional points that make up the layer point cloud are converted into three-dimensional coordinates (X, Y, Z coordinates).

[0070] Subsequently, in step S610, clustering is performed on the layer point cloud restored in three dimensions in step S609 to generate a three-dimensional map for each depression 702. This completes the flow of the second identification process.

[0071] Then, the depression specifying unit 222 outputs the three-dimensional map generated by the first specifying process or the second specifying process to the volume calculation unit 223.

[0072] Here, another method for identifying depression 702 in the second identification process is, for example, to identify depression 702 using the image of color data shown in Fig. 9 or the image of projection data shown in Fig. 10 without performing the depression search described above. As yet another method, a depression search method may be employed in which a cross-sectional area (the outline of the depression surface) is detected from the projection data using an algorithm such as edge detection, and a search is performed to determine whether a lower layer exists within the area surrounded by the cross-sectional area.

[0073] Returning to FIG. 3 , in S306, the volume calculation unit 223 calculates (estimates) the volume of the depression 1 based on the three-dimensional map generated in steps S400 and S600. If there is occlusion, the volume calculation unit 223 performs a first calculation process to interpolate a group of missing interpolation points obtained from the three-dimensional map of the depression 1 identified by the depression identification unit 222 into the missing portion 903 created by the occlusion, and then calculates the volume of the depression 1. If there is no occlusion, the volume calculation unit 223 performs a second calculation process to calculate the volume of the depression 1 from the three-dimensional map of the depression 1 identified by the depression identification unit 222. The volume calculation unit 223 selects either the first calculation process or the second calculation process depending on whether there is an influence of occlusion and calculates the volume.

[0074] 11 is a diagram showing how a measurement space is measured by a three-dimensional sensor when there is an influence of occlusion. In the following explanation, a case where a depression 1 formed on the surface of a feature 10 shown in FIG. 11 is the specific target will be explained.

[0075] As shown in FIG. 11, if there is an obstacle between the three-dimensional sensor 20 and the depression 1 to be identified, occlusion occurs, where the obstacle in front hides part of the depression 1 behind it, making it impossible to see. As a result, the area of ​​depression 1 behind the obstacle cannot be measured by the three-dimensional sensor 20. In the example shown in FIG. 11, the three-dimensional sensor 20 is placed in front of the depression 1 (on the left side in FIG. 11), and the front part of the feature 10 that exists between the three-dimensional sensor 20 and the depression 1 is the obstacle. When there is an effect of occlusion, the three-dimensional map acquired from the depression identification unit 222 will have some point cloud data missing due to the occlusion.

[0076] In this way, when the measurement data 100 has a missing portion 903 that does not include point cloud data, even if the volume is calculated for the 3D map acquired from the depression identification unit 222, the volume of the occlusion portion cannot be obtained, and therefore the accurate volume of the entire depression 1 cannot be calculated. Therefore, the depression measurement device 200 calculates the volume of the depression 1 by automatically switching between the first calculation process (step S800) and the second calculation process (step S900) depending on whether or not there is an influence of occlusion. Note that the user may visually check the measurement data 100 to determine whether or not there is an influence of occlusion, and then manually switch between the first calculation process and the second calculation process.

[0077] The flow of the first calculation process will be described with reference to Fig. 12. Fig. 12 is a flowchart showing the flow of the first calculation process in the volume calculation unit. As shown in Fig. 12, when it is confirmed that there is an influence of occlusion, the recess measuring device 200 executes the first calculation process shown in steps S801 to S804.

[0078] First, in step S801, outliers are removed as necessary from the point cloud data included in the three-dimensional map 904 of the depression 1 identified by the depression identification unit 222 in step S305.

[0079] Next, in step S802, the point cloud data included in the 3D map from which the outliers have been removed in step S801 is searched for the lowest point P1, which is the 3D point including the Z coordinate at the deepest position (lowest point search), and the 3D coordinates of the lowest point P1 found by the lowest point search are obtained.

[0080] Next, in step S803, a completed 3D map is generated by performing point group interpolation to interpolate a missing interpolation point group in the missing portion 903. Note that a completed 3D map is generated for each identified depression 1.

[0081] Here, there are two methods for performing the point group interpolation in step S803. First, the first method will be described with reference to Fig. 13 and Fig. 14. Fig. 13 is an image diagram before the missing portion is interpolated with the missing interpolation point group given by the lowest point P1. Fig. 14 is an image diagram after the missing portion is interpolated with the missing interpolation point group given by the lowest point P1.

[0082] 13 has a missing portion 903 in which at least a portion of the point cloud data is missing on the Y-axis side (left side in FIG. 13) of the central axis when the X-axis passing through the lowest point P1 included in the 3D map 904 is taken as the central axis. That is, the missing portion 903 shown in FIG. 13 does not include the actual lowest point (lowest point P1). In this case, the lowest point P1 found in step S802 is assumed to be the bottom surface (vertically lower surface) of the identified depression 1.

[0083] The first method is suitable when the lowest point P1 found in step S802 matches the actual lowest point. The fact that the found lowest point P1 matches the actual lowest point means that the actual lowest point is not missing, and that the actual lowest point is not included in the missing portion 903. When the first method is applied, the three-dimensional coordinates of the missing interpolation point group are given by the lowest point P1 found in step S802.

[0084] FIG. 14 shows a three-dimensional map 905 in which a group of missing interpolated points given by the lowest point P1 is interpolated into a missing portion 903. The group of missing interpolated points given by the lowest point P1 is point cloud data that is symmetrical about the central axis to the point cloud data located in the range of depth L (FIG. 13) from the central axis to the far side of the Y axis (right side in FIGS. 13 and 14) in the three-dimensional map 904. In other words, the group of missing interpolated points is point cloud data located in the range of depth L from the central axis to the near side of the Y axis (left side in FIGS. 13 and 14). The three-dimensional map 905 generated by interpolating the group of missing interpolated points given by the lowest point P1 into the missing portion 903 is a completed three-dimensional map. In this way, the shape of the missing portion 903 shown in FIG. 13 can be estimated from the three-dimensional map acquired from the depression identification unit 222, and a completed three-dimensional map can be generated.

[0085] On the other hand, the second method is suitable when the lowest point P1 found in step S802 does not match the actual lowest point. When the found lowest point P1 does not match the actual lowest point, it means that the actual lowest point is missing, and that the actual lowest point is included in the missing portion 903. When the second method is applied, the three-dimensional coordinates of the missing interpolation point group are given by the inclination angle of the depression 1.

[0086] The second method will now be described with reference to Fig. 15 and Fig. 16. Fig. 15 is an image diagram of the missing portion before interpolation of the missing interpolation points given by the inclination angle θ of the depression. Fig. 16 is an image diagram of the missing portion after interpolation of the missing interpolation points given by the inclination angle θ of the depression.

[0087] The measurement data 100 shown in Figure 15 has a missing portion 903 where the actual lowest point corresponding to the bottom portion of the depression 1 is missing due to occlusion. That is, the missing portion 903 shown in Figure 15 includes the actual lowest point. In this case, the lowest point P1 searched for in step S802 is a 3D point located vertically above the actual lowest point. Therefore, if a completed 3D map is generated using the same method as the first method using the lowest point P1 included in the 3D map 904 shown in Figure 15 and the volume is calculated, the error with respect to the actual volume of the depression 1 may increase.

[0088] Therefore, when the actual lowest point is missing, as shown in Fig. 16, the tilt angle θ of depression 1 is calculated from the point cloud data of 3D map 904 before point cloud interpolation is performed. The tilt angle θ can be calculated, for example, using the 3D coordinates of each 3D point located on the farthest side and the nearest side as seen from 3D sensor 20 in 3D map 904. Note that the farthest side as seen from 3D sensor 20 is the right side in Figs. 15 and 16, and the near side as seen from the 3D sensor is the left side in Figs. 15 and 16. The tilt angle θ of depression 1 calculated here is the angle between the inclined surface of depression 1 and the horizontal direction.

[0089] Then, a group of missing interpolation points representing the shape along the inclination angle θ from both ends of the Y axis of the top surface portion of the depression 1 is interpolated. This determines the virtual lowest point P2. A three-dimensional map 905 generated by interpolating the group of missing interpolation points given by the inclination angle θ into the missing portion 903 is a completed three-dimensional map. In this way, the shape of the missing portion 903 shown in FIG. 15 can be estimated from the three-dimensional map acquired from the depression identification unit 222, and a completed three-dimensional map can be generated. Note that a completed three-dimensional map is generated for each identified depression 1.

[0090] Next, in step S804, the volume of the completed 3D map is calculated. The volume can be found, for example, by creating a convex hull of the point cloud data of the completed 3D map. Here, the convex hull is the smallest convex set that contains a given set. The convex hull can be created, for example, by an algorithm using the Quickhull method. A convex hull created by an algorithm using the Quickhull method is composed of multiple tetrahedrons. If the base area of ​​a tetrahedron is S and the height is h, the volume Vi of the tetrahedron can be calculated from the following equation (1) for the volume of a triangular pyramid. Vi=Sh / 3 Equation (1)

[0091] By calculating the volume of the tetrahedrons that make up the created convex hull in this way, the volume of the completed 3D map can be calculated.

[0092] On the other hand, in the second calculation process, the volume is calculated for the three-dimensional map acquired from the depression identification unit 222, without interpolating the missing interpolation point group. The volume can be calculated for the three-dimensional map acquired from the depression identification unit 222, for example, by creating a convex hull of the point cloud data of the three-dimensional map acquired from the depression identification unit 222 and calculating the volume of the tetrahedron that makes up the created convex hull. The method for calculating the volume using a convex hull is the same as the calculation method described in the first calculation process.

[0093] Instead of the above-described method of creating a convex hull, the volume of the depression 1 may be calculated for voxel data obtained by processing a three-dimensional map used for volume calculation into cubes called voxels.

[0094] The volume calculation unit 223 then outputs the volume calculation result generated by the first calculation process or the second calculation process to the display unit 232. Note that by acquiring measurement data 100 that is less affected by occlusion, missing portions 903 are reduced, and the accuracy of volume calculation can be improved. One method for acquiring measurement data 100 that is less affected by occlusion is to measure the measurement space from various directions while moving the 3D sensor 20. Another method for acquiring measurement data 100 that is less affected by occlusion is to install multiple 3D sensors 20 in the measurement space and extract the measurement data 100 that is least affected by occlusion from the multiple data acquired from the multiple 3D sensors 20.

[0095] Returning to FIG. 4, in S307, the display unit 232 acquires the calculation result from the volume calculation unit 223. Having acquired the calculation result, the display unit 232 displays the calculation result. FIG. 17 is an image diagram after the depression is measured by the depression measurement device shown in FIG. 3. As shown in FIG. 17, the calculation result may include, in addition to information about the volume V of the depression 1, various information about the measurement data 100, the target area 503, the identified depression 1, the dimensions of the depression 1 (height H, width W, depth D), etc.

[0096] The depression measurement process according to this embodiment can be implemented on a terminal such as a PC (Personal Computer) or tablet used by a user. Fig. 18 is a block diagram showing the hardware configuration of the depression measurement device shown in Fig. 3. As shown in Fig. 18, the depression measurement device 200 has an input / output interface 110, a memory 120, and a processor 130.

[0097] The input / output interface 110 is used to communicate with any other device. For example, the input / output interface 110 may be used to acquire the measurement data 100 from the three-dimensional sensor 20, or may be used to output data related to the calculation results to another device.

[0098] The memory 120 is configured, for example, by a combination of a volatile memory and a non-volatile memory. The memory 120 is used to store software (computer programs) including one or more instructions executed by the processor 130, data used for various processes of the depression measuring device 200, and the like.

[0099] The processor 130 reads and executes software (computer program) from the memory 120, thereby performing the processing of the depression measurement device 200 described above using the flowchart shown in Fig. 4. Here, the depression measurement program causes the computer to execute the following processes: acquiring point cloud data of a measurement space obtained by measuring, with the 3D sensor 20, a measurement space including at least a portion of a depression 1 formed on the surface of the feature 10; specifying a target area 503 including the depression 1, the volume of which is to be calculated, in the point cloud data of the measurement space; identifying the depression 1 based on the point cloud data of the target area 503; and calculating the volume of the depression 1 based on the point cloud data of the identified depression 1. In addition, the depression measurement program causes the computer to execute the process of identifying the depression 1 by selecting either a first identification method for identifying the depression 1 (502) from a level ground 501 having a surface unevenness amount less than a preset threshold, or a second identification method for identifying the depression 1 (702) from an uneven ground 701 having a surface unevenness amount equal to or greater than a threshold.

[0100] The processor 130 may be, for example, a microprocessor, a microprocessor unit (MPU), a central processing unit (CPU), etc. The processor 130 may include multiple processors.

[0101] The above-described program executed by the depression measuring apparatus 200 can be stored using various types of non-transitory computer-readable media and supplied to a computer. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memories), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memories)). The program may also be supplied to a computer by various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer-readable media can supply the program to a computer via wired communication paths such as electric wires and optical fibers, or via wireless communication paths. The above describes the embodiments.

[0102] Here, a related technology to the above is a technology in which, when measuring the depths of multiple depressions, the depressions to be detected are visually confirmed from point cloud data and measurements are performed on each of the depressions detected through visual confirmation.However, such a technology has the problem of increasing costs.

[0103] Furthermore, for example, in a technique for measuring a specific depth in a depression, it is necessary to acquire point cloud data of the entire depression in order to measure the dimensions and volume of the entire depression. However, with such a technique, if a portion of the depression is missing due to occlusion or the like, it is not possible to calculate the dimensions and volume of the entire depression using only the point cloud data of that portion, which may result in the depression being unable to be measured.

[0104] In contrast, in this embodiment, the target region 503 containing the depression 1 whose volume is to be calculated can be arbitrarily specified, which makes it easy to identify and measure the depression 1, reducing costs and increasing the accuracy of identifying and measuring the depression 1. Furthermore, by arbitrarily specifying the target region 503 containing the depression 1 whose volume is to be calculated, it is possible to calculate not only the entire volume of the depression 1 but also the partial volume of the depression 1. Therefore, according to this embodiment, it is possible to simulate volume measurement.

[0105] Furthermore, in this embodiment, even if a portion of the depression 1 is missing due to occlusion or the like, the depression 1 can be measured by interpolating a missing interpolation point group in the missing portion 903 based on the point cloud data of at least a portion of the depression 1.

[0106] In this embodiment, the depression 1 can be identified and measured whether the surface on which the depression 1 exists is level ground 501 or uneven ground 701. Therefore, the present invention can be applied to cases where it is required to identify and measure the depression 1 in an environment with a complex shape.

[0107] The depression measurement device, depression measurement method, and depression measurement program described in the above embodiments can be used for managing the progress of excavation work in the construction industry, monitoring the excavation volume of excavation work in the gas and electricity industries, inspecting and monitoring roads and equipment in the civil engineering industry, monitoring abnormalities in farmland in agriculture and forestry, and monitoring the water volume of reservoirs, etc. [Explanation of symbols]

[0108] 1, 502, 702 recesses 10 Features 20 3D Sensor 100 measurement data 110 Input / Output Interface 120 memory 130 processors 200 Depression measurement device 201 Measurement data acquisition unit 210 Pretreatment section 211 Format Conversion Unit 212 Noise removal section 213 Angle conversion unit 220 Measurement Unit 221 Target area specification section 222 Depression Identification Section 223 Volume calculation unit 230 User Interface Section 231 Operation section 232 Display section 501 Land leveling 502a Top part 503 Target Area 701 Uneven terrain 903 Missing part 904, 905 3D Map P1, P2 lowest point

Claims

1. a measurement data acquisition unit that acquires point cloud data of a measurement space obtained by measuring the measurement space including at least a part of a depression formed on the surface of the feature using a three-dimensional sensor; a target area designation unit that designates a target area including the depression whose volume is to be calculated for the point cloud data of the measurement space; a depression identifying unit that identifies the depression based on point cloud data of the target area; a volume calculation unit that calculates a volume of the depression based on point cloud data of the identified depression, The depression identifying unit is A depression measurement device that identifies the depression by selecting either a first identification process that identifies the depression from a level ground where the amount of surface unevenness is less than a predetermined threshold, or a second identification process that identifies the depression from an uneven ground where the amount of surface unevenness is greater than or equal to the threshold.

2. In the first identification process, A process of removing planar portions from the point cloud data of the target area; generating plane-removed clusters by performing clustering on the point cloud data from which the plane portions have been removed; a process of interpolating point cloud data of an upper surface portion of the depression for each of the plane removal clusters; The depression measuring device according to claim 1 , wherein the depression measuring device performs the following.

3. In the second identification process, A process of layering a depressed surface of the depression that is perpendicular to the depth direction of the depression with respect to the point cloud data of the target area; a process of acquiring point cloud data for each layer obtained by layering the depression surface, and sequentially searching the lower layers present within the depression surface for each layer cluster generated by sequentially adding the point cloud data for each layer from the upper layer to the generated search clusters; The depression measuring device according to claim 1 or 2, wherein the depression measuring device performs the following.

4. The volume calculation unit If there is an occlusion in the recess that cannot be measured by the three-dimensional sensor, a first calculation process for calculating a volume of the depression after interpolating a missing portion caused by the occlusion using a missing interpolation point group obtained from the point cloud data of the depression; If there is no occlusion, performing a second calculation process of calculating a volume of the depression based on the point cloud data of the depression identified by the depression identification unit; The depression measuring device according to any one of claims 1 to 3.

5. The volume calculation unit The depression measuring device according to any one of claims 1 to 4, wherein at least one of the height, width, and depth dimensions of the identified depression is calculated.

6. a step of acquiring point cloud data of a measurement space obtained by measuring a measurement space including at least a part of a depression formed on a surface of a feature using a three-dimensional sensor; specifying a target region including the depression whose volume is to be calculated for the point cloud data of the measurement space; Identifying the depression based on point cloud data of the target area; and calculating a volume of the depression based on point cloud data of the identified depression, In the step of identifying the depression, A depression measurement method for identifying the depression by selecting either a first identification method for identifying the depression from a level ground where the amount of surface unevenness is less than a predetermined threshold value, or a second identification method for identifying the depression from an uneven ground where the amount of surface unevenness is greater than or equal to the threshold value.

7. A dent measurement program executed by a computer, a process of acquiring point cloud data of a measurement space obtained by measuring the measurement space including at least a part of a depression formed on the surface of a feature using a three-dimensional sensor; A process of specifying a target region including the depression whose volume is to be calculated for the point cloud data of the measurement space; A process of identifying the depression based on point cloud data of the target area; and calculating a volume of the depression based on point cloud data of the identified depression; In the process of identifying the depression, A depression measurement program that executes a process of identifying the depression by selecting either a first identification method that identifies the depression from a level ground where the amount of surface unevenness is less than a predetermined threshold value, or a second identification method that identifies the depression from an uneven ground where the amount of surface unevenness is greater than or equal to the threshold value.

Images