Estimation device, estimation method, and estimation program

Abstract

An estimation device comprising: a division unit that uses a predetermined space division method to divide point cloud data to be modeled into voxel groups each formed from a plurality of voxels in accordance with the depth of a specified hierarchical level; an estimation unit that performs model estimation using the divided voxel groups and outputs an estimation result obtained by modeling the point cloud data; a range update unit that uses the divided voxel groups and the output estimation result to obtain a set of intersections and obtain partial voxel groups in which a processing subject of the voxel groups has been updated; and an iterative processing unit that repeats the model estimation and the update of the partial voxel groups until the hierarchical level of the depth of the process subject satisfies a predetermined condition.

Description

[Technical Field]

[0001] The disclosed technology relates to an estimation device, an estimation method, and an estimation program. [Background technology]

[0002] Conventionally, real space is measured using a measuring device such as LiDAR (Light Detection and Ranging) and expressed as three-dimensional point cloud data (see, for example, Non-Patent Document 1). By converting real space positions into data in this way, the data can be used for the maintenance and management of social infrastructure. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Graziosi, D., Nakagami, O., Kuma, S., Zaghetto, A., Suzuki, T., & Tabatabai, A., "An overview of ongoing point cloud compression standardization activities: video-based (V-PCC) and geometry-based (G-PCC)", MPEG Point Cloud Compression, April 7, 2020 Summary of the Invention [Problem to be solved by the invention]

[0004] When modeling to utilize point cloud data and performing model estimation, there is a problem in that the processing costs become enormous as the size of the point cloud increases.

[0005] The disclosed technology has been made in consideration of the above points, and aims to provide an estimation device, an estimation method, and an estimation program that can reduce processing costs and achieve highly efficient and fast model estimation. [Means for solving the problem]

[0006] A first aspect of the present disclosure is an estimation device including: a division unit that divides point cloud data to be modeled into voxel groups each consisting of a plurality of voxels according to a specified hierarchical depth using a predetermined spatial division method; an estimation unit that performs model estimation using the divided voxel groups and outputs estimation results that model the point cloud data; a range update unit that uses the divided voxel groups and the output estimation results to find a set of intersections to find a partial voxel group by updating the processing target of the voxel group; and an iterative processing unit that repeats the model estimation and the updating of the partial voxel group until the hierarchical depth of the processing target satisfies a predetermined condition.

[0007] A second aspect of the present disclosure is an estimation method in which a computer executes the following processes: dividing point cloud data to be modeled into voxel groups consisting of multiple voxels according to a specified hierarchical depth using a predetermined spatial division method; performing model estimation using the divided voxel groups; outputting estimation results that model the point cloud data; using the divided voxel groups and the output estimation results, determining a set of intersections to determine a partial voxel group by updating the processing target of the voxel group; and repeating the model estimation and updating of the partial voxel group until the hierarchical depth of the processing target satisfies a predetermined condition.

[0008] A third aspect of the present disclosure is an estimation program that causes a computer to execute the following processes: divide point cloud data to be modeled into voxel groups, each consisting of a plurality of voxels, according to a specified hierarchical depth using a predetermined spatial division method; perform model estimation using the divided voxel groups; output estimation results that model the point cloud data; use the divided voxel groups and the output estimation results to find a set of intersections to find a partial voxel group by updating the processing target of the voxel group; and repeat the model estimation and updating of the partial voxel group until the hierarchical depth of the processing target satisfies a predetermined condition. [Effects of the Invention]

[0009] According to the disclosed technology, it is possible to reduce processing costs and achieve highly efficient and fast model estimation. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an example of modeling using model estimation of point cloud data according to a conventional method. [Figure 2] FIG. 2 is a block diagram showing the hardware configuration of the estimation device. [Figure 3] FIG. 3 is a block diagram showing the functional configuration of the estimation device. [Figure 4] FIG. 4 is a flowchart showing the flow of the estimation process performed by the estimation device. [Figure 5] Figure 5 shows the voxel group before and after estimation when the specified hierarchical depth is six levels. [Figure 6] Figure 6 shows the estimated voxels when the specified hierarchical depth is eight levels. [Figure 7] FIG. 7 is a table showing a comparison of the number of estimation targets and model estimation time between the conventional method and the present method. DETAILED DESCRIPTION OF THE INVENTION

[0011] An example of an embodiment of the disclosed technology will be described below with reference to the drawings. Note that the same or equivalent components and parts in each drawing are given the same reference numerals. Also, the dimensional proportions in the drawings are exaggerated for the convenience of explanation and may differ from the actual proportions.

[0012] Figure 1 shows an example of modeling using model estimation of point cloud data using a conventional method. In the conventional method, modeling was performed by simply performing RANSAC model estimation from point cloud data. This type of model estimation consumes excessive memory and processing time, resulting in huge costs.

[0013] Therefore, in this embodiment, highly efficient model estimation is achieved by combining the octree hierarchical structure of point cloud data with model estimation. In the method of this embodiment, a group of voxels obtained by octree-dividing point cloud data to be modeled is subjected to model estimation for each layer in turn, with a predetermined start and end layer, and the process of excluding voxels outside the model for that layer is repeated. This reduces processing costs and enables highly efficient and fast model estimation.

[0014] The configuration of this embodiment will be described below. Fig. 2 is a block diagram showing the hardware configuration of the estimation device 100.

[0015] 2, the estimation device 100 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage 14, an input unit 15, a display unit 16, and a communication interface (I / F) 17. Each component is connected to each other via a bus 19 so as to be able to communicate with each other.

[0016] The CPU 11 is a central processing unit that executes various programs and controls each part. That is, the CPU 11 reads a program from the ROM 12 or the storage 14 and executes the program using the RAM 13 as a work area. The CPU 11 controls each of the above components and performs various arithmetic processing in accordance with the program stored in the ROM 12 or the storage 14. In this embodiment, an estimation program is stored in the ROM 12 or the storage 14.

[0017] The ROM 12 stores various programs and various data. The RAM 13 temporarily stores programs or data as a working area. The storage 14 is configured with a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) and stores various programs including the operating system and various data.

[0018] The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used to perform various inputs.

[0019] The display unit 16 is, for example, a liquid crystal display, and displays various information. The display unit 16 may function as the input unit 15 by adopting a touch panel system.

[0020] The communication interface 17 is an interface for communicating with other devices such as terminals, etc. For this communication, for example, a wired communication standard such as Ethernet (registered trademark) or FDDI, or a wireless communication standard such as 4G, 5G, or Wi-Fi (registered trademark) is used.

[0021] Next, each functional configuration of the estimation device 100 will be described. Fig. 3 is a block diagram showing the configuration of the estimation device of this embodiment. Each functional configuration is realized when the CPU 11 reads out an estimation program stored in the ROM 12 or storage 14, deploys it in the RAM 13, and executes it. As shown in Fig. 3, the estimation device 100 includes a point cloud data storage unit 102, a division unit 110, an extraction unit 112, an estimation unit 114, a range update unit 116, and an iterative processing unit 118.

[0022] Point cloud data of the estimation target is stored in the point cloud data storage unit 102. In the processing of each unit of the estimation device 100, the point cloud data is read from the point cloud data storage unit 102 and processed.

[0023] The dividing unit 110 receives point cloud data as input and divides the point cloud data into voxel groups each consisting of a plurality of voxels according to a specified hierarchical depth using an octree space division method. The octree space division method is a method of recursively dividing a three-dimensional space in which the point cloud data exists so that the spaces of eight child nodes belong to the space of a parent node. The specified hierarchical depth (depth) is defined as d, and the voxel group (i.e., each voxel) corresponding to the hierarchical depth d is defined as V(d). The hierarchical depth d may be specified by input from the user. Note that the octree space division method is just one example, and any method capable of dividing a point cloud into voxel groups can be used.

[0024] The extraction unit 112 extracts a set C(d) of center points of each voxel by thinning the point group by extracting center points from each voxel of the voxel group V(d) of the layer d to be processed, which is a divided voxel group. Each center point of the set extracted by the extraction unit 112 corresponds to each voxel of the voxel group. Note that the processing by the extraction unit 112 may be omitted under certain conditions, such as when the number of points included in the voxel group is small and below a certain number, and the voxel group V(d) may be used to perform model estimation, which will be described later.

[0025] The estimation unit 114 performs model estimation using the set C(d) of center points extracted by the extraction unit 112, and outputs an estimation result P(d) that models the point cloud data. The estimation result P(d) is output to the range update unit 116 and also stored in the model storage unit 104. Note that P(d) is a parameter of the model. For example, when modeling is a plane model, if the equation of the plane is ax+by+cz+d=0, then the actual state of P is the parameters {a, b, c, d}. In this embodiment, RANSAC is used as the method for model estimation, but any method that can model point cloud data may be used.

[0026] The range update unit 116 uses the divided voxel group V(d) and the output estimation result P(d) to obtain a set of intersections between V(d) and V(d), and obtains a partial voxel group V'(d) by updating the voxel group processing target. Through the processing of the range update unit 116, a partial voxel group V'(d) is obtained in which voxels outside the model of layer d have been excluded. In other words, as the layer becomes deeper with each iteration, the voxels in the range of the processing target are narrowed down.

[0027] The iterative processing unit 118 repeats the extraction by the extraction unit 112, the model estimation by the estimation unit 114, and the updating of the voxel group by the range update unit 116 until the partial voxel group V'(d) satisfies a predetermined condition. The iterative processing unit 118 counts up the layer depth d (d=d+1) to the depth of the next layer, creates the voxel group V(d) of the next layer from the partial voxel group V'(d), and repeats the process.

[0028] Next, the operation of the estimation device 100 will be described. Fig. 4 is a flowchart showing the flow of estimation processing by the estimation device 100. The estimation processing is performed by the CPU 11 reading out an estimation processing program from the ROM 12 or storage 14, expanding it into the RAM 13, and executing it. The estimation device 100 receives designation of point cloud data to be estimated and the hierarchical depth from the user, reads out the point cloud data from the point cloud data storage unit 102, and performs the following processing.

[0029] In step S100, the CPU 11, as the division unit 110, receives point cloud data as input and divides the point cloud data into voxel groups V(d) each consisting of a plurality of voxels according to the specified hierarchical depth using an octree spatial division method.

[0030] In step S102, the CPU 11, as the extraction unit 112, extracts a set C(d) of center points of each voxel obtained by thinning out the point group by extracting center points from each voxel of the divided voxel group V(d) of the layer d to be processed.

[0031] In step S104, the CPU 11 functions as the estimation unit 114 to perform model estimation using the set C(d) of extracted center points, and outputs an estimation result P(d) obtained by modeling the point cloud data.

[0032] In step S106, the CPU 11, as the range update unit 116, uses the divided voxel group V(d) and the output estimation result P(d) to obtain the set of intersections between V(d) and P(d) and obtains a partial voxel group V'(d) by updating the processing target of the voxel group.

[0033] In step S108, the CPU 11, functioning as the iterative processing unit 118, determines whether the partial voxel group V'(d) satisfies a condition. The condition is satisfied when, for example, the set of intersections, i.e., the number of points in the partial voxel group V'(d), falls below the number of samples required for model estimation. The required number of samples may be specified as a termination condition according to a predetermined required accuracy. In the case of a planar model, the required number of samples is "3" or the like. If the condition is satisfied, the estimation result of the model estimated in step S104 and the hierarchical depth d are finally output, and the processing ends. If the condition is not satisfied, the process proceeds to step S110.

[0034] In step S110, the CPU 11, functioning as the iterative processing unit 118, counts up the layer depth d (d=d+1) to the depth of the next layer, and creates a voxel group V(d) of the next layer from the partial voxel group V'(d).Then, the process returns to step S102 and repeats the process.

[0035] As described above, the estimation device 100 of this embodiment can realize highly efficient and fast model estimation while suppressing processing costs.

[0036] Figure 5 shows the voxel group before and after estimation when the specified hierarchical depth is 6. The specified hierarchical depth is d=6. Figure 5 (a) shows the voxel group before estimation, and the modeled voxel group after estimation. The voxel group in (b) is the modeled voxel group estimated from (a). Since (a) is before estimation, all point clouds such as buildings and roads are included in the voxels. In (b), the plane has been modeled, and it can be seen that the point clouds in the high parts of the buildings are not included in the voxel group.

[0037] FIG. 6 shows a group of voxels after estimation when the specified hierarchical depth is 8 levels. The specified hierarchical depth is d=8. The example shown in FIG. 6 shows a group of voxels resulting from estimation obtained by iteratively modeling up to 8 levels. The voxel size is reduced to 1 / 64 of that of 6 levels, and the point cloud contained in the voxel approaches a plane more closely. The plane of the estimation result is depicted, and it can be seen that most of the point cloud of a large building is not contained in the voxel. In other words, the voxel contains something other than a building and represents an area close to a road. The method of this embodiment enables such object detection, i.e., model estimation, at high speed.

[0038] Figure 7 shows a comparison of the number of points to be estimated and the model estimation time between the conventional method and our method. The comparison uses the same point cloud data, but with our method, the octree layers d = 4 and d = 6 are specified for the specified layer depth d. With the conventional method, the number of points to be estimated is approximately 3 million, and the model estimation time is 25 minutes, 46 seconds, or 529 msec. In contrast, with our method, when the octree layer d = 4 is specified, the number of points to be estimated is 531, and the model estimation time is 1 msec. With our method, when the octree layer d = 6 is specified, the number of points to be estimated is 20,195, and the model estimation time is 7 msec. As shown above, our method uses an octree structure, which significantly reduces the number of points to be estimated and also speeds up the time required for model estimation.

[0039] The method of this embodiment can also accommodate more complex shapes by increasing the number of model types. For example, layers 4-6 can accommodate planar models, and layers 7-9 can accommodate cylindrical models.

[0040] Furthermore, the voxel exclusion process by the range update unit 116 can be used as a point cloud segmentation function. The excluded voxels are considered as a separate set, and hierarchical model estimation is applied to enable segmentation of a wide-area point cloud.

[0041] In the above embodiment, the estimation process executed by the CPU after reading the software (program) may be executed by various processors other than the CPU. Examples of such processors include dedicated electrical circuits, such as programmable logic devices (PLDs) (such as field-programmable gate arrays (FPGAs)) whose circuit configuration can be changed after manufacture, graphics processing units (GPUs), and application-specific integrated circuits (ASICs) that are processors with circuit configurations specifically designed to execute specific processes. The estimation process may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). The hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor devices.

[0042] In the above embodiment, the estimation program is pre-stored (installed) in the storage 14, but the present invention is not limited to this. The program may be provided in a form stored in a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The program may also be downloaded from an external device via a network.

[0043] The following additional notes are provided regarding the above-described embodiments.

[0044] (Additional note 1) Memory and at least one processor coupled to said memory; Including, The processor: Dividing the point cloud data to be modeled into voxel groups each consisting of a plurality of voxels according to a specified hierarchical depth using a predetermined spatial division method; performing model estimation using the divided voxel group, and outputting an estimation result in which the point cloud data is modeled; Using the divided voxel group and the output estimation result, a set of intersections is obtained to obtain a partial voxel group in which the processing target of the voxel group is updated; repeating the model estimation and the updating of the partial voxel group until the depth layer to be processed satisfies a predetermined condition; The estimation device is configured as follows.

[0045] (Additional note 2) A non-transitory storage medium storing a program executable by a computer to perform an estimation process, Dividing the point cloud data to be modeled into voxel groups each consisting of a plurality of voxels according to a specified hierarchical depth using a predetermined spatial division method; performing model estimation using the divided voxel group, and outputting an estimation result in which the point cloud data is modeled; Using the divided voxel group and the output estimation result, a set of intersections is obtained to obtain a partial voxel group in which the processing target of the voxel group is updated; repeating the model estimation and the updating of the partial voxel group until the depth layer to be processed satisfies a predetermined condition; Non-transitory storage medium. [Explanation of symbols]

[0046] 100 Estimator 102 Point cloud data storage unit 104 Model Memory Unit 110 Split section 112 Extraction part 114 Estimation Department 116 Range Update Section 118 Iterative Processing Unit

Claims

1. A division unit that divides point cloud data in a three-dimensional space that is the subject of modeling using the RANSAC model estimation method into voxel groups each consisting of a plurality of voxels according to the depth, using an octree space division method that recursively divides the three-dimensional space in which the point cloud data exists so that the spaces of eight child nodes belong to the space of a parent node, for each hierarchy level according to the specified hierarchical depth; an estimation unit that performs estimation by the model estimation method using the divided voxel group, and outputs an estimation result in which the point cloud data is modeled so as to draw at least a plane; a range update unit that uses the divided voxel group and the output estimation result to obtain a set of intersections and obtain a partial voxel group by updating the processing target of the voxel group; an iterative processing unit that repeats the model estimation and the update of the partial voxel group until the depth layer to be processed satisfies a predetermined condition; An estimation device comprising:

2. an extraction unit that extracts a set of center points by thinning the point cloud by extracting a center point from each of the divided voxels of the voxel group; the estimation unit performs model estimation using the set corresponding to the voxel group and outputs the estimation result; The estimation device according to claim 1 , wherein the iterative processing unit repeats the extraction, the model estimation, and the update of the voxel group until the depth specification satisfies a predetermined condition.

3. Point cloud data in a three-dimensional space to be modeled using the RANSAC model estimation method is divided into voxel groups each consisting of multiple voxels according to the depth, using a space division method using an octree that recursively divides the three-dimensional space in which the point cloud data exists so that the space of eight child nodes belongs to the space of a parent node, for each hierarchy level according to the specified hierarchical depth; performing estimation by the model estimation method using the divided voxel group, and outputting an estimation result in which the point cloud data is modeled so as to draw at least a plane; Using the divided voxel group and the output estimation result, a set of intersections is obtained to obtain a partial voxel group in which the processing target of the voxel group is updated; repeating the model estimation and the updating of the partial voxel group until the depth layer to be processed satisfies a predetermined condition; A method of estimating that the processing is performed by a computer.

4. Point cloud data in a three-dimensional space to be modeled using the RANSAC model estimation method is divided into voxel groups each consisting of a plurality of voxels according to the depth, using a space division method using an octree that recursively divides the three-dimensional space in which the point cloud data exists so that the space of eight child nodes belongs to the space of a parent node, for each hierarchy level according to the depth of the specified hierarchy; performing estimation by the model estimation method using the divided voxel group, and outputting an estimation result in which the point cloud data is modeled so as to draw at least a plane; Using the divided voxel group and the output estimation result, a set of intersections is obtained to obtain a partial voxel group in which the processing target of the voxel group is updated; repeating the model estimation and the updating of the partial voxel group until the depth layer to be processed satisfies a predetermined condition; An estimation program that causes a computer to execute the process.

Images