Data structure, estimation device, and method

Abstract

One aspect of a data structure according to the present invention comprises a plurality of basic voxel maps that are configured for each hierarchy obtained by multi-component tree structuring of point group data to be modeled, and configured for each region obtained by dividing a plane on which each voxel including the point group data is projected. Each of the plurality of basic voxel maps includes: a first voxel set intersecting a plane estimated from partial point group data corresponding to a region in each layer; and a neighbor link indicating a second voxel set in another basic voxel map configured in a neighboring region of the same layer. The data structure is used for comparison processing for comparing the first voxel set in a certain basic voxel map to the second voxel set indicated by a neighbor link, and integration processing for integrating the first voxel set and the second voxel set when the result of the comparison processing is a match.

Description

Data structure, estimation device and method

[0001] One aspect of the present invention relates to a data structure, an estimation apparatus and a method.

[0002] In recent years, 3D point cloud collection systems using mobile objects such as cars and drones equipped with measurement devices such as LiDAR (Light Detection and Ranging) have been developed. 3D point cloud data is increasingly being applied to real-world digital twins, urban development, infrastructure maintenance, and natural disaster prevention measures. However, 3D point cloud data imposes a heavy processing load due to its enormous data size. For example, when processing 3D point cloud data for visualization or structural analysis, the amount of calculation required is large, so there is a demand for faster processing.

[0003] A common approach to speeding up the process is to use a model fitting algorithm called Random Sample Consensus (RANSAC), which can estimate shapes using point cloud data extracted from 3D point cloud data (see, for example, Non-Patent Document 1). Also, the Point Cloud Library (PCL), which implements RANSAC for 3D point clouds, is widely known (see, for example, Non-Patent Document 2).

[0004] On the other hand, there is known an estimation method for quickly estimating a detailed shape model from 3D point cloud data based on RANSAC and PCL (see, for example, Non-Patent Document 3). This estimation method quickly estimates a detailed shape model, which is a voxel set with a resolution corresponding to the hierarchical level, by structuring the 3D point cloud data into an octree, as shown in Fig. 12, and applying RANSAC hierarchically to the resulting voxel set. Figs. 13 and 14 are diagrams showing the initial octree-structured level and the plane estimation results, respectively.

[0005] Furthermore, this estimation method extends the estimation of a uniform plane model to the estimation of a plurality of divided plane models, and by integrating the plurality of plane models, it is possible to estimate even undulating shapes. Specifically, a voxel map is used to divide the estimation range of the point cloud. For example, as shown in FIG. 15, a voxel map M ｎis generated by mapping the center coordinates of each 3D voxel shown in FIG. 14 onto a 2D plane. This voxel map M ｎ As shown in Fig. 16, the point cloud data can be divided into multiple ranges by dividing the image into regions selected in frame FL. By applying RANSAC independently and hierarchically to the point cloud data for each divided region, a planar model, which is a set of voxels for each layer, is estimated for each region.

[0006] Then, planar models (voxel sets) of adjacent regions are searched and compared from the planar models corresponding to the region and layer pair, and adjacent planar models of the appropriate layer are integrated to approximately represent the undulating shape.

[0007] M. Fischler and R. Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, Vol. 24, No. 6, pp. 381-395, 1981. Radu Bogdan Rusu and Steve Cousins. 3D is here: Point Cloud Library (PCL). In IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China, May 9-13, 2011. IEEE. Xiaojun Wu, Ryuichi Tanida, and Jun Shimamura. Interactive visualization of large-scale 3D point cloud data based on hierarchical structures. ITE Technical Report, Vol. 47, No. 9, pp. 43-46, March 2023.

[0008] Although there are no particular problems with the estimation method described above, the inventors' investigations have revealed that there is room for improvement in that the search process for searching for adjacent voxel sets in an appropriate hierarchy from the enormous number of voxel sets corresponding to the combination of region and hierarchy is heavy.

[0009] The present invention has been made in consideration of the above circumstances, and provides a technique that can reduce the load of the process of searching for a voxel set for each region.

[0010] To solve the above problems, one aspect of a data structure according to the present invention is a hierarchical voxel map data structure used in a computer having a processing unit and a storage unit, and stored in the storage unit. The data structure includes a plurality of basic voxel maps, each set for a hierarchy obtained by structuring point cloud data to be modeled into a multilevel tree, and each set for a region obtained by dividing a plane onto which each voxel including the point cloud data is projected. Each of the plurality of basic voxel maps includes, for each hierarchy, a first voxel set that intersects with a plane estimated from partial point cloud data corresponding to the region, and an adjacent link indicating a second voxel set in another basic voxel map set in an adjacent region at the same hierarchy. The data structure is used for comparison processing and integration processing. The comparison processing is a processing in which the processing unit compares a first voxel set in a certain basic voxel map among the plurality of basic voxel maps with a second voxel set indicated by an adjacent link in the certain basic voxel map. The integration process is a process in which the processing unit integrates the first voxel set and the second voxel set when the results of the comparison process are consistent.

[0011] According to one aspect of the present invention, it is possible to easily compare a first voxel set in a certain layer with a second voxel set in an adjacent region in the same layer using an adjacent link, thereby reducing the load of searching voxel sets for each region.

[0012] That is, according to one aspect of the present invention, it is possible to reduce the load of the process of searching for a voxel set for each region.

[0013] FIG. 1 is a block diagram showing an example of the configuration of an estimation device according to an embodiment of the present invention. FIG. 2 is a block diagram showing an example of the configuration of a processing unit shown in FIG. 1. FIG. 3 is a schematic diagram for explaining a hierarchical voxel map according to an embodiment. FIG. 4 is a schematic diagram for explaining links at the same level in the hierarchical voxel map of FIG. 3. FIG. 5 is a schematic diagram for explaining links at the same level and adjacent levels in the hierarchical voxel map of FIG. 3. FIG. 6 is a flowchart for explaining an example of operation in an embodiment. FIG. 7 is a schematic diagram for explaining an example of operation in an embodiment. FIG. 8 is a schematic diagram for explaining an example of operation in an embodiment. FIG. 9 is a flowchart for explaining an example of operation in step ST50 of FIG. 6. FIG. 10 is a flowchart for explaining an example of operation in step ST56 of FIG. 9. FIG. 11 is a flowchart for explaining an example of operation in step ST60 of FIG. 6. FIG. 12 is a diagram showing an example of a general 3D point cloud. FIG. 13 is a diagram showing an initial level when the 3D point cloud of FIG. 12 is structured as an octree. Fig. 14 is a diagram showing the result of plane estimation when RANSAC is applied hierarchically from the initial layer in Fig. 13. Fig. 15 is a diagram showing an example of a voxel map mapped onto a 2D plane from a set of voxels in the result of plane estimation in Fig. 14. Fig. 16 is a diagram showing an example of an operation of dividing the voxel map in Fig. 15.

[0014] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the term "estimation device" may be appropriately replaced with any other term such as an image processing device or an information processing device. Similarly, the term "voxel set" may be appropriately replaced with "voxel group." In other words, terms with the same meaning may be appropriately replaced.

[0015] 1 is a block diagram showing an example of the configuration of an estimation device according to an embodiment of the present invention. The estimation device 100 is, for example, a personal computer. Note that the estimation device 100 may also be configured as a server computer installed on the web or in the cloud.

[0016] The estimating device 100 includes a processing unit 110 that uses a hardware processor such as a central processing unit (CPU), and is connected to the processing unit 110 via a bus 150 with a storage unit having a program storage unit 120 and a data storage unit 130, and an input / output interface (hereinafter, interface will be abbreviated as I / F) unit 140. The estimating device 100 is an example of a computer that includes a processing unit and a storage unit.

[0017] The input / output I / F unit 140 is connected to the input device 102 and the display device 104 .

[0018] The input device 102 is composed of, for example, a keyboard and a mouse, and is used by the user to input information specifying the 3D point cloud data that the user wishes to acquire, commands for instructing the estimation apparatus 100 to perform various operations, and the like.

[0019] The display device 104 is used to display information indicating various operating states of the estimation apparatus 100 and information necessary for input operations, as well as to appropriately display point cloud data and images based on a voxel set. The display device 104 is an example of a display.

[0020] The input / output I / F unit 140 may include a communication interface. By including the communication interface, even if a data transmission device capable of transmitting 3D point cloud data, such as a data storage device that stores 3D point cloud data or a measurement device that measures 3D point cloud data, is located far away, the 3D point cloud data transmitted from the data transmission device can be acquired via a network.

[0021] 2, the processing unit 110 includes a preprocessing unit 111, a generating unit 112, a dividing unit 113, and an estimating unit 114. The processing unit 110 is not limited to the processes of the above-mentioned units 111 to 114, and can execute any process in response to an operation by an operator.

[0022] Each of the above units 111 to 114 is realized by causing a hardware processor of the processing unit 110 to execute an application program stored in the program storage unit 120. Note that some or all of the above units 111 to 114 may be realized using hardware such as an LSI (Large Scale Integration) or an ASIC (Application Specific Integrated Circuit).

[0023] For example, the pre-processing unit 111 sets an initial layer of an octree structure for the point cloud data to be modeled, sets a voxel set corresponding to the point cloud data of that layer, and estimates a plane of that layer from the voxel set.

[0024] The generator 112 generates a voxel map by, for example, projecting the central coordinates of each voxel in the voxel set onto the plane.

[0025] The dividing unit 113 obtains partial point cloud data corresponding to regions obtained by dividing the voxel map, for example, by spatially dividing the point cloud data.

[0026] The estimation unit 114 estimates point cloud data of a deeper hierarchy based on a voxel set that intersects with a plane estimated from a voxel set corresponding to the partial point cloud data, for example, for each region. The estimation unit 114 also performs processes using data structures, such as comparison processes and integration processes, which will be described later.

[0027] The program storage unit 120 is configured, for example, by combining a nonvolatile memory such as a solid-state drive (SSD) that can be written and read as needed with a nonvolatile memory such as a read-only memory (ROM) as a storage medium. It stores middleware such as an operating system (OS) as well as application programs necessary for executing various control operations according to an embodiment. Hereinafter, the OS and each application program will be collectively referred to as a program. The program may be installed on the computer in advance from a network or a non-transitory computer-readable storage medium, or may be pre-recorded on the computer. In either case, the program is executed by the processor to cause the computer to function as the estimation device 100. The processor may also be referred to as a processing circuit.

[0028] The data storage unit 130 is, for example, a combination of a nonvolatile memory, such as an SSD, that can be written and read at any time, and a volatile memory, such as a random access memory (RAM). The data storage unit 130 stores data such as 3D point cloud data and the data structure of a hierarchical voxel map 200 established from the 3D point cloud data, as shown in FIG. 3 . The data stored in the data storage unit 130 is used by the computer functioning as the estimation apparatus 100. The 3D point cloud data and the hierarchical voxel map 200 may be acquired from an external storage device, or may be acquired via a network from another terminal device, a server computer on the web, or a cloud. Similarly, the 3D point cloud data and the data structure of the hierarchical voxel map 200 may be acquired by the computer in advance from a network or a non-transitory computer-readable storage medium, or may be recorded in advance in the computer. The data storage unit 130 is an example of a storage unit.

[0029] Here, the data structure of the hierarchical voxel map 200 comprises a plurality of basic voxel maps 201, which are set for each hierarchy obtained by structuring the point cloud data to be modeled into an octree structure, and which are set for each area obtained by dividing a plane onto which each voxel containing the point cloud data is projected.

[0030] Each of the plurality of basic voxel maps 201 is a first voxel set V that intersects with a plane estimated from partial point cloud data corresponding to a region for each layer. ｉ ｎ and a second voxel set V in another basic voxel map 201 set in an adjacent region of the same layer. Ｕ ｎ , V Ｄ ｎ , V Ｌ ｎ , V Ｒ ｎ The neighboring links indicate the second voxel set V Ｕ ｎ , V Ｄ ｎ , V Ｌ ｎ , V Ｒ ｎ The basic voxel map 201 is information that can individually identify the first voxel set V, for example, a file name or a voxel set ID. ｉ ｎ Image files that use voxel data such as these as image data and include link information such as adjacent links in the accompanying information can be used as appropriate. In addition to adjacent links, hierarchical links and split source links, which will be described later, can also be used as appropriate as link information. Image data may be called entity data. The accompanying information may be called metadata.

[0031] The superscript n of a voxel set represents the layer n. ｉ ｎ The subscript i in ｎ is divided into k regions, and represents the i-th region i (i=1, 2, ..., k). Ｕ ｎ , V Ｄ ｎ , VＬ ｎ , V Ｒ ｎ The subscripts U, D, L, and R in ｉ ｎ Of the four adjacent regions on the plane, the regions in the up (U), down (D), left (L), and right (R) directions represent the regions in the +y (U), -y (D), -x (L), and +x (R) directions on the xy plane. For example, as shown in Figures 3 and 4, the adjacent link makes it possible to search for voxel sets of adjacent regions U, D, L, and R on the same layer n. In Figure 4, the first voxel set V ｉ ｎ V ６ ｎ , the second adjacent voxel set V Ｕ ｎ , V Ｄ ｎ , V Ｌ ｎ , V Ｒ ｎ are V １０ ｎ , V ２ ｎ , V ５ ｎ , V ７ ｎ In addition, although k=16 in FIG. 4, the value of k is not limited to this.

[0032] In addition, each of the plurality of basic voxel maps 201 is a third voxel set V ｉ ｎ－１ , V ｉ ｎ＋１ Among the hierarchical links of the layer n, the hierarchical link to the adjacent layer (n-1) may be called a parent hierarchical link. Similarly, among the hierarchical links of the layer n, the hierarchical link to the adjacent layer (n+1) may be called a child hierarchical link. The hierarchical link is a link to the third voxel set V ｉ ｎ－１ , V ｉ ｎ＋１The hierarchical link is information that can individually identify each voxel set, for example, a file name or a voxel set ID. As shown in Figures 3 and 5, the hierarchical link makes it possible to search for voxel sets in adjacent layers (n-1, n+1) of the same region i. In Figure 5, the first voxel set V ｉ ｎ V ２ ｎ In this case, the third voxel set V ｉ ｎ－１ , V ｉ ｎ＋１ are V ２ ｎ－１ , V ２ ｎ＋１ This becomes:

[0033] Each of the plurality of basic voxel maps is a voxel set V ｎ The division source link may further include a division source link indicating the division source voxel set V ｎ The information is information that can identify the voxel set V, for example, a file name or a voxel set ID. ｎ When is divided into k regions i, the voxel set of each region i is V １ ｎ , V ２ ｎ , ..., V ｋ ｎ The voxel set V １ ｎ , V ２ ｎ , ..., V ｋ ｎ is the voxel set V ｉ ｎ (i=1, 2, ..., k). Also, the voxel set V ｎ is the voxel set {V ｎ} may also be expressed as

[0034] Each of the plurality of basic voxel maps 201 is a voxel set V ｉ ｎ The parameter may further include a parameter (not shown) corresponding to the voxel set V ｉ ｎmay be a plane parameter representing the plane on which the voxel set V ｉ ｎ Alternatively, the parameter may be a normal vector to a plane where the voxel set intersects with the voxel set. However, since the voxel set does not necessarily represent a plane, the parameter is not limited to a plane parameter, a normal vector, or the like. For example, the parameter may be a geometric parameter related to the geometric shape represented by the voxel set, or an evaluation parameter related to the number of points included in the voxel set. The evaluation parameter indicates that the greater the number of points, the higher the reliability of the result (e.g., a plane) estimated from the point cloud. Note that the plane parameter is expressed based on a plane equation representing the plane where the voxel set intersects, i.e., a plane model modeled from point cloud data. Here, if the plane equation is ax+by+cz+d=0, then the plane parameters can be, for example, {a, b, c, d} defining the plane equation. The plane normal vector can be, for example, the coefficients (a, b, c) of the plane equation.

[0035] Here, the data structure is used for, for example, a comparison process and an integration process. The comparison process is performed by comparing a first voxel set V ｉ ｎ and a second voxel set V indicated by an adjacent link in a certain basic voxel map 201. Ｕ ｎ , V Ｄ ｎ , V Ｌ ｎ , or V Ｒ ｎ The processing unit 110 compares the first voxel set V ｉ ｎ The first parameter (e.g., normal vector) corresponding to the second voxel set V Ｕ ｎ , V Ｄ ｎ , V Ｌ ｎ , or V Ｒ ｎThe processing unit 110 may calculate a difference between the first voxel set V and a second parameter (e.g., a normal vector) corresponding to the first voxel set V and compare the difference with a threshold value. ｉ ｎ and the second voxel set V Ｕ ｎ , V Ｄ ｎ , V Ｌ ｎ , or V Ｒ ｎ The processing unit 110 integrates the above.

[0036] Furthermore, the data structure may appropriately use, for example, a search process and a division process. In the search process, when the result of the comparison process is inconsistent, the third voxel set V indicated by the hierarchical link in another basic voxel map 201 is divided into two. ｉ ｎ－１ In the division process, if the results of the comparison process are inconsistent, the processing unit 110 divides the first voxel set V ｉ ｎ is used as the division source and divided into multiple regions, and the first voxel set V ｉ ｎ This is the process of setting the original link to the split link.

[0037] Next, an example of the operation of the estimation device configured as described above will be explained using Figures 6 to 11. The following explanation will be given in the order of (1) segmentation estimation algorithm (Figures 6 to 8), (2) hierarchical RANSAC (Figure 9), (3) creation of basic voxel map (Figure 10), and (4) processing using the data structure of the hierarchical voxel map (Figure 11).

[0038] (1) Segmentation Estimation Algorithm As shown in FIG. 6, the processing unit 110 sets an initial layer n of the octree structure for the point cloud data to be modeled, and calculates the point cloud data {W ｎ}, the voxel set {V ｎ} is set (step ST10). ｎ} is the point cloud data W ｎHere, we will provide a supplementary explanation of the octree structure. As shown on the left side of Figure 7, the three-dimensional space is divided into a set of voxels, for example, one voxel V ｎ－１ , and divide it equally into eight voxels. ｎ By repeating this process, the three-dimensional space is divided into multiple voxels V ｎ－１ ~V ｎ＋２ The point cloud data {W ｎ－１} is structured as an octree, as shown on the right side of FIG. ｎ－１ ~V ｎ＋２ For the point W ｎ－１ ~W ｎ＋２ Depending on the presence or absence of ｎ－１ ~V ｎ＋２ That is, the point cloud data {W ｎ－１}, {W ｎ}, {W ｎ＋１}, {W ｎ＋２} into a multi-level voxel set {V ｎ－１}, {V ｎ}, {V ｎ＋１}, {V ｎ＋２}. In FIG. 7, the voxel set {V ｎ－１}, {V ｎ}, {V ｎ＋１}, {V ｎ＋２ The number of elements in the voxel set {V ｎ－１}, {V ｎ}, {V ｎ＋１}, {V ｎ＋２ The number of elements in the point cloud data W ｎ－１ , W ｎ , W ｎ＋１ , W ｎ＋２ Voxel V containing ｎ－１ , V ｎ , V ｎ＋１ , V ｎ＋２ In an octree structure, the deeper the layer, the finer the voxels become, and the higher the resolution. For example, shallower layers use fewer voxels to represent coarse point cloud data, while deeper layers use more voxels to represent more detailed point cloud data.

[0039] Returning to FIG. 6, after step ST10, the processing unit 110 ｎ} to the plane P ｎ Specifically, the processing unit 110 estimates the voxel set {V ｎ}, the plane P ｎ (Step ST20). RANSAC is a model fitting algorithm that can estimate a shape using point cloud data randomly extracted from 3D point cloud data. That is, in step ST20, a voxel set {V ｎ} represents the plane P ｎ The shape of the

[0040] After step ST20, the processing unit 110 calculates the voxel set {V ｎ Each voxel V ｎ The center coordinates of the plane P ｎ By projecting onto each voxel V ｎ A voxel map Mn, which is a 2D map of the above, is generated (step ST30).

[0041] After step ST30, the processing unit 110 ｎ Specifically, the processing unit 110 obtains partial point cloud data corresponding to the divided regions of the voxel map M by spatially dividing the point cloud data. ｎ is divided into partial images {r ｎｉ} corresponding to the subgroup of points {w ｎｉ} is calculated (step ST40). ｎ is divided into 16 regions of 4 rows and 4 columns, and 16 partial images r ｎ１ ~r ｎ１６ Each partial image r ｎ１ ~r ｎ１６ is the set of points w ｎ１ ~w ｎ１６ Corresponds to.

[0042] Returning to FIG. 6, after step ST40, the processing unit 110 calculates a plane P estimated from the partial point cloud data for each region. ｎSpecifically, the processing unit 110 estimates point cloud data of a layer n+1, which is a layer n deeper than the layer n, based on a set of voxels that intersect with ｎｉ} (step ST50). The details of step ST50 will be described later.

[0043] After step ST50, the processing unit 110 integrates the results of step ST50 and outputs the obtained result (step ST60). The details of step ST60 will be described later.

[0044] (2) Hierarchical RANSAC In (2), details of step ST50 will be described with reference to Fig. 9. This step ST50 includes steps ST51 to ST58. These will be described in order below.

[0045] The processing unit 110 calculates the partial point group {w ｎｉ}, the corresponding voxel sub-set {V ｎｉ} is set (step ST51). ｎｉ} is the partial point cloud data w ｎｉ is the set of voxels that contains

[0046] After step ST51, the processing unit 110 calculates the partial voxel set {V ｎｉ}, and perform RANSAC on the domain plane P ｎｉ is estimated (step ST52).

[0047] After step ST52, the processing unit 110 calculates the partial voxel set {V ｎｉ}, the domain plane P ｎｉ The set of voxels {C ｎ} is obtained (step ST53).

[0048] After step ST53, the processing unit 110 calculates the voxel set {C ｎ} is divided into a voxel set {C ｎ＋１} is obtained (step ST54).

[0049] After step ST54, the processing unit 110 calculates the voxel set {C ｎ＋１}, the domain plane P ｎｉ The subgroup of points {w（ｎ＋１）ｉ} is obtained (step ST55).

[0050] After step ST55, the processing unit 110 creates a basic voxel map 201 from the hierarchical voxel map 200 (step ST56). The details of step ST56 will be described later.

[0051] After step ST56, the processing unit 110 determines whether the depth of the layer satisfies the condition (specified number of times) (step ST57). If the result of the determination in step ST57 is no, the layer n is updated to n+1, and the processing of steps ST51 to ST57 is repeated.

[0052] On the other hand, if the result of the determination in step ST57 is that the condition is satisfied, the processing unit 110 ｎｉ If the result of the determination in step ST58 is NO, the area i is updated to i+1 and the processes in steps ST51 to ST58 are repeated.

[0053] On the other hand, if the result of the determination in step ST58 is that the process has been executed, the processing unit 110 ends step ST50 and proceeds to step ST60.

[0054] (3) Creation of Basic Voxel Map In (3), details of step ST56 will be described using FIG. 10. This step ST56 includes steps ST56-1 to ST56-5. Note that the execution order of steps ST56-2 to ST56-5 may be changed as appropriate. The following describes each step in order.

[0055] The processing unit 110 calculates a voxel set {C ｎ} on the plane P ｎ Voxel set V ｉ ｎ (step ST56-1).

[0056] After step ST56-1, the processing unit 110 sets the set voxel set V ｉ ｎ(Step ST56-2). The parameters are set for the voxel set V ｉ ｎ The plane P where ｎ is the normal vector to

[0057] After step ST56-2, the processing unit 110 divides the original voxel set V ｎ About V ｉ ｎ The division source link is set (step ST56-3).

[0058] After step ST56-3, the processing unit 110 calculates the adjacent voxel set V Ｕ ｎ , V Ｄ ｎ , V Ｌ ｎ , V Ｒ ｎ (Step ST56-4). ｉ ｎ If V is at one of the four corners, there are two adjacent voxel sets. ｉ ｎ If is one of the four sides, there are three adjacent voxel sets.

[0059] After step ST56-4, the processing unit 110 calculates the voxel set V ｉ ｎ＋１ (Step ST56-5) In this way, the basic voxel map 201 for the layer n and the region i is created and stored in the data storage unit 130, as shown in FIG.

[0060] After step ST56-5, the processing section 110 ends step ST56 and proceeds to step ST57.

[0061] (4) Processing Using Hierarchical Voxel Map Data Structure In (4), step ST60 will be described in detail with reference to Fig. 11. This step ST60 includes steps ST61 to ST67. These will be described in order below.

[0062] The processing unit 110 reads out each basic voxel map 201 of layer n from the data storage unit 130 (step ST61). Generally, the appropriate layer n is known based on the measurement target of the 3D point cloud data and the user's experience, and is often specified by the user. Step ST61 is based on the assumption that the appropriate layer n is known. If the appropriate layer n is unknown, it is necessary to search for the appropriate layer n using hierarchical links and adjacent links.

[0063] After step ST61, the processing unit 110 calculates the first voxel set V in a certain basic voxel map 201 among the plurality of basic voxel maps. ｉ ｎ and a second voxel set V indicated by an adjacent link in a certain basic voxel map 201. Ｕ ｎ , V Ｄ ｎ , V Ｌ ｎ , or V Ｒ ｎ For example, the processing unit 110 compares the first voxel set V ｉ ｎ The first normal vector corresponding to the plane where the two voxel sets V intersect is Ｕ ｎ The angle formed with the second parameter normal vector corresponding to the plane on which the vector intersects is calculated, and the angle formed with the second parameter normal vector is compared with a threshold value.

[0064] After step ST62, the processing unit 110 determines whether the comparison result is a match (step ST63). For example, the processing unit 110 determines that the two angles match if the angle is equal to or smaller than a threshold value.

[0065] If the comparison result in step ST63 is not consistent, the processing unit 110 executes an alternative process (step ST64). As the alternative process, for example, a search process and a division process can be used as appropriate. The search process is performed by dividing the third voxel set V indicated by the hierarchical link in another basic voxel map 201. ｉ ｎ－１ The search process described here is a process of searching for two voxel sets V ｉｎ , V Ｕ ｎ If the voxel set V ｉ ｎ－１ However, the search process is not limited to this. ｉ ｎ The voxel set V of multiple regions is divided into ｇ ｎ (g=1, 2, ..., h, 2≦h≦k) and divide the first voxel set V ｉ ｎ The division process is a process of setting the original link to the original link. ｉ ｎ , V Ｕ ｎ In this case, when the voxel set V is not consistent, it is possible to generate a voxel set of a narrower divided region. ｉ ｎ If the number of points included in is less than a reference value, the search process may be performed from the viewpoint of increasing the number of points and increasing the reliability, and in other cases, the division process may be performed. However, the use of both processes is not limited to this.

[0066] On the other hand, if the comparison result is a match as a result of the determination in step ST63, the processing unit 110 ｉ ｎ and the second voxel set V Ｕ ｎ , V Ｄ ｎ , V Ｌ ｎ , or V Ｒ ｎ are integrated (step ST65).

[0067] After step ST65, the processing unit 110 calculates the voxel set V ｉ ｎIt is determined whether or not integration of (i=1, 2, ..., k) has been completed (step ST66). If the result of the determination in step ST66 is NO, the processing unit 110 returns to step ST62 and repeats the processes of steps ST62 to ST66 for the remaining basic voxel maps 201.

[0068] On the other hand, if the result of the determination in step ST66 is that the process has ended, the processing unit 110 ｉ ｎ (i=1, 2, . . . , k) is integrated and output (step ST67), and the process ends.

[0069] As described above, according to one embodiment, the data structure of the hierarchical voxel map 200 is used in the estimation device 100 including the processing unit 110 and the data storage unit 130, and is stored in the data storage unit 130. The data structure includes a plurality of basic voxel maps 201, which are set for each layer obtained by structuring the point cloud data to be modeled into an octree structure, and which are set for each region obtained by dividing a plane onto which each voxel including the point cloud data is projected. Each of the plurality of basic voxel maps 201 is a first voxel set V that intersects with a plane estimated from partial point cloud data corresponding to a region for each layer. ｉ ｎ and a second voxel set V in another basic voxel map set in an adjacent region of the same layer. Ｕ ｎ , V Ｄ ｎ , V Ｌ ｎ , V Ｒ ｎ The data structure is used for comparison and integration. The comparison is performed by comparing a first voxel set V in a base voxel map 201 among the plurality of base voxel maps 201. ｉ ｎ and a second voxel set V indicated by an adjacent link in a certain basic voxel map 201. Ｕ ｎ , V Ｄ ｎ , V Ｌ ｎ , V Ｒ ｎThe processing unit 110 compares the first voxel set V ｉ ｎ and the second voxel set V Ｕ ｎ , V Ｄ ｎ , V Ｌ ｎ , V Ｒ ｎ This is a process in which the processing unit 110 integrates the first voxel set V ｉ ｎ and the second voxel set V of the adjacent regions U, D, L, and R of the same layer n. Ｕ ｎ , V Ｄ ｎ , V Ｌ ｎ , V Ｒ ｎ Therefore, it is possible to easily compare the voxel sets in each region using adjacent links, thereby reducing the load of the process of searching for the voxel sets in each region.

[0070] Additionally, in the conventional technology, a huge number of voxel sets corresponding to pairs of regions and layers are stored in memory without being associated with each other. That is, in the conventional technology, a huge number of voxel sets are stored without any of the adjacent links, parent layer links, child layer links, or split source links shown in FIG. 3 . On the other hand, in the conventional technology, to meaningfully integrate these voxel sets, it is necessary to compare the planar models of each adjacently related voxel set. However, in the conventional technology, there is no explicit description of the adjacent relationship, and to compare the models, the CPU must search for adjacent voxel sets at the appropriate layer from the huge number of voxel sets. Therefore, in the conventional technology, a complex search process is required, which incurs a huge processing cost and places a heavy load on the CPU. In contrast, according to one embodiment, the processing unit 110 can call adjacent voxel sets at the same layer using the adjacent links, thereby reducing the load on the CPU's search process.

[0071] According to one embodiment, each of the plurality of basic voxel maps 201 is a third voxel set V in the basic voxel map 201 of the adjacent layer set in the adjacent layer (n-1, n+1) of the same region i. ｉ ｎ－１ , V ｉ ｎ＋１ If the comparison result is not consistent, the data structure may further include a hierarchical link indicating the third voxel set V indicated by the hierarchical link in the other basic voxel map 201. ｉ ｎ－１ , V ｉ ｎ＋１ In this case, in addition to the above-described effects, if the results of the comparison process are inconsistent, it is possible to search for a voxel set in an adjacent layer.

[0072] According to one embodiment, each of the plurality of base voxel maps 201 is a voxel set V ｎ The data structure may further include a split source link indicating the first voxel set V ｉ ｎ is used as the division source and divided into multiple regions, and the first voxel set V ｉ ｎ In this case, in addition to the above-mentioned effect, when the results of the comparison process are inconsistent, the voxel set V ｉ ｎ can be further divided.

[0073] Furthermore, according to one embodiment, the hierarchical voxel map 200 explicitly describes adjacent links, hierarchical links, and split source links to related and adjacent voxel sets, thereby enabling the direct execution of the following processes (a) to (e), thereby enabling efficient detailed model estimation.

[0074] (a) Comparison of estimation results of adjacent regions (processing related to adjacent links) (b) Control of division of child layers for each region (processing related to division source links) (c) Integration of multiple regions (processing related to adjacent links) (d) Search for adjacent regions across layers (processing related to hierarchical links and adjacent links) (e) Point cloud division across layers (processing related to hierarchical links) Furthermore, according to one embodiment, each of the multiple basic voxel maps 201 is a voxel set V ｉ ｎ The parameters may further include parameters corresponding to the voxel set V ｉ ｎ may be a plane parameter representing the plane on which the voxel set V ｉ ｎ The comparison process may be performed by using the normal vector of the first voxel set V ｉ ｎ The first parameter and the second voxel set V Ｕ ｎ , V Ｄ ｎ , V Ｌ ｎ , or V Ｒ ｎ In this case, by comparing the parameters, the comparison process can be performed more easily than when comparing the voxel sets directly.

[0075] Other Embodiments The functional configuration of the estimation device 100, its processing procedures and processing contents, the types and uses of data, and the like can be modified in various ways without departing from the spirit of the present invention.

[0076] For example, in one embodiment, an octree structure is used as a method for structuring point cloud data, but this is not limiting. In other words, any multi-tree structure that structures point cloud data with two or more branches can be used as appropriate.

[0077] In one embodiment, for example, a voxel map M ｎHowever, the present invention can also be applied to other three-dimensional models. For example, in the case of a cylindrical model, the voxel map M is created on a two-dimensional plane obtained by developing the cylindrical model. ｎ For other models, a planar expansion may be performed using the texture mapping technique used in the field of computer graphics (CG).

[0078] Also for example, in one embodiment, the voxel map M ｎ However, the number of divided regions is not limited to 16. For example, the number of divided regions may be other than 16.

[0079] In addition, for example, in the embodiment, the basic voxel map 201 is created in step ST56, but this is not limiting. ｎ Based on the region i into which the image is divided, the layer n, and the designated number of times of hierarchical RANSAC (depth of layer n), a template of each basic voxel map 201 is created. In step ST53, the voxel set {C ｎ}, the voxel set {C ｎ} is added to the template of the basic voxel map 201 of the region i and the layer n as a voxel set V ｉ ｎ That is, any executable procedure can be used as the procedure for creating the basic voxel map 201.

[0080] For example, in one embodiment, step ST61 is performed to read each basic voxel map 201 on layer n. However, this is not limiting. For example, a certain basic voxel map 201 on layer n may be read in step ST61, and a second voxel set indicated by an adjacent link in the certain basic voxel map 201 may be read in step ST62. That is, the basic voxel maps 201 may be read sequentially. Alternatively, a number of basic voxel maps 201 not exceeding the capacity of RAM may be read in step ST61, and after the integration of the voxel sets in the read basic voxel maps 201 is completed, each of the other basic voxel maps 201 may be read. That is, the reading procedure for the basic voxel maps 201 may be a combination of batch reading and sequential reading. In this way, any feasible procedure can be used to read the basic voxel maps 201.

[0081] In addition, for example, in one embodiment, step ST61 is performed to read each basic voxel map 201 of layer n, but this is not limiting. For example, after reading one basic voxel map 201 of an arbitrary layer, the hierarchical links in the read basic voxel map 201 may be traced in order to read the basic voxel map 201 of the desired layer n.

[0082] Although the embodiments of the present invention have been described in detail above, the above description is merely an example of the present invention in every respect. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. In other words, when implementing the present invention, specific configurations according to the embodiments may be appropriately adopted.

[0083] In short, this invention is not limited to the above-described embodiments, and in the implementation stage, the components can be modified and embodied without departing from the spirit of the invention. Furthermore, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be omitted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined.

[0084] REFERENCE SIGNS LIST 100... Estimation device 102... Input device 104... Display device 110... Processing unit 111... Preprocessing unit 112... Generation unit 113... Division unit 114... Estimation unit 120... Program storage unit 130... Data storage unit 140... Input / output I / F unit 150... Bus 200... Hierarchical voxel map 201... Basic voxel map M ｎ …Voxel map r ｎ１ ~r ｎ１６ ...Partial image V ｎ－１ ~V ｎ＋２ …Voxel W ｎ－１ ~W ｎ＋２ …point cloud

Claims

1. A data structure of a hierarchical voxel map used in a computer having a processing unit and a storage unit, and stored in the storage unit, comprising a plurality of basic voxel maps set for each level obtained by structuring point cloud data to be modeled into a multilevel tree, and set for each area obtained by dividing a plane onto which each voxel containing the point cloud data is projected, wherein each of the plurality of basic voxel maps includes: a first voxel set that intersects with a plane estimated from partial point cloud data corresponding to the area for each level; and an adjacent link indicating a second voxel set in another basic voxel map set in an adjacent area at the same level, the data structure being used for a comparison process in which the processing unit compares a first voxel set in a basic voxel map of the plurality of basic voxel maps with a second voxel set indicated by an adjacent link in the basic voxel map; and an integration process in which the processing unit integrates the first voxel set and the second voxel set if the results of the comparison process are consistent.

2. The data structure according to claim 1, wherein each of the plurality of basic voxel maps further includes a hierarchical link indicating a third voxel set in a basic voxel map of an adjacent layer set in an adjacent layer of the same region, and when the results of the comparison process are inconsistent, the hierarchical link is used in a search process in which the processing unit searches for the third voxel set indicated by the hierarchical link in the other basic voxel map.

3. The data structure according to claim 1, wherein each of the plurality of basic voxel maps further includes a division source link indicating a voxel set to be divided, and when the results of the comparison process are inconsistent, the data structure is used in a division process in which the processing unit divides the first voxel set into a plurality of regions using the first voxel set as the division source link.

4. A preprocessing unit that sets an initial layer of a multi-layer tree structure for point cloud data to be modeled, sets a voxel set corresponding to the point cloud data of the layer, and estimates a plane of the layer from the voxel set; a generation unit that generates a voxel map by projecting the center coordinates of each voxel of the voxel set onto the plane; a division unit that obtains partial point cloud data corresponding to regions obtained by dividing the voxel map by spatial division of the point cloud data; an estimation unit that estimates point cloud data of a layer deeper in the layer based on a voxel set that intersects with the plane estimated from the partial point cloud data for each of the regions; and a storage unit that stores a data structure that includes a plurality of basic voxel maps that are set for each layer obtained by structuring the target point cloud data into a multi-layer tree and that are set for each of the divided regions, wherein each of the plurality of basic voxel maps includes: a first voxel set that intersects with the plane estimated from the partial point cloud data for each layer; and an adjacent link that indicates a second voxel set in another basic voxel map that is set in an adjacent region of the same layer. and an estimation device used for a comparison process in which the estimation unit compares a first voxel set in a basic voxel map among the plurality of basic voxel maps with a second voxel set indicated by an adjacent link in the basic voxel map, and an integration process in which the estimation unit integrates the first voxel set and the second voxel set when a result of the comparison process is consistent.

5. A method executed by an estimation device, comprising: setting an initial layer of a multi-level tree structure for point cloud data to be modeled, setting a voxel set corresponding to the point cloud data of that layer, and estimating a plane of that layer from that voxel set; generating a voxel map by projecting the center coordinates of each voxel of the voxel set onto that plane; obtaining partial point cloud data corresponding to regions obtained by dividing the voxel map by spatially dividing the point cloud data; and estimating point cloud data of a layer deeper in the hierarchy for each region based on a voxel set that intersects with the plane estimated from the voxel set corresponding to the partial point cloud data. a data structure including a plurality of basic voxel maps set for each layer obtained by structuring the target point cloud data into a multilevel tree and set for each of the divided areas, each of the plurality of basic voxel maps including a first voxel set that intersects with a plane estimated from the partial point cloud data for each layer and an adjacent link indicating a second voxel set in another basic voxel map set in an adjacent area of ​​the same layer; a comparison between the first voxel set in a basic voxel map of the plurality of basic voxel maps and the second voxel set indicated by the adjacent link in the basic voxel map; and, if a result of the comparison is consistent, merging the first voxel set and the second voxel set.

Images