Point group processing device and point group processing program

Abstract

A point group processing device according to one embodiment of the present invention is provided with a classification unit and a thinning unit. The classification unit: performs point group recognition on a three-dimensional point group acquired by measuring a real space; and allocates, to each measurement point in the three-dimensional point group, classification results indicating an object corresponding to the relevant measurement point. The thinning unit thins measurement points from the three-dimensional point group according to the classification results.

Description

Point cloud processing device and point cloud processing program

[0001] One aspect of the present invention relates to a point cloud processing device and a point cloud processing program.

[0002] In recent years, a technology for representing a three-dimensional space and the objects and people within that space using a three-dimensional point cloud, which is a collection of measurement points acquired by measuring objects and people using a distance sensor such as LiDAR (Light Detection and Ranging), has come to be used for a variety of purposes. In a technology for representing a three-dimensional space using such a three-dimensional point cloud, it is possible to construct a three-dimensional space that more closely resembles the real space by using images from a camera to color each measurement point (see, for example, Non-Patent Document 1).

[0003] This technology of representing three-dimensional space using three-dimensional point clouds provides three-dimensional information at the time of measurement, making it possible to construct three-dimensional space at a lower cost than constructing it from photographs.

[0004] “Launch of the ‘TENGUN Ogijima Project’ to Promote Regional Co-creation – Joint Study Begins Aiming to Create and Expand a Connected Population through a Photorealistic ‘Ogijima’ Metaverse Realized by IOWN,” [online], November 15, 2022, NTTR&D Forum 2022, [Retrieved April 25, 2024], Internet <URL: https: / / group.ntt / jp / newsrelease / 2022 / 11 / 15 / 221115b.html>

[0005] The amount of data in a 3D point cloud is often enormous. Therefore, arranging the point cloud in the line of sight from a certain location in 3D space and displaying each measurement point on the screen takes a very long time unless the computer has high specifications. Furthermore, if the line of sight is changed while the data is being displayed, the 3D point cloud must be read and displayed again, which reduces usability.

[0006] To address this issue, a method has been adopted in which the measured 3D point cloud is thinned out to reduce the amount of data and then used to represent the 3D space. However, if measurement points are uniformly thinned out from the 3D point cloud, there are cases in which the walls of buildings and other objects are visible through the cloud, which creates a sense of incongruity for the observer.

[0007] The present invention has been made in consideration of the above circumstances, and aims to provide a technology that makes it possible to thin out a 3D point cloud so that it does not look unnatural from the scenery that is actually seen.

[0008] In order to solve the above problems, one aspect of the present invention provides a point cloud processing device that includes a classification unit and a thinning unit. The classification unit performs point cloud recognition on a 3D point cloud acquired by measuring real space, and assigns, to each measurement point in the 3D point cloud, a classification result that indicates an object corresponding to the measurement point. The thinning unit thins out measurement points from the 3D point cloud in accordance with the classification result.

[0009] According to one aspect of the present invention, by classifying the type of object a point cloud represents and thinning the point cloud according to the classification results, it is possible to provide a technology that makes it possible to eliminate the visibility of building walls and the like, and to thin out a three-dimensional point cloud so that it does not look out of place compared to the actual scenery.

[0010] FIG. 1 is a block diagram showing an example of the hardware configuration of a point cloud processing device according to a first embodiment of the present invention. FIG. 2 is a block diagram showing an example of the software configuration of the point cloud processing device. FIG. 3 is a diagram showing an example of a 3D point cloud to which point cloud recognition processing is performed. FIG. 4 is a diagram showing an example of the point cloud recognition result of the 3D point cloud of FIG. 3. FIG. 5 is a schematic diagram showing an example of thinning out a 3D point cloud obtained by measuring a wall. FIG. 6 is a schematic diagram showing an example of thinning out a 3D point cloud obtained by measuring a shrub. FIG. 7 is a diagram for explaining bottom-up selection. FIG. 8 is a flowchart showing an example of the processing procedure and processing content of point cloud processing executed by a control unit of the point cloud processing device. FIG. 9 is a block diagram showing an example of the software configuration of a point cloud processing device according to a second embodiment of the present invention. FIG. 10 is a flowchart showing an example of the processing procedure and processing content of point cloud processing executed by a control unit of the point cloud processing device according to the second embodiment.

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0012] 1 and 2 are block diagrams showing an example of the hardware and software configurations of a point cloud processing device 1 according to a first embodiment of the present invention. The point cloud processing device 1 may be a server computer connected to a network NW including the Internet, or may be a user computer such as a personal computer (PC), smartphone, or tablet terminal that can be connected to the network NW.

[0013] The point cloud processing device 1 includes a control unit 11, to which a storage unit having a program storage unit 12 and a data storage unit 13, a communication interface unit 14, and an input / output interface unit 15 are connected via a bus 16. In the figure, "interface" is abbreviated as "IF."

[0014] The control unit 11 is a hardware processor such as a CPU (Central Processing Unit). For example, the CPU can execute multiple information processes simultaneously by using a multi-core and multi-threaded CPU. The control unit 11 may include multiple hardware processors.

[0015] Under the control of the control unit 11, the communication interface unit 14 transmits and receives information to and from other devices using a communication protocol defined by the network NW.

[0016] An input device 171 and an output device 172 are connected to the input / output interface unit 15. Note that the point cloud processing apparatus 1 does not necessarily have to be equipped with all of these devices, and some of the devices may be connected as external devices.

[0017] The input device 171 is used by a user of the point cloud processing device 1 to input instructions and information necessary for the operation of the point cloud processing device 1. If the point cloud processing device 1 is a personal computer, the input device 171 includes, for example, a keyboard, a pointing device such as a mouse or a touchpad, etc. If the point cloud processing device 1 is a smartphone or tablet terminal, the input device 171 includes, for example, a touch panel and operation buttons arranged on the display screen of the output device 172. The input device 171 may also include a reader device for reading information from a recording medium storing various information such as programs and data. Furthermore, the input device 171 may also include a microphone for capturing audio, or a camera for capturing video, such as a webcam or a general video camera. The input device 171 may also include a distance sensor such as LiDAR.

[0018] The output device 172 includes a display such as a liquid crystal monitor, an organic EL (Electro Luminescence) monitor, a projector screen, or a head-mounted display that displays various information generated by the point cloud processing device 1. The output device 172 may also include a speaker that transmits various information to the user by sound. Furthermore, the output device 172 may include a writer device that writes the information generated by the point cloud processing device 1 to a recording medium.

[0019] The program storage unit 12 is configured, for example, by combining a nonvolatile memory that can be written to and read from as needed, such as a hard disk drive (HDD), a solid state drive (SSD), or an EEPROM (registered trademark) (electrically erasable programmable read-only memory), as a storage medium, with a nonvolatile memory such as a read-only memory (ROM). The program storage unit 12 stores a point cloud processing program, which is an application program required for operation as the point cloud processing device according to the first embodiment of the present invention, in addition to middleware such as an operating system (OS). Hereinafter, the OS and each application program will be collectively referred to as the program.

[0020] The data storage unit 13 is, for example, a combination of a nonvolatile memory that can be written to and read from as needed, such as an HDD, SSD, EEPROM, or memory card, as a storage medium, and a volatile memory, such as a RAM (Random Access Memory). The data storage unit 13 includes, in its storage area, a three-dimensional point cloud storage unit 131, a parameter storage unit 132, and a generated data storage unit 133 as storage units necessary for implementing the first embodiment.

[0021] The three-dimensional point cloud storage unit 131 stores a three-dimensional point cloud. The three-dimensional point cloud is acquired by measuring an object or a person using a distance sensor such as LiDAR, and each three-dimensional measurement point of the three-dimensional point cloud has three-dimensional position information (x, y, z) about the surface of the object or person. In addition, each three-dimensional measurement point of the three-dimensional point cloud may be accompanied by color information captured by a camera during measurement with the distance sensor as attribute information.

[0022] The parameter storage unit 132 stores various parameters, the details of which will be described later.

[0023] The generated data storage unit 133 stores various types of generated data generated by the control unit 11. Details of this generated data will also be described later.

[0024] The control unit 11 includes, as processing function units according to the first embodiment of the present invention, an input information acquisition unit 111, a three-dimensional point cloud recognition unit 112, a clustering execution unit 113, a point cloud reduction unit 114, a viewpoint acquisition unit 115, and an output generation unit 116. All of these processing function units 111 to 116 are realized by causing a hardware processor of the control unit 11 to execute a point cloud processing program according to the first embodiment of the present invention, which is stored in the program storage unit 12.

[0025] The point cloud processing program may be stored in advance in the program storage unit 12, or may be read out when necessary from a storage medium storing the point cloud processing program and stored in the program storage unit 12. Alternatively, the point cloud processing program may be downloaded from a program server (not shown) or the like and stored in the program storage unit 12. At least a part of the processing functions of at least one of the processing function units 111 to 116 may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), instead of being realized by the point cloud processing program and a hardware processor of the control unit 11.

[0026] The input information acquisition unit 111 acquires input information received via the network NW by the communication interface unit 14 or input information input from the input device 171 via the input / output interface unit 15, and stores the input information in the data storage unit 13. Specifically, the input information acquisition unit 111 acquires a three-dimensional point cloud and stores it in the three-dimensional point cloud storage unit 131. The input information acquisition unit 111 acquires parameters and stores them in the parameter storage unit 132. Note that some of the parameters can be included in advance as default values ​​in the point cloud processing program stored in the program storage unit 12, rather than being specified and input from another device via the network NW or by a user using the input device 171. The input information acquisition unit 111 can also acquire parameters from the point cloud processing program and store them in the parameter storage unit 132. Of course, such parameters can be used by including them in the part of the point cloud processing program that realizes the functional processing units that use them: the 3D point cloud recognition unit 112, the clustering execution unit 113, and the point cloud reduction unit 114, without having to be acquired by the input information acquisition unit 111 and stored in the parameter memory unit 132.

[0027] The three-dimensional point cloud recognition unit 112 applies point cloud recognition to the three-dimensional point cloud stored in the three-dimensional point cloud storage unit 131 based on the parameters stored in the parameter storage unit 132, and assigns a classification result to each measurement point. Point cloud recognition techniques are disclosed, for example, in Japanese Patent No. 7424509. Specifically, the parameters used by the three-dimensional point cloud recognition unit 112 include a point cloud recognition model to be used. The three-dimensional point cloud recognition unit 112 adds the classification result as attribute information to each measurement point of the three-dimensional point cloud stored in the three-dimensional point cloud storage unit 131.

[0028] FIG. 3 shows an example of a three-dimensional point cloud subjected to point cloud recognition processing, and FIG. 4 shows an example of the point cloud recognition results of this three-dimensional point cloud. Point cloud recognition uses machine learning using real data to determine what type of object the point cloud represents (buildings, power lines, trees, the ground, etc.). In particular, objects that generally occur infrequently (such as mooring posts, shrine gates, and convex mirrors on Ogijima Island, as listed in Non-Patent Document 1) can be accurately identified by transferring a feature space previously trained on a large number of images. Note that in FIGS. 3 and 4, the black areas indicate areas without measurement points, and the automobiles in FIG. 4 indicate areas where objects could not be identified.

[0029] The clustering execution unit 113 performs clustering of each measurement point using the classification results by the 3D point cloud recognition unit 112. Specifically, the clustering execution unit 113 creates clusters, which are groups of measurement points for each classification result (wall, ground, tree, ...), from the 3D point cloud to which the classification results are added as attribute information and which is stored in the 3D point cloud storage unit 131, based on the parameters stored in the parameter storage unit 132. The parameters used by the clustering execution unit 113 include a clustering method and the number of clusters. The clustering execution unit 113 stores the created clusters, which are the clustering results, in the generated data storage unit 133.

[0030] The point cloud reduction unit 114 performs a thinning process on a cluster containing point clouds of a classification to be thinned. Specifically, based on parameters stored in the parameter storage unit 132, the point cloud reduction unit 114 performs a thinning process on 3D point clouds that belong to the cluster to be thinned out of the clusters stored in the generated data storage unit 133 and to which the classification results stored in the 3D point cloud storage unit 131 have been added. The parameters used by the point cloud reduction unit 114 include the classification to be thinned out and the thinning rate. Note that the thinning rate may be the same value for all classifications to be thinned out, or may be a different value for each classification to be thinned out. The point cloud reduction unit 114 stores the 3D point clouds after thinning in the 3D point cloud storage unit 131.

[0031] FIG. 5 is a schematic diagram showing an example of thinning a 3D point cloud obtained by measuring a wall. The diagram on the left shows the point cloud before thinning, and the diagram on the right shows the point cloud after thinning. For example, when measuring a wall using LiDAR, the scan is evenly distributed, often resulting in a 3D point cloud with regularly spaced measurement points, as shown in the diagram on the left. Here, if the thinning rate is set to 0.5, i.e., half the original amount, gaps that did not exist before become noticeable, and the sense of uniformity is lost, as shown in the diagram on the right. Thus, for objects with many flat surfaces, such as walls and buildings, thinning the point cloud makes the shape difficult to understand, so not thinning or using a low thinning rate will result in a less unnatural appearance.

[0032] FIG. 6 is a schematic diagram showing an example of thinning a 3D point cloud obtained by measuring a shrub. Because shrubs have complex shapes, for example, when measuring a wall using LiDAR, there are areas where the scan hits and areas where it doesn't, and as shown in the diagram on the left, a 3D point cloud is often obtained in which the measurement points are arranged irregularly. In such cases, even if the thinning rate is set to 0.5, i.e., thinning is halved, as shown in the diagram on the right, the random impression remains unchanged, and thinning does not look unnatural. Thus, for objects with uneven surfaces such as trees and rocky areas, thinning is not a problem because changes in shape are difficult to notice.

[0033] Therefore, the point cloud reduction unit 114 checks whether each of the clusters stored in the generated data storage unit 133 is a cluster that corresponds to the classification to be thinned out stored in the parameter storage unit 132, and if the cluster corresponds to the registered classification, it deletes the three-dimensional point cloud in accordance with the thinning rate stored in the parameter storage unit 132. If the cluster does not correspond to the registered classification, that is, if the cluster is not a cluster to be thinned out, thinning is not performed.

[0034] The method for selecting points to be thinned out can be random selection. For example, if the number of measurement points belonging to a cluster in the 3D point cloud is N and the thinning rate is 0.5, N / 2 measurement points are randomly selected.

[0035] Alternatively, measurement points to be deleted may be selected using a bottom-up selection method. FIG. 7 is a diagram illustrating this bottom-up selection method. In bottom-up selection, a hierarchical graph, similar to a tournament table called a dendrogram, is created, and the number of measurement points included in each layer is selected according to a threshold value, which in this embodiment is the thinning rate stored in the parameter storage unit 132. For example, if the number of measurement points belonging to a cluster is 12 and the thinning rate is 0.7, the number is 12 × 0.7 = 8.4, with seven being the closest. Therefore, seven measurement points are selected (shaded points in FIG. 7 ), and the remaining five measurement points are selected as deletion targets. The seven measurement points are selected randomly, one at a time, within each subcluster. For example, in the case of FIG. 7 , if a subcluster has three measurement points, one is selected from the three. If a subcluster has one measurement point, that one is selected.

[0036] The viewpoint acquisition unit 115 acquires viewpoint information sent via the network NW received by the communication interface unit 14, or viewpoint information input from the input device 171 via the input / output interface unit 15. The viewpoint information includes a reference point and a viewing direction. The viewing direction can be expressed by a line-of-sight direction and a viewing angle. The reference point is an arbitrary point in a three-dimensional space expressed using a three-dimensional point cloud, and the line-of-sight direction is the point viewed from this reference point. The viewpoint acquisition unit 115 supplies the acquired viewpoint information to the output generation unit 116.

[0037] The output generation unit 116 generates an image representing a three-dimensional space viewed from a reference point using the thinned three-dimensional point cloud stored in the three-dimensional point cloud storage unit 131 based on the viewpoint information supplied from the viewpoint acquisition unit 115. Specifically, the output generation unit 116 sets a measurement point cloud in the thinned three-dimensional point cloud that is included in the range of the field of view angle when viewed in the line of sight direction from the reference point as a processing target point cloud. The coordinates of the processing target point cloud are coordinates in three-dimensional space expressed using the three-dimensional point cloud for each measurement point of the processing target point cloud. The coordinates (x, y, z) of each measurement point of this processing target point cloud and the coordinates (a, b, c) of the reference point, which is the viewing location, are coordinate values ​​in the xyz coordinate system, which is the LiDAR coordinate system (hereinafter referred to as the measurement coordinate system). In contrast, in the generated image representing the three-dimensional space viewed from the reference point, the coordinate value of the reference point is (0,0,0) in the XYZ coordinate system, which is a coordinate system of the user's viewpoint (hereinafter referred to as the user coordinate system), in which the line of sight direction is the Y coordinate and the direction of gravity is the Z coordinate. The output generation unit 116 generates an image representing the three-dimensional space viewed from the reference point by converting the coordinates of each measurement point of the processing target point cloud based on the reference point and the line of sight direction. The output generation unit 116 transmits the generated image representing the three-dimensional space viewed from the reference point to another device via the network NW using the communication interface unit 14, or outputs it to the output device 172 via the input / output interface unit 15. For example, if the output device 172 is a display, the image representing the three-dimensional space viewed from the reference point is displayed. Furthermore, if the output device 172 is a writer, the image representing the three-dimensional space viewed from the reference point is written to a recording medium as a file.

[0038] (Operation Example) Next, a description will be given of an operation example of the point cloud processing device 1 configured as described above. Fig. 8 is a flowchart showing an example of the processing procedure and processing content of point cloud processing executed by the control unit 11 in accordance with the point cloud processing program stored in the program storage unit 12 of the point cloud processing device 1. The point cloud processing shown in this flowchart is executed in response to a processing start instruction sent via the network NW received by the communication interface unit 14 or input from the input device 171 via the input / output interface unit 15.

[0039] As shown in FIG. 8 , the control unit 11 of the point cloud processing device 1 first operates as an input information acquisition unit 111 to acquire input information transmitted from another device of any source via the network NW or input from the input device 171 via the input / output interface unit 15, and stores the information in the data storage unit 13 (step S11). Specifically, if the input information is a three-dimensional point cloud, the control unit 11 stores it in the three-dimensional point cloud storage unit 131. If the input information is parameters, the control unit 11 stores them in the parameter storage unit 132. The parameters may include the point cloud recognition model to be used, the clustering method, the number of clusters, the thinning target classification, and the thinning rate. Alternatively, the control unit 11 may acquire the parameters from a point cloud processing program stored in the program storage unit 12.

[0040] Thereafter, the control unit 11 operates as the 3D point cloud recognition unit 112 to recognize the 3D point cloud (step S12). Specifically, the control unit 11 applies point cloud recognition to the 3D point cloud stored in the 3D point cloud storage unit 131 based on the parameters (point cloud recognition model) stored in the parameter storage unit 132, and assigns a classification result to each measurement point. The control unit 11 adds the classification result as attribute information to each measurement point of the 3D point cloud stored in the 3D point cloud storage unit 131.

[0041] Then, the control unit 11 operates as the clustering execution unit 113 to perform clustering (step S13). Specifically, the control unit 11 clusters the 3D point clouds to which the classification results are added as attribute information, stored in the 3D point cloud storage unit 131, based on the parameters (clustering method, number of clusters) stored in the parameter storage unit 132. The control unit 11 stores the clustering results in the generated data storage unit 133.

[0042] Thereafter, the control unit 11 operates as the point cloud reduction unit 114 and deletes the point clouds (step S14). Specifically, based on the parameters (thinning target classification, thinning rate) stored in the parameter storage unit 132, the control unit 11 deletes the point clouds, to which the classification results stored in the 3D point cloud storage unit 131 have been added, that belong to the clusters to be thinned out of the clusters stored in the generated data storage unit 133, in accordance with the thinning rate. The control unit 11 stores the thinned 3D point clouds in the 3D point cloud storage unit 131.

[0043] In this way, the control unit 11 thins out the measurement points of the point cloud of the cluster to be thinned out from all the three-dimensional point clouds acquired in the processing of step S11.

[0044] Then, the control unit 11 operates as the viewpoint acquisition unit 115 and determines whether viewpoint information transmitted from another device via the network NW or input from the input device 171 via the input / output interface unit 15 has been acquired (step S15).

[0045] If it is determined in the processing of step S15 that viewpoint information has not been acquired, the control unit 11 determines whether or not to terminate this point cloud processing (step S16). The control unit 11 can make this termination determination by determining whether or not an end instruction has been received via the network NW and received by the communication interface unit 14, or an end instruction has been input from the input device 171 via the input / output interface unit 15. If it is determined not to terminate this point cloud processing, the control unit 11 proceeds to the processing of step S15.

[0046] Furthermore, if it is determined in the processing of step S15 that viewpoint information has been acquired, the control unit 11 operates as the output generation unit 116 to generate an output image representing the three-dimensional space viewed from the reference point from each measurement point of the thinned-out processing target point cloud (step S17). At this time, if color information is added as attribute information of each measurement point, the control unit 11 can generate a color image by coloring each measurement point according to the color information.

[0047] Then, the control unit 11 outputs the generated image representing the three-dimensional space viewed from the reference point to another device via the network NW using the communication interface unit 14, or to the output device 172 via the input / output interface unit 15 (step S18). Thereafter, the control unit 11 proceeds to the process of step S16.

[0048] In this way, the control unit 11 repeats the processes of steps S15 to S18, and when it is determined in the process of step S16 that this point cloud process is to be ended, the control unit 11 ends the process shown in this flowchart.

[0049] (Actions and Effects) As described above, in the point cloud processing device 1 according to the first embodiment, the 3D point cloud recognition unit 112 performs point cloud recognition on a 3D point cloud acquired by measuring real space, assigns a classification result indicating the object corresponding to each measurement point in the 3D point cloud, and the clustering execution unit 113 and the point cloud reduction unit 114 thin out the measurement points from the 3D point cloud according to the classification result. Thus, the 3D point cloud recognition unit 112 is an example of a classification unit, and the clustering execution unit 113 and the point cloud reduction unit 114 are examples of a thinning unit. According to the point cloud processing device 1 according to the first embodiment, by classifying the object of the point cloud and thinning out the point cloud according to the classification result, it is possible to prevent the walls of buildings and the like from being visible through the 3D point cloud, and to thin out the 3D point cloud so as not to look out of place in an actual landscape.

[0050] In the point cloud processing device 1 according to the first embodiment, the clustering execution unit 113 uses the classification results to create clusters, which are groups of measurement points that indicate the same object, and the point cloud reduction unit 114 thins out the measurement points in the created groups that correspond to the classification to be thinned out. Thus, the clustering execution unit 113 is an example of a group creation unit. Therefore, according to the point cloud processing device 1 according to the first embodiment, the need for thinning is determined for each cluster, which is a group of multiple measurement points with the same classification result, and thinning is performed accordingly. Therefore, there is no need to check the classification results of each measurement point one by one to determine whether thinning is necessary, which speeds up processing.

[0051] Furthermore, in the point cloud processing device 1 according to the first embodiment, the point cloud reduction unit 114 thins out the measurement points of a group according to a thinning rate determined for each classification to be thinned. Therefore, according to the point cloud processing device 1 according to the first embodiment, thinning is performed at a thinning rate determined for each cluster classification, enabling thinning appropriate for the object. For example, because objects such as trees and rocks have inherent gaps and complex shapes, increasing the thinning rate does not significantly change the shape and is unlikely to create an unnatural appearance, thereby significantly reducing the amount of data in the 3D point cloud. Furthermore, for objects such as buildings and walls, where measurement points are often arranged in a regular pattern, increasing the thinning rate can make the gaps more noticeable and cause a loss of uniformity, creating an unnatural appearance. Therefore, the thinning rate can be reduced to reduce the amount of data without creating an unnatural appearance.

[0052] Furthermore, in the point cloud processing device 1 according to the first embodiment, the 3D point cloud recognition unit 112 performs point cloud recognition on all measurement points of the acquired 3D point cloud and assigns classification results. Therefore, according to the point cloud processing device 1 according to the first embodiment, when the output generation unit 116 generates an output image representing a 3D space, it can generate the output image based on the thinned 3D point cloud rather than the acquired 3D point cloud, making it possible to generate an output image that quickly follows changes in viewpoint.

[0053] Second Embodiment The same reference numerals as those in the first embodiment are used to designate the same configurations and processes as those in the first embodiment, and descriptions thereof will be omitted, and only the portions that differ from the first embodiment will be described.

[0054] 9 is a block diagram showing an example of the software configuration of the point cloud processing apparatus 1 according to the second embodiment of the present invention. The control unit 11 in the second embodiment further includes an extraction unit 117 as a processing function unit according to the second embodiment of the present invention.

[0055] The extraction unit 117 extracts a point cloud to be processed from the 3D point cloud stored in the 3D point cloud storage unit 131 based on the viewpoint information acquired by the viewpoint acquisition unit 115, and stores the extracted 3D point cloud in the generated data storage unit 133. The point cloud to be processed corresponds to the point cloud used by the output generation unit 116 when generating an image representing a 3D space viewed from a reference point. In this embodiment, the 3D point cloud recognition unit 112, the clustering execution unit 113, the point cloud reduction unit 114, and the output generation unit 116 perform processing on this extracted 3D point cloud (hereinafter referred to as the extracted 3D point cloud).

[0056] That is, the 3D point cloud recognition unit 112 applies point cloud recognition to the extracted 3D point cloud stored in the generated data storage unit 133 based on the parameters stored in the parameter storage unit 132, and assigns a classification result to each measurement point. The 3D point cloud recognition unit 112 adds the classification result to each measurement point of the extracted 3D point cloud stored in the generated data storage unit 133 as attribute information.

[0057] The clustering execution unit 113 clusters the extracted 3D point cloud to which the classification results are added as attribute information, which is stored in the generated data storage unit 133, based on the parameters stored in the parameter storage unit 132. The clustering execution unit 113 stores the clustering results in the generated data storage unit 133.

[0058] The point cloud reduction unit 114 performs a thinning process on the extracted three-dimensional point cloud, to which the classification result stored in the generated data storage unit 133 has been added, which belongs to the cluster to be thinned out of the clusters stored in the generated data storage unit 133, based on the parameters stored in the parameter storage unit 132. The point cloud reduction unit 114 stores the extracted three-dimensional point cloud after thinning in the generated data storage unit 133.

[0059] Based on the viewpoint information supplied from the viewpoint acquisition unit 115, the output generation unit 116 generates an image representing the three-dimensional space as seen from a reference point using the extracted three-dimensional point cloud after thinning stored in the generated data storage unit 133.

[0060] 10 is a flowchart showing an example of the processing procedure and processing content of point cloud processing executed by the control unit 11 of the point cloud processing device 1 according to the second embodiment. After acquiring input information in the processing of step S11 and storing it in the data storage unit 13, in this embodiment, a determination is made as to whether or not viewpoint information has been acquired, which is the processing of step S15.

[0061] If it is determined in the processing of step S15 that viewpoint information has been acquired, the control unit 11 operates as an extraction unit 117, extracts the point cloud to be processed from the three-dimensional point cloud stored in the three-dimensional point cloud memory unit 131 based on the acquired viewpoint information, and stores the extracted three-dimensional point cloud in the generated data memory unit 133 (step S19).

[0062] Thereafter, the control unit 11 proceeds to the process of step S12 and recognizes the 3D point cloud. In this embodiment, the control unit 11 applies point cloud recognition to the extracted 3D point cloud stored in the generated data storage unit 133 based on the parameters (point cloud recognition model) stored in the parameter storage unit 132, and assigns a classification result to each measurement point. The control unit 11 adds the classification result as attribute information to each measurement point of the extracted 3D point cloud stored in the generated data storage unit 133.

[0063] Next, in the process of step S13, the control unit 11 clusters the extracted 3D point cloud to which the classification results are added as attribute information and which is stored in the generated data storage unit 133, based on the parameters (clustering method, number of clusters) stored in the parameter storage unit 132. The control unit 11 stores the clustering results in the generated data storage unit 133.

[0064] Thereafter, in the processing of step S14, the control unit 11 performs point cloud deletion in accordance with the thinning rate for the extracted 3D point cloud, to which the classification results stored in the generated data storage unit 133 have been added, that belongs to the cluster to be thinned out of the clusters stored in the generated data storage unit 133, based on the parameters (thinning target classification, thinning rate) stored in the parameter storage unit 132. The control unit 11 stores the extracted 3D point cloud after thinning in the generated data storage unit 133.

[0065] In this way, the control unit 11 thins out the measurement points of the point cloud of the cluster to be thinned out from all the extracted three-dimensional point clouds extracted in the process of step S19.

[0066] Then, in this embodiment, the control unit 11 proceeds to processing in step S17, and generates an output image representing the three-dimensional space as seen from the reference point from each measurement point of the extracted three-dimensional point cloud after thinning.

[0067] (Actions and Effects) As described above, in the point cloud processing device 1 to which the point cloud processing device of the second embodiment is applied, it is possible to thin out the three-dimensional point cloud so that it does not look unnatural from the scenery that is actually seen, just like in the first embodiment.

[0068] Furthermore, in the point cloud processing device 1 according to the second embodiment, the extraction unit 117 extracts only the 3D point clouds required for creating an output image from the input 3D point cloud, and the 3D point cloud recognition unit 112 performs point cloud recognition on the extracted 3D point cloud extracted by the extraction unit 117 as the processing target, and assigns the classification result. Therefore, according to the point cloud processing device 1 according to the second embodiment, the processing target can be limited to a small number of measurement points, thereby enabling faster processing.

[0069] [Other Embodiments] The present invention is not limited to the above-described embodiments.

[0070] For example, in the processing of step S11 in the flowcharts shown in Figures 7 and 10, a three-dimensional point cloud and parameters are acquired as input information, but some of these may be acquired before the point cloud processing shown in these flowcharts begins.

[0071] Furthermore, the point cloud processing program stored in the program storage unit 12 may be transferred in a state where it is stored in the computer constituting the point cloud processing device 1, or may be transferred in a state where it is not stored in the computer. In the latter case, the point cloud processing program may be transferred via the network NW, or may be transferred in a state where it is recorded on a recording medium. The recording medium is a non-transitory tangible medium. The recording medium is a computer-readable medium. The form of the recording medium is not important as long as it is a medium that can store a program and is computer-readable, such as a CD-ROM or a memory card.

[0072] Although the embodiments of the present invention have been described in detail above, the above description is merely an example of the present invention in every respect. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. In other words, when implementing the present invention, specific configurations according to the embodiments may be appropriately adopted.

[0073] In short, this invention is not limited to the above-described embodiments, and the components can be modified and embodied in practice without departing from the spirit of the invention. Furthermore, various inventions can be created by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be omitted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined.

[0074] 1... Point cloud processing device 11... Control unit 12... Program storage unit 13... Data storage unit 14... Communication interface unit 15... Input / output interface unit 16... Bus 111... Input information acquisition unit 112... 3D point cloud recognition unit 113... Clustering execution unit 114... Point cloud reduction unit 115... Viewpoint acquisition unit 116... Output generation unit 117... Extraction unit 131... 3D point cloud storage unit 132... Parameter storage unit 133... Generated data storage unit 171... Input device 172... Output device

Claims

1. A point cloud processing device comprising: a classification unit that performs point cloud recognition on a three-dimensional point cloud obtained by measuring real space and assigns to each measurement point in the three-dimensional point cloud a classification result indicating the object corresponding to that measurement point; and a thinning unit that thins out the measurement points from the three-dimensional point cloud according to the classification result.

2. The point cloud processing device according to claim 1, wherein the thinning unit includes: a group creation unit that uses the classification results to create groups of measurement points that indicate the same object; and a point cloud reduction unit that thins out the measurement points of groups that correspond to the classification to be thinned out from the created groups.

3. The point cloud processing device according to claim 2, wherein the point cloud reduction unit thins out the measurement points of the group in accordance with a thinning rate determined for each classification to be thinned out.

4. A point cloud processing program that causes a processor to perform the following steps: performing point cloud recognition on a three-dimensional point cloud obtained by measuring real space, and assigning to each measurement point in the three-dimensional point cloud a classification result indicating the object corresponding to that measurement point; and thinning out the measurement points from the three-dimensional point cloud according to the classification result.

Images