Point cloud processing device and point cloud processing program

Abstract

A point cloud processing device according to one aspect of the present invention is provided with a calculating unit and an adjusting unit. The calculating unit calculates a distance from an arbitrary reference point in three-dimensional space based on a three-dimensional point cloud acquired by measuring a real space, for each measurement point of a processing target point cloud in the three-dimensional point cloud. The adjusting unit adjusts the size of each measurement point of the processing target point cloud to become smaller as the distance calculated by the calculating unit decreases and larger as the distance increases.

Description

Point cloud processing device and point cloud processing program

[0001] One aspect of the present invention relates to a point cloud processing device and a point cloud processing program.

[0002] In recent years, a technology for representing a three-dimensional space and the objects and people within that space using a three-dimensional point cloud, which is a collection of measurement points acquired by measuring objects and people using a distance sensor such as LiDAR (Light Detection and Ranging), has come to be used for a variety of purposes. In a technology for representing a three-dimensional space using such a three-dimensional point cloud, it is possible to construct a three-dimensional space that more closely resembles the real space by using images from a camera to color each measurement point (see, for example, Non-Patent Document 1).

[0003] This technology of representing three-dimensional space using three-dimensional point clouds provides three-dimensional information at the time of measurement, making it possible to construct three-dimensional space at a lower cost than constructing it from photographs.

[0004] “Launch of the ‘TENGUN Ogijima Project’ to Promote Regional Co-creation – Joint Study Begins Aiming to Create and Expand a Connected Population through a Photorealistic ‘Ogijima’ Metaverse Realized by IOWN,” [online], November 15, 2022, NTTR&D Forum 2022, [Retrieved April 25, 2024], Internet <URL: https: / / group.ntt / jp / newsrelease / 2022 / 11 / 15 / 221115b.html>

[0005] In technology that uses 3D point clouds to represent 3D space, it is necessary to set the size of each measurement point appropriately. That is, when a group of measurement points is lined up in the line of sight from a certain location in 3D space, the farther away they are from this location, the denser they appear to be. Therefore, if the size of each measurement point is set to a small value, nearby measurement points will become almost invisible. Conversely, if the size of each measurement point is set to a large value, nearby measurement points will overlap too much and become difficult to see.

[0006] This invention has been made in consideration of the above circumstances, and aims to provide a technology that makes it possible to represent three-dimensional space using three-dimensional point clouds without reducing visibility.

[0007] In order to solve the above problems, one aspect of the present invention provides a point cloud processing device that includes a calculation unit and an adjustment unit. The calculation unit calculates the distance from an arbitrary reference point in three-dimensional space for each measurement point of a target point cloud in the three-dimensional point cloud based on a three-dimensional point cloud acquired by measuring real space. The adjustment unit adjusts the size of each measurement point of the target point cloud so that the closer the distance calculated by the calculation unit is, the smaller the size of the measurement point of the target point cloud is, and the larger the size of the measurement point is, so that the farther the distance is.

[0008] According to one aspect of the present invention, the size of each measurement point in the point cloud to be processed is adjusted according to the distance from an arbitrary reference point, thereby providing a technology that enables the representation of three-dimensional space using a three-dimensional point cloud without reducing visibility.

[0009] FIG. 1 is a block diagram showing an example of the hardware configuration of a point cloud processing device according to a first embodiment of the present invention. FIG. 2 is a block diagram showing an example of the software configuration of the point cloud processing device. FIG. 3 is a schematic diagram for explaining the distance in the line of sight direction from a reference point of a processing target point cloud. FIG. 4 is a schematic diagram showing an XY plane set on a processing target point cloud in a three-dimensional point cloud. FIG. 5 is a schematic diagram showing the size and appearance of each measurement point on the XY plane when a three-dimensional space is displayed based on the processing target point cloud. FIG. 6 is a schematic diagram showing size adjustment of the processing target point cloud according to distance. FIG. 7 is a flowchart showing an example of the processing procedure and processing content of point cloud processing executed by a control unit of the point cloud processing device. FIG. 8 is a diagram showing an example of a display of an image representing a three-dimensional space when the size of the processing target point cloud is uniformly reduced. FIG. 9 is a diagram showing an example of a display of an image representing a three-dimensional space when the size of the processing target point cloud is uniformly increased. FIG. 10 is a diagram showing an example of a display of an image representing a three-dimensional space when the size of the processing target point cloud is adjusted according to distance. FIG. 11 is a block diagram showing an example of the software configuration of a point cloud processing device according to a second embodiment of the present invention. FIG. 12 is a flowchart showing an example of the processing procedure and processing content of point cloud processing executed by the control unit of the point cloud processing apparatus according to the second embodiment.

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0011] 1 and 2 are block diagrams showing an example of the hardware and software configurations of a point cloud processing device 1 according to a first embodiment of the present invention. The point cloud processing device 1 may be a server computer connected to a network NW including the Internet, or may be a user computer such as a personal computer (PC), smartphone, or tablet terminal that can be connected to the network NW.

[0012] The point cloud processing device 1 includes a control unit 11, to which a storage unit having a program storage unit 12 and a data storage unit 13, a communication interface unit 14, and an input / output interface unit 15 are connected via a bus 16. In the figure, "interface" is abbreviated as "IF."

[0013] The control unit 11 is a hardware processor such as a CPU (Central Processing Unit). For example, the CPU can execute multiple information processes simultaneously by using a multi-core and multi-threaded CPU. The control unit 11 may include multiple hardware processors.

[0014] Under the control of the control unit 11, the communication interface unit 14 transmits and receives information to and from other devices using a communication protocol defined by the network NW.

[0015] An input device 171 and an output device 172 are connected to the input / output interface unit 15. Note that the point cloud processing apparatus 1 does not necessarily have to be equipped with all of these devices, and some of the devices may be connected as external devices.

[0016] The input device 171 is used by a user of the point cloud processing device 1 to input instructions and information necessary for the operation of the point cloud processing device 1. If the point cloud processing device 1 is a personal computer, the input device 171 includes, for example, a keyboard, a pointing device such as a mouse or a touchpad, etc. If the point cloud processing device 1 is a smartphone or tablet terminal, the input device 171 includes, for example, a touch panel and operation buttons arranged on the display screen of the output device 172. The input device 171 may also include a reader device for reading information from a recording medium storing various information such as programs and data. Furthermore, the input device 171 may also include a microphone for capturing audio, or a camera for capturing video, such as a webcam or a general video camera. The input device 171 may also include a distance sensor such as LiDAR.

[0017] The output device 172 includes a display such as a liquid crystal monitor, an organic EL (Electro Luminescence) monitor, a projector screen, or a head-mounted display that displays various information generated by the point cloud processing device 1. The output device 172 may also include a speaker that transmits various information to the user by sound. Furthermore, the output device 172 may include a writer device that writes the information generated by the point cloud processing device 1 to a recording medium.

[0018] The program storage unit 12 is configured, for example, by combining a nonvolatile memory that can be written to and read from as needed, such as a hard disk drive (HDD), a solid state drive (SSD), or an EEPROM (registered trademark) (electrically erasable programmable read-only memory), as a storage medium, with a nonvolatile memory such as a read-only memory (ROM). The program storage unit 12 stores a point cloud processing program, which is an application program required for operation as the point cloud processing device according to the first embodiment of the present invention, in addition to middleware such as an operating system (OS). Hereinafter, the OS and each application program will be collectively referred to as the program.

[0019] The data storage unit 13 is, for example, a combination of a nonvolatile memory that can be written to and read from as needed, such as an HDD, SSD, EEPROM, or memory card, as a storage medium, and a volatile memory, such as a RAM (Random Access Memory). The data storage unit 13 includes, in its storage area, a three-dimensional point cloud storage unit 131, a camera image storage unit 132, and a parameter storage unit 133 as storage units necessary for implementing the first embodiment.

[0020] The three-dimensional point cloud storage unit 131 stores a three-dimensional point cloud. The three-dimensional point cloud is acquired by measuring an object or a person using a distance sensor such as LiDAR, and each three-dimensional measurement point of the three-dimensional point cloud has three-dimensional position information (x, y, z) about the surface of the object or person.

[0021] The camera image storage unit 132 stores camera images. The camera images are images of objects or people captured using a camera when a 3D point cloud is acquired using a distance sensor. That is, there is a correspondence between the 3D point cloud stored in the 3D point cloud storage unit 131 and the camera images stored in the camera image storage unit 132.

[0022] The parameter storage unit 133 stores various parameters, the details of which will be described later.

[0023] The control unit 11 includes, as processing function units according to the first embodiment of the present invention, an input information acquisition unit 111, a processing target point cloud determination unit 112, a distance calculation unit 113, an output size adjustment unit 114, and an output generation unit 115. All of these processing function units 111 to 115 are realized by causing a hardware processor of the control unit 11 to execute a point cloud processing program according to the first embodiment of the present invention, which is stored in the program storage unit 12.

[0024] The point cloud processing program may be stored in advance in the program storage unit 12, or may be read out when necessary from a storage medium storing the point cloud processing program and stored in the program storage unit 12. Alternatively, the point cloud processing program may be downloaded from a program server (not shown) or the like and stored in the program storage unit 12. At least a part of the processing functions of at least one of the processing function units 111 to 115 may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), instead of being realized by the point cloud processing program and a hardware processor of the control unit 11.

[0025] The input information acquisition unit 111 acquires input information received via the network NW by the communication interface unit 14 or input information input from the input device 171 via the input / output interface unit 15, and stores the input information in the data storage unit 13. Specifically, the input information acquisition unit 111 acquires a three-dimensional point cloud and stores it in the three-dimensional point cloud storage unit 131. The input information acquisition unit 111 acquires camera images and stores them in the camera image storage unit 132. The input information acquisition unit 111 acquires parameters and stores them in the parameter storage unit 133. Note that some of the parameters can be included in advance as default values ​​in the point cloud processing program stored in the program storage unit 12, rather than being specified and input from another device via the network NW or by a user using the input device 171. The input information acquisition unit 111 can also acquire parameters from the point cloud processing program and store them in the parameter storage unit 133. Of course, such parameters can be used by including them in the part of the point cloud processing program that realizes the functional processing units that use them, that is, the processing target point cloud determination unit 112 and the output size adjustment unit 114, without having to be acquired by the input information acquisition unit 111 and stored in the parameter memory unit 133.

[0026] The processing target point cloud determination unit 112 determines a processing target point cloud, which is a measurement point cloud to be processed, from the 3D point clouds stored in the 3D point cloud storage unit 131 based on parameters stored in the parameter storage unit 133. The parameters used by the processing target point cloud determination unit 112 specifically include a reference point, a viewpoint (line of sight direction), and a field of view angle. The reference point is an arbitrary point in a 3D space represented using the 3D point cloud, and the viewpoint is a point viewed from this reference point. The control unit 11 generates display data for displaying the 3D space as viewed from this reference point. The processing target point cloud determination unit 112 determines, as the processing target point cloud, a measurement point cloud in the 3D point cloud that is included in the range of the field of view when viewed from the reference point to the viewpoint. The processing target point cloud determination unit 112 supplies the coordinates of the determined processing target point cloud to the distance calculation unit 113 and the output size adjustment unit 114. The coordinates of the processing target point cloud can be coordinates in a three-dimensional space expressed using a three-dimensional point cloud for each measurement point of the point cloud that forms the boundary of the range of the field of view angle.

[0027] The distance calculation unit 113 calculates the distance from the reference point for each measurement point of the processing target point group from the three-dimensional point group stored in the three-dimensional point group storage unit 131, based on the coordinates of the processing target point group supplied from the processing target point group determination unit 112 and the reference point, which is a parameter stored in the parameter storage unit 133. The distance calculation unit 113 then supplies the calculated distance for each measurement point of the processing target point group to the output size adjustment unit 114.

[0028] The coordinates (x, y, z) of each measurement point in the processing target point cloud and the coordinates (a, b, c) of the reference point, which is the viewing location, are known. These coordinates are coordinate values ​​in the xyz coordinate system (hereinafter referred to as the measurement coordinate system) of the LiDAR (and camera). Therefore, the distance calculation unit 113 can calculate the distance of each measurement point from the reference point using Euclidean distance, etc.

[0029] Alternatively, the distance calculation unit 113 may calculate this distance as the distance in the line of sight from the reference point. FIG. 3 is a schematic diagram for explaining the distance in the line of sight from the reference point of the processing target point cloud. In FIG. 3, white circles represent each measurement point Po of the processing target point cloud that is within the field of view with respect to the line of sight direction VD, in which the viewpoint is viewed from an arbitrary reference point Pref in a three-dimensional space represented using a three-dimensional point cloud. FIG. 3 shows each measurement point Po in an XYZ coordinate system, which is a coordinate system of the user's viewpoint (hereinafter referred to as the user coordinate system), in which the line of sight direction VD is the Y coordinate and the direction of gravity is the Z direction. In this user coordinate system, the coordinate value of the reference point Pref is (0,0,0). The distance calculation unit 113 converts the coordinate values ​​(x, y, z) of the measurement coordinate system into coordinate values ​​(X, Y, Z) of the user coordinate system for each measurement point Po of the processing target point group based on the coordinates of the processing target point group supplied from the processing target point group determination unit 112 and the reference point and viewpoint (line of sight direction VD) parameters stored in the parameter storage unit 133, and can use the Y coordinate value in the user coordinate system as the distance L in the line of sight direction VD. In FIG. 3, in order to improve the distinguishability, the reference point Pref is projected onto Z = -n (n is determined by the viewing angle) to obtain a projected reference point Pref pro The distance L in the line of sight direction VD may be rephrased as the distance in the depth direction when looking from the reference point Pref in the line of sight direction VD.

[0030] For the sake of explanation, FIG. 4 shows an XY plane PL set on the processing target point group in the three-dimensional point group. XY This is a schematic diagram showing the XY plane PL XY includes a plurality of measurement points Po. FIG. 5 shows the XY plane PL when displaying a three-dimensional space based on the processing target point group. XY 4, which is in the line of sight direction VD from the reference point Pref in the three-dimensional space. XY When the above processing target point group is arranged, as shown in the center of Figure 5, the further back it is, that is, the farther it is from the reference point Pref, the more densely the measurement points Po appear to be arranged. Therefore, as shown in the bottom of Figure 5, the size of each measurement point Po is reduced to a small measurement point Po. SIf you adjust it to display, the measurement point Po S On the other hand, as shown in the upper part of FIG. 5, when the size of each measurement point Po is increased to a large measurement point Po, L If you adjust it to display, the point cloud near the reference point Pref will be displayed as the nearby measurement point Po L The XY plane has been explained here, but the same applies to the XZ plane.

[0031] Therefore, in this embodiment, the output size adjustment unit 114 adjusts the size of the measurement point Po in accordance with the distance from the reference point Pref calculated by the distance calculation unit 113 to each measurement point Po of the processing target point group.

[0032] The output size adjustment unit 114 adjusts the size of each measurement point Po of the processing target point cloud stored in the 3D point cloud storage unit 131 based on the coordinates of the processing target point cloud supplied from the processing target point cloud determination unit 112, the distance of each measurement point Po of the processing target point cloud supplied from the distance calculation unit 113, and the parameters stored in the parameter storage unit 133. Specifically, the parameters used by the output size adjustment unit 114 are the number of divisions N, distance Li (i = 1, 2, ..., N-1), and size Si (i = 1, 2, ..., N). The number of divisions N is the number by which the three-dimensional space within the field of view is divided according to the distance from the reference point Pref. In the example of FIG. 3, N = 3. The distance Li is the distance divided relative to the number of divisions N. In the example of FIG. 3, for example, L1 = 20 m and L2 = 500 m can be set to divide the space into three distance ranges: close, middle, and long. As a result, the three-dimensional space within the field of view is divided into a short distance range of less than 20 m from the reference point Pref, a medium distance range of 20 m to 500 m, and a long distance range of 500 m or more. For example, if the distance of each measurement point Po from the reference point Pref of the processing target point group is the distance in the line of sight VD, then as shown in FIG. XZ Space closer than 1, this XZ plane PL XZ 1 to the XZ plane PL 500m from the reference point Pref XZ Space closer than 2, this XZ plane PLXZ The space is divided into spaces from 2 to further away. The size Si is a specific numerical value of the size of the measurement point Po included in the i-th space. FIG. 6 is a schematic diagram showing the size adjustment of the processing target point cloud according to distance by the output size adjustment unit 114. When the processing target point cloud is divided into three distance ranges as in the example of FIG. 3, the size of each measurement point Po of the processing target point cloud before adjustment shown in the upper part of FIG. 6 becomes small in the short distance range as shown in the lower part of FIG. S and in the long distance range, the measurement point Po L The output size adjustment unit 114 supplies the point cloud to be processed, the size of which has been adjusted in this way, to the output generation unit 115.

[0033] The output generation unit 115 colors each measurement point Po of the size-adjusted processing target point cloud supplied from the output size adjustment unit 114 based on the camera image stored in the camera image storage unit 132. The output generation unit 115 then converts the coordinates of each measurement point Po of the size-adjusted and colored processing target point cloud based on the reference point Pref and the viewpoint (line of sight direction VD), which are parameters stored in the parameter storage unit 133, to generate an image representing a three-dimensional space viewed from the reference point Pref. The output generation unit 115 transmits the generated image representing the three-dimensional space viewed from the reference point Pref to another device via the network NW via the communication interface unit 14, or outputs it to the output device 172 via the input / output interface unit 15. For example, if the output device 172 is a display, an image representing the three-dimensional space viewed from the reference point Pref is displayed. Furthermore, if the output device 172 is a writer, the image representing the three-dimensional space viewed from the reference point Pref is written to a recording medium as a file.

[0034] (Operation Example) Next, a description will be given of an operation example of the point cloud processing device 1 configured as described above. Fig. 7 is a flowchart showing an example of the processing procedure and processing content of point cloud processing executed by the control unit 11 in accordance with the point cloud processing program stored in the program storage unit 12 of the point cloud processing device 1. The point cloud processing shown in this flowchart is executed in response to a processing start instruction sent via the network NW received by the communication interface unit 14 or input from the input device 171 via the input / output interface unit 15.

[0035] As shown in FIG. 7 , the control unit 11 of the point cloud processing device 1 first operates as an input information acquisition unit 111 to acquire input information transmitted from another device of any source via the network NW or information input from the input device 171 via the input / output interface unit 15, and stores the information in the data storage unit 13 (step S101). Specifically, if the input information is a three-dimensional point cloud, the control unit 11 stores it in the three-dimensional point cloud storage unit 131. If the input information is a camera image, the control unit 11 stores it in the camera image storage unit 132. If the input information is a parameter, the control unit 11 stores it in the parameter storage unit 133. The parameter may include a reference point Pref, a viewpoint (line of sight direction VD), a field of view angle, a division number N, a distance Li, and a size Si. Alternatively, the control unit 11 may acquire the parameter from a point cloud processing program stored in the program storage unit 12.

[0036] Thereafter, the control unit 11 checks whether the reference point Pref and the line of sight direction VD have been acquired (step S102). If the reference point Pref and the line of sight direction VD have not been acquired, the control unit 11 proceeds to the process of step S101.

[0037] On the other hand, if the reference point Pref and the line of sight direction VD have already been acquired, the control unit 11 operates as a processing target point group determination unit 112 and determines a processing target point group from the three-dimensional point group stored in the three-dimensional point group memory unit 131 based on the reference point, viewpoint (line of sight direction), and field of view angle (step S103).

[0038] Thereafter, the control unit 11 operates as a distance calculation unit 113 to calculate the distance from the reference point Pref for each measurement point Po in the determined processing target point group (step S104). The control unit 11 can calculate this distance using Euclidean distance or the like. Alternatively, the control unit 11 may calculate the distance L in the line of sight VD for each measurement point Po based on the coordinates of each measurement point Po in the processing target point group, the coordinates of the reference point, and the viewpoint (line of sight direction VD).

[0039] Then, the control unit 11 operates as an output size adjustment unit 114 and adjusts the size of the measurement points Po in accordance with the distance from the reference point Pref to each measurement point Po of the processing target point group based on the number of divisions N, the distance Li, and the size Si (step S105).

[0040] After the size of each measurement point Po of the processing target point cloud has been adjusted in this way, the control unit 11 operates as the output generation unit 115 to generate an output image representing the three-dimensional space viewed from the reference point Pref (step S106). At this time, the control unit 11 can color each measurement point Po of the processing target point cloud based on the camera image stored in the camera image storage unit 132.

[0041] Then, the control unit 11 outputs the image representing the three-dimensional space as seen from the generated reference point Pref to another device via the network NW via the communication interface unit 14, or to the output device 172 via the input / output interface unit 15 (step S107).

[0042] Thereafter, the control unit 11 determines whether or not to terminate this point cloud processing (step S108). The control unit 11 can make this termination determination by determining whether or not there is a termination instruction received by the communication interface unit 14 via the network NW, or a termination instruction input from the input device 171 via the input / output interface unit 15. If it is determined not to terminate this point cloud processing, the control unit 11 proceeds to the processing of step S101. On the other hand, if it is determined to terminate this point cloud processing, the processing shown in this flowchart is terminated.

[0043] 8 and 9 are diagrams illustrating examples of an image representing a three-dimensional space displayed on a display serving as the output device 172. FIG. 8 illustrates a case in which the size of each measurement point Po in the processing target point cloud is uniformly reduced, FIG. 9 illustrates a case in which the size of each measurement point Po is uniformly increased, and FIG. 10 illustrates a case in which the size of each measurement point Po in the processing target point cloud is adjusted according to distance. As shown in FIG. 8 , when the size of each measurement point Po in the processing target point cloud is uniformly reduced, the image results in some buildings and other structures not being visible. Furthermore, as shown in FIG. 9 , when the size of each measurement point Po in the processing target point cloud is uniformly increased, the foreground measurement points become too large, reducing visibility. In contrast, when the size of each measurement point Po in the processing target point cloud is adjusted according to distance as in this embodiment, buildings and scenery in the foreground and in the background are displayed naturally, as shown in FIG. 10 .

[0044] (Operations and Effects) As described above, in the point cloud processing device 1 according to the first embodiment, the distance calculation unit 113 calculates the distance from an arbitrary reference point Pref in the three-dimensional space for each measurement point Po of the target point group in the three-dimensional point cloud based on the three-dimensional point cloud acquired by measuring the real space. The output size adjustment unit 114 adjusts the size of each measurement point Po of the target point group so that the closer the distance calculated by the distance calculation unit 113, the smaller the size and the farther the distance. Thus, the distance calculation unit 113 is an example of a calculation unit, and the output size adjustment unit 114 is an example of an adjustment unit. According to the point cloud processing device 1 according to the first embodiment, the size of each measurement point Po of the three-dimensional point cloud is adjusted according to the distance, making it easier to achieve a size of the measurement point Po that corresponds to the sense of distance. Therefore, the measurement point cloud size can be set according to the sense of distance, thereby solving problems of reduced visibility, such as partial invisibility or difficulty in seeing the foreground. In other words, according to the point cloud processing device 1 according to the first embodiment, it is possible to represent a three-dimensional space using a three-dimensional point cloud without reducing visibility.

[0045] In the point cloud processing device 1 according to the first embodiment, the distance calculation unit 113 calculates the distance from the reference point Pref to each measurement point Po in the processing target point cloud as the Euclidean distance between the coordinate value of each measurement point Po and the coordinate value of the reference point Pref. Therefore, according to the point cloud processing device 1 according to the first embodiment, the distance can be easily calculated by a simple calculation.

[0046] Alternatively, in the point cloud processing device 1 according to the first embodiment, the distance calculation unit 113 calculates the distance of each measurement point Po in the processing target point cloud from the reference point Pref as the distance in the line of sight VD from the reference point Pref. Therefore, according to the point cloud processing device 1 according to the first embodiment, it is possible to calculate the distance in the depth direction when looking from the reference point Pref in the line of sight VD.

[0047] Furthermore, in the point cloud processing device 1 according to the first embodiment, the output size adjustment unit 114 adjusts the size of each measurement point Po in the point cloud to be processed for each distance range specified by the distance Li, which is a parameter, to a size Si specified according to the distance range. Therefore, according to the point cloud processing device 1 according to the first embodiment, it is possible to adjust the size of multiple measurement points Po collectively, which eliminates the need to calculate the size to be adjusted for each individual measurement point Po according to its distance, thereby enabling faster processing.

[0048] Furthermore, in the point cloud processing device 1 according to the first embodiment, the processing target point cloud determiner 112 determines each measurement point Po in the 3D point cloud that is included within a predetermined range of field of view from the reference point Pref as the processing target point cloud. Therefore, according to the point cloud processing device 1 according to the second embodiment, it is possible to limit the processing target point cloud, thereby enabling faster processing.

[0049] Second Embodiment The same reference numerals as those in the first embodiment are used to designate the same configurations and processes as those in the first embodiment, and descriptions thereof will be omitted, and only the portions that differ from the first embodiment will be described.

[0050] 11 is a block diagram showing an example of the software configuration of a point cloud processing device 1 according to a second embodiment of the present invention. The control unit 11 in this second embodiment includes a processing target point cloud extraction unit 116 as a processing function unit according to the second embodiment of the present invention, instead of the processing target point cloud determination unit 112 of the first embodiment. Furthermore, the distance calculation unit 113 and the output size adjustment unit 114 are not connected to the 3D point cloud storage unit 131.

[0051] The processing target point cloud extraction unit 116 determines and extracts a processing target point cloud from the three-dimensional point clouds stored in the three-dimensional point cloud storage unit 131 based on the reference point Pref, the viewpoint (line of sight direction VD), and the field of view angle, which are parameters stored in the parameter storage unit 133. The processing target point cloud extraction unit 116 supplies the extracted processing target point cloud to the distance calculation unit 113 and the output size adjustment unit 114.

[0052] The distance calculation unit 113 calculates the distance from the reference point for each measurement point Po of the processing target point group supplied from the processing target point group determination unit 112, based on the reference point, which is a parameter stored in the parameter storage unit 133, or based on the reference point and the viewpoint (line of sight direction VD). The distance calculation unit 113 supplies the calculated distance from each measurement point Po of the processing target point group to the output size adjustment unit 114.

[0053] The output size adjustment unit 114 adjusts the size of each measurement point Po of the processing target point cloud supplied from the processing target point cloud extraction unit 116, based on the distance of each measurement point Po of the processing target point cloud supplied from the distance calculation unit 113, and the division number N, distance Li (i = 1, 2, ..., N-1), and size Si (i = 1, 2, ..., N), which are parameters stored in the parameter storage unit 133. The output size adjustment unit 114 supplies the processing target point cloud whose size has been adjusted to the output generation unit 115.

[0054] 12 is a flowchart showing an example of the procedure and content of the point cloud processing executed by the control unit 11 of the point cloud processing device 1 according to the second embodiment. In this embodiment, when it is determined in step S102 that the reference point Pref and the line of sight direction VD have been acquired, the control unit 11 operates as the processing target point cloud extraction unit 116 and extracts a processing target point cloud from the three-dimensional point cloud stored in the three-dimensional point cloud storage unit 131 based on the reference point, the viewpoint (line of sight direction), and the field of view angle (step S111).

[0055] Thereafter, the control unit 11 operates as the distance calculation unit 113 to calculate the distance from the reference point Pref for each measurement point Po of the processing target point group (step S104). Note that in this embodiment, the processing target point group is the point group extracted in step S111.

[0056] The control unit 11 then operates as an output size adjustment unit 114 to adjust the size of each measurement point Po in the processing target point cloud in accordance with the distance from the reference point Pref to the measurement point Po in accordance with the division number N, the distance Li, and the size Si (step S105). Note that in this embodiment, the processing target point cloud is the point cloud extracted in step S111.

[0057] (Actions and Effects) As described above, in the point cloud processing device 1 to which the point cloud processing device of the second embodiment is applied, as in the first embodiment, the size of each measurement point Po of the three-dimensional point cloud is adjusted according to the distance, so that it is possible to represent three-dimensional space using the three-dimensional point cloud without reducing visibility.

[0058] In the point cloud processing device 1 according to the second embodiment, the processing target point cloud extraction unit 116 extracts each measurement point Po in the 3D point cloud that is within a predetermined range of field of view from the reference point Pref as the processing target point cloud, and the distance calculation unit 113 and output size adjustment unit 114 perform distance calculations and size adjustments for each measurement point Po in the extracted processing target point cloud. Thus, according to the point cloud processing device 1 according to the second embodiment, as in the first embodiment, the processing target point cloud can be limited, thereby enabling faster processing. Furthermore, the distance calculation unit 113 and output size adjustment unit 114 do not need to read out each processing target point cloud from the 3D point cloud storage unit 131, thereby further speeding up processing.

[0059] [Other Embodiments] The present invention is not limited to the above-described embodiments.

[0060] For example, in the processing of step S101 in the flowcharts shown in Figures 7 and 12, the three-dimensional point cloud, the camera image, and the parameters of the reference point Pref, the viewpoint (line of sight direction VD), the viewing angle, the number of divisions N, the distance Li, and the size Si are acquired as input information, but some of these may be acquired before the start of the point cloud processing shown in these flowcharts. In the processing of step S101, it is sufficient to acquire at least the reference point Pref and the viewpoint (line of sight direction VD) as input information.

[0061] Furthermore, the point cloud processing program stored in the program storage unit 12 may be transferred in a state where it is stored in the computer constituting the point cloud processing device 1, or may be transferred in a state where it is not stored in the computer. In the latter case, the point cloud processing program may be transferred via the network NW, or may be transferred in a state where it is recorded on a recording medium. The recording medium is a non-transitory tangible medium. The recording medium is a computer-readable medium. The form of the recording medium is not important as long as it is a medium that can store a program and is computer-readable, such as a CD-ROM or a memory card.

[0062] Although the embodiments of the present invention have been described in detail above, the above description is merely an example of the present invention in every respect. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. In other words, when implementing the present invention, specific configurations according to the embodiments may be appropriately adopted.

[0063] In short, this invention is not limited to the above-described embodiments, and the components can be modified and embodied in practice without departing from the spirit of the invention. Furthermore, various inventions can be created by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be omitted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined.

[0064] DESCRIPTION OF SYMBOLS 1...Point cloud processing device 11...Control unit 12...Program storage unit 13...Data storage unit 14...Communication interface unit 15...Input / output interface unit 16...Bus 111...Input information acquisition unit 112...Processing target point cloud determination unit 113...Distance calculation unit 114...Output size adjustment unit 115...Output generation unit 116...Processing target point cloud extraction unit 131...3D point cloud storage unit 132...Camera image storage unit 133...Parameter storage unit 171...Input device 172...Output device L, Li...Distance N...Number of divisions NW...Network PL XY ...XY plane PL XZ 1. PL XZ 2...XY plane Po, Po L , Po S …Measurement point Pref…Reference point Pref pro ...Projection reference point VD...View direction

Claims

1. A point cloud processing device comprising: a calculation unit that calculates the distance from an arbitrary reference point in three-dimensional space based on a three-dimensional point cloud obtained by measuring real space, for each measurement point of a point cloud to be processed in the three-dimensional point cloud; and an adjustment unit that adjusts the size of each measurement point of the point cloud to be processed so that the closer the distance calculated by the calculation unit is, the smaller the size of each measurement point of the point cloud to be processed, and the farther the distance is, the larger the size of each measurement point to be processed.

2. The point cloud processing device according to claim 1, wherein the calculation unit calculates the distance from the reference point in the line of sight direction as the distance of each of the measurement points of the processing target point cloud.

3. A point cloud processing device according to claim 1 or 2, wherein the adjustment unit adjusts the size of each measurement point of the processing target point cloud for each specified distance range to a size specified according to the distance range.

4. A point cloud processing program that causes a processor to execute the following steps: calculating the distance from an arbitrary reference point in three-dimensional space for each measurement point of a target point group in the three-dimensional point cloud obtained by measuring real space; and adjusting the size of each measurement point of the target point group so that the closer the calculated distance is, the smaller the size becomes, and the farther the calculated distance is, the larger the size becomes.

Images