Point cloud data processing device, point cloud data processing method and computer readable recording medium

Abstract

The present invention provides a point cloud data processing device, a point cloud data processing method and a program, which can easily and efficiently group point cloud data without using a two-dimensional graphic of an object as an object. The point cloud data processing device (11) comprises: an image data acquisition unit (13) for acquiring image data of an object; a point cloud data acquisition unit (15) for acquiring point cloud data; an identification unit (23) for identifying an object based on image data, acquiring an area of ​​the object and determining attribute information of the object; and an attribute assignment unit (25) for selecting point cloud data belonging to an area of ​​the object in the point cloud data and assigning the determined attribute information to the selected point cloud data.

Description

Technical Field

[0001] The present invention relates to a point cloud data processing device, a point cloud data processing method and a program, and in particular to a technology for classifying point cloud data and assigning attribute information. Background Art

[0002] There is known a technique for acquiring point cloud data composed of a plurality of points representing three-dimensional information on the surface of an object by utilizing reflection on the surface of the object using a laser scanner or the like. The acquired point cloud data is applied in various ways.

[0003] For example, by grouping the points constituting the acquired point cloud data for each object, a three-dimensional model of the object can be generated from the point cloud data.

[0004] Patent Document 1 describes acquiring point cloud data of an object and grouping the point cloud data using a pre-created two-dimensional graphic of the object.

[0005] Previous technical literature

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Publication No. 2013-97489 Summary of the invention

[0008] Technical issues to be solved by the invention

[0009] In Patent Document 1, point clouds are grouped by using a two-dimensional graphic of an object, but the two-dimensional graphic of the object may not be available to the user. For example, there may be a case where the two-dimensional graphic of the object does not exist, or there may be a case where the two-dimensional graphic of the object exists but the shape of the object is changed when acquiring point cloud data of the object, and there may be a difference between the two-dimensional graphic and the object.

[0010] The present invention is completed in view of such a situation, and its purpose is to provide a point cloud data processing device, a point cloud data processing method and a program that can easily and efficiently group point cloud data without using a two-dimensional graphic of an object as an object.

[0011] Means for solving technical problems

[0012] As one aspect of the present invention for achieving the above-mentioned purpose, a point cloud data processing device comprises: an image data acquisition unit, which acquires image data of an object; a point cloud data acquisition unit, which acquires point cloud data, the point cloud data having a positional relationship corresponding to the image data and representing three-dimensional information of multiple points on the outer surface of the object; a recognition unit, which recognizes the object based on the image data, and acquires the area of ​​the object and determines the attribute information of the object; and an attribute assignment unit, which selects the point cloud data of the area belonging to the object in the point cloud data, and assigns the determined attribute information to the selected point cloud data.

[0013] According to this aspect, the region of the object and the attribute information of the determined object can be obtained based on the image data, and the point cloud data belonging to the region of the determined object can be selected and the attribute information can be assigned. Therefore, even if the two-dimensional graphics of the object are not prepared in advance, the image data of the object can be obtained and the point cloud data can be easily and efficiently selected to assign the attribute information.

[0014] Preferably, when the image data contains image data of multiple objects, the recognition unit obtains the area of ​​each object and determines the attribute information of each object for each object, and the attribute assignment unit selects the point cloud data of the area belonging to each object in the point cloud data for each object, and assigns the attribute information of each object to the point cloud data selected for each object.

[0015] Preferably, when the image data includes image data of an object having multiple partial areas, the recognition unit identifies the area of ​​each partial area, obtains the area of ​​each partial area and determines the attribute information of each partial area, and the attribute assigning unit selects point cloud data belonging to each partial area in the point cloud data according to each partial area, and assigns the attribute information of each partial area to the point cloud data selected according to each partial area.

[0016] Preferably, the point cloud data processing device has a point cloud data supplement unit, which detects the missing part of the point cloud data belonging to the area of ​​the object based on at least the area of ​​the object identified by the identification unit, and interpolates the point cloud data of the detected missing part, and the attribute assignment unit assigns attribute information to the supplemented point cloud data.

[0017] Preferably, the image data acquisition unit acquires a photographic image obtained by imaging the object as the image data.

[0018] Preferably, the point cloud data processing device includes an image data generating unit that generates image data based on the point cloud data, and the image data acquiring unit acquires the image data generated by the image data generating unit.

[0019] Preferably, the recognition unit includes a recognizer that performs machine learning or deep learning.

[0020] Preferably, the point cloud data processing device includes a three-dimensional model generating unit that generates a three-dimensional model of the object based on the point cloud data to which the attribute information is given.

[0021] Preferably, the image data and the point cloud data are respectively acquired by devices having the same optical axis.

[0022] Preferably, the image data and the point cloud data are respectively acquired by devices whose relative positions are known.

[0023] Another aspect of the present invention for achieving the above-mentioned purpose is a point cloud data processing method, which includes: a step of acquiring image data of an object; a step of acquiring point cloud data, which has a positional relationship corresponding to the image data and represents three-dimensional information of multiple points on the outer surface of the object; a step of identifying the object based on the image data, and acquiring the area of ​​the object and determining the attribute information of the object; and a step of selecting point cloud data of an area of ​​the object in the point cloud data, and assigning the determined attribute information to the selected point cloud data.

[0024] As another aspect of the present invention for achieving the above-mentioned purpose, a program enables a computer to execute a point cloud data processing procedure including the following steps: a step of acquiring image data of an object; a step of acquiring point cloud data, which has a positional relationship corresponding to the image data and represents three-dimensional information of multiple points on the outer surface of the object; a step of identifying the object based on the image data, and acquiring the area of ​​the object and determining the attribute information of the object; and a step of selecting point cloud data of an area belonging to the object in the point cloud data, and assigning the determined attribute information to the selected point cloud data.

[0025] Effects of the Invention

[0026] According to the present invention, the area of ​​an object and the attribute information of the determined object can be acquired based on image data, and the point cloud data of the area belonging to the determined object can be selected and assigned the attribute information. Therefore, even if a two-dimensional graphic of the object is not prepared in advance, the image data of the object can be acquired and the point cloud data can be easily and efficiently selected to assign the attribute information. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a conceptual diagram showing one aspect of acquiring image data and point cloud data.

[0028] Figure 2 This is a conceptual diagram for explaining a laser scanner and an imaging device mounted on a three-dimensional measuring device.

[0029] Figure 3 This is a conceptual diagram illustrating the corresponding positional relationship between a photographic image and point cloud data.

[0030] Figure 4This is a block diagram showing an example of the functional structure of a point cloud data processing device.

[0031] Figure 5 This is a diagram for explaining recognition of the object A through segmentation performed by the recognition unit.

[0032] Figure 6 This is a diagram for explaining a point Q to which attribute information is assigned.

[0033] Figure 7 This is a flowchart illustrating a point cloud data processing method using a point cloud data processing device.

[0034] Figure 8 This is a block diagram showing an example of the functional structure of a point cloud data processing device.

[0035] Fig. 9 This figure explains the supplement of point cloud data.

[0036] Fig.10 This is a block diagram showing an example of the functional structure of a point cloud data processing device.

[0037] Fig.11 This is a diagram showing a photographic image in which a triangular prism-shaped object is the subject. DETAILED DESCRIPTION

[0038] Hereinafter, preferred embodiments of the point cloud data processing device, point cloud data processing method, and program according to the present invention will be described with reference to the accompanying drawings.

[0039] Figure 1 The point cloud data is obtained by the three-dimensional measuring device 1 and input to the point cloud data processing device 11 (refer to Figure 4 ) image data (photographic image 5) and point cloud data 7 (refer to Figure 4 ) is a conceptual diagram of one aspect. In addition, Figure 2 It is a conceptual diagram for explaining the laser scanner 101 and the imaging device (camera) 102 mounted in the three-dimensional measuring device 1 .

[0040] like Figure 1 As shown, the three-dimensional measuring device 1 acquires point cloud data of an object A as a measurement target and a photographic image 5 with the object A as a subject.

[0041] The three-dimensional measuring device 1 is a laser scanner 101 (see Figure 2 ) emits a laser pulse and measures the distance based on the time it takes for the laser pulse to reflect on the surface of the object A to be measured and return. In addition, the laser scanner 101 acquires the angular information of the emission direction of the laser pulse with respect to the rotation center of the laser scanner 101 (in Figure 1The horizontal direction a and the vertical direction β are recorded in it). Then, based on the measured distance and the angle information of the emission direction of the laser pulse, the three-dimensional information of the reflection point of the laser pulse is obtained. The laser scanner 101 obtains the three-dimensional information of multiple points by changing the horizontal direction a and the vertical direction β of the emission direction of the laser pulse at a prescribed interval. Here, the point cloud data is data representing the three-dimensional information of multiple points on the outer surface of the object to be measured, and each point constituting the point cloud data has three-dimensional information (three-dimensional coordinates). The laser scanner 101 is not limited to the time-of-flight method, and can also obtain point cloud data by a phase difference method or a triangulation method.

[0042] The three-dimensional measuring device 1 is equipped with an imaging device 102 having an optical axis L (see Figure 2 ) to acquire a photographic image 5. The photographic device 102 includes a camera element such as a CCD (Charge Coupled Device) image sensor (not shown). The camera element has a plurality of pixels composed of photoelectric exchange elements arranged two-dimensionally in the x direction (horizontal direction) and the y direction (vertical direction), and a color filter (not shown) having a two-dimensional Bayer arrangement of filters such as R (red), G (green) and B (blue) is provided on the upper surface of the plurality of pixels. Moreover, a photographic image 5 of a color image is acquired by the photographic device 102. In addition, the photographic image acquired by the photographic device 102 is not limited to a color image, and may also be a black and white image.

[0043] The three-dimensional measuring device 1 is provided with a photographing device 102 on a laser scanner 101, and the positional relationship between the laser scanner 101 and the photographing device 102 is known. In addition, the positional relationship between the laser scanner 101 and the photographing device 102 is not particularly limited as long as it is known. For example, the laser scanner 101 and the photographing device 102 may be provided to have the same optical axis. In addition, the three-dimensional measuring device 1 can simultaneously or sequentially acquire point cloud data and the photographic image 5.

[0044] In the three-dimensional measuring device 1, the laser scanner 101 acquires point cloud data, but the device for acquiring point cloud data is not limited to the laser scanner 101. For example, a ToF (Time-of-Flight) camera or a stereo camera may be used to acquire point cloud data.

[0045] Figure 3 This is to explain the relationship between the photographic image 5 and the point cloud data 7 (see Figure 4 ) have a corresponding positional relationship. Photographic image 5 is image data in which pixels P are arranged two-dimensionally. Pixels P have R, G, and B values ​​respectively. Figure 3, a pixel P of coordinates (Px, Py) in the photographic image 5 is shown, and a point Q having a positional relationship corresponding to the pixel P is shown. The point Q constitutes the point cloud data 7, and has information on the three-dimensional coordinates (x, y, z) as position information, the R, G, B values ​​of the corresponding pixel P, and the reflection intensity (I). In addition, since the photographic image 5 and the point cloud data 7 have a corresponding positional relationship, the corresponding relationship between the pixel P and the point Q can be obtained.

[0046] Next, the point cloud data processing device 11 to which the captured image 5 and the point cloud data 7 acquired as described above are input will be described.

[0047] <First Embodiment>

[0048] Figure 4 1 is a block diagram showing a functional configuration example of the point cloud data processing device 11 of the first embodiment. The point cloud data processing device 11 is constituted by, for example, a computer. The point cloud data processing device 11 may be mounted on the three-dimensional measuring device 1 or may be constituted by a computer separate from the three-dimensional measuring device 1 .

[0049] The point cloud data processing device 11 includes an image data acquisition unit 13 , a point cloud data acquisition unit 15 , a control unit 17 , a display control unit 19 , and a storage unit 21 .

[0050] The image data acquisition unit 13 acquires, as image data, a photographed image 5 obtained by photographing the object A. The image data acquisition unit 13 acquires the photographed image 5 by wire or wirelessly.

[0051] The point cloud data acquisition unit 15 acquires the point cloud data 7. The point cloud data acquisition unit 15 acquires the point cloud data 7 by wire or wirelessly.

[0052] The captured image (image data) 5 and the point cloud data 7 obtained by the image data acquisition unit 13 and the point cloud data acquisition unit 15 have a corresponding positional relationship.

[0053] The control unit 17 is constituted by a CPU (Central Processing Unit) (not shown) mounted on a computer. The control unit 17 includes a recognition unit 23 and an attribute assignment unit 25 .

[0054] The recognition unit 23 recognizes the object based on the photographic image 5, and obtains the area of ​​the object and the attribute information for identifying the object. The recognition unit 23 can obtain the area of ​​the object in the photographic image and the attribute information for identifying the object by various methods. For example, the recognition unit 23 is composed of a recognizer that implements machine learning or deep learning, and obtains the area of ​​the object and the attribute information for identifying the object by segmenting the photographic image. In addition, existing models such as FCN (Fully Convolutional Network), SegNet, Pix2Pix, etc. can be used in segmentation, and models corresponding to the recognition method of object A performed by the recognition unit 23 can also be individually produced. In addition, the learning of the recognition unit 23 can use deep learning, or machine learning in a broad sense.

[0055] Figure 5 This figure explains the recognition of object A by segmentation performed by the recognition unit 23. Here, object A is composed of object B and object C, and the recognition unit 23 receives the captured image 5 as input, and acquires the region of object B and attribute information of object B, and the region of object C and attribute information of object C.

[0056] The recognition unit 23 is a recognizer that implements machine learning. Figure 5 (A) is segmented into a photographic image 5, such as Figure 5 As shown in (B), determine the area of ​​object B (in Figure 5 (B) is indicated by a dotted line), obtains the attribute information of the area (Pole), and determines the area of ​​object C (in Figure 5 (B) is indicated by a dotted line), and the attribute information (Base) of the area is obtained. Figure 5 In the above description, the object B and the object C are recognized based on the photographic image 5 , but the present invention is not limited to this. For example, the recognition unit 23 may acquire the region of the object A and the attribute information of the object A based on the photographic image 5 .

[0057] Back to Figure 4, the attribute assigning unit 25 selects the point cloud data belonging to the area of ​​object B in the point cloud data 7, and assigns attribute information (Pol e) to the selected point cloud data. Specifically, first, the attribute assigning unit 25 selects the point cloud data composed of the points Q corresponding to the pixels P in the area of ​​object B in the photographic image 5. The pixels P of the photographic image 5 correspond to the points Q constituting the point cloud data. Therefore, by determining the area of ​​object B in the photographic image 5, the point cloud data corresponding to object B in the point cloud data 7 can be selected. Thereafter, the attribute assigning unit 25 assigns attribute information (Pol e) or attribute information (Base) to the selected point cloud data. In addition, the attribute information of objects B and C is not limited to the attributes associated with the shapes of objects B and C. For example, as attribute information, they can also be named (labeled) just to distinguish objects B and C, such as "b" or "c".

[0058] Figure 6 This is a diagram for explaining the point Q to which the attribute information (Pole) is assigned. The point cloud data of the object B is shown in the enlarged diagram BK, and the three-dimensional coordinates (x, y, z), the R, G, B values ​​of the corresponding pixel P and the reflection intensity I, and the attribute information "Label: 'Pole'" recognized from the photographic image 5 are assigned to the point Q constituting the point cloud data. By assigning attribute information to the point Q in this way, the point cloud data can be grouped according to each attribute. In addition, although the description is omitted, the attribute information (Base) is also assigned to the point corresponding to the object C in the same way.

[0059] Back to Figure 4 The storage unit 21 stores the image data and the point cloud data 7. In addition, in order to realize various functions of the point cloud data processing device 11, various programs executed by the CPU are stored.

[0060] The display control unit 19 is constituted by a CPU, and controls display on the monitor 9. For example, the monitor 9 displays point cloud data and / or image data.

[0061] Furthermore, the point cloud data processing device 11 may include an instruction receiving unit (not shown) including a keyboard or a mouse for receiving instructions from a user.

[0062] Figure 7 It is a flowchart explaining the point cloud data processing method using the point cloud data processing device 11 .

[0063] First, the image data acquisition unit 13 of the point cloud data processing device 11 acquires the photographic image 5 as image data (step S10). Then, the point cloud data acquisition unit 15 acquires the point cloud data 7 (step S11). Next, the recognition unit 23 of the point cloud data processing device 11 recognizes the object based on the image data, and acquires the area of ​​the object and the attribute information for determining the object (step S12). Then, the attribute assignment unit 25 of the point cloud data processing device 11 selects the point cloud data of the area of ​​the object belonging to the photographic image 5, and assigns the attribute information to the selected point cloud data (step S13).

[0064] As described above, according to this embodiment, the region of an object and attribute information for determining the object can be obtained based on a photographic image (image data), and point cloud data can be selected and attribute information can be assigned based on the determined region of the object and the attribute information of the object. Thus, even if a two-dimensional graphic of the object is not prepared in advance, this aspect can obtain a photographic image of the object and easily and efficiently select point cloud data to assign attribute information.

[0065] In the above-mentioned embodiment, the hardware structure of the processing unit that performs various processing is various processors as shown below. The various processors include a general-purpose processor that functions as various processing units by executing software (programs), a processor that can change the circuit structure after manufacturing, such as a programmable logic device (PLD), and a processor that has a circuit structure specially designed for executing specific processing such as an ASIC (Application Specific Integrated Circuit), such as a dedicated circuit.

[0066] A processing unit can be composed of one of these various processors, or it can be composed of two or more processors of the same or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA). In addition, multiple processing units can be composed of one processor. As an example of using one processor to constitute multiple processing units, first, there is a method such as represented by a computer such as a client or server, which uses a combination of one or more CPUs and software to constitute a processor, and the processor functions as multiple processing units. Secondly, there is a method such as represented by a system on chip (System On Chip: SoC), which uses a processor that implements the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip. In this way, various processing units are composed of one or more of the above-mentioned various processors as a hardware structure.

[0067] Furthermore, more specifically, the hardware structure of these various processors is a circuit (circuitry) in which circuit elements such as semiconductor elements are combined.

[0068] The above structures and functions can be appropriately implemented by any hardware, software or a combination of the two. For example, the present invention can also be applied to a program that causes a computer to execute the above processing steps (processing procedures), a computer-readable recording medium (non-temporary recording medium) recording such a program, or a computer capable of installing such a program.

[0069] <Second Embodiment>

[0070] Next, a second embodiment is described. In this embodiment, when there is a missing portion in the point cloud data, the missing portion is interpolated, and then a three-dimensional model is generated based on the supplemented point cloud data.

[0071] Figure 8 1 is a block diagram showing an example of the functional configuration of the point cloud data processing device 11 according to the present embodiment. Figure 4 Parts that have already been described are denoted by the same symbols and their description is omitted.

[0072] The point cloud data processing device 11 includes an image data acquisition unit 13 , a point cloud data acquisition unit 15 , a control unit 17 , a display control unit 19 , and a storage unit 21 .

[0073] The control unit 17 includes a recognition unit 23 , an attribute assignment unit 25 , a point cloud data supplementation unit 27 , and a three-dimensional model generation unit 29 .

[0074] The point cloud data supplementation unit 27 supplements the point cloud data 43 having a positional relationship corresponding to the photographic image (image data) 41 acquired by the point cloud data acquisition unit 15. That is, when there is a missing portion in the point cloud data 43 acquired by the point cloud data acquisition unit 15, the point cloud data supplementation unit 27 interpolates the missing portion. Specifically, based on at least the area of ​​the object recognized by the recognition unit 23, the missing portion of the point cloud data 43 belonging to the area of ​​the object is detected, and the point cloud data of the detected missing portion is interpolated. In addition, the point cloud data supplementation unit 27 can also use the attribute information of the object D determined by the recognition unit 23 to supplement the point cloud data 43. The point cloud data supplementation unit 27 interpolates the point cloud data of the missing portion in the point cloud data 43 by a well-known technique.

[0075] The three-dimensional model generation unit 29 generates a three-dimensional model of the object based on the point cloud data 43 to which the attribute information is assigned. The three-dimensional model generation unit 29 can generate a three-dimensional model based on the point cloud data 43 acquired by the point cloud data acquisition unit 15 or the point cloud data 43 supplemented by the point cloud data supplement unit 27. In addition, when a three-dimensional model is generated based on the point cloud data 43, the information (attribute information) on which object's surface each point of the point cloud data 43 is a point is powerful information in making a correct polygon mesh or performing correct texture mapping. In addition, the three-dimensional model generation performed by the three-dimensional model generation unit 29 uses a well-known technology.

[0076] Fig. 9 This is a diagram for explaining the supplementation of the point cloud data 43 by the point cloud data supplementation unit 27 . Fig. 9 (A) shows a photographic image 41 acquired by the image data acquisition unit 13, Fig. 9 (B) shows the point cloud data 43 acquired by the point cloud data acquisition unit 15. In addition, in the photographic image 41, the object D which is a tube is photographed as a subject.

[0077] like Fig. 9 As shown in (A), at a part of the object D, light is reflected regularly, and there is a glossy portion 45A with high brightness. In addition, at the portion corresponding to the glossy portion 45A of the object D, the laser pulse emitted to obtain the point cloud data is reflected regularly, and the return light of the laser pulse cannot be received. Moreover, the laser scanner 101 cannot obtain the point cloud data at the portion corresponding to the glossy portion 45A. Therefore, Fig. 9 As shown in (B), a missing portion 45B is generated in the point cloud data 43 of the object D.

[0078] Even in the case of the photographic image 41 of the object D having the glossy portion 45A, the recognition unit 23 can recognize that the glossy portion 45A is a part of the object D. Furthermore, the point cloud data supplementation unit 27 recognizes the missing portion 45B of the point cloud data 43 of the object D based on the region of the object D or the region of the object D and the attribute information of the object D acquired by the recognition unit 23, and performs linear interpolation or spline interpolation using nearby point cloud data, thereby calculating the point cloud data of the missing portion 45B. In addition, in the case of an object having a cylindrical outer surface such as a pipe, the point cloud data in the missing portion 45B can be interpolated based on the curvature of the nearby point cloud data and the basis vector of the cylinder. Thus, the point cloud data supplementation unit 27 can interpolate the missing portion 45B more accurately.

[0079] As described above, the point cloud data supplementation unit 27 interpolates the missing part of the point cloud data based on the area of ​​the object, or the area of ​​the object and the attribute information of the object, so that more accurate supplementation can be performed. In addition, when the three-dimensional model generation unit 29 generates a three-dimensional model using the supplemented point cloud data, a more accurate three-dimensional model can be generated.

[0080] <Third Embodiment>

[0081] Next, the third embodiment will be described. In the first and second embodiments described above, the photographic image 5 (or the photographic image 41 ) is used as image data. However, in the present embodiment, image data other than the photographic image is acquired.

[0082] Fig.10 1 is a block diagram showing an example of the functional configuration of the point cloud data processing device 11 according to the present embodiment. Figure 4 Parts that have already been described are denoted by the same symbols and their description is omitted.

[0083] The point cloud data processing device 11 includes an image data acquisition unit 13, a point cloud data acquisition unit 15, a control unit 17, a display control unit 19, and a storage unit 21. Figure 4 Parts that have already been described are denoted by the same symbols and their description is omitted.

[0084] The control unit 17 includes a recognition unit 23 , an attribute imparting unit 25 , and an image data generating unit 35 .

[0085] The image data generating unit 35 generates image data based on the point cloud data 7. Each point constituting the point cloud data 7 has a three-dimensional coordinate, so the image data generating unit 35 can generate image data by projecting the three-dimensional coordinate of each point onto a predetermined plane. At this time, image data to which a pixel value (density) is assigned according to the reflection intensity or distance of each point can be synthesized. Then, the image data acquiring unit 13 acquires the image data generated by the image data generating unit 35.

[0086] By generating image data from the point cloud data 7 in this way, image data having a positional relationship corresponding to the point cloud data 7 can be obtained even without acquiring a photographic image, and the point cloud data can be easily and efficiently classified and assigned attribute information.

[0087] <Fourth Embodiment>

[0088] Next, a fourth embodiment is described. In this embodiment, point cloud data is grouped for each partial region of an object.

[0089] When the image data includes image data of an object having a plurality of partial regions, the recognition unit 23 of the present embodiment recognizes the area of ​​each partial region and acquires the area of ​​each partial region and attribute information of each partial region for specifying the partial region.

[0090] The attribute assigning unit 25 classifies the point cloud data of the regions belonging to the respective partial regions in the point cloud data into the respective partial regions, and assigns the attribute information of the respective partial regions to the point cloud data classified into the respective partial regions.

[0091] Fig.11 : is a diagram showing a photographic image 51 in which an object D, which is a triangular prism, is taken as a subject. The recognition unit 23 determines the area (partial area) of the surface E, the area (partial area) of the surface F, and the area (partial area) of the surface G of the object D based on the photographic image 51. In addition, the recognition unit 23 obtains attribute information for determining each partial area of ​​the surface E, the surface F, and the surface G of the object D from the photographic image 51. In addition, for example, the recognition unit 23 may also determine the area of ​​the surface E, the surface F, and the surface G of the object D by a recognizer that implements machine learning or deep learning to extract the surface (including planes and curved surfaces) of the object, and obtain the attribute information (surface E, surface F, surface G) of each partial area. In addition, for example, the recognition unit 23 may also determine the area of ​​the surface E, the surface F, and the surface G of the object D by image processing that extracts the contour of the object, and obtain the attribute information of each partial area.

[0092] As described above, by grouping the point cloud data for each partial region of the object, it is possible to more accurately perform grouping corresponding to the object.

[0093] Although the examples of the present invention have been described above, it goes without saying that the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the present invention.

[0094] Explanation of symbols

[0095] 1. Three-dimensional measuring device

[0096] 5 Photographic images

[0097] 7Point cloud data

[0098] 9 Monitor

[0099] 11Point cloud data processing device

[0100] 13 Image data acquisition unit

[0101] 15Point cloud data acquisition department

[0102] 17 Control Unit

[0103] 19Display control unit

[0104] 21 Storage

[0105] 23. Identification Department

[0106] 25 Attribute Assignment Department

[0107] 27 Point Cloud Data Supplement Department

[0108] 29 Three-dimensional model generation department

[0109] 35 Image data generation unit

Claims

1. A point cloud data processing device, comprising: an image data acquisition unit that acquires image data of an object; a point cloud data acquisition unit that acquires point cloud data having a positional relationship corresponding to the image data and representing three-dimensional information of a plurality of points on the outer surface of the object; an identification unit, which identifies the object based on the image data, and obtains the area of ​​the object and determines the attribute information of the object; an attribute assigning unit, which selects point cloud data belonging to a region of the object from among the point cloud data, and assigns the acquired attribute information to the selected point cloud data; as well as a point cloud data supplementing unit, which detects a missing portion of the point cloud data belonging to the area of ​​the object based at least on the area of ​​the object recognized by the recognition unit and the attribute information, and interpolates the point cloud data of the detected missing portion, The attribute imparting unit also imparts the attribute information to the supplemented point cloud data. The point cloud data processing device includes a three-dimensional model generating unit configured to generate a three-dimensional model of the object based on the point cloud data supplemented by the point cloud data supplementing unit and to which the attribute information is added.

2. The point cloud data processing device according to claim 1, wherein: When the image data includes the image data of a plurality of the objects, the recognition unit acquires the area of ​​each object and specifies the attribute information of each object for each of the objects. The attribute imparting unit selects, for each of the objects, the point cloud data of a region belonging to each of the objects in the point cloud data, and imparts the attribute information of each of the objects to the point cloud data selected for each of the objects.

3. The point cloud data processing device according to claim 1 or 2, wherein: When the image data includes the image data of the object having a plurality of partial regions, the recognition unit recognizes the region of each partial region, acquires the region of each partial region and determines attribute information of each partial region, The attribute assigning unit selects the point cloud data of the area belonging to each of the partial areas among the point cloud data for each of the partial areas, and assigns the attribute information of each of the partial areas to the point cloud data selected for each of the partial areas.

4. The point cloud data processing device according to claim 1 or 2, wherein: The image data acquisition unit acquires a photographic image obtained by photographing the object as the image data.

5. The point cloud data processing device according to claim 1 or 2, wherein: The point cloud data processing device includes an image data generating unit that generates the image data based on the point cloud data. The image data acquisition unit acquires the image data generated by the image data generation unit.

6. The point cloud data processing device according to claim 1 or 2, wherein: The recognition unit includes a recognizer that performs machine learning or deep learning.

7. The point cloud data processing device according to claim 1 or 2, wherein: The image data and the point cloud data are respectively acquired through devices having the same optical axis.

8. The point cloud data processing device according to claim 1 or 2, wherein: The image data and the point cloud data are respectively acquired through devices whose relative positions are known.

9. A point cloud data processing method, comprising: The step of acquiring image data of an object; a step of acquiring point cloud data, wherein the point cloud data has a positional relationship corresponding to the image data and represents three-dimensional information of a plurality of points on the outer surface of the object; A step of identifying the object based on the image data, acquiring the area of ​​the object and determining the attribute information of the object; A step of selecting point cloud data belonging to a region of the object from the point cloud data, and assigning the acquired attribute information to the selected point cloud data; as well as a step of detecting a missing portion of the point cloud data belonging to the area of ​​the object based at least on the identified area of ​​the object and the attribute information, and interpolating the point cloud data of the detected missing portion, The step of assigning the attribute information also assigns the attribute information to the supplemented point cloud data. A three-dimensional model of the object is generated based on the point cloud data that has been supplemented and endowed with the attribute information.

10. A computer-readable recording medium storing a program for causing a computer to execute a point cloud data processing procedure comprising the following steps: The step of acquiring image data of an object; a step of acquiring point cloud data, wherein the point cloud data has a positional relationship corresponding to the image data and represents three-dimensional information of a plurality of points on the outer surface of the object; A step of identifying the object based on the image data, acquiring the area of ​​the object and determining the attribute information of the object; A step of selecting point cloud data belonging to a region of the object from the point cloud data, and assigning the acquired attribute information to the selected point cloud data; as well as a step of detecting a missing portion of the point cloud data belonging to the area of ​​the object based at least on the identified area of ​​the object and the attribute information, and interpolating the point cloud data of the detected missing portion, The step of assigning the attribute information also assigns the attribute information to the supplemented point cloud data. A three-dimensional model of the object is generated based on the point cloud data that has been supplemented and endowed with the attribute information.

Images