Point cloud data processing device, point cloud data processing method and computer readable recording medium
By associating point cloud data with image data and using machine learning recognizers, the problem of inefficient allocation of point cloud data attribute information in complex shapes and overlapping object scenes is solved, and efficient and accurate attribute information is achieved, suitable for complex shapes and overlapping objects with point cloud data processing.
Patent Information
- Application Number
- CN202080096126.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-02-18
- Filing Date
- 2020-12-07
- Publication Date
- 2024-12-17
- Estimated Expiration
- 2040-12-07
AI Technical Summary
In the case of measuring complex shapes or overlapping objects, it is inefficient and prone to errors when manually assigning attribute information to point cloud data, especially when objects are arranged in complex manner with each other, it is difficult for the prior art to achieve efficient and accurate attribute information assignment.
By associating point cloud data with image data, the processor uses three-dimensional rotation, movement, and zoom-in and display, and selecting the area corresponding to the specified point in the image data, automatically assigning attribute information to the point cloud data, and using machine learning recognizer Auxiliary area selection and attribute correction.
It achieves efficient and accurate assignment of attribute information to point cloud data, reduces manual operation time, and improves data processing efficiency and accuracy in complex shapes and overlapping objects scenes.
Smart Images
Figure CN115088020B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a point cloud data processing device, a point cloud data processing method and a computer-readable recording medium storing a program, and in particular to a technology for assigning attribute information to each point constituting point cloud data. Background Art
[0002] There is known a technique for acquiring point cloud data composed of a plurality of points (point cloud) representing three-dimensional information on the surface of an object using reflection on the surface of the object by a laser scanner etc. In recent years, various forms of application of this point cloud data have been considered.
[0003] On the other hand, the measured point cloud data itself has a large volume, which becomes a problem from the perspective of operation when using the point cloud data. Therefore, the point cloud data is modeled (such as a solid model, a surface model, etc.) to reduce the volume of the point cloud data.
[0004] Here, when modeling point cloud data, it is necessary to assign (group) attribute information to each point cloud constituting the point cloud data of each measured object. For example, when measuring multiple objects, it is necessary to assign attribute information to each point cloud of the object.
[0005] When attribute information is assigned to a point cloud in this manner, for example, a user manually selects each point to identify a point cloud corresponding to an object, and the same attribute information is assigned to the identified point cloud.
[0006] In the technology described in Patent Document 1, one point included in the region of the object is specified on the image data, the region of the object is determined based on the specified one point, and point clouds corresponding to the determined region of the object are grouped.
[0007] Previous technical literature
[0008] Patent Literature
[0009] Patent Document 1: Japanese Patent Application Publication No. 2012-83157 Summary of the invention
[0010] Technical issues to be solved by the invention
[0011] Here, the measurement object measured by the laser scanner is not only a simple-shaped object, but also a complex structure composed of multiple objects or objects overlapped and arranged in a complex manner. For example, when measuring piping, sometimes multiple pipes are arranged in an overlapping manner. Even in such a case, it is necessary to give attribute information to the point cloud of each pipe.
[0012] In the case of such a measurement object, it takes a lot of time for the user to manually select each point and assign attribute information. In addition, when the user selects a point, an erroneous selection may occur. Therefore, in the case where the measurement object is complicatedly composed of multiple objects or the objects overlap and are complicatedly arranged, the method in which the user manually selects each point cannot perform efficient work.
[0013] In addition, as described in Patent Document 1, in the technology of specifying a point cloud with attribute information by designating an object in two-dimensionally displayed image data, overlapping objects may be hidden behind other objects in the two-dimensionally displayed image data, and thus cannot be accurately selected. Therefore, in the technology described in Patent Document 1, attribute information may not be accurately assigned to the point cloud.
[0014] The present invention has been made in view of such circumstances, and an object of the present invention is to provide a point cloud data processing device, a point cloud data processing method, and a program that can efficiently and accurately assign attribute information to a point cloud.
[0015] Means for solving technical problems
[0016] A point cloud data processing device as one mode of the present invention for achieving the above-mentioned purpose comprises a memory and a processor, wherein the memory stores point cloud data representing three-dimensional information of multiple points on the surface of an object and multiple image data obtained by photographing the object, and at least associates the position of a pixel of any one of the multiple image data with the points constituting the point cloud data, wherein the processor causes the point cloud data to be displayed on a display unit in a manner that allows three-dimensional rotation, three-dimensional movement, and enlargement and reduction, accepts designation of a designated point in the point cloud data displayed on the display unit, selects an area of the object object including an area corresponding to the designated point in the image data, and assigns the same attribute information to the points corresponding to the area of the object object in the point cloud data.
[0017] According to this method, point cloud data is displayed on a display unit in a manner that allows three-dimensional rotation, three-dimensional movement, and enlargement and reduction, and the designation of a designated point is accepted in the point cloud data. Furthermore, the method selects an area of the object object including an area corresponding to the designated point in the image data, and assigns the same attribute information to the points corresponding to the area. Therefore, the method can efficiently and accurately assign attribute information to the point cloud data.
[0018] Preferably, the processor selects a region of the target object in the image data using a recognizer that has performed machine learning.
[0019] Preferably, the processor displays image data of a region corresponding to the designated point on the display unit after receiving the designation of the designated point.
[0020] Preferably, the processor combines the plurality of image data and displays the combined image data on the display unit.
[0021] Preferably, the processor receives selection of a region of the target object in the image data displayed on the display unit.
[0022] Preferably, the processor performs correction of adding attribute information to the point cloud data to which the attribute information is added by the identifier that has performed machine learning.
[0023] Preferably, the processor displays the point cloud data to which the attribute information is assigned on the display unit, receives a correction related to the assignment of the attribute information, and applies the correction to the assigned attribute information.
[0024] Preferably, the processor acquires the image data and the point cloud data acquired by a device having the same optical axis.
[0025] A point cloud data processing method as another embodiment of the present invention is a point cloud data processing method of a point cloud data processing device, wherein the point cloud data processing device includes a memory and a processor, wherein the memory stores point cloud data representing three-dimensional information of multiple points on the surface of an object and multiple image data obtained by photographing the object, and at least associates the position of a pixel of any one of the multiple image data with the points constituting the point cloud data, wherein the processor performs: a step of displaying the point cloud data on a display unit in a manner that allows three-dimensional rotation, three-dimensional movement, and enlargement and reduction; a step of accepting a designation of a designated point in the point cloud data displayed on the display unit; a step of selecting an area of the object object including an area corresponding to the designated point in the image data; and a step of assigning the same attribute information to points corresponding to the area of the object object in the point cloud data.
[0026] As another embodiment of the present invention, a program causes a point cloud data processing device to execute a point cloud data processing method, wherein the point cloud data processing device includes a memory and a processor, wherein the memory stores point cloud data representing three-dimensional information of multiple points on the surface of an object and multiple image data obtained by photographing the object, and at least associates the position of a pixel of any one of the multiple image data with the points constituting the point cloud data, wherein the program causes the processor to perform: a step of displaying the point cloud data on a display unit in a manner that allows three-dimensional rotation, three-dimensional movement, and enlargement and reduction; a step of accepting a designation of a designated point in the point cloud data displayed on the display unit; a step of selecting an area of an object object including an area corresponding to the designated point in the image data; and a step of assigning the same attribute information to points corresponding to the area of the object object in the point cloud data.
[0027] Effects of the Invention
[0028] According to the present invention, point cloud data is displayed on a display unit in a manner that allows three-dimensional rotation, three-dimensional movement, and enlargement and reduction. In the point cloud data, designation of a designated point is accepted, and an area of the object object including the area corresponding to the designated point is selected in the image data, and the same attribute information is assigned to the points corresponding to the area, thereby enabling the point cloud to be assigned attribute information efficiently and accurately. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 This is a conceptual diagram showing one method of acquiring image data and point cloud data using a three-dimensional measuring device.
[0030] Figure 2 This is a conceptual diagram for explaining the laser scanner and imaging device installed in the three-dimensional measuring machine.
[0031] Figure 3 This is a conceptual diagram for explaining a situation in which image data and point cloud data have a corresponding positional relationship.
[0032] Figure 4 This is a block diagram showing an example of the functional structure of a point cloud data processing device.
[0033] Figure 5 This is a flowchart showing a point cloud data processing method performed using a point cloud data processing device.
[0034] Figure 6 A diagram schematically showing point cloud data displayed on a monitor.
[0035] Figure 7 It is a diagram schematically showing image data.
[0036] Figure 8 A diagram showing image data displayed on a monitor.
[0037] Fig. 9 This is a diagram for explaining point cloud data to which attribute information has been assigned.
[0038] Fig.10 is a diagram showing a point cloud displayed on the monitor.
[0039] Fig.11 is a diagram showing a point cloud displayed on the monitor. DETAILED DESCRIPTION
[0040] Hereinafter, preferred embodiments of the point cloud data processing device, point cloud data processing method and program according to the present invention will be described with reference to the accompanying drawings.
[0041] Figure 1 The point cloud data obtained by the three-dimensional measuring machine 1 and input to the point cloud data processing device 11 (refer to Figure 4)'s image data 5 and point cloud data 7.
[0042] like Figure 1 As shown, the three-dimensional measuring machine 1 acquires point cloud data of an object A to be measured and image data 5 with the object A as a subject. The object A is composed of an object D and an object E.
[0043] The laser scanner 101 (see Figure 2 ) is a time-of-flight laser scanner that emits laser pulses and measures distance based on the time it takes for the laser pulses to reflect and return from the surface of the object A to be measured. In addition, the laser scanner 101 acquires angle information (in Figure 1 , the horizontal direction α and the vertical direction β are recorded). Then, the three-dimensional information of the reflection point of the laser pulse is obtained based on the measured distance and the angle information of the emission direction of the laser pulse. The laser scanner 101 obtains the three-dimensional information of multiple points by changing the horizontal direction α and the vertical direction β of the emission direction of the laser pulse at a prescribed interval. Here, the point cloud data is data representing the three-dimensional information of multiple points on the surface of the object to be measured, and each point constituting the point cloud data has three-dimensional information (three-dimensional coordinates). The laser scanner 101 is not limited to a laser scanner based on the time-of-flight method, but can also be a laser scanner that obtains point cloud data by a phase difference method or a triangulation method.
[0044] The three-dimensional measuring machine 1 is equipped with a photographing device 102 having an optical axis L (see Figure 2 ) to obtain image data 5. The photographing device 102 includes a CCD (Charge Coupled Device) or other imaging element (not shown). The imaging element has a plurality of pixels composed of photoelectric conversion elements arranged two-dimensionally in the x direction (horizontal direction) and the y direction (vertical direction), and a color filter (not shown) having a two-dimensional Bayer arrangement of filters such as R (red), G (green) and B (blue) is provided on the upper surface of the plurality of pixels. Furthermore, the image data 5 of a color image is obtained by the photographing device 102. In addition, the image data obtained by the photographing device 102 is not limited to a color image, and may also be a monochrome image.
[0045] Figure 2 1 is a conceptual diagram for explaining a laser scanner 101 and an imaging device (camera) 102 mounted on the three-dimensional measuring machine 1 .
[0046] exist Figure 2In the example shown, the laser scanner 101 and the imaging device 102 are arranged with the optical axis L being the same using a half mirror 103. In addition, the positional relationship between the laser scanner 101 and the imaging device 102 is not particularly limited as long as it is known. For example, the imaging device 102 may be arranged above the laser scanner 101 without setting the optical axis to be the same. In addition, the three-dimensional measuring device 1 can simultaneously or sequentially acquire the point cloud data 7 and the image data 5.
[0047] In the three-dimensional measuring device 1, the laser scanner 101 acquires point cloud data, but the means for acquiring point cloud data is not limited to the laser scanner 101. For example, the point cloud data may be acquired using a ToF (Time-of-Flight) camera or a stereo camera.
[0048] Figure 3 2 is a conceptual diagram for explaining a situation in which image data 5 and point cloud data 7 have a corresponding positional relationship. Figure 3 In the example shown, a plurality of image data 5 (image data 5A to 5D) are acquired. For example, the three-dimensional measuring device 1 is rotated in the translation direction to acquire a plurality of image data 5A to 5D in different imaging directions. In the following description, the image data 5A is described.
[0049] The image data 5A is image data in which pixels P are arranged two-dimensionally. The pixels P have values for each of R, G, and B. Figure 3 , a pixel P of coordinates (Px, Py) in image data 5A is shown, and a point Q having a positional relationship corresponding to pixel P is shown. Point Q constitutes point cloud data 7, and has three-dimensional coordinates (x, y, z) as position information, and information on the R, G, B values of the corresponding pixel P and the reflection intensity (I). In addition, since image data 5A and point cloud data 7 have a corresponding positional relationship, the corresponding relationship between pixel P and point Q can be obtained. In addition, image data 5B to 5D also have a corresponding positional relationship between pixels and points as described above.
[0050] Next, the point cloud data processing device 11 to which the image data (image data 5A to 5D) 5 and the point cloud data 7 acquired as described above are input will be described.
[0051] Figure 4 1 is a block diagram showing a functional configuration example of the point cloud data processing device 11. The point cloud data processing device 11 is constituted by, for example, a computer. The point cloud data processing device 11 may be mounted on the three-dimensional measuring machine 1 or may be constituted by a computer separate from the three-dimensional measuring machine 1.
[0052] The point cloud data processing device 11 includes an image data acquisition unit 13 , a point cloud data acquisition unit 15 , a control unit 17 , a display control unit 19 , and a storage unit (memory) 21 .
[0053] The image data acquisition unit 13 acquires, as image data, a plurality of image data 5A to 5D obtained by photographing the object A. The image data acquisition unit 13 may acquire the plurality of image data 5A to 5D individually, or may acquire composite image data obtained by compositely combining the image data 5A to 5D.
[0054] The point cloud data acquisition unit 15 acquires the point cloud data 7. The point cloud data 7 corresponding to each of the image data 5A to 5D is acquired. In addition, the point cloud data 7 corresponding to each of the image data 5A to 5D may be synthesized and acquired similarly to the image data 5A to 5D.
[0055] The image data acquisition unit 13 and the point cloud data acquisition unit 15 acquire data by wire or wirelessly. The image data 5A to 5D and the point cloud data 7 acquired by the image data acquisition unit 13 and the point cloud data acquisition unit 15 are acquired by the three-dimensional measuring device 1 described above and have corresponding positional relationships.
[0056] The control unit 17 is realized by a CPU (Central Processing Unit) (processor) (not shown) mounted on the computer executing a program stored in the storage unit 21. The control unit 17 includes a designation receiving unit 23, an area selecting unit 25, and an attribute assigning unit 27. In addition, the display control unit 19 also realizes its function by the CPU executing a program stored in the storage unit 21.
[0057] The storage unit 21 stores the point cloud data 7 and the image data 5A to 5D in association with the positions of pixels and points.
[0058] Figure 5 1 is a flowchart showing a point cloud data processing method performed using the point cloud data processing device 11. The point cloud data processing method is performed by executing a program stored in the storage unit 21.
[0059] First, the display control unit 19 displays the point cloud data 7 stored in the storage unit 21 on the monitor (display unit) 9 (step S10: point cloud data display step). Thereafter, the designation receiving unit 23 receives the designation of a designated point in the point cloud data 7 displayed on the monitor 9 (step S11: designation step). Next, the area selection unit 25 selects an area of the object including the area corresponding to the designated point in the image data (step S12: selection step). Then, the attribute assigning unit 27 assigns the same attribute information to the point cloud data 7 corresponding to the area of the object selected in the image data (step S13: attribute assigning step).
[0060] Below, each step is described in detail.
[0061] <Point cloud data display and specification steps>
[0062] The point cloud data display step (step S10) is performed by the display control unit 19. In addition, the designation step (step S11) is performed by the designation receiving unit 23. The display control unit 19 displays the point cloud data 7 stored in the storage unit 21 on the monitor 9.
[0063] Figure 6 1 is a diagram schematically showing the point cloud data 7 displayed on the monitor 9. The dots shown in the diagram schematically show the points of the point cloud data 7, and the outline of the object A is shown by a dotted line (imaginary line).
[0064] The display control unit 19 displays the point cloud data 7 on the monitor 9. Since each point constituting the point cloud data 7 has a three-dimensional coordinate, each point is displayed on the XYZ coordinate system shown in the figure based on the three-dimensional coordinate. The point cloud data 7 displayed on the monitor 9 is displayed in a manner that can be rotated, moved, and enlarged and reduced in three dimensions. The user can use a fixed-point device to rotate and move the point cloud data 7 in three dimensions, and enlarge and reduce it. In this way, the point cloud data 7 is rotated, moved, and enlarged and reduced in three dimensions, so that the user can accurately specify the designated point F. In the case where the measurement object is complexly composed of multiple objects or the objects overlap and are complexly configured, the user can accurately specify the designated point by rotating and moving the point cloud data 7 in three dimensions, and enlarging and reducing it. The user specifies point F using a fixed-point device. The designation receiving unit 23 receives the coordinate value of point F and accepts the designation.
[0065] <Selection Steps>
[0066] The selection step (step S12) is performed by the region selection unit 25. The region selection unit 25 selects a region of the target object including the region corresponding to the designated point F in the image data 5.
[0067] Figure 7 Schematically shows the image data 5A. The image data 5A has a region G corresponding to the designated point F in the point cloud data 7. The region G is composed of one or more pixels of the image data 5A. As described above, each point constituting the point cloud data 7 has a corresponding positional relationship with each pixel of the image data 5A. Therefore, in the image data 5A, by selecting the region of the object including the region G, it is possible to specify the point cloud obtained by the surface reflection of the object D including the designated point F. The selection of the region of the object including the region G is described below.
[0068] First, a case where a region of a target object is manually selected from the image data 5A will be described.
[0069] Figure 8 5A is a diagram showing image data 5A displayed on the monitor 9. Figure 8In the description, the case where the image data 5A is displayed is described, but a composite image of the image data 5A to 5D may be displayed.
[0070] The display control unit 19 displays the image data 5A on the monitor 9. The display control unit 19 displays the region G in the image data 5A so that the user can confirm it. For example, the display control unit 19 sets the color of the region G to a color different from the surrounding color, or displays only the region G in a flashing manner, so as to notify the user of the position of the region G. The user observes the image data 5A displayed on the monitor 9, and selects the region R of the target object with a pointing device. The region selection unit 25 accepts the user's selection of the region R and selects the region R. In addition, the region R is the region of the object D, and since the user can confirm the object D in the image data 5A, the region R can be easily selected.
[0071] In addition, Figure 8 In the above description, the user manually selects the region R, but the present invention is not limited thereto. For example, the region selection unit 25 is composed of a recognizer that implements machine learning, and the region R can be selected by segmenting the image data 5A using the recognizer.
[0072] The recognizer recognizes the object D based on the image data 5A and selects the region R of the object D. The recognizer is composed of a recognizer that implements machine learning or deep learning, and specifies the region of the object by segmenting the image data 5A. In addition, existing models such as FCN (Fully Convolutional Network), SegNet, Pix2Pix, etc. can be used for segmentation, or a model that matches the recognition method of the object D performed by the recognizer can be independently constructed. In addition, the learning of the recognizer can use deep learning, or machine learning in a broad sense.
[0073] <Attribute Assignment Steps>
[0074] The attribute assigning step (step S13) is performed by the attribute assigning unit 27. The attribute assigning unit 27 assigns the same attribute information to the points corresponding to the region R in the point cloud data 7. Here, the attribute information refers to information indicating the surface of the object on which each point is reflected, in other words, information indicating the object to which each point corresponds. Since each pixel of the image data 5A corresponds to each point of the point cloud data 7, the attribute assigning unit 27 can specify the point cloud corresponding to the region R of the image data 5A and assign the attribute information (D). Here, since the region R is the region corresponding to the object D in the image data 5A, the point cloud assigned with the attribute information (D) becomes a point cloud reflected on the surface of the object D.
[0075] Fig. 9It is a diagram for explaining the point cloud data 7 to which attribute information (D) is given.
[0076] exist Fig. 9 In the point cloud data 7 shown, the same attribute information (D) is assigned to the point cloud U corresponding to the region R. Points assigned with the attribute information (D) are shown as black circles. The point cloud U is composed of point clouds assigned with the attribute information (D).
[0077] In this way, by assigning attribute information (D) to the point cloud U corresponding to the object D, the point cloud U can be easily used when creating a solid model or surface model corresponding to the object D. Specifically, the attribute information (D) can be used to extract only the point cloud U from the point cloud data 7 to generate a solid model or a surface model.
[0078] As described above, in this embodiment, a designated point is designated in the point cloud data 7, and a region R including the region G corresponding to the designated point is selected on the image data. Then, a point cloud U corresponding to the region R selected on the image data is identified in the point cloud data 7, and attribute information (D) is assigned to the point cloud U. The point cloud U assigned with the attribute information (D) is a point cloud reflected on the surface of the object D. Thus, this method can efficiently and accurately assign attribute information (D) to each point corresponding to the object D among the points constituting the point cloud data 7.
[0079] <Others>
[0080] In the above description, a case where a part of the point cloud U of the point cloud data 7 is grouped is described. However, the grouping of the point cloud U is not always done well. For example, in the selection of the region R in the image data 5A in the above description, when the selection of the boundary portion is slightly deviated, sometimes the attribute information cannot be accurately assigned. In this embodiment, in such a case, the attribute information is corrected. In this embodiment, the control unit 17 is provided with a correction unit (not shown) to correct the assigned attribute information.
[0081] Fig.10 9 is a diagram of a point cloud to which attribute information W is assigned. Fig.10 In the case shown, attribute information W is given to point cloud U corresponding to object D, but attribute information W is also given to point clouds other than point cloud corresponding to object D (point cloud UE).
[0082] The display control unit 19 displays the point cloud (point cloud U and point cloud UE) to which the attribute information W is assigned on the monitor 9. The user confirms the displayed point cloud, determines that the point cloud UE does not correspond to the object D, selects the point cloud UE with the pointing device, and corrects the attribute information W assigned to the point cloud UE. The correction unit accepts the correction by the user and corrects the attribute information W assigned to the point cloud UE.
[0083] Fig.11 9 is a diagram showing a point cloud to which attribute information W is assigned and displayed on the monitor 9. Fig.11 In the case shown, attribute information W is assigned to a point cloud U corresponding to an object D, but a missing region (region UH) exists in the point cloud to which attribute information W is assigned.
[0084] The display control unit 19 displays the point cloud U to which the attribute information W is assigned on the monitor 9. The user confirms the displayed area UH and adds point clouds using a pointing device to fill the area UH. Thus, the correction unit adds point clouds having the attribute information W to the area UH.
[0085] In the above example, the user manually corrects the attribute information that has been assigned, but the present invention is not limited to this example. For example, the temporarily assigned attribute information may be corrected using a recognizer that has performed machine learning.
[0086] As described above, in the present embodiment, since correction is made to the temporarily assigned attribute information, it is possible to more accurately assign attribute information to the point cloud.
[0087] In the above-mentioned embodiment, the hardware structure of the processing unit that performs various processing is various processors (processors) as shown below. Among the various processors, there are general-purpose processors that execute software (programs) and function as various processing units, namely CPUs, FPGAs (Field Programmable Gate Arrays) and other processors whose circuit structures can be changed after manufacturing, namely programmable logic devices (PLDs), ASICs (Application Specific Integrated Circuits) and other processors with circuit structures specially designed for performing specific processing, namely dedicated circuits, etc.
[0088] A processing unit can be composed of one of these various processors, or it can be composed of two or more processors of the same or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA). In addition, multiple processing units can be composed of one processor. As an example of using one processor to constitute multiple processing units, first, there is a form such as a computer such as a user end or a server, which uses a combination of one or more CPUs and software to constitute a processor, and the processor functions as multiple processing units. Secondly, there is a form such as a system on chip (System On Chip: SoC) and the like, which uses a processor that implements the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip. In this way, various processing units are composed of one or more of the above-mentioned various processors as a hardware structure.
[0089] Furthermore, more specifically, the hardware structure of these various processors is a circuit (circuitry) in which circuit elements such as semiconductor elements are combined.
[0090] The above structures and functions can be appropriately implemented by any hardware, software, or a combination of the two. For example, the present invention can also be applied to a program that causes a computer to execute the above processing steps (processing procedures), a computer-readable recording medium (non-temporary recording medium) that records such a program, or a computer that can install such a program.
[0091] As mentioned above, although the example of this invention was demonstrated, this invention is not limited to the said embodiment, It goes without saying that various deformation|transformation is possible within the range which does not deviate from the spirit of this invention.
[0092] Explanation of symbols
[0093] 1: 3D measuring instrument
[0094] 5: Image data
[0095] 7: Point cloud data
[0096] 9: Monitor
[0097] 11: Point cloud data processing device
[0098] 13: Image data acquisition unit
[0099] 15: Point cloud data acquisition department
[0100] 17: Control Department
[0101] 19: Display control unit
[0102] 21: Storage
[0103] 23: Designated receiving department
[0104] 25: Area Selection Department
[0105] 27: Attribute Assignment
[0106] 101: Laser Scanner
[0107] 102: Photographic Installation
[0108] 103: Half mirror
Claims
1. A point cloud data processing device, comprising a memory and a processor, wherein the memory stores point cloud data representing three-dimensional information of a plurality of points on a surface of an object and a plurality of image data obtained by photographing the object, and associates at least the position of a pixel of any one of the plurality of image data with a point constituting the point cloud data, wherein: The processor The point cloud data is displayed on a display unit in a manner that allows three-dimensional rotation, three-dimensional movement, and enlargement and reduction. receiving designation of a designated point in the point cloud data displayed on the display unit, In the image data, a region of the target object including a region corresponding to the designated point is selected, assigning the same attribute information to the points in the point cloud data corresponding to the area of the target object, The attribute information is information indicating to which object each point corresponds.
2. The point cloud data processing device according to claim 1, wherein: The processor selects a region of the target object in the image data using a recognizer that has performed machine learning.
3. The point cloud data processing device according to claim 1 or 2, wherein: The processor, upon receiving the designation of the designated point, causes the display unit to display the image data of the area corresponding to the designated point.
4. The point cloud data processing device according to claim 3, wherein: The processor synthesizes the plurality of image data and displays the synthesized image data on the display unit.
5. The point cloud data processing device according to claim 3, wherein: The processor receives selection of a region of the target object in the image data displayed on the display unit.
6. The point cloud data processing device according to claim 1 or 2, wherein: The processor corrects the attribute information by using a recognizer that has performed machine learning to the point cloud data to which the attribute information has been assigned.
7. The point cloud data processing device according to claim 1 or 2, wherein: The processor displaying the point cloud data to which the attribute information is assigned on a display unit, A correction related to the assigned attribute information is received, and the assigned attribute information is corrected.
8. The point cloud data processing device according to claim 1 or 2, wherein: The processor acquires the image data and the point cloud data acquired by a device having a same optical axis.
9. A point cloud data processing method, which is a point cloud data processing method of a point cloud data processing device, the point cloud data processing device having a memory and a processor, the memory storing point cloud data representing three-dimensional information of multiple points on the surface of an object and multiple image data obtained by photographing the object, and at least associating the position of a pixel of any one of the multiple image data with the points constituting the point cloud data, wherein: The processor performs: The step of displaying the point cloud data on a display unit in a manner that allows three-dimensional rotation, three-dimensional movement, and enlargement and reduction; a step of accepting designation of a designated point in the point cloud data displayed on the display unit; A step of selecting a region of the target object including a region corresponding to the designated point in the image data; as well as a step of assigning the same attribute information to the points corresponding to the area of the target object in the point cloud data, The attribute information is information indicating to which object each point corresponds.
10. A computer-readable recording medium storing a program, wherein the program causes a point cloud data processing device to execute a point cloud data processing method, wherein the point cloud data processing device comprises a memory and a processor, wherein the memory stores point cloud data representing three-dimensional information of a plurality of points on a surface of an object and a plurality of image data obtained by photographing the object, and at least associates the position of a pixel of any one of the plurality of image data with a point constituting the point cloud data, wherein: The program causes the processor to: The step of displaying the point cloud data on a display unit in a manner that allows three-dimensional rotation, three-dimensional movement, and enlargement and reduction; a step of accepting designation of a designated point in the point cloud data displayed on the display unit; A step of selecting a region of the target object including a region corresponding to the designated point in the image data; as well as A step of assigning the same attribute information to the points corresponding to the region of the target object in the point cloud data, wherein the attribute information is information indicating to which object each point corresponds.