Data processing device, data processing system, data processing method, and data processing program

Abstract

A decomposition unit (113): decomposes, for individual objects present in a relevant region, point cloud data in source data that includes the point cloud data, which is obtained by a positioning device for measuring the position of a reflection point by irradiating the relevant region with irradiation light and receiving reflected light reflected at the reflection point, and image data for the relevant region, the image data being such that individual pieces of point data constituting the point cloud data and pixels are associated with each other; and generates decomposition data that is point cloud data for each object. A corresponding data identification unit (116) identifies decomposition data that pertains to a designated object, among the items of decomposition data for each object, and partial data that is configured from pixels associated with the decomposition data pertaining to the designated object, among the items of image data included in the source data.

Description

Data processing device, data processing system, data processing method and data processing program

[0001] The present disclosure relates to a technique for handling three-dimensional point cloud data.

[0002] A positioning device such as a LiDAR camera is used to acquire three-dimensional point cloud data of an object. LiDAR stands for Light Detection and Ranging. For example, Patent Document 1 describes acquiring point cloud data using a positioning device and measuring the size of an object.

[0003] Japanese Patent Application Laid-Open No. 2023-023348

[0004] When point cloud data is acquired using a positioning device, point cloud data is acquired for the entire space illuminated by light. In other words, point cloud data is acquired not only for the target object for which point cloud data is to be acquired, but also for surrounding objects, floors, walls, etc. It is conceivable to use the point cloud data of a target object to consider layouts when the target object is placed in another location. However, extracting point cloud data for a target object from point cloud data that also includes surrounding objects requires processing and cropping using dedicated software, which is time-consuming. The present disclosure aims to easily extract point cloud data for a target object from point cloud data that also includes surrounding objects, etc.

[0005] The data processing device according to the present disclosure includes: point cloud data obtained by a positioning device that measures the position of a reflection point by irradiating a target area with irradiation light and receiving reflected light reflected at the reflection point; and original data including image data about the target area, in which each point data constituting the point cloud data is associated with a pixel, and a decomposition unit that decomposes the point cloud data into object-by-object to generate decomposed data that is point cloud data for each object; and a corresponding data identification unit that identifies decomposed data for a specified object from the decomposed data for each object generated by the decomposition unit and partial data composed of pixels associated with the decomposed data for the specified object from the image data included in the original data.

[0006] In the present disclosure, original data, which is point cloud data obtained by a positioning device, is decomposed for each object present in a target area to generate decomposed data, which is point cloud data for each object. Then, partial data composed of pixels associated with the decomposed data for a specified object is identified. This allows the point cloud data for the specified object and the partial data for the specified object in the image data to be obtained.

[0007] FIG. 1 is a configuration diagram of a data processing system 100 according to embodiment 1. FIG. 1 is a configuration diagram of a data processing device 10 according to embodiment 1. FIG. 2 is a flowchart of data registration processing according to embodiment 1. FIG. 3 is a flowchart of data extraction processing according to embodiment 1. FIG. 4 is an explanatory diagram of decomposition processing according to embodiment 1. FIG. 5 is a flowchart of data editing processing according to embodiment 1. FIG. 6 is an explanatory diagram of a usage example of the data processing system 100 according to embodiment 1. FIG. 7 is an explanatory diagram of the data structures of original data 51 and decomposed data according to embodiment 1. FIG. 8 is a configuration diagram of a data processing system 100 according to embodiment 2. FIG. 9 is a configuration diagram of a data processing device 10 according to embodiment 2. FIG. 10 is a configuration diagram of an imaging device 20 according to embodiment 2. FIG. 11 is a configuration diagram of an operation terminal 30 according to embodiment 2. FIG. 11 is a flowchart of data registration processing according to embodiment 2. FIG. 12 is a flowchart of data extraction processing according to embodiment 2. FIG. 13 is a flowchart of data editing processing according to embodiment 2.

[0008] First Embodiment *** Description of Configuration *** The configuration of a data processing system 100 according to the first embodiment will be described with reference to Fig. 1. The data processing system 100 according to the first embodiment includes a data processing device 10. In the first embodiment, the data processing device 10 is a computer such as a smartphone or a tablet terminal.

[0009] The configuration of a data processing device 10 according to the first embodiment will be described with reference to Fig. 2. The data processing device 10 is a computer. The data processing device 10 includes the following hardware: a processor 11, a memory 12, a storage 13, a communication interface 14, a camera 15, a positioning device 16, and a display device 17. The processor 11 is connected to other hardware via signal lines and controls the other hardware.

[0010] The processor 11 is an IC that performs processing. IC stands for Integrated Circuit. Specific examples of the processor 11 include a CPU, a DSP, and a GPU. CPU stands for Central Processing Unit. DSP stands for Digital Signal Processor. GPU stands for Graphics Processing Unit.

[0011] The memory 12 is a storage device that temporarily stores data. Specific examples of the memory 12 include SRAM and DRAM. SRAM stands for Static Random Access Memory. DRAM stands for Dynamic Random Access Memory.

[0012] The storage 13 is a storage device that stores data. A specific example of the storage 13 is an SSD. SSD is an abbreviation for Solid State Drive. The storage 13 may also be a portable recording medium such as an SD (registered trademark) memory card, CompactFlash (registered trademark), NAND flash, a flexible disk, an optical disk, a compact disk, a Blu-ray (registered trademark) disk, or a DVD. SD is an abbreviation for Secure Digital. DVD is an abbreviation for Digital Versatile Disk.

[0013] The communication interface 14 is an interface for communicating with external devices. Specific examples of the communication interface 14 include Ethernet (registered trademark), USB, and HDMI (registered trademark) ports. USB stands for Universal Serial Bus. HDMI stands for High-Definition Multimedia Interface.

[0014] The camera 15 is a photographing device such as an optical camera for acquiring image data.

[0015] The positioning device 16 is a device that measures the position of a reflection point by irradiating a target area with irradiation light and receiving the light reflected by the reflection point.

[0016] The display device 17 is a device such as an LCD for displaying information. LCD stands for Liquid Crystal Display.

[0017] The data processing device 10 includes, as functional components, an imaging unit 111, a measurement unit 112, a decomposition unit 113, a display unit 114, a designation receiving unit 115, a corresponding data identification unit 116, a data extraction unit 117, and an editing unit 118. The functions of each functional component of the data processing device 10 are realized by software. The storage 13 stores programs that realize the functions of each functional component of the data processing device 10. These programs are loaded into the memory 12 by the processor 11 and executed by the processor 11. In this way, the functions of each functional component of the data processing device 10 are realized.

[0018] 2 shows only one processor 11. However, there may be a plurality of processors 11, and the plurality of processors 11 may cooperate to execute programs that realize the respective functions.

[0019] ***Description of Operation*** The operation of the data processing system 100 according to the first embodiment will be described with reference to Figures 3 to 8. The operating procedure of the data processing system 100 according to the first embodiment corresponds to the data processing method according to the first embodiment. Furthermore, the program that realizes the operation of the data processing system 100 according to the first embodiment corresponds to the data processing program according to the first embodiment.

[0020] The operations of the data processing system 100 include a data registration process, a data extraction process, and a data editing process. The data registration process is a process of acquiring image data and point cloud data of a target area and registering them as original data 51. The data extraction process is a process of identifying image data and point cloud data for a specified object from the original data 51 and extracting them as editing data 52. The data editing process is a process of performing editing using the editing data 52.

[0021] The data registration process according to the first embodiment will be described with reference to Fig. 3. (Step S11: Photographing process) The photographing unit 111 photographs the target area using the camera 15 and acquires image data of the target area. At this time, the user may acquire multiple image data of the target area by photographing a video of the target area while moving the data processing device 10.

[0022] (Step S12: Measurement Processing) Simultaneously with the photographing in step S11, the measurement unit 112 measures the positions of the reflection points by irradiating the target area with irradiation light and receiving the reflected light reflected from the reflection points using the positioning device 16. In this way, the measurement unit 112 acquires point cloud data indicating the positions of multiple reflection points in the target area.

[0023] (Step S13: Registration Process) The measurement unit 112 associates each pixel of the one or more pieces of image data acquired in step S11 with each piece of point data of the point cloud data acquired in step S12. Then, the measurement unit 112 writes the associated point cloud data and image data to the storage 13 as original data 51. Associating each pixel of the image data with each piece of point data of the point cloud data means associating pixels and point data for the same position. By simultaneously acquiring image data and point cloud data, it is possible to associate pixels and point data for the same position.

[0024] The data extraction process according to the first embodiment will be described with reference to Fig. 4. (Step S21: Processing Request Acceptance Process) The decomposition unit 113 accepts a processing request for the original data 51. If multiple pieces of original data 51 are stored in the storage 13, the processing request accepts a processing request including a designation of the original data 51 to be processed. At this time, the multiple pieces of original data 51 may be displayed to accept a selection from the user.

[0025] (Step S22: Decomposition Processing) The decomposition unit 113 sets the original data 51 specified in step S21 as the original data 51 to be processed. The decomposition unit 113 decomposes the point cloud data in the original data 51 to be processed for each object present in the target area, and generates decomposed data, which is point cloud data for each object. The decomposition unit 113 registers the decomposed data in the storage 13 in association with the original data 51. For example, as shown in FIG. 5 , it is assumed that objects A, B, and C exist in the target area. In this case, the decomposition unit 113 detects objects A, B, and C present in the target area, and decomposes the point cloud data into objects A, B, C, and others (floors, walls, etc.). As a result, decomposed data, which is point cloud data for object A, object B, and object C, is generated.

[0026] Specifically, the decomposition unit 113 removes floor and wall point data from the point cloud data. Using existing surface estimation technology, the decomposition unit 113 can identify floor and wall portions. The decomposition unit 113 clusters the remaining point data based on rules based on the distance between objects. The decomposition unit 113 can cluster the remaining point data using the X-means method or the like. The decomposition unit 113 logically decomposes each piece of point cloud data separated by clustering into decomposed data. Alternatively, the decomposition unit 113 recognizes objects using a learning model generated by deep learning and logically decomposes each point cloud into decomposed data. The decomposition unit 113 also logically decomposes the removed floor and wall portions into decomposed data. Here, "logical decomposition" refers to a state in which each piece of point data is assigned a label that allows identification of the decomposed data, which is point cloud data for each object. The decomposition unit 113 then generates a physically decomposed OBJ file from the logically decomposed decomposed data. OBJ files will be described later.

[0027] (Step S23: Display Processing) The display unit 114 displays the original data 51 specified in step S21 on the display device 17. Specifically, the display unit 114 pastes the image data included in the original data 51 in accordance with the positions of each point data of the point cloud data included in the original data 51. As a result, the image data is combined with the point cloud data, and a 3D model of the target area is generated. Then, the display unit 114 displays the 3D model of the target area.

[0028] (Step S24: Designation Receiving Process) The designation receiving unit 115 receives designation of an object to be processed from one or more objects included in the 3D model displayed in step S23. For example, the object to be processed is designated by touching the object to be processed. At this time, the designation receiving unit 115 also receives designation of whether the request is a cut request to cut and extract data of the designated object, or a delete request to delete data of the designated object and extract the remaining data.

[0029] (Step S25: Corresponding Data Identification Process) The corresponding data identification unit 116 identifies the decomposed data for the specified object specified in step S24 from the decomposed data generated in step S22. The corresponding data identification unit 116 can identify the decomposed data for the specified object by identifying the decomposed data including the point data of the touched position. Furthermore, the corresponding data identification unit 116 identifies partial data composed of pixels associated with the decomposed data for the specified object from the image data included in the original data to be processed.

[0030] (Step S26: Extraction Process) The data extraction unit 117 extracts editing data 52 from the decomposed data and partial data for the specified object identified in step S25. The data extraction unit 117 determines whether a cut request or a delete request was specified in step S24. If a cut request was specified, the data extraction unit 117 extracts the decomposed data and partial data for the specified object as editing data 52. If a delete request was specified, the data extraction unit 117 extracts the decomposed data and partial data for the specified object removed from the original data 51 as editing data 52. Here, when the decomposed data is removed from the point cloud data included in the original data 51, point data for the portion where the decomposed data was located is missing. The data extraction unit 117 may fill in the missing point data using point data of the walls and floor. Specifically, the data extraction unit 117 may fill in the missing point data by extending the walls and floor assuming that the walls and floor are flat. Similarly, the data extraction unit 117 may also complement missing portions of the image data using image data of the walls and floor.

[0031] The data editing process according to the first embodiment will be described with reference to Figure 6. (Step S31: Data Reading Process) The editing unit 118 reads the editing data 52 extracted in step S26. The display unit 114 then displays the read editing data 52. At this time, the display unit 114 displays a three-dimensional model synthesized by pasting the image data included in the editing data 52 in accordance with the positions of each point data of the point cloud data included in the editing data 52.

[0032] The data extraction process may be executed multiple times, resulting in multiple pieces of editing data 52 being extracted. In this case, the editing unit 118 reads the multiple pieces of editing data 52. The display unit 114 then displays the multiple pieces of editing data 52 that have been read on a single screen. That is, a 3D model obtained by combining image data with point cloud data for each piece of editing data 52 is displayed on a single screen. At this time, the display unit 114 displays the multiple pieces of editing data 52 at the same scale. That is, the display unit 114 displays each piece of editing data 52 after matching its scale to a reference scale. In the data registration process shown in FIG. 3 , the point cloud data acquired by the same data processing device 10 have the same scale, so there is no need to match the scales. However, point cloud data acquired by different data processing devices 10 may have different scales, so matching becomes necessary.

[0033] Furthermore, the editing unit 118 may load a 3D model created by CAD or the like in addition to the editing data 52. CAD stands for Computer-Aided Design. In this case, the display unit 114 displays the loaded editing data 52 and the 3D model created by CAD or the like on a single screen. That is, the 3D model obtained by combining image data with point cloud data for the editing data 52 and the 3D model created by CAD or the like are displayed on a single screen. In this case, the display unit 114 displays the scale of the editing data 52 and the scale of the 3D model created by CAD or the like together.

[0034] (Step S32: Editing process) The editing unit 118 accepts editing of the editing data 52 etc. displayed in step S31. When a plurality of three-dimensional models of the editing data 52 are displayed, or when a three-dimensional model of the editing data 52 and a three-dimensional model created by CAD or the like are displayed, the editing unit 118 accepts editing of combination data that combines a plurality of three-dimensional models.

[0035] An example of use of the data processing system 100 according to the first embodiment will be described with reference to FIG. 7 . First, in the data registration process, data about rooms A and B is acquired and registered as original data 51. In the data extraction process, object X is designated as a specified object in the original data 51 about room A, and a cut request is issued. As a result, point cloud data and image data about object X are extracted as editing data 52. Furthermore, all objects are designated as specified objects in the original data 51 about room B, and a delete request is issued. As a result, the point cloud data of all objects is removed from room B, and the remaining point cloud data and image data corresponding to the remaining point cloud data are extracted as editing data 52. In other words, the point cloud data and image data of room B in an empty state after all objects have been removed are extracted as editing data 52. In the data editing process, the editing data 52 about object X and the editing data 52 about room B are displayed on a single screen. In other words, a three-dimensional model of object X and a three-dimensional model of room B in an empty state are displayed on a single screen. Then, the 3D model of object X is moved and superimposed on the 3D model of room B. This makes it possible to check what state object X will be in when placed in room B. It is also possible to duplicate the 3D model of object X and check what state multiple objects X will be in when placed in room B.

[0036] Here, the 3D model of object X and the 3D model of empty room B are both generated from the original data 51 registered in the data registration process. However, either the 3D model of object X or the 3D model of empty room B may be a 3D model created using CAD or the like. In other words, it is also possible to combine and edit a 3D model obtained by extracting point cloud data and image data of some objects, or a 3D model obtained by excluding point cloud data of some objects, with a 3D model created using CAD or the like.

[0037] Use example 2 will be described. This is used when checking whether object Y placed in space C can be placed in space B by passing through space A. New original data 51 is generated by a request to combine original data 51 for space A with original data 51 for space B. Then, object Y is designated as a specified object in the original data 51 for space C, and a cut request is issued. As a result, point cloud data and image data for object Y are extracted as editing data 52. In the data editing process, the editing data 52 for object Y and the editing data 52 in which spaces A and B are combined are displayed on a single screen. That is, a three-dimensional model of object Y and a three-dimensional model in which spaces A and B are combined are displayed on a single screen. Then, the three-dimensional model of object Y is moved so that it is superimposed on the three-dimensional model in which spaces A and B are combined. This makes it possible to check what state object Y will appear in if it is placed in the space in which spaces A and B are combined. It is also possible to duplicate the three-dimensional model of object Y and check what state it will be in when multiple objects Y are placed in a space where space A and space B are combined. In this case, the three-dimensional model of object Y does not have to be generated from original data 51, but may be a three-dimensional model created using CAD or the like. As explained in connection with space A and space B, original data 51 of three or more spaces can also be combined in accordance with the carry-in route of object Y. This makes it possible to check whether object Y can be carried in and installed along the carry-in route from the carry-in entrance to the installation location.

[0038] Use example 3 will be described. This is used when checking whether object Z placed in space C can be placed in place of object ZZ that is located in space A and is scheduled to be removed. Object ZZ is designated as a specified object in the original data 51 for space A, and a deletion request is issued. As a result, the point cloud data of object ZZ is removed from space A, and the remaining point cloud data and image data corresponding to the remaining point cloud data are extracted as editing data 52. Then, object Z is designated as a specified object in the original data 51 for space C, and a cut request is issued. As a result, the point cloud data and image data for object Z are extracted as editing data 52. In the data editing process, the editing data 52 for object Z and the editing data 52 in which the point cloud data of object ZZ has been removed from space A are displayed on a single screen. That is, a three-dimensional model of object Z and a three-dimensional model in which object ZZ has been removed from space A are displayed on a single screen. Then, the three-dimensional model of object Z is moved so that it is superimposed on the three-dimensional model in which object ZZ has been removed from space A. This makes it possible to check what state the object Z will be in when placed in space A. It is also possible to duplicate the three-dimensional model of object Z and check what state a plurality of objects Z will be in when placed in space A. In this case, the three-dimensional model of object Z does not necessarily have to be one generated from the original data 51, but may be a three-dimensional model created by CAD or the like.

[0039] The data structures of the original data 51 and decomposed data according to the first embodiment will be described with reference to FIG. 8 . The original data 51 includes an OBJ file, an MTL file, a texture file, an image file, and a thumbnail file. The OBJ file is a three-dimensional data file indicating coordinate data of a three-dimensional point cloud, etc., and contains information indicating at which position in the three-dimensional data image data is to be applied as a texture based on the texture coordinates. The MTL file is a file describing information required for texture mapping. The texture file is image data of a texture to be applied to the three-dimensional point cloud indicated by the OBJ file according to the MTL file. The image file is image data acquired in step S11 and is configured, for example, as a JPG file. The thumbnail file is data obtained by reducing the image data indicated by the image file. As for the decomposed data, an OBJ file is set for each object in the original data 51. The decomposition data is set for each object as an OBJ file, so when a specified object is identified, the OBJ file of the specified object is identified, and the information held in the OBJ file identifies partial data, which is image data of the specified object corresponding to the three-dimensional data of the OBJ file.

[0040] ***Effects of First Embodiment*** As described above, the data processing system 100 according to the first embodiment decomposes the original data 51, which is point cloud data obtained by the positioning device 16, for each object present in the target area, to generate decomposed data, which is point cloud data for each object. The data processing system 100 then identifies the decomposed data for the specified object and partial data composed of pixels associated with the decomposed data. This allows the point cloud data for the specified object and a partial image of the specified object in the image data to be obtained. In particular, by simply specifying the specified object, it is possible to obtain the point cloud data for the specified object and a partial image of the specified object in the image data, without having to enclose the specified object or the like.

[0041] Since point cloud data and partial images of the specified object are obtained, it becomes easy to combine and edit a three-dimensional model of the specified object with other three-dimensional models.

[0042] Furthermore, when displaying multiple 3D models on a single screen, the data processing system 100 according to the first embodiment displays the 3D models at the same scale. This allows for easy layout adjustments, etc. For example, if the designated object is a large object such as furniture, the data processing system 100 can be used to capture an image of an area including the entrance to the room and the planned installation location of the furniture as a target area. This makes it possible to confirm whether the furniture can be delivered along a delivery route from the entrance to the planned installation location, which is difficult to see visually. It is also possible to confirm whether the furniture can be installed in the planned installation location. Other designated objects include vending machines, various manufacturing equipment, floor-mounted air conditioners, and air conditioners installed on the wall or ceiling of a room. The data processing system 100 according to the first embodiment allows for confirmation of the delivery route and the planned installation location when installing a new designated object in the planned installation location or when replacing an existing designated object in the planned installation location.

[0043] Second Embodiment The second embodiment differs from the first embodiment in that the data processing system 100 is made up of a plurality of devices. In the second embodiment, this difference will be explained, and explanation of the same points will be omitted.

[0044] ***Description of Configuration*** The configuration of a data processing system 100 according to the second embodiment will be described with reference to FIG. 9 . The data processing system 100 includes a data processing device 10, an image capturing device 20, and an operation terminal 30. In the second embodiment, the data processing device 10 is a computer such as a server. The image capturing device 20 is a computer such as a smartphone or a tablet terminal. The operation terminal 30 is a computer such as a smartphone, a tablet terminal, or a PC. PC stands for Personal Computer. The data processing device 10, the image capturing device 20, and the operation terminal 30 are connected via a network 40. The image capturing device 20 and the operation terminal 30 may be implemented by a single computer. For example, a single smartphone may implement both the image capturing device 20 and the operation terminal 30.

[0045] The configuration of a data processing device 10 according to the second embodiment will be described with reference to Fig. 10. The data processing device 10 differs from the data processing device 10 shown in Fig. 2 in that the data processing device 10 does not include, as hardware, a camera 15, a positioning device 16, and a display device 17. The data processing device 10 also differs from the data processing device 10 shown in Fig. 2 in that the data processing device 10 does not include, as functional components, an imaging unit 111 and a measurement unit 112.

[0046] The configuration of the image capturing device 20 according to the second embodiment will be described with reference to Fig. 11. The image capturing device 20 includes the following hardware components: a processor 21, a memory 22, a storage 23, a communication interface 24, a camera 25, and a positioning device 26. The processor 21 is connected to other hardware components via signal lines and controls the other hardware components. The processor 21, the memory 22, the storage 23, the communication interface 24, the camera 25, and the positioning device 26 are similar to the processor 11, the memory 12, the storage 13, the communication interface 14, the camera 15, and the positioning device 16, respectively.

[0047] The photographing device 20 includes, as functional components, a photographing unit 211, a measuring unit 212, and a communication unit 213. The functions of each functional component of the photographing device 20 are realized by software. The storage 23 stores a program that realizes the function of each functional component of the photographing device 20. This program is read into the memory 22 by the processor 21 and executed by the processor 21. In this way, the function of each functional component of the photographing device 20 is realized.

[0048] The configuration of the operation terminal 30 according to the second embodiment will be described with reference to Fig. 12. The operation terminal 30 includes the following hardware: a processor 31, a memory 32, a storage 33, a communication interface 34, and a display device 35. The processor 31 is connected to other hardware via signal lines and controls the other hardware. The processor 31, memory 32, storage 33, communication interface 34, and display device 35 are similar to the processor 11, memory 12, storage 13, communication interface 14, and display device 17, respectively.

[0049] The operation terminal 30 includes, as functional components, an instruction unit 311 and a communication unit 312. The functions of each functional component of the operation terminal 30 are realized by software. The storage 33 stores a program that realizes the function of each functional component of the operation terminal 30. This program is read into the memory 32 by the processor 31 and executed by the processor 31. In this way, the function of each functional component of the operation terminal 30 is realized.

[0050] ***Description of Operation*** The operation of the data processing system 100 according to the second embodiment will be described with reference to FIGS.

[0051] The data registration process according to the second embodiment will be described with reference to Fig. 13. (Step S11A: Photography Process) As in step S11 of Fig. 3, the photography unit 211 of the photography device 20 photographs a target area to be photographed using the camera 25 and acquires image data of the target area.

[0052] (Step S12A: Measurement Process) As in step S12 of FIG. 3, the measurement unit 212 of the image capturing device 20 uses the positioning device 26 to acquire point cloud data indicating the positions of a plurality of reflection points in the target area.

[0053] (Step S13A: Registration Processing) The communication unit 213 of the imaging device 20 associates each pixel of the one or more pieces of image data acquired in step S11A with each point data of the point cloud data acquired in step S12A, and transmits the resulting data to the data processing device 10 as original data 51. As a result, the original data 51 is written to the storage 13 of the data processing device 10.

[0054] The data extraction process according to the second embodiment will be described with reference to Fig. 14. (Step S21A: Processing Request Acceptance Process) The instruction unit 311 of the operation terminal 30 accepts a processing request for the original data 51, similar to step S21 in Fig. 4. Then, the communication unit 312 of the operation terminal 30 transmits the processing request to the data processing device 10.

[0055] (Step S22A: Decomposition processing) As in step S22 of Figure 4, the decomposition unit 113 of the data processing device 10 generates decomposed data, which is point cloud data for each object, using the original data 51 specified in step S21A as the original data 51 to be processed, and registers it in the storage 13.

[0056] (Step S23A: Display Process) As in step S23 of FIG. 4, the display unit 114 of the data processing device 10 transmits the original data 51 specified in step S21A to the operation terminal 30 and displays it on the display device 35.

[0057] (Step S24A: Designation Receiving Process) As in step S24 of FIG. 4 , the instruction unit 311 of the operation terminal 30 accepts designation of an object to be processed from among one or more objects included in the 3D model displayed in step S23A. At this time, the instruction unit 311 also accepts designation of whether the request is a cut request or a delete request. Then, the communication unit 312 of the operation terminal 30 transmits information indicating the object to be processed and information indicating whether the request is a cut request or a delete request to the data processing device 10.

[0058] (Step S25A: Corresponding Data Identification Process) The corresponding data identification unit 116 of the data processing device 10 identifies decomposed data for the specified object specified in step S24A from the decomposed data generated in step S22A, similar to step S25 in Fig. 4. Furthermore, the corresponding data identification unit 116 identifies partial data composed of pixels associated with the decomposed data for the specified object.

[0059] (Step S26A: Extraction Process) The data extraction unit 117 of the data processing device 10 extracts the editing data 52 from the decomposed data and partial data for the specified object identified in step S25A, similar to step S26 in Fig. 4. The data extraction unit 117 may transmit the editing data 52 to the operation terminal 30 in response to a request from the operation terminal 30.

[0060] The data editing process according to the second embodiment will be described with reference to Fig. 15. (Step S31: Data Reading Process) The editing unit 118 of the data processing device 10 reads the editing data 52 extracted in step S26, similar to step S31 in Fig. 6. The display unit 114 then transmits the read editing data 52 to the operation terminal 30 and displays it on the display device 35.

[0061] (Step S32: Editing Process) The editing unit 118 of the data processing device 10 accepts editing of the editing data 52 etc. displayed in step S31, similar to step S32 in Fig. 6. Here, the editing unit 118 accepts editing from the operation terminal 30.

[0062] ***Effects of Embodiment 2*** As described above, the data processing system 100 according to Embodiment 2 is realized by the data processing device 10, the imaging device 20, and the operation terminal 30. Even in this case, it is possible to achieve the same effects as those of the data processing system 100 according to Embodiment 1.

[0063] 14, by transmitting the editing data 52 to the operation terminal 30, it is possible to perform editing using the editing data 52 on the operation terminal 30 side. For example, it is also possible to read the editing data 52 into CAD software and perform editing.

[0064] ***Other Configurations*** <Modification 1> In the first embodiment, each functional component is realized by software. However, in Modification 1, each functional component may be realized by hardware. The differences between Modification 1 and the first embodiment will be described below.

[0065] When each functional component is realized by hardware, the data processing device 10 includes an electronic circuit in place of the processor 11, the memory 12, and the storage 13. The electronic circuit is a dedicated circuit for realizing the functions of each functional component, the memory 12, and the storage 13.

[0066] Similarly, when each functional component is realized by hardware, the imaging device 20 includes an electronic circuit instead of the processor 21, the memory 22, and the storage 23. The electronic circuit is a dedicated circuit that realizes the functions of each functional component, the memory 22, and the storage 23.

[0067] Similarly, when each functional component is realized by hardware, the operation terminal 30 includes an electronic circuit in place of the processor 31, the memory 32, and the storage 33. The electronic circuit is a dedicated circuit that realizes the functions of each functional component, the memory 32, and the storage 33.

[0068] Possible electronic circuits include a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, a logic IC, a GA, an ASIC, and an FPGA. GA stands for Gate Array. ASIC stands for Application Specific Integrated Circuit. FPGA stands for Field-Programmable Gate Array. Each functional component may be realized by a single electronic circuit, or each functional component may be distributed across multiple electronic circuits.

[0069] <Modification 2> As a modification 2, some of the functional components may be realized by hardware, and other functional components may be realized by software.

[0070] The processors 11, 21, and 31, the memories 12, 22, and 32, the storages 13, 23, and 33, and the electronic circuits are collectively referred to as a processing circuit. In other words, the functions of the functional components are realized by the processing circuit.

[0071] Furthermore, the term "unit" in the above description may be read as a "circuit," "step," "procedure," "process," or "processing circuit."

[0072] Aspects of the present disclosure are summarized below as appendices. (Appendix 1) A data processing system comprising: a decomposition unit that decomposes point cloud data in original data, the original data including point cloud data obtained by a positioning device that measures positions of reflection points by irradiating a target area with irradiation light and receiving reflected light reflected from the reflection points, for each object present in the target area, the point cloud data including image data for the target area in which each point data constituting the point cloud data is associated with a pixel, to generate decomposed data that is point cloud data for each object; and a correspondence data identification unit that identifies decomposed data for a specified object from the decomposed data for each object generated by the decomposition unit and partial data consisting of pixels associated with the decomposed data for the specified object from image data included in the original data. (Appendix 2) The data processing system according to Appendices 1, further comprising: a data extraction unit that extracts the decomposed data and the partial data for the specified object identified by the correspondence data identification unit as editing data. (Supplementary Note 3) The data processing system according to Supplementary Note 1, further comprising: a data extraction unit that extracts, as editing data, data obtained by excluding from the original data the decomposed data and the partial data for the designated object identified by the corresponding data identification unit. (Supplementary Note 4) The data processing system according to Supplementary Note 2 or 3, further comprising: a display unit that displays a three-dimensional model obtained by combining the image data with the point cloud data included in the original data, and a designation receiving unit that receives designation of the designated object from the three-dimensional model displayed by the display unit. (Supplementary Note 5) The data processing system according to Supplementary Note 2 or 3, further comprising: a display unit that simultaneously displays the editing data extracted by the data extraction unit and other point cloud data different from the point cloud data, and an editing unit that receives editing of combined data that combines the editing data and the other point cloud data displayed by the display unit.(Supplementary Note 6) The data processing system according to Supplementary Note 5, wherein the display unit displays the editing data and the other point cloud data in a state where the scale of the editing data and the scale of the other point cloud data are matched. (Supplementary Note 7) The data processing system according to any one of Supplements 1 to 6, wherein the decomposition unit generates decomposed data for each object by removing point data of a floor from the point cloud data and clustering the remaining point data. (Supplementary Note 8) The data processing system according to any one of Supplements 1 to 7, wherein the decomposition unit generates the decomposed data for each object as a separate file. (Supplementary Note 9) A data processing method in which a computer decomposes point cloud data in original data, which includes point cloud data obtained by a positioning device that measures the position of a reflection point by irradiating a target area with irradiation light and receiving reflected light reflected from the reflection point, and image data about the target area, in which each point data constituting the point cloud data is associated with a pixel, for each object present in the target area to generate decomposed data, which is point cloud data for each object, and the computer identifies decomposed data for a specified object from the decomposed data for each object and partial data composed of pixels associated with the decomposed data for the specified object from the image data included in the original data. (Supplementary Note 10) A data processing program that causes a computer to execute the following steps: a decomposition process that decomposes point cloud data obtained by a positioning device that measures the position of a reflection point by irradiating a target area with irradiation light and receiving reflected light reflected from the reflection point, and original data that includes image data about the target area, the image data being image data in which each point data constituting the point cloud data is associated with a pixel, for each object present in the target area, to generate decomposed data that is point cloud data for each object; and a corresponding data identification process that identifies decomposed data for a specified object from the decomposed data for each object generated by the decomposition process, and partial data composed of pixels that are associated with the decomposed data for the specified object from image data included in the original data.

[0073] The embodiments and modifications of the present disclosure have been described above. Some of these embodiments and modifications may be combined and implemented. Furthermore, one or more of them may be implemented partially. Note that the present disclosure is not limited to the above embodiments and modifications, and various modifications are possible as needed.

[0074] 100 Data processing system, 10 Data processing device, 11 Processor, 12 Memory, 13 Storage, 14 Communication interface, 15 Camera, 16 Positioning device, 17 Display device, 111 Photography unit, 112 Measurement unit, 113 Decomposition unit, 114 Display unit, 115 Designation reception unit, 116 Corresponding data identification unit, 117 Data extraction unit, 118 Editing unit, 20 Photography device, 21 Processor, 22 Memory, 23 Storage, 24 Communication interface, 25 Camera, 26 Positioning device, 211 Photography unit, 212 Measurement unit, 213 Communication unit, 30 Operation terminal, 31 Processor, 32 Memory, 33 Storage, 34 Communication interface, 35 Display device, 311 Instruction unit, 312 Communication unit, 51 Original data, 52 Editing data.

Claims

1. A data processing device comprising: point cloud data obtained by a positioning device that measures the position of a reflection point by irradiating a target area with irradiation light and receiving reflected light reflected from the reflection point; and original data including image data about the target area in which each point data constituting the point cloud data is associated with a pixel; a decomposition unit that decomposes the point cloud data into object-by-object data to generate decomposed data that is point cloud data for each object; and a corresponding data identification unit that identifies decomposed data for a specified object from the decomposed data for each object generated by the decomposition unit and partial data composed of pixels associated with the decomposed data for the specified object from the image data included in the original data.

2. A data processing device connected to an image capturing device, wherein the image capturing device comprises a communication unit that transmits to the data processing device original data including point cloud data obtained by a positioning device that measures the position of a reflection point by irradiating a target area with irradiation light and receiving reflected light reflected from the reflection point, and image data about the target area, where each point data constituting the point cloud data is associated with a pixel; the data processing device comprises: a decomposition unit that decomposes the point cloud data in the original data transmitted from the image capturing device for each object present in the target area to generate decomposed data that is point cloud data for each object; and a corresponding data identification unit that identifies decomposed data for a specified object from the decomposed data for each object generated by the decomposition unit and partial data composed of pixels that are associated with the decomposed data for the specified object from the image data included in the original data.

3. The data processing device according to claim 1 or 2 further comprises: an instruction receiving unit that receives either a cut request to cut and extract data about a specified object or a delete request to delete the data about the specified object and extract the remaining data; and a data extraction unit that uses the partial data identified by the corresponding data identification unit to extract data as editing data depending on whether the instruction received by the instruction receiving unit indicates a cut request or a delete request.

4. A data processing device as described in claim 3, wherein the data extraction unit extracts the decomposed data and the partial data for the specified object as the editing data when the instruction indicates a cut request, and extracts data obtained by excluding the decomposed data and the partial data for the specified object from the original data as the editing data when the instruction indicates a delete request.

5. The data processing device according to claim 3, further comprising a display unit that displays a three-dimensional model obtained by combining the image data with the point cloud data included in the original data, and the instruction receiving unit receives the designation of the specified object from the three-dimensional model displayed by the display unit.

6. The data processing device according to claim 3, further comprising: a display unit that simultaneously displays the editing data extracted by the data extraction unit and other point cloud data different from the point cloud data; and an editing unit that accepts editing of combined data that combines the editing data displayed by the display unit with the other point cloud data.

7. A data processing device according to claim 6, wherein the display unit displays the editing data and the other point cloud data in a state where the scale of the editing data matches the scale of the other point cloud data.

8. A data processing system comprising an imaging device and a data processing device, wherein the imaging device comprises a communication unit that transmits to the data processing device original data including point cloud data obtained by a positioning device that measures the positions of reflection points by irradiating a target area with irradiation light and receiving reflected light reflected from the reflection points, and image data about the target area, where each point data constituting the point cloud data is associated with a pixel; and the data processing device comprises: a decomposition unit that decomposes the point cloud data in the original data transmitted from the imaging device for each object present in the target area to generate decomposed data that is point cloud data for each object; and a corresponding data identification unit that identifies decomposed data for a specified object from the decomposed data for each object generated by the decomposition unit and partial data composed of pixels that are associated with the decomposed data for the specified object from the image data included in the original data.

9. The data processing system according to claim 8, further comprising an operation terminal, wherein the operation terminal comprises: an instruction unit that receives either a cut request to cut and extract data about a specified object or a delete request to delete data about the specified object and extract the remaining data; and a communication unit that transmits request information indicating whether the instruction received by the instruction unit is a cut request or a delete request to the data processing device; and wherein the data processing device further comprises: a data extraction unit that uses the partial data identified by the corresponding data identification unit to extract, as data to be edited, data according to whether the request information transmitted by the communication unit indicates a cut request or a delete request.

10. A data processing system as described in claim 9, wherein the data extraction unit extracts the decomposed data and the partial data for the specified object as the editing data when the request information indicates a cut request, and extracts data obtained by excluding the decomposed data and the partial data for the specified object from the original data as the editing data when the request information indicates a delete request.

11. The data processing system according to claim 9, wherein the data processing device further comprises a display unit that displays a three-dimensional model obtained by combining the image data with the point cloud data contained in the original data, and the instruction unit accepts designation of the specified object from the three-dimensional model displayed by the display unit.

12. The data processing system according to claim 9, further comprising: a display unit that simultaneously displays the editing data extracted by the data extraction unit and other point cloud data different from the point cloud data; and an editing unit that accepts editing of combined data that combines the editing data displayed by the display unit with the other point cloud data.

13. A data processing system according to claim 12, wherein the display unit displays the editing data and the other point cloud data with the scale of the editing data and the scale of the other point cloud data matched.

14. A data processing method in which an imaging device transmits to a data processing device original data including point cloud data obtained by a positioning device that measures the position of a reflection point by irradiating a target area with irradiation light and receiving reflected light reflected from the reflection point, and image data for the target area in which each point data constituting the point cloud data is associated with a pixel; the data processing device decomposes the point cloud data in the original data transmitted from the imaging device for each object present in the target area to generate decomposed data that is point cloud data for each object; and the data processing device identifies decomposed data for a specified object from the decomposed data for each object and partial data composed of pixels associated with the decomposed data for the specified object from the image data included in the original data.

15. A data processing program for a data processing device connected to an image capturing device, wherein the image capturing device has a communication unit that transmits to the data processing device original data including point cloud data obtained by a positioning device that measures the positions of reflection points by irradiating a target area with irradiation light and receiving reflected light reflected from the reflection points, and image data for the target area, in which each point data constituting the point cloud data is associated with a pixel, and the data processing program causes a computer to function as a data processing device that performs a decomposition process that decomposes the point cloud data in the original data transmitted from the image capturing device for each object present in the target area to generate decomposed data that is point cloud data for each object, and a corresponding data identification process that identifies decomposed data for a specified object from the decomposed data for each object generated by the decomposition process, and partial data composed of pixels that are associated with the decomposed data for the specified object from the image data included in the original data.

Images