Data processing device, data processing system, data processing method and data processing program

Abstract

To easily extract point cloud data of a target object from point cloud data including surrounding objects and the like. [Solution] A decomposition unit 113 decomposes the point cloud data in the original data, which includes point cloud data obtained by a positioning device that measures the positions of reflection points by irradiating a target area with irradiation light and receiving reflected light from the reflection points, and image data about the target area in which each point data constituting the point cloud data is associated with a pixel, for each object present in the target area, to generate decomposed data that is point cloud data for each object. A corresponding data identification unit 116 identifies the decomposed data for a specified object from the decomposed data for each object and partial data composed of pixels associated with the decomposed data for the specified object from the image data included in the original data.

Description

[Technical Field]

[0001] The present disclosure relates to a technique for handling three-dimensional point cloud data. [Background technology]

[0002] Three-dimensional point cloud data of an object is acquired using a positioning device such as a LiDAR camera. LiDAR stands for Light Detection and Ranging. For example, Patent Document 1 describes acquiring point cloud data using a positioning device and measuring the size of the object. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-023348 Summary of the Invention [Problem to be solved by the invention]

[0004] When point cloud data is acquired using a positioning device, point cloud data is acquired for the entire space illuminated by light, meaning that point cloud data is acquired not only for the object for which point cloud data is to be acquired, but also for surrounding objects, floors, walls, etc. It is possible to use point cloud data of a target object to consider the layout when the target object is placed in another location. However, extracting point cloud data of the target object from point cloud data that also includes surrounding objects requires processing and cutting using dedicated software, which is time-consuming. An object of the present disclosure is to enable easy extraction of point cloud data about a target object from point cloud data that also includes surrounding objects, etc. [Means for solving the problem]

[0005] The data processing device according to the present disclosure includes: a decomposition unit that decomposes the point cloud data in original data, which includes point cloud data obtained by a positioning device that measures the positions of reflection points by irradiating a target area with irradiation light and receiving reflected light reflected from the reflection points, and image data about the target area, the point cloud data being image data in which each point data constituting the point cloud data is associated with a pixel, for each object present in the target area, to generate decomposed data which is point cloud data for each object; a corresponding data specifying unit that specifies decomposed data for a designated object from the decomposed data for each object generated by the decomposing unit and partial data composed of pixels that are associated with the decomposed data for the designated object from the image data included in the original data; Equipped with. [Effects of the Invention]

[0006] In the present disclosure, original data, which is point cloud data obtained by a positioning device, is decomposed for each object present in a target area to generate decomposed data, which is point cloud data for each object. Then, partial data composed of pixels associated with the decomposed data for a specified object is identified. This allows the point cloud data for the specified object and the partial data for the specified object in the image data to be obtained. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a configuration diagram of a data processing system 100 according to a first embodiment. [Figure 2] 1 is a configuration diagram of a data processing device 10 according to a first embodiment. [Figure 3] 10 is a flowchart of a data registration process according to the first embodiment. [Figure 4] 4 is a flowchart of a data extraction process according to the first embodiment. [Figure 5] FIG. 3 is an explanatory diagram of a decomposition process according to the first embodiment. [Figure 6] 10 is a flowchart of a data editing process according to the first embodiment. [Figure 7] FIG. 1 is an explanatory diagram of an example of use of the data processing system 100 according to the first embodiment. [Figure 8] FIG. 2 is an explanatory diagram of the data structure of original data 51 and decomposed data according to the first embodiment. [Figure 9] FIG. 10 is a configuration diagram of a data processing system 100 according to a second embodiment. [Figure 10] FIG. 10 is a configuration diagram of a data processing device 10 according to a second embodiment. [Figure 11] FIG. 10 is a configuration diagram of an imaging device 20 according to a second embodiment. [Figure 12] FIG. 10 is a configuration diagram of an operation terminal 30 according to a second embodiment. [Figure 13] 10 is a flowchart of a data registration process according to the second embodiment. [Figure 14] 10 is a flowchart of a data extraction process according to the second embodiment. [Figure 15] 10 is a flowchart of a data editing process according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Embodiment 1 ***Configuration Description*** The configuration of a data processing system 100 according to the first embodiment will be described with reference to FIG. The data processing system 100 according to the first embodiment includes a data processing device 10. In the first embodiment, the data processing device 10 is a computer such as a smartphone or a tablet terminal.

[0009] The configuration of the data processing device 10 according to the first embodiment will be described with reference to FIG. The data processing device 10 is a computer. The data processing device 10 includes the following hardware: a processor 11, a memory 12, a storage 13, a communication interface 14, a camera 15, a positioning device 16, and a display device 17. The processor 11 is connected to other hardware via signal lines and controls the other hardware.

[0010] The processor 11 is an IC that performs processing. IC stands for Integrated Circuit. Specific examples of the processor 11 include a CPU, a DSP, and a GPU. CPU stands for Central Processing Unit. DSP stands for Digital Signal Processor. GPU stands for Graphics Processing Unit.

[0011] The memory 12 is a storage device that temporarily stores data. Specific examples of the memory 12 include SRAM and DRAM. SRAM stands for Static Random Access Memory. DRAM stands for Dynamic Random Access Memory.

[0012] The storage 13 is a storage device that stores data. A specific example of the storage 13 is an SSD. SSD is an abbreviation for Solid State Drive. The storage 13 may also be a portable recording medium such as an SD (registered trademark) memory card, CompactFlash (registered trademark), NAND flash, a flexible disk, an optical disk, a compact disk, a Blu-ray (registered trademark) disk, or a DVD. SD is an abbreviation for Secure Digital. DVD is an abbreviation for Digital Versatile Disk.

[0013] The communication interface 14 is an interface for communicating with an external device. Specific examples of the communication interface 14 include Ethernet (registered trademark), USB, HDD, etc. It is an MI (registered trademark) port. USB is an abbreviation for Universal Serial Bus. HDMI is an abbreviation for High-Definition Multimedia Interface.

[0014] The camera 15 is a photographing device such as an optical camera for acquiring image data.

[0015] The positioning device 16 is a device that measures the position of a reflection point by irradiating a target area with irradiation light and receiving the light reflected by the reflection point.

[0016] The display device 17 is a device for displaying information, such as an LCD, which stands for Liquid Crystal Display.

[0017] The data processing device 10 includes, as functional components, an imaging unit 111, a measurement unit 112, a decomposition unit 113, a display unit 114, a designation receiving unit 115, a corresponding data identification unit 116, a data extraction unit 117, and an editing unit 118. The functions of each functional component of the data processing device 10 are realized by software. Storage 13 stores programs that realize the functions of each functional component of data processing device 10. These programs are loaded into memory 12 by processor 11 and executed by processor 11. In this way, the functions of each functional component of data processing device 10 are realized.

[0018] 2 shows only one processor 11. However, there may be a plurality of processors 11, and the plurality of processors 11 may cooperate to execute programs that realize the respective functions.

[0019] ***Explanation of Operation*** The operation of the data processing system 100 according to the first embodiment will be described with reference to FIGS. The operation procedure of data processing system 100 according to embodiment 1 corresponds to the data processing method according to embodiment 1. Furthermore, the program that realizes the operation of data processing system 100 according to embodiment 1 corresponds to the data processing program according to embodiment 1.

[0020] The operations of the data processing system 100 include a data registration process, a data extraction process, and a data editing process. The data registration process is a process of acquiring image data and point cloud data of a target area and registering them as original data 51. The data extraction process is a process of identifying image data and point cloud data for a specified object from the original data 51 and extracting them as editing data 52. The data editing process is a process of performing editing using the editing data 52.

[0021] The data registration process according to the first embodiment will be described with reference to FIG. (Step S11: Photographing process) The photographing unit 111 photographs the target area using the camera 15 and acquires image data of the target area. At this time, the user may acquire multiple image data of the target area by photographing a video of the target area while moving the data processing device 10.

[0022] (Step S12: Measurement process) At the same time as capturing an image in step S11, the measurement unit 112 measures the positions of the reflection points by irradiating the target area with irradiation light and receiving the reflected light from the reflection points using the positioning device 16. In this way, the measurement unit 112 acquires point cloud data indicating the positions of multiple reflection points in the target area.

[0023] (Step S13: Registration process) The measurement unit 112 associates each pixel of the one or more pieces of image data acquired in step S11 with each point data of the point cloud data acquired in step S12. Then, the measurement unit 112 writes the associated point cloud data and image data to the storage 13 as original data 51. Corresponding each pixel of the image data with each point data of the point cloud data means that pixels and point data at the same position are associated with each other. By simultaneously acquiring image data and point cloud data, it is possible to associate pixels and point data at the same position with each other.

[0024] The data extraction process according to the first embodiment will be described with reference to FIG. (Step S21: Processing request acceptance process) The decomposition unit 113 accepts a processing request for the original data 51. When a plurality of pieces of original data 51 are stored in the storage 13, the processing request including the specification of the original data 51 to be processed is accepted. At this time, the plurality of pieces of original data 51 may be displayed to allow the user to select one.

[0025] (Step S22: Decomposition processing) The decomposition unit 113 sets the original data 51 specified in step S21 as the original data 51 to be processed. The decomposition unit 113 decomposes the point cloud data in the original data 51 to be processed for each object present in the target region to generate decomposed data, which is point cloud data for each object. The decomposition unit 113 registers the decomposed data in the storage 13 in association with the original data 51. For example, suppose that an object A, an object B, and an object C exist in the target area as shown in Fig. 5. In this case, the decomposition unit 113 detects each of the objects A, B, and C that exist in the target area, and decomposes the point cloud data into each of the objects A, B, C, and others (floors, walls, etc.). As a result, decomposed data that is the point cloud data of the object A, decomposed data that is the point cloud data of the object B, and decomposed data that is the point cloud data of the object C are generated.

[0026] Specifically, the decomposition unit 113 removes floor and wall point data from the point cloud data. Using existing technology for estimating surfaces, the decomposition unit 113 can identify floor and wall portions. The decomposition unit 113 clusters the remaining point data based on rules based on the distance between objects. The decomposition unit 113 can cluster the remaining point data by using the X-means method or the like. The decomposition unit 113 logically decomposes each point cloud data separated by clustering into decomposed data. Alternatively, the decomposition unit 113 recognizes an object using a learning model generated by deep learning, and logically decomposes each point cloud into decomposed data. The decomposition unit 113 also logically decomposes the removed floor and wall parts into decomposed data. Here, logical decomposition refers to a state in which a label is assigned to each point data so that the decomposed data, which is point cloud data for each object, can be identified. The decomposition unit 113 then generates a physically decomposed OBJ file from the logically decomposed decomposed data. The OBJ file will be described later.

[0027] (Step S23: Display processing) The display unit 114 displays the original data 51 specified in step S21 on the display device 17. Specifically, the display unit 114 pastes the image data included in the original data 51 in accordance with the positions of the point data of the point cloud data included in the original data 51. In this way, the image data is combined with the point cloud data, and a three-dimensional model of the target area is generated. Then, the display unit 11 4 displays a three-dimensional model of the target area.

[0028] (Step S24: Designation acceptance process) Designation receiving unit 115 receives designation of an object to be processed from one or more objects included in the 3D model displayed in step S23. For example, the object to be processed is designated by touching the object to be processed. At this time, the designation receiving unit 115 also receives designation as to whether the request is a cut request to cut and extract the data of the designated object, or a delete request to delete the data of the designated object and extract the remaining data.

[0029] (Step S25: Corresponding data identification process) The corresponding data identification unit 116 identifies the decomposed data for the specified object specified in step S24 from the decomposed data generated in step S22. The corresponding data identification unit 116 can identify the decomposed data for the specified object by identifying the decomposed data including the point data of the touched position. Furthermore, the corresponding data specifying unit 116 specifies partial data made up of pixels that are associated with the decomposed data for the specified object from the image data included in the original data to be processed.

[0030] (Step S26: Extraction process) The data extraction unit 117 extracts the editing data 52 from the decomposed data and partial data for the designated object identified in step S25. The data extraction unit 117 determines whether a cut request or a delete request was specified in step S24. If a cut request was specified, the data extraction unit 117 extracts the decomposed data and partial data for the specified object as the editing data 52. If a delete request was specified, the data extraction unit 117 extracts the decomposed data and partial data for the specified object, which are removed from the original data 51, as the editing data 52. Here, when the decomposed data is removed from the point cloud data included in the original data 51, point data for the portion where the decomposed data was located is missing. The data extraction unit 117 may fill in the missing point data using point data of the walls and floor. Specifically, the data extraction unit 117 may fill in the missing point data by extending the walls and floor assuming that the walls and floor are flat. Similarly, the data extraction unit 117 may fill in the missing point data for the missing portion of the image data using image data of the wall and floor portions.

[0031] The data editing process according to the first embodiment will be described with reference to FIG. (Step S31: Data reading process) The editing unit 118 reads the editing data 52 extracted in step S26. The display unit 114 then displays the read editing data 52. At this time, the display unit 114 displays a three-dimensional model synthesized by pasting the image data included in the editing data 52 in accordance with the positions of the point data of the point cloud data included in the editing data 52.

[0032] There are cases where the data extraction process is executed multiple times and multiple pieces of editing data 52 are extracted. In this case, the editing unit 118 reads the multiple pieces of editing data 52. The display unit 114 then displays the multiple pieces of editing data 52 that have been read on one screen. In other words, a 3D model obtained by combining image data with point cloud data for each piece of editing data 52 is displayed on one screen. At this time, the display unit 114 displays the multiple pieces of editing data 52 at the same scale. That is, the display unit 114 displays each piece of editing data 52 after matching the scale of each piece of editing data 52 with a reference scale. In the data registration process shown in FIG. 3, the point cloud data acquired by the same data processing device 10 Since the scales are the same, there is no need to match the scales, but point cloud data acquired by different data processing devices 10 may have different scales, and it may become necessary to match the scales.

[0033] Furthermore, the editing unit 118 may read a 3D model created by CAD or the like in addition to the editing data 52. CAD stands for Computer-Aided Design. In this case, the display unit 114 displays the read editing data 52 and the 3D model created by CAD or the like on one screen. That is, the 3D model obtained by combining image data with point cloud data for the editing data 52 and the 3D model created by CAD or the like are displayed on one screen. At this time, the display unit 114 displays the scale of the editing data 52 together with the scale of the three-dimensional model created by CAD or the like.

[0034] (Step S32: Editing process) The editing unit 118 accepts editing of the editing data 52 etc. displayed in step S31. When a plurality of three-dimensional models of the editing data 52 are displayed, or when a three-dimensional model of the editing data 52 and a three-dimensional model created by CAD or the like are displayed, the editing unit 118 accepts editing of combination data that combines a plurality of three-dimensional models.

[0035] An example of use of the data processing system 100 according to the first embodiment will be described with reference to FIG. First, in the data registration process, data on rooms A and B is acquired and registered as original data 51. In the data extraction process, object X is designated as a designated object in original data 51 for room A, and a cut request is issued. As a result, point cloud data and image data for object X are extracted as editing data 52. Furthermore, all objects are designated as designated objects in original data 51 for room B, and a delete request is issued. As a result, the point cloud data of all objects is removed from room B, and the remaining point cloud data and image data corresponding to the remaining point cloud data are extracted as editing data 52. In other words, the point cloud data and image data of room B in an empty state from which all objects have been removed are extracted as editing data 52. In the data editing process, editing data 52 for object X and editing data 52 for room B are displayed on one screen. That is, a 3D model of object X and a 3D model of empty room B are displayed on one screen. The 3D model of object X is then moved and superimposed on the 3D model of room B. This makes it possible to check what state the object X will be in if it is placed in room B. It is also possible to duplicate the 3D model of object X and check what state the object will be in if multiple objects X are placed in room B.

[0036] Here, the 3D model of object X and the 3D model of empty room B are both generated from original data 51 registered in the data registration process. However, either the 3D model of object X or the 3D model of empty room B may be a 3D model created using CAD or the like. In other words, it is possible to combine and edit a 3D model obtained by extracting point cloud data and image data of some objects, or a 3D model obtained by excluding point cloud data of some objects, with a 3D model created using CAD or the like.

[0037] The data configurations of the original data 51 and the decomposed data according to the first embodiment will be described with reference to FIG. The original data 51 includes an OBJ file, an MTL file, a texture file, an image file, and a thumbnail file. The OBJ file contains coordinate data of a group of three-dimensional points. The MTL file is a three-dimensional data file indicating the texture coordinates, etc., and contains information on which position in the three-dimensional data image data is to be pasted as a texture. The MTL file is a file that describes the information required when performing texture mapping. The texture file is image data of the texture to be pasted onto the three-dimensional point cloud indicated by the OBJ file according to the MTL file. The image file is image data acquired in step S11, and is configured in JPG format, for example. The thumbnail file is data that has been reduced from the image data indicated by the image file. The decomposed data is set for each object as an OBJ file in the original data 51. Since the decomposed data is set for each object as an OBJ file, once a designated object is identified, the OBJ file of the designated object is identified, and the information held in the OBJ file identifies partial data, which is image data of the designated object corresponding to the three-dimensional data of the OBJ file.

[0038] ***Effects of the First Embodiment*** As described above, data processing system 100 according to the first embodiment decomposes original data 51, which is point cloud data obtained by positioning device 16, into each object present in the target area, to generate decomposed data, which is point cloud data for each object. Data processing system 100 then identifies the decomposed data for the specified object and partial data composed of pixels associated with the decomposed data. This results in obtaining point cloud data for the specified object and a partial image of the specified object from the image data. In particular, by simply specifying a specified object, it is possible to obtain point cloud data for the specified object and a partial image of the specified object from the image data, without having to enclose the specified object or the like.

[0039] Since point cloud data and partial images of the specified object are obtained, it becomes easy to combine and edit a 3D model of the specified object with other 3D models.

[0040] Furthermore, when displaying multiple 3D models on one screen, the data processing system 100 according to the first embodiment adjusts the scale of each 3D model before displaying them, which makes it easy to perform layout adjustments, etc. For example, if the specified object is a large object such as furniture, by using the data processing system 100 to capture an image of an area including the entrance to the room and the planned installation location of the furniture as the target area, it is possible to check whether the furniture can be brought in along the delivery route from the entrance to the planned installation location, which is difficult to check visually.It is also possible to check whether the furniture can be installed in the planned installation location.

[0041] Embodiment 2 The second embodiment differs from the first embodiment in that the data processing system 100 is configured by a plurality of devices. In the second embodiment, this difference will be explained, and explanation of the same points will be omitted.

[0042] ***Configuration Description*** The configuration of a data processing system 100 according to the second embodiment will be described with reference to FIG. The data processing system 100 includes a data processing device 10, an imaging device 20, and an operation terminal 30. In the second embodiment, the data processing device 10 is a computer such as a server. The imaging device 20 is a computer such as a smartphone or a tablet terminal. The operation terminal 30 is a computer such as a smartphone, a tablet terminal, or a PC. PC is an abbreviation for personal computer. The data processing device 10, the photographing device 20, and the operation terminal 30 are connected via a network 40. It continues. The photographing device 20 and the operation terminal 30 may be implemented by a single computer. For example, both the photographing device 20 and the operation terminal 30 may be implemented by a single smartphone.

[0043] The configuration of a data processing device 10 according to the second embodiment will be described with reference to FIG. 2 in that the data processing device 10 does not include, as hardware, the camera 15, the positioning device 16, and the display device 17. The data processing device 10 also differs from the data processing device 10 shown in FIG. 2 in that the data processing device 10 does not include, as functional components, the photographing unit 111 and the measuring unit 112.

[0044] The configuration of the imaging device 20 according to the second embodiment will be described with reference to FIG. The photographing device 20 includes the following hardware: a processor 21, a memory 22, a storage 23, a communication interface 24, a camera 25, and a positioning device 26. The processor 21 is connected to other hardware via signal lines and controls the other hardware. The processor 21, memory 22, storage 23, communication interface 24, camera 25, and positioning device 26 are similar to the processor 11, memory 12, storage 13, communication interface 14, camera 15, and positioning device 16, respectively.

[0045] The photographing device 20 includes, as functional components, a photographing unit 211, a measuring unit 212, and a communication unit 213. The functions of the functional components of the photographing device 20 are realized by software. The storage 23 stores a program that realizes the function of each functional component of the image capturing device 20. The program is read into the memory 22 by the processor 21 and executed by the processor 21. In this way, the function of each functional component of the image capturing device 20 is realized.

[0046] The configuration of the operation terminal 30 according to the second embodiment will be described with reference to FIG. The operation terminal 30 includes the following hardware: a processor 31, a memory 32, a storage 33, a communication interface 34, and a display device 35. The processor 31 is connected to other hardware via signal lines and controls the other hardware. The processor 31, memory 32, storage 33, communication interface 34, and display device 35 are similar to the processor 11, memory 12, storage 13, communication interface 14, and display device 17, respectively.

[0047] The operation terminal 30 includes, as functional components, an instruction unit 311 and a communication unit 312. The functions of the functional components of the operation terminal 30 are realized by software. The storage 33 stores a program that realizes the function of each functional component of the operation terminal 30. The program is read into the memory 32 by the processor 31 and executed by the processor 31. In this way, the function of each functional component of the operation terminal 30 is realized.

[0048] ***Explanation of Operation*** The operation of the data processing system 100 according to the second embodiment will be described with reference to FIGS.

[0049] The data registration process according to the second embodiment will be described with reference to FIG. (Step S11A: Photographing process) As in step S11 of FIG. 3, the photographing unit 211 of the photographing device 20 photographs the target area using the camera 25 and acquires image data of the target area.

[0050] (Step S12A: Measurement processing) The measurement unit 212 of the image capturing device 20 uses the positioning device 26 to acquire point cloud data indicating the positions of a plurality of reflection points in the target area, similar to step S12 in FIG.

[0051] (Step S13A: Registration process) The communication unit 213 of the photographing device 20 associates each pixel of the one or more pieces of image data acquired in step S11A with each point data of the point cloud data acquired in step S12A, and transmits the resulting data to the data processing device 10 as original data 51. As a result, the original data 51 is written to the storage 13 of the data processing device 10.

[0052] The data extraction process according to the second embodiment will be described with reference to FIG. (Step S21A: Processing request acceptance process) 4, the instruction unit 311 of the operation terminal 30 receives a processing request for the original data 51. Then, the communication unit 312 of the operation terminal 30 transmits the processing request to the data processing device 10.

[0053] (Step S22A: Decomposition processing) As in step S22 of Figure 4, the decomposition unit 113 of the data processing device 10 generates decomposed data, which is point cloud data for each object, using the original data 51 specified in step S21A as the original data 51 to be processed, and registers it in the storage 13.

[0054] (Step S23A: Display processing) The display unit 114 of the data processing device 10 transmits the original data 51 designated in step S21A to the operation terminal 30 and displays it on the display device 35, similar to step S23 in FIG.

[0055] (Step S24A: Designation acceptance process) 4, instruction unit 311 of operation terminal 30 accepts designation of an object to be processed from among one or more objects included in the 3D model displayed in step S23A. At this time, instruction unit 311 also accepts designation of whether the request is a cut request or a delete request. Then, communication unit 312 of operation terminal 30 transmits information indicating the object to be processed and information indicating whether the request is a cut request or a delete request to data processing device 10.

[0056] (Step S25A: Corresponding data identification process) 4, the corresponding data identification unit 116 of the data processing device 10 identifies the decomposed data for the specified object specified in step S24A from the decomposed data generated in step S22A. Furthermore, the corresponding data identification unit 116 identifies partial data composed of pixels associated with the decomposed data for the specified object.

[0057] (Step S26A: Extraction process) The data extraction unit 117 of the data processing device 10 extracts the editing data 52 from the decomposed data and partial data for the designated object identified in step S25A, similar to step S26 in FIG. The data extraction unit 117 may transmit the editing data 52 to the operation terminal 30 in response to a request from the operation terminal 30 .

[0058] The data editing process according to the second embodiment will be described with reference to FIG. (Step S31: Data reading process) The editing unit 118 of the data processing device 10 reads the editing data 52 extracted in step S26, similar to step S31 in Fig. 6. Then, the display unit 114 displays the read editing data 52. The collected data 52 is transmitted to the operation terminal 30 and displayed on the display device 35 .

[0059] (Step S32: Editing process) The editing unit 118 of the data processing device 10 accepts editing of the edit data 52 etc. displayed in step S31, similar to step S32 in Fig. 6. Here, the editing unit 118 accepts editing from the operation terminal 30.

[0060] ***Effects of the Second Embodiment*** As described above, the data processing system 100 according to the second embodiment is realized by the data processing device 10, the photographing device 20, and the operation terminal 30. Even in this case, it is possible to achieve the same effects as those of the data processing system 100 according to the first embodiment.

[0061] 14, by transmitting the editing data 52 to the operation terminal 30, it is also possible to perform editing using the editing data 52 on the operation terminal 30 side. For example, it is also possible to read the editing data 52 into CAD software and perform editing.

[0062] ***Other Configurations*** <Variation 1> In the first embodiment, each functional component is realized by software. However, as a first modification, each functional component may be realized by hardware. The differences between the first embodiment and the first modification will be described below.

[0063] When each functional component is realized by hardware, the data processing device 10 includes an electronic circuit in place of the processor 11, the memory 12, and the storage 13. The electronic circuit is a dedicated circuit for realizing the functions of each functional component, the memory 12, and the storage 13.

[0064] Similarly, when each functional component is realized by hardware, the imaging device 20 includes an electronic circuit instead of the processor 21, the memory 22, and the storage 23. The electronic circuit is a dedicated circuit for realizing the functions of each functional component, the memory 22, and the storage 23.

[0065] Similarly, when each functional component is realized by hardware, the operation terminal 30 includes an electronic circuit in place of the processor 31, the memory 32, and the storage 33. The electronic circuit is a dedicated circuit for realizing the functions of each functional component, the memory 32, and the storage 33.

[0066] Possible electronic circuits include single circuits, composite circuits, programmed processors, parallel programmed processors, logic ICs, GAs, ASICs, and FPGAs. GA stands for Gate Array. ASIC stands for Application Specific Integrated Circuit. FPGA stands for Field-Programmable Gate Array. Each functional component may be realized by one electronic circuit, or each functional component may be realized by distributing it among a plurality of electronic circuits.

[0067] <Variation 2> As a second modification, some of the functional components may be realized by hardware, and other functional components may be realized by software.

[0068] The processors 11, 21, and 31, the memories 12, 22, and 32, the storages 13, 23, and 33, and the electronic circuits are collectively referred to as a processing circuit. In other words, the functions of the functional components are realized by the processing circuit.

[0069] Furthermore, the term "unit" in the above description may be read as a "circuit," "step," "procedure," "process," or "processing circuit."

[0070] Various aspects of the present disclosure are summarized below as appendices. (Appendix 1) a decomposition unit that decomposes the point cloud data in original data, which includes point cloud data obtained by a positioning device that measures the positions of reflection points by irradiating a target area with irradiation light and receiving reflected light reflected from the reflection points, and image data about the target area, the point cloud data being image data in which each point data constituting the point cloud data is associated with a pixel, for each object present in the target area, to generate decomposed data which is point cloud data for each object; a corresponding data specifying unit that specifies decomposed data for a designated object from the decomposed data for each object generated by the decomposing unit and partial data composed of pixels that are associated with the decomposed data for the designated object from the image data included in the original data; A data processing system comprising: (Appendix 2) The data processing system further comprises: a data extraction unit that extracts the decomposed data and the partial data for the designated object identified by the corresponding data identification unit as editing data; 2. The data processing system of claim 1, comprising: (Appendix 3) The data processing system further comprises: a data extraction unit that extracts, as editing data, data obtained by excluding the decomposed data and the partial data for the designated object identified by the corresponding data identification unit from the original data; 2. The data processing system of claim 1, comprising: (Appendix 4) The data processing system further comprises: a display unit that displays a three-dimensional model obtained by combining the image data with the point cloud data included in the original data; a designation receiving unit that receives designation of the designated object from the three-dimensional model displayed by the display unit; 4. The data processing system of claim 2 or 3, comprising: (Appendix 5) The data processing system further comprises: a display unit that simultaneously displays the editing data extracted by the data extraction unit and other point cloud data different from the point cloud data; an editing unit that accepts editing of combined data that is a combination of the editing data displayed by the display unit and the other point cloud data; 4. The data processing system of claim 2 or 3, comprising: (Appendix 6) The display unit displays the editing data and the other point cloud data in a state where the scale of the editing data and the scale of the other point cloud data match. 6. The data processing system of claim 5. (Appendix 7) The decomposition unit generates decomposed data for each object by removing floor point data from the point cloud data and clustering the remaining point data. 7. A data processing system according to any one of claims 1 to 6. (Appendix 8) The decomposition unit generates the decomposition data for each of the objects as a separate file. 8. A data processing system according to any one of claims 1 to 7. (Appendix 9) a computer decomposes the point cloud data in original data, which includes point cloud data obtained by a positioning device that measures the positions of reflection points by irradiating a target area with irradiation light and receiving reflected light reflected from the reflection points, and image data about the target area, in which each point data constituting the point cloud data is associated with a pixel, for each object present in the target area, to generate decomposed data, which is point cloud data for each object; A data processing method in which a computer identifies decomposition data for a specified object from the decomposition data for each object and partial data consisting of pixels associated with the decomposition data for the specified object from the image data included in the original data. (Appendix 10) a decomposition process for decomposing the point cloud data in original data, which includes point cloud data obtained by a positioning device that measures the positions of reflection points by irradiating a target area with irradiation light and receiving reflected light reflected from the reflection points, and image data about the target area, in which each point data constituting the point cloud data is associated with a pixel, for each object present in the target area, to generate decomposed data, which is point cloud data for each object; a corresponding data specifying process for specifying decomposed data for a designated object among the decomposed data for each object generated by the decomposition process, and partial data composed of pixels associated with the decomposed data for the designated object among the image data included in the original data; A data processing program that causes a computer to execute the following.

[0071] The embodiments and modifications of the present disclosure have been described above. Some of these embodiments and modifications may be combined and implemented. Also, one or more of them may be implemented partially. Note that the present disclosure is not limited to the above embodiments and modifications, and various modifications are possible as needed. [Explanation of symbols]

[0072] 100 data processing system, 10 data processing device, 11 processor, 12 memory, 13 storage, 14 communication interface, 15 camera, 16 positioning device, 17 display device, 111 photographing unit, 112 measurement unit, 113 resolution unit, 114 display unit, 115 designation reception unit, 116 corresponding data identification unit, 117 data extraction unit, 118 editing unit, 20 photographing device, 21 processor, 22 memory, 23 storage, 24 communication interface, 25 camera, 26 positioning device, 211 photographing unit, 212 measurement unit, 213 communication unit, 30 operation terminal, 31 processor, 32 memory, 33 storage, 34 communication interface, 35 display device, 311 instruction unit, 312 communication unit, 51 original data, 52 editing data.

Claims

1. A data processing device connected to an imaging device, The imaging device is a communication unit that transmits to the data processing device original data including point cloud data obtained by a positioning device that measures the positions of reflection points by irradiating a target area with irradiation light and receiving reflected light reflected from the reflection points, and image data about the target area, the image data being image data in which each point data constituting the point cloud data is associated with a pixel; The data processing device includes: a decomposition unit that decomposes the point cloud data in the original data transmitted from the image capturing device for each object present in the target area to generate decomposed data that is point cloud data for each object; a corresponding data specifying unit that specifies decomposed data for a designated object from the decomposed data for each object generated by the decomposing unit and partial data composed of pixels that are associated with the decomposed data for the designated object from the image data included in the original data; A data processing device comprising:

2. The data processing device further comprises: an instruction receiving unit that receives either a cut request to cut and extract data about a designated object or a delete request to delete data about the designated object and extract remaining data; a data extraction unit that uses the partial data identified by the corresponding data identification unit to extract, as editing data, data according to whether the instruction accepted by the instruction acceptance unit indicates a cut request or a delete request. The data processing apparatus of claim 1 , comprising:

3. When the instruction indicates a cut request, the data extraction unit extracts the decomposed data and the partial data for the designated object as the editing data, and when the instruction indicates a deletion request, extracts data obtained by excluding the decomposed data and the partial data for the designated object from the original data as the editing data.

3. The data processing device according to claim 2.

4. The data processing device further comprises: a display unit that displays a three-dimensional model obtained by combining the image data with the point cloud data included in the original data; Equipped with The data processing device according to claim 2 , wherein the instruction receiving unit receives a designation of the designated object from the three-dimensional model displayed by the display unit.

5. The data processing device further comprises: a display unit that simultaneously displays the editing data extracted by the data extraction unit and other point cloud data different from the point cloud data; an editing unit that accepts editing of combined data that is a combination of the editing data displayed by the display unit and the other point cloud data; The data processing device according to claim 2 , comprising:

6. The display unit displays the editing data and the other point cloud data in a state where the scale of the editing data and the scale of the other point cloud data match.

6. A data processing device according to claim 5.

7. A data processing system comprising an imaging device and a data processing device, The imaging device is a communication unit that transmits to the data processing device original data including point cloud data obtained by a positioning device that measures the positions of reflection points by irradiating a target area with irradiation light and receiving reflected light reflected from the reflection points, and image data about the target area, the image data being image data in which each point data constituting the point cloud data is associated with a pixel; The data processing device includes: a decomposition unit that decomposes the point cloud data in the original data transmitted from the image capturing device for each object present in the target area to generate decomposed data that is point cloud data for each object; a corresponding data specifying unit that specifies decomposed data for a designated object from the decomposed data for each object generated by the decomposing unit and partial data composed of pixels that are associated with the decomposed data for the designated object from the image data included in the original data; A data processing system comprising:

8. the data processing system further comprises an operation terminal; The operation terminal an instruction unit that receives either a cut request to cut and extract data about a designated object or a delete request to delete data about the designated object and extract remaining data; a communication unit that transmits request information indicating whether the instruction received by the instruction unit is a cut request or a delete request to the data processing device; Equipped with The data processing device further comprises: a data extraction unit that uses the partial data identified by the corresponding data identification unit to extract, as editing data, data according to whether the request information transmitted by the communication unit indicates a cut request or a delete request.

8. The data processing system of claim 7, comprising:

9. The data extraction unit extracts the decomposed data and the partial data for the designated object as the editing data when the request information indicates a cut request, and extracts data obtained by excluding the decomposed data and the partial data for the designated object from the original data as the editing data when the request information indicates a deletion request.

9. The data processing system of claim 8.

10. The data processing device further comprises: a display unit that displays a three-dimensional model obtained by combining the image data with the point cloud data included in the original data; Equipped with 9. The data processing system according to claim 8, wherein the instruction unit accepts designation of the designated object from the three-dimensional model displayed by the display unit.

11. The data processing device further comprises: a display unit that simultaneously displays the editing data extracted by the data extraction unit and other point cloud data different from the point cloud data; an editing unit that accepts editing of combined data that is a combination of the editing data displayed by the display unit and the other point cloud data; 9. The data processing system of claim 8, comprising:

12. The display unit displays the editing data and the other point cloud data in a state where the scale of the editing data and the scale of the other point cloud data match.

12. The data processing system of claim 11.

13. an imaging device transmits original data to a data processing device, the original data including point cloud data obtained by a positioning device that measures the positions of reflection points by irradiating a target area with irradiation light and receiving reflected light reflected from the reflection points, and image data for the target area, the image data being image data in which each point data constituting the point cloud data is associated with a pixel; a data processing device that decomposes the point cloud data in the original data transmitted from the imaging device for each object present in the target area to generate decomposed data that is point cloud data for each object; A data processing method in which a data processing device identifies decomposition data for a specified object from the decomposition data for each object and partial data consisting of pixels associated with the decomposition data for the specified object from the image data included in the original data.

14. A data processing program for a data processing device connected to an imaging device, The imaging device is a communication unit that transmits to the data processing device original data including point cloud data obtained by a positioning device that measures the positions of reflection points by irradiating a target area with irradiation light and receiving reflected light reflected from the reflection points, and image data about the target area, the image data being image data in which each point data constituting the point cloud data is associated with a pixel; The data processing program a decomposition process for decomposing the point cloud data in the original data transmitted from the image capturing device into decomposed data for each object present in the target area, which is point cloud data for each object; a corresponding data specifying process for specifying decomposed data for a designated object among the decomposed data for each object generated by the decomposition process, and partial data composed of pixels associated with the decomposed data for the designated object among the image data included in the original data; A data processing program that causes a computer to function as a data processing device that performs the above.

Images