Data processing device

Abstract

A data processing device (20) comprises a control unit that three-dimensionally models a continuous shape in a pipeline by sequentially coupling point group data (61) by using a point group matching method, the point group data (61) being obtained through a plurality of observations using a sensor for observing a partial shape in the pipeline as a point group while moving in the pipeline. In the point group of the point group data (61) of a coupling destination and the point group of the point group data (61) of a coupling target, said coupling includes excluding a point corresponding to the outside of a common range in the pipeline, from point group matching targets.

Description

Data Processing Unit

[0001] The present disclosure relates to a data processing device.

[0002] Non-Patent Document 1 proposes a method of combining point clouds acquired using a laser scanner to create a continuous three-dimensional model of an underground road tunnel.

[0003] Yasuhiro Mitani, "Maintenance and Management of Road Structures Using 3D Data," [online], January 25, 2018, Public Works Research Institute New Technology Showcase 2018 in Fukuoka, [Retrieved March 8, 2024], Internet <URL: https: / / www.pwri.go.jp / jpn / results / tec-info / siryou / 2017 / fukuokasc / pdf / SC2018_fukuoka04.pdf>

[0004] Japan is facing a shrinking workforce and aging social infrastructure. To address these issues, DX (Digital Transformation) is being promoted for infrastructure operations and maintenance / management. "DX" stands for digital transformation. One part of DX is 3D modeling, which digitally reproduces the shape and location of social infrastructure facilities. Point clouds, which contain coordinate information, are widely used for 3D modeling. 3D modeling using point clouds involves connecting intermittently acquired point clouds by overlaying the position, orientation, and geometric features of the point clouds at the time of photographing them, to create a continuous 3D model of the object.

[0005] Ground-based social infrastructure facilities can be efficiently modeled in 3D using MMS, which combines point clouds and satellite positioning, or drones. "MMS" is an abbreviation for Mobile Mapping System.

[0006] Obtaining location information underground using satellite positioning is difficult, and many underground facilities are tunnel-shaped and lack distinctive geometric features. As a result, 3D modeling of underground social infrastructure facilities has not progressed as much as aboveground social infrastructure facilities. In particular, underground social infrastructure facilities include a vast number of buried pipelines, which are narrower than road tunnels and have almost no landmarks for point cloud matching. While 3D modeling of such buried pipelines is required, it is difficult with existing technologies such as those disclosed in Non-Patent Document 1. For example, the diameter of a telecommunications pipeline is approximately 8 cm, and approximately 600,000 km of such pipelines exist underground.

[0007] The present disclosure has been made in view of the above circumstances, and aims to accurately create a three-dimensional model of the continuous shape inside a pipeline.

[0008] A data processing device according to one embodiment includes a control unit that uses a point cloud matching technique to sequentially combine point cloud data obtained through multiple observations by a sensor that observes the partial shapes within the pipeline as a point cloud while moving within the pipeline, thereby creating a three-dimensional model of the continuous shape within the pipeline, and the combining includes excluding points that correspond outside the common range within the pipeline in each of the point cloud of the point cloud data to be combined and the point cloud of the point cloud data to be combined from the point cloud matching targets.

[0009] According to the present disclosure, it is possible to accurately create a three-dimensional model of the continuous shape inside a pipeline.

[0010] FIG. 1 is a diagram showing the configuration of a 3D modeling system according to an embodiment; FIG. 2 is a diagram showing an example of a pipeline between manholes; FIG. 3 is a block diagram showing the configurations of a data processing device and a measuring device provided in the 3D modeling system; FIG. 4 is a diagram showing data obtained by the measuring device and operations performed by the data processing device; FIG. 5 is a diagram showing an example of attitude / coordinate calculation during point cloud photography; FIG. 6 is a diagram showing an example of a point cloud plot; FIG. 7 is a diagram showing an example of point cloud matching pre-processing; FIG. 8 is a diagram showing an example of point cloud matching; FIG. 9 is a diagram showing an example of the result of point cloud matching; FIG. 10 is a diagram showing an example of the next point cloud plot; FIG. 11 is a diagram showing an example of 3D modeling of the entire length of a pipeline; FIG. 12 is a diagram showing a case where multiple pipelines are laid.

[0011] An embodiment will be described below with reference to the drawings.

[0012] In each drawing, the same or corresponding parts are denoted by the same reference numerals. In the description of this embodiment, the description of the same or corresponding parts will be omitted or simplified as appropriate.

[0013] The configuration of a three-dimensional modeling system 10 according to this embodiment will be described with reference to FIG.

[0014] The three-dimensional modeling system 10 includes a data processing device 20 and a measurement device 30. The three-dimensional modeling system 10 is a system in which the measurement device 30 scans a pipeline 11, and the data processing device 20 creates a three-dimensional model of the continuous shape of the pipeline 11 from its start point to its end point.

[0015] The conduit 11 is laid between manholes 13 underground as shown in FIG. 2 . Ducts 14 leading to the manholes 13 are formed at both ends of the conduit 11. In this embodiment, the conduit 11 is a narrow underground conduit with little distinctive shape. The underground conduit is, for example, a communication conduit through which a communication cable passes, or a power transmission conduit through which a power transmission line passes. The diameter of the underground conduit is 1 m or less, and even a small one is 10 cm or less. The diameter of the underground conduit may vary along its axial direction due to dimensional errors. When the underground conduit is laid, it may be bent, causing unevenness in the underground conduit. After the underground conduit is laid, its position or shape may change due to the surrounding earth pressure.

[0016] The data processing device 20 is used by the worker 12 inside the manhole 13. The data processing device 20 is, for example, a general-purpose computer such as a PC, or a dedicated computer. "PC" is an abbreviation for personal computer. The data processing device 20 creates a three-dimensional model of the pipeline 11 using point clouds. Specifically, the data processing device 20 combines point clouds acquired inside a narrow underground pipeline with few geometric features to create a continuous three-dimensional model. In this embodiment, use of the data processing device 20 enables the efficiency of buried object design, burial consultations, test excavation, and construction supervision.

[0017] The data processing device 20 communicates with the measurement device 30. In this embodiment, the data processing device 20 is connected to the measurement device 30 via an optical media converter 40 and a communication cable 41, and performs optical communication with the measurement device 30. As a modified example, the data processing device 20 may perform electrical communication with the measurement device 30. The data processing device 20 may also perform wireless communication with the measurement device 30.

[0018] The measuring device 30 moves within the pipeline 11. In this embodiment, the measuring device 30 is pulled between the ducts 14 by a pulling cable 51. The data processing device 20 may detect the moving distance of the measuring device 30 via an encoder 50 attached to the pulling cable 51. The pulling cable 51 is manually wound by the operator 12. Alternatively, the pulling cable 51 may be automatically wound by a motor. As a modified example, the measuring device 30 may be self-propelled.

[0019] The configuration of a data processing device 20 according to this embodiment will be described with reference to FIG.

[0020] The data processing device 20 includes a control unit 21 , a storage unit 22 , a communication unit 23 , an input unit 24 , and an output unit 25 .

[0021] The control unit 21 includes at least one processor, at least one programmable circuit, at least one dedicated circuit, or any combination thereof. The processor is a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for specific processing. "CPU" is an abbreviation for central processing unit. "GPU" is an abbreviation for graphics processing unit. An example of the programmable circuit is an FPGA. "FPGA" is an abbreviation for field-programmable gate array. An example of the dedicated circuit is an ASIC. "ASIC" is an abbreviation for application specific integrated circuit. The control unit 21 controls each part of the data processing device 20 and executes processing related to the operation of the data processing device 20.

[0022] The storage unit 22 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or any combination thereof. The semiconductor memory is, for example, a RAM, a ROM, or a flash memory. "RAM" is an abbreviation for random access memory. "ROM" is an abbreviation for read only memory. RAM is, for example, an SRAM or a DRAM. "SRAM" is an abbreviation for static random access memory. "DRAM" is an abbreviation for dynamic random access memory. ROM is, for example, an EEPROM. "EEPROM" is an abbreviation for electrically erasable programmable read only memory. Flash memory is, for example, an SSD. "SSD" is an abbreviation for solid-state drive. Magnetic memory is, for example, an HDD. "HDD" is an abbreviation for hard disk drive. The storage unit 22 functions, for example, as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 22 stores data used in the operation of the data processing device 20 and data obtained by the operation of the data processing device 20 .

[0023] The communication unit 23 includes at least one communication module. The communication module is, for example, a module that complies with an optical communication standard. The communication unit 23 communicates with the measuring device 30 via the optical media converter 40. The communication unit 23 receives data used in the operation of the data processing device 20 and transmits data obtained by the operation of the data processing device 20.

[0024] The input unit 24 includes at least one input device. The input device is, for example, a physical key, a capacitive key, a pointing device, a touch screen integrated with a display, a visible light camera, or a microphone. The input unit 24 accepts an operation to input data used for the operation of the data processing device 20. The input unit 24 may be connected to the data processing device 20 as an external input device instead of being provided in the data processing device 20. As a connection interface, an interface compatible with standards such as USB, HDMI (registered trademark), or Bluetooth (registered trademark) can be used. "USB" is an abbreviation for Universal Serial Bus. "HDMI (registered trademark)" is an abbreviation for High-Definition Multimedia Interface.

[0025] The output unit 25 includes at least one output device. The output device is, for example, a display or a speaker. The display is, for example, an LCD or an organic EL display. "LCD" is an abbreviation for liquid crystal display. "EL" is an abbreviation for electroluminescent. The output unit 25 outputs data obtained by the operation of the data processing device 20. The output unit 25 may be connected to the data processing device 20 as an external output device instead of being provided in the data processing device 20. The connection interface may be an interface compatible with standards such as USB, HDMI (registered trademark), or Bluetooth (registered trademark).

[0026] The functions of the data processing device 20 are realized by executing a program according to this embodiment on a processor serving as the control unit 21. That is, the functions of the data processing device 20 are realized by software. The program causes a computer to execute the operations of the data processing device 20, thereby causing the computer to function as the data processing device 20. That is, the computer functions as the data processing device 20 by executing the operations of the data processing device 20 in accordance with the program.

[0027] The program can be stored on a non-transitory computer-readable medium. Examples of the non-transitory computer-readable medium include flash memory, magnetic recording devices, optical disks, magneto-optical recording media, and ROMs. The program can be distributed by selling, transferring, or lending portable media such as SD cards, DVDs, or CD-ROMs that store the program. "SD" is an abbreviation for Secure Digital. "DVD" is an abbreviation for digital versatile disc. "CD-ROM" is an abbreviation for compact disc read only memory. The program can also be distributed by storing it in the storage of a server and transferring it from the server to another computer. The program can also be provided as a program product.

[0028] A computer temporarily stores a program stored on a portable medium or transferred from a server in its main storage device. The computer then reads the program stored in the main storage device with its processor and executes processing in accordance with the read program. The computer may also read the program directly from the portable medium and execute processing in accordance with the program. The computer may also execute processing in accordance with the received program each time a program is transferred from the server to the computer. Processing may also be executed using a so-called ASP-type service that realizes functions simply by issuing execution instructions and obtaining results, without transferring the program from the server to the computer. "ASP" is an abbreviation for application service provider. A program is information used for processing by a computer and includes something equivalent to a program. For example, data that is not a direct instruction to a computer but has properties that define computer processing falls under the category of "something equivalent to a program."

[0029] Some or all of the functions of the data processing device 20 may be realized by a programmable circuit or a dedicated circuit as the control unit 21. In other words, some or all of the functions of the data processing device 20 may be realized by hardware.

[0030] The configuration of a measurement device 30 according to this embodiment will be described with reference to FIG.

[0031] The measurement device 30 includes a sensor 31 , an attitude measurement device 32 , a distance measurement device 33 , a data storage 34 , a battery 35 , and an optical media converter 36 .

[0032] The sensor 31 is, for example, a TOF camera. "TOF" is an abbreviation for time-of-flight. The attitude measuring device 32 is, for example, an IMU. "IMU" is an abbreviation for inertial measurement unit. The distance measuring device 33 is, for example, an encoder or an odometer. The data storage 34 is, for example, a mini PC. The battery 35 is, for example, a storage battery. The optical media converter 36 communicates with the data processing device 20 via an optical media converter 40.

[0033] An outline of this embodiment will be described with reference to FIG.

[0034] When the data processing device 20 receives the posture data 62, the travel distance data 63, and the point cloud data 61 acquired by the measurement device 30 scanning between the ducts 14 in the pipeline 11, the data processing device 20 creates a three-dimensional model of the shape inside the pipeline 11. Specifically, the data processing device 20 assigns coordinates and postures to the point cloud data 61 acquired for each image capture by the sensor 31. When performing a matching process between the first point cloud data acquired in the first image capture and the second point cloud data acquired in the second image capture, the data processing device 20 performs pre-processing and then repeats the matching process to reproduce the continuous shape between the manholes 13 inside the pipeline 11 from the point cloud data 61 acquired in the multiple images captures. The pre-processing is a process of masking areas outside the common range between the first point cloud data and the second point cloud data and adding markers to common points between the first point cloud data and the second point cloud data. According to this embodiment, the pre-processing can improve the accuracy of point cloud matching, making it possible to accurately reproduce the shape inside the pipeline 11. Moreover, it is possible to create a 3D model of the pipeline 11 without excavation. It is also possible to obtain the position of the pipeline 11, i.e., its coordinates. This makes it possible to improve the efficiency of buried object design, burial consultations, trial excavation, and on-site supervision, and to reduce accidents during excavation work.

[0035] In this embodiment, the measuring device 30 is movable along the inner surface of the pipeline 11. While moving within the pipeline 11, the measuring device 30 observes the shape of the interior of the pipeline 11 as a point cloud using the sensor 31 to acquire point cloud data 61. The measuring device 30 measures its posture at the time the point cloud data 61 was acquired using the posture measuring device 32 to acquire posture data 62. The measuring device 30 measures the travel distance at the time the point cloud data 61 was acquired using the distance measuring device 33 to acquire the travel distance data 63. The measuring device 30 stores the point cloud data 61, posture data 62, and travel distance data 63 in the data storage 34. The measuring device 30 synchronizes the point cloud data 61, posture data 62, and travel distance data 63 via the data storage 34. The worker 12 can monitor the logging status of the measuring device 30 using the data processing device 20 inside the manhole 13. The worker 12 can check the consistency of the travel distance data 63 by passing the towing cable 51 through the encoder 50 inside the manhole 13. According to this embodiment, the measurement device 30 is unitized, and data acquisition can be automated and synchronized, thereby shortening the scanning time inside the pipeline 11.

[0036] The operation of the data processing device 20 according to this embodiment will be described with reference to Fig. 4. The operation described below corresponds to the 3D modeling method according to this embodiment. That is, the 3D modeling method according to this embodiment includes steps S1 to S5 shown in Fig. 4.

[0037] S1 is a step of calculating the posture / coordinates during point cloud photography. As shown in Figure 5, when the X axis is set to the right of the measurement device 30, the Y axis is set to the downward direction of the measurement device 30, and the Z axis is set to the front direction of the measurement device 30, the measurement device 30 can move along the inner surface of the pipeline 11, so the movement direction can always be approximated to the negative direction of the Z axis, i.e., the back direction of the measurement device 30. Therefore, the control unit 21 of the data processing device 20 converts the trajectory of the measurement device 30 into coordinates based on the posture and movement distance.

[0038] Specifically, the initial position is (X ０ , Y ０ , Z ０ ), the initial posture (Yaw ０ , Pitch ０ , Roll０ ), the first distance traveled is D １ , the posture after movement (Yaw １ , Pitch １ , Roll １ ), the control unit 21 of the data processing device 20 determines the position after movement (X １ , Y １ , Z １ ) is calculated as a relative coordinate based on the previous coordinate as follows: １ =X ０ + (-D １ ) sin (Yaw １ -Yaw ０ )cos(Pitch １ -Pitch ０ ) Y １ = Y ０ + (-D １ )sin(Pitch １ -Pitch ０ ) Z １ =Z ０ + (-D １ ) cos(Yaw １ -Yaw ０ )cos(Pitch １ -Pitch ０ )

[0039] The initial position is the position of the measurement device 30 at the time when the first point cloud is captured by the sensor 31. The initial orientation is the orientation of the measurement device 30 at the time when the first point cloud is captured by the sensor 31. The initial movement distance is the movement distance of the measurement device 30 from the time when the first point cloud is captured by the sensor 31 to the time when the second point cloud is captured by the sensor 31. The orientation after movement is the orientation of the measurement device 30 at the time when the second point cloud is captured by the sensor 31.

[0040] The control unit 21 of the data processing device 20 identifies the posture at each time point by referring to the posture data 62. The control unit 21 of the data processing device 20 identifies the distance traveled from the previous time point to each time point by referring to the travel distance data 63.

[0041] The control unit 21 of the data processing device 20 determines the positions after subsequent movements in the same manner as the position after the first movement. For example, the last movement distance is Dｎ , the final posture (Yaw ｎ , Pitch ｎ , Roll ｎ ), the control unit 21 of the data processing device 20 determines the last position (X ｎ , Y ｎ , Z ｎ ) is calculated as a relative coordinate based on the previous coordinate as follows: ｎ =X ｎ－１ + (-D ｎ ) sin (Yaw ｎ -Yaw ｎ－１ )cos(Pitch ｎ -Pitch ｎ－１ ) Y ｎ = Y ｎ－１ + (-D ｎ )sin(Pitch ｎ -Pitch ｎ－１ ) Z ｎ =Z ｎ－１ + (-D ｎ ) cos(Yaw ｎ -Yaw ｎ－１ )cos(Pitch ｎ -Pitch ｎ－１ )

[0042] When the measurement device 30 moves a certain distance, the sensor 31 observes a point cloud, that is, when the point cloud data 61 is acquired, the total moving distance D １ , D ２ , ..., D ｎ will be the same distance.

[0043] The settings of the X-axis, Y-axis, and Z-axis may differ depending on the coordinate system of the device used as the measurement device 30.

[0044] According to this embodiment, coordinate calculation is possible underground where satellite positioning is not possible. The travel distance may be calculated by integrating acceleration twice, but in this embodiment, a directly measured value is used, thereby making it possible to reduce errors.

[0045] S2 is a step of plotting point clouds. For example, as shown in FIG. 6, the control unit 21 of the data processing device 20 plots the point cloud captured first and the point cloud captured second in the same three-dimensional space as a first point cloud 71 and a second point cloud 72, respectively. Then, the control unit 21 of the data processing device 20 assigns coordinates and orientations to the first point cloud 71 and the second point cloud 72. The coordinates assigned to the first point cloud 71 are (X ０ , Y ０ , Z ０ The orientation assigned to the first point group 71 is (Yaw ０ , Pitch ０ , Roll ０ The coordinates assigned to the second point group 72 are (X １ , Y １ , Z １ ), which are relative coordinates based on the coordinates assigned to the first point group 71. The orientation assigned to the second point group 72 is (Yaw １ -Yaw ０ , Pitch １ -Pitch ０ , Roll １ -Roll ０ )

[0046] The control unit 21 of the data processing device 20 refers to the point cloud data 61 to identify the point cloud photographed at each time point.

[0047] According to this embodiment, before matching point clouds, the coordinates and orientation acquired / calculated by the orientation measuring device 32 and the distance measuring device 33 are assigned to the point cloud photographed by the sensor 31, and the point cloud is plotted, thereby improving the accuracy of subsequent point cloud matching.

[0048] S3 is the step of pre-processing for point cloud matching. As shown in FIG. 6 , the connections between the plotted point clouds are not smooth. Therefore, it is possible to combine the point clouds using a point cloud combining method such as ICP. However, if a general point cloud combining method is applied without pre-processing, the two point clouds have nearly identical geometric features, which may result in the computer misrecognizing them as images taken at the same location and combining them in different ranges. "ICP" stands for iterative closest point. In this embodiment, as shown in FIG. 7 , in order to match the first point cloud 71 and the second point cloud 72 only in the common range between A and B, the control unit 21 of the data processing device 20 applies a mask 81 to areas other than the common range. The control unit 21 of the data processing device 20 further estimates the common points between the first point cloud 71 and the second point cloud 72 and assigns a marker 82. In the example shown in FIG. 7 , the marker 82 includes an artificially created spherical point cloud having a diameter equal to the inner diameter of the pipeline 11. As a variant, the mark 82 may further include an artificially created linear point cloud that is located at least partially within the spherical point cloud and extends in the vertical direction. Here, the "vertical direction" does not refer to the up and down direction in the three-dimensional space in which the first point cloud 71 and the second point cloud 72 are plotted, but refers to the up and down direction in the real space in which the pipeline 11 exists.

[0049] According to this embodiment, by providing a mask 81 and a mark 82 outside the common range, it becomes possible to combine the point clouds along the shape of the pipeline 11 .

[0050] S4 is a step of point cloud matching. As shown in FIG. 8 , the control unit 21 of the data processing device 20 matches the common area marked with markers 82 while applying a mask 81, and then removes the mask 81 and markers 82 as shown in FIG. 9 . Specifically, the control unit 21 of the data processing device 20 matches the common area of ​​the first point cloud 71 with the common area of ​​the second point cloud 72, and superimposes the first point cloud 71 and the second point cloud 72 to integrate them into a single point cloud. At this time, the matching itself is performed only on the common area, but the integration is performed on the entire first point cloud 71 and the entire second point cloud 72. That is, by superimposing the first point cloud 71 and the second point cloud 72, the control unit 21 of the data processing device 20 reassigns the corrected coordinates and orientation to the integrated point cloud 72A.

[0051] According to this embodiment, by performing pre-processing and then matching the point clouds, it is possible to accurately reproduce the shape inside the pipeline 11.

[0052] Steps S1 to S4 are repeated. For example, the control unit 21 of the data processing device 20 determines the second position (X ２ , Y ２ , Z ２ ) is calculated, in S2, as shown in FIG. 10, the third point cloud photographed is plotted as a third point cloud 73 in the same three-dimensional space as the integrated point cloud 72A. Then, the control unit 21 of the data processing device 20 assigns coordinates and orientation to the third point cloud 73. The coordinates assigned to the third point cloud 73 are (X ２ , Y ２ , Z ２ ), which are relative coordinates based on the coordinates assigned to the integrated point group 72A. The orientation assigned to the third point group 73 is (Yaw ２ -Yaw １ , Pitch ２ -Pitch １ , Roll ２ -Roll １ The procedure of the point cloud matching pre-processing in S3 and the procedure of point cloud matching in S4 are the same as those for the first point cloud 71 and the second point cloud 72.

[0053] According to this embodiment, it is possible to accurately reproduce the shape of the continuous pipeline 11 from a plurality of point clouds by repeating the process.

[0054] S5 is a step of creating a three-dimensional model of the entire length of the pipeline 11. In S5, the control unit 21 of the data processing device 20 continuously reproduces the shape of the pipeline 11 from its start point to its end point as a three-dimensional model 90, as shown in FIG.

[0055] According to this embodiment, the shape of the pipeline 11 from its start point to its end point can be accurately reproduced in three dimensions.

[0056] As described above, the control unit 21 of the data processing device 20 uses a point cloud matching technique to sequentially combine point cloud data 61 obtained through multiple observations by the sensor 31, which observes the partial shapes of the interior of the pipeline 11 as a point cloud while moving within the pipeline 11, to create a 3D model of the continuous shape within the pipeline 11. The term "combining" includes excluding points that correspond to outside the common range within the pipeline 11 in each of the point clouds of the point cloud data 61 to be combined and the point cloud of the point cloud data 61 to be combined from the target of point cloud matching. For example, in FIG. 7 , in the second point cloud 72 corresponding to the point cloud of the point cloud data 61 to be combined, points closer to the end point of the pipeline 11 than one end B of the common range are excluded from the target of point cloud matching by a mask 81. In FIG. 7 , in the first point cloud 71 corresponding to the point cloud data 61 to be combined, points closer to the start point of the pipeline 11 than the other end A of the common range are excluded from the target of point cloud matching by a mask 81. Therefore, even when a general point cloud matching method such as ICP is used, the first point cloud 71 and the second point cloud 72 are unlikely to be combined in different ranges, allowing for accurate 3D modeling of the continuous shape inside the pipeline 11. The size of the common range is set according to the travel distance for each observation. For example, if the travel distance from the time when the first point cloud 71 is observed to the time when the second point cloud 72 is observed is 20 cm, the mask 81 is applied to 20 cm of the front side of the first point cloud 71 and 20 cm of the back side of the second point cloud 72.

[0057] In this embodiment, "combining" includes performing point cloud matching after inserting a point cloud of marks corresponding to common positions within the pipeline 11 into each of the point clouds of the point cloud data 61 to be combined and the point clouds of the point cloud data 61 to be combined. The point cloud of marks is, for example, a spherical point cloud having the same diameter as the diameter inside the pipeline 11. For example, in FIG. 7 , a spherical point cloud whose center is located at one end B of the common range and whose diameter is the same as the inner diameter of the pipeline 11 is assigned as a mark 82 to the second point cloud 72 corresponding to the point cloud of the point cloud data 61 to be combined. In FIG. 7 , a spherical point cloud whose center is located at one end B of the common range and whose diameter is the same as the inner diameter of the pipeline 11 is also assigned as a mark 82 to the first point cloud 71 corresponding to the point cloud data 61 to be combined. Because the marks 82 have the same characteristics, even when a general point cloud matching method such as ICP is used, the first point cloud 71 and the second point cloud 72 are less likely to be combined in different ranges, making it possible to accurately create a 3D model of the continuous shape inside the pipeline 11. The number of points included in the spherical point cloud of the marks 82 is, for example, 5,000, but may be adjusted appropriately depending on the granularity of the point cloud data 61.

[0058] In addition to a spherical point cloud, a linear point cloud extending vertically through the center of the spherical point cloud may be assigned as the mark 82 to each of the second point cloud 72 and the first point cloud 71. The linear point cloud may be shorter than the inner diameter of the pipeline 11, may have the same length as the inner diameter of the pipeline 11, or may be longer than the inner diameter of the pipeline 11. When the length of the linear point cloud is longer than the inner diameter of the pipeline 11, i.e., longer than the diameter of the spherical point cloud, the mark 82 resembles a spherical point cloud impaled on a skewer-shaped point cloud. The number of points included in the skewer-shaped point cloud is, for example, 5,000, but may be adjusted as appropriate. The length of the skewer-shaped point cloud is, for example, equivalent to 1 meter when the diameter of the pipeline 11 is approximately 8 cm, but may be adjusted as appropriate. When a linear point cloud, such as a comb-shaped point cloud, is also applied, the continuous shape inside the pipeline 11 can be three-dimensionally modeled with higher accuracy than when only a spherical point cloud is used as the marker 82. Specifically, when a linear point cloud extending in the vertical direction is also applied, twisting in the axial direction can be suppressed, improving accuracy.

[0059] In this embodiment, "combining" includes plotting the point clouds of the point cloud data 61 to be combined and the point clouds of the point cloud data 61 to be combined in the same three-dimensional space in a positional relationship corresponding to the difference between the attitude and travel distance of the sensor 31 at the time when the point cloud of the point cloud data 61 to be combined was observed and the attitude and travel distance of the sensor 31 at the time when the point cloud of the point cloud data 61 to be combined was observed, and then performing point cloud matching. In FIG. 6 , a second point cloud 72 corresponding to the point cloud of the point cloud data 61 to be combined and a first point cloud 71 corresponding to the point cloud data 61 to be combined are plotted in the same three-dimensional space in a positional relationship corresponding to the difference between the attitude and travel distance of the sensor 31 at the time when the second point cloud 72 was observed and the attitude and travel distance of the sensor 31 at the time when the first point cloud 71 was observed. This allows for more accurate three-dimensional modeling of the continuous shape inside the pipeline 11.

[0060] The pipeline 11 may be laid in a single line, or in multiple lines as shown in Fig. 12. In either case, the position of the iron cover 15 of the manhole 13 can be confirmed from the ground surface, and the relative positions of the ducts 14, which are the start and end points of the pipeline 11, from the iron cover 15 can also be obtained, so that the absolute coordinates of the start and end points of the pipeline 11 can be identified by surveying. The absolute coordinates are a combination of latitude, longitude, and altitude.

[0061] According to this embodiment, by specifying the coordinates of the start point and the end point of the pipeline 11, it is possible to improve the accuracy in specifying the position of the pipeline 11. The position of the pipeline 11 can be easily output to any topographical map or other drawing that can be plotted by latitude, longitude, and altitude.

[0062] Even when multiple pipelines 11 are laid, the relative positions of the individual pipelines 11 are determined by design, so if the coordinates of any one pipeline 11 can be identified, the positions of the other pipelines 11 laid along the same route can be calculated. Therefore, it is not necessary to measure all of the pipelines 11, and efficient position confirmation is possible.

[0063] In this embodiment, the measurement device 30 includes only one sensor 31 such as a TOF camera, but two may be provided, one at the front and one at the back. For example, by installing two TOF cameras, the point cloud acquisition area can be expanded. As a result, the accuracy of the three-dimensional model can also be improved.

[0064] The present disclosure is not limited to the above-described embodiments. For example, two or more blocks shown in the block diagram may be integrated, or one block may be divided. Two or more steps shown in the flowchart may be executed in parallel or in a different order, instead of being executed in chronological order as described, depending on the processing capabilities of the device executing each step, or as needed. Other modifications are possible within the scope of the present disclosure.

[0065] 10 3D modeling system 11 Pipe 12 Worker 13 Manhole 14 Duct 15 Iron cover 20 Data processing device 21 Control unit 22 Memory unit 23 Communication unit 24 Input unit 25 Output unit 30 Measuring device 31 Sensor 32 Attitude measuring device 33 Distance measuring device 34 Data storage 35 Battery 36 Optical media converter 40 Optical media converter 41 Communication cable 50 Encoder 51 Towing cable 61 Point cloud data 62 Attitude data 63 Travel distance data 71 First point cloud 72 Second point cloud 72A Integrated point cloud 73 Third point cloud 81 Mask 82 Marker 90 3D model

Claims

1. A data processing device having a control unit that uses a point cloud matching technique to sequentially combine point cloud data obtained from multiple observations by a sensor that observes the partial shapes within the pipeline as a point cloud while moving within the pipeline, thereby creating a three-dimensional model of the continuous shape within the pipeline, wherein the combining includes excluding points that correspond outside the common range within the pipeline in each of the point cloud of the point cloud data to be combined and the point cloud of the point cloud data to be combined from the target of point cloud matching.

2. The data processing device according to claim 1, wherein the combining step includes inserting a point cloud of landmarks corresponding to a common position within the pipeline into each of the point clouds of the point cloud data to be combined and the point cloud of the point cloud data to be combined, and then performing point cloud matching.

3. The data processing device according to claim 2, wherein the landmark point cloud includes a spherical point cloud having the same diameter as the diameter inside the pipeline.

4. A data processing device according to any one of claims 1 to 3, wherein the combining step includes plotting the point cloud of the point cloud data to be combined and the point cloud of the point cloud data to be combined in the same three-dimensional space in a positional relationship corresponding to the difference between the attitude and travel distance of the sensor at the time when the point cloud of the point cloud data to be combined was observed and the attitude and travel distance of the sensor at the time when the point cloud of the point cloud data to be combined was observed, and then performing point cloud matching.

Images