Point cloud joining device, point cloud joining system, point cloud joining method, and point cloud joining program

Abstract

To correct self position attitude information of a point group so that the point group corresponding to the same object is joined.SOLUTION: A tie point acquisition unit 113 that acquires, as tie points, a transverse section characteristic point that is a common characteristic point in a transverse section of a target object and a longitudinal section characteristic point that is a common characteristic point in a longitudinal section of the target object, from a first laser point cloud that is a three dimensional point cloud obtained by a first laser scanner and a second laser point cloud that is a three dimensional point cloud obtained by a second laser scanner that measures the same target object in a posture different from that of the first laser scanner; A deviation amount calculation unit 114 that calculates a posture deviation amount between a transverse section characteristic point in the first laser point cloud and a transverse section characteristic point in the second laser point cloud, and calculates a posture deviation amount between a longitudinal section characteristic point in the first laser point cloud and a longitudinal section characteristic point in the second laser point cloud, and a self-position and posture correction unit 115 that generates self-position and posture correction information obtained by correcting self-position and posture information on the basis of the posture deviation amount.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a technique for joining multiple point clouds obtained by a mobile mapping system (MMS). [Background technology]

[0002] As a conventional point cloud joining device and method, Patent Document 1 discloses a device that extracts tie points from a reference laser point cloud contained in reference measurement data and a target laser point cloud contained in target measurement data, and corrects the self-position contained in the target measurement data so that the positions of the tie points match. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7412651 Summary of the Invention [Problem to be solved by the invention]

[0004] The conventional point cloud stitching device described above does not correct point clouds based on differences in measurement orientation, which means that two point clouds with different orientations at the time of measurement cannot be stitched together to form point clouds corresponding to the same object, resulting in misalignment.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a point cloud joining device that can correct the self-position and posture information of multiple point clouds that have different postures at the time of measurement, using the point cloud of one laser scanner as a reference so that point clouds corresponding to the same object are joined. [Means for solving the problem]

[0006] The point cloud joining device of the present disclosure is a tie point acquisition unit that acquires, as tie points, cross-sectional feature points that are common feature points in cross-sections of the object and longitudinal-sectional feature points that are common feature points in longitudinal sections of the object from a first laser point cloud that is a three-dimensional point cloud obtained by a first laser scanner and a second laser point cloud that is a three-dimensional point cloud obtained by a second laser scanner that measures the same object in a different attitude from the first laser scanner; a deviation amount calculation unit that calculates a posture deviation amount between a cross-sectional feature point in the first laser point cloud and a cross-sectional feature point in the second laser point cloud, and calculates a posture deviation amount between a longitudinal cross-sectional feature point in the first laser point cloud and a longitudinal cross-sectional feature point in the second laser point cloud; a self-position and attitude correction unit that corrects self-position and attitude information of the second laser point cloud based on the attitude deviation amount; A point cloud joining device comprising: [Effects of the Invention]

[0007] According to the present disclosure, it is possible to correct the self-position and orientation information of the point cloud so that point clouds corresponding to the same object are joined. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. [Figure 2] FIG. 1 is a diagram illustrating a configuration of a measurement device. [Figure 3] FIG. 2 is a top view corresponding to FIG. [Figure 4] FIG. 2 is a diagram illustrating a configuration of a measurement vehicle. [Figure 5] FIG. 1 is a configuration diagram of a point group joining device. [Figure 6] FIG. 2 is a diagram showing information contained in an auxiliary storage device. [Figure 7] This is an illustration of how to measure a pedestrian bridge. [Figure 8] This is an image diagram of when measuring a suspension wire and a trolley wire. [Figure 9] 10 is a flowchart showing the operation of the point cloud joining device. [Figure 10]FIG. 10 is an operational flow diagram of the tie point acquisition unit in the tie point acquisition process. [Figure 11] FIG. 10 is a diagram showing an example of a user input screen. [Figure 12] 10 is an example of a point cloud displayed on an output device when there is an error in the self-position. [Figure 13] 10 is an example of a point cloud displayed on an output device when there is an attitude error. [Figure 14] 10 is an example of a screen displayed on an output device. [Figure 15] 10 is an example of a screen displayed on an output device. [Figure 16] 10 is an example of a screen displayed on an output device. [Figure 17] FIG. 10 is a diagram illustrating an example of calculating a deviation amount. [Figure 18] FIG. 10 is a diagram showing an example of an input screen that displays a measurement point cloud. [Figure 19] This is an illustration of how to measure a pedestrian bridge. [Figure 20] FIG. 10 is a diagram illustrating an example of a point cloud acquired for the same object. [Figure 21] FIG. 10 is a diagram showing an example in which point groups relating to the same object are joined together. DETAILED DESCRIPTION OF THE INVENTION

[0009] MMS uses a laser scanner to measure the target object. When using a single laser scanner, only a portion of the target object can be measured because the surface it hits varies depending on the direction of the laser irradiation, and point clouds may be missing depending on the surface condition of the surface being measured. For example, when measuring the distance between trolleys and suspension wires on railway overhead lines, if the measurement surface of the overhead wire is dirty, part of the measurement may not be possible, making it impossible to measure the distance in that part. For this reason, MMSs with multiple laser scanners are used.

[0010] When measuring a point cloud using multiple laser scanners, it is necessary to overlap the point clouds acquired by each laser scanner. Generally, a point cloud is determined by the direction and distance of the measurement points relative to the laser scanner's own position and orientation. Furthermore, the laser scanner's own position and orientation are acquired using signals received from positioning satellites by a measurement vehicle equipped with a laser scanner. Therefore, while the position of a point cloud measured when the satellite is visible can be determined accurately, the position of a point cloud measured when the satellite is not visible is ambiguous. As a result, a problem occurs in which point clouds of the same object acquired by multiple laser scanners in sections where the satellite is not visible do not overlap accurately.

[0011] One way to solve this problem is to treat the measurement vehicle as a rigid body and correct its own position and orientation by resolving the inconsistencies in the point clouds obtained from each laser scanner. However, there are limits to the accuracy of the correction because slight distortion occurs in the vehicle body due to rail joints, steps, or right and left turns. Furthermore, if errors occur in the measurements made by the laser scanners, it is difficult to solve the problem by improving the accuracy of obtaining the self-position and orientation. The point cloud joining device disclosed herein corrects the self-position and orientation information of other point clouds so that point clouds corresponding to the same object can be joined using the point cloud of one laser scanner as a reference.

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the following description, like components are denoted by like reference numerals, and their names and functions are the same or similar. Therefore, detailed descriptions thereof may be omitted.

[0013] Embodiment 1 ***Configuration Description*** The measurement vehicle 200 that acquires the point clouds to be joined by the point cloud joining device 100 will be described.

[0014] Fig. 1 is a side view of the measurement vehicle 200. The x direction in Fig. 1 is called the traveling direction or forward direction of the measurement vehicle 200. Furthermore, the surface directly facing the measurement vehicle 200 when viewed from the front is called the front surface, and the surface directly facing the measurement vehicle 200 when viewed from the rear is called the rear surface.

[0015] The measurement vehicle 200 is mounted on a road-rail vehicle 800 or a trolley, and measures the three-dimensional point cloud while the road-rail vehicle 800 or the trolley is moving. The measurement vehicle 200 itself may perform measurements while traveling.

[0016] A roof carrier 201 is attached to the top surface of the measurement vehicle 200. A rear carrier 203 is attached to the rear surface. A measurement device 2010 is attached via the roof carrier 201 and the rear carrier 203. FIG. 2 is a diagram showing the configuration of the measurement device. The measurement device 2010 includes a first laser scanner 210, a second laser scanner 220, a third laser scanner 230, a fourth laser scanner 240, an overhead line measurement camera 250, an antenna 260, an all-around camera 270, a front camera 280, and a rear camera 290.

[0017] The first laser scanner 210 is mounted tilted forward so as to perform measurements diagonally downward in front and diagonally upward in rear relative to the front end of the measurement vehicle 200. The second laser scanner 220 is mounted tilted backward so as to perform measurements diagonally upward in front and diagonally downward in rear relative to the rear end of the measurement vehicle 200.

[0018] Fig. 3 is a top view corresponding to Fig. 1. The first laser scanner 210 and the second laser scanner 220 may be pulled out perpendicular to the traveling direction by a slide mechanism 202 during measurement.

[0019] The third laser scanner 230 measures vertically above and below, for example, measuring the ceiling of a tunnel or a railway track. The third laser scanner 230 is also called a vertical laser.

[0020] The fourth laser scanner 240 performs measurements by scanning around the y-axis within a plane parallel to the traveling direction, that is, within the xz plane in Fig. 1. The fourth laser scanner 240 is also called a horizontal laser.

[0021] The overhead line measurement camera 250 captures an image above the measurement vehicle 200. The objects captured by the overhead line measurement camera 250 include objects measured by at least one of the first laser scanner 210, the second laser scanner 220, and the fourth laser scanner 240.

[0022] The antenna 260 acquires the self-position information of the measurement vehicle 200 through communication with a positioning satellite.

[0023] The omnidirectional camera 270 captures images of the entire periphery of the measurement vehicle 200. The surrounding images captured by the omnidirectional camera 270 include, for example, railroad rail surfaces, footbridges, guardrails, signs, road markings, walls, pillars, other features, overhead lines, and buildings.

[0024] The front camera 280 captures an image in front of the measurement vehicle 200. The front camera 280 may be mounted so that it can be turned left and right. The front camera 280 captures images of, for example, the road surface, the rail surface of a railway, a footbridge, a guardrail, signs, road markings, walls, pillars, other features, overhead lines, and buildings. When the front camera 280 captures an image of the rail surface of a railway, a light may be provided close to the front camera 280 to illuminate the imaging unit of the front camera 280.

[0025] The rear camera 290 captures an image of the rear underside of the measurement vehicle 200. The rear camera 290 captures an image of, for example, a road surface or a rail surface of a railway. When the rear camera 290 captures an image of a rail surface of a railway, a light that illuminates the imaging unit of the rear camera 290 may be provided close to the rear camera 290.

[0026] In addition, the measurement device 2010 may further include another camera that captures images of the surroundings of the measurement vehicle 200. Furthermore, the imaging direction of the other camera is not specified, and it may be above, diagonally above, or diagonally below the measurement vehicle 200.

[0027] Fig. 4 is a diagram showing the configuration of the measurement vehicle 200. Fig. 4(a) is a left side view of the measurement vehicle 200. Fig. 4(b) is a top view of the measurement vehicle 200. Fig. 4(c) is a right side view of the measurement vehicle 200.

[0028] Fig. 4(d) is a rear view of the measurement vehicle 200 when the first laser scanner 210 is not pulled out by the slide mechanism 202. In Fig. 4(d), the second laser scanner 220 and the overhead line measurement camera 250 are omitted. Fig. 4(e) is a rear view of the measurement vehicle 200 when the first laser scanner 210 is pulled out by the slide mechanism 202. In Fig. 4(e), the second laser scanner 220 and the overhead line measurement camera 250 are omitted.

[0029] Fig. 4(f) is a rear view of the measurement vehicle 200 when the first laser scanner 210 and the second laser scanner 220 are not pulled out by the slide mechanism 202. In Fig. 4(f), the first laser scanner 210, the second laser scanner 220, and the overhead contact line measurement camera 250 are omitted.

[0030] FIG. 4(g) is a rear view of the measurement vehicle 200 when the second laser scanner 220 is not pulled out by the slide mechanism 202. The first laser scanner 210 is omitted from FIG. 4(g). FIG. 4(h) is a rear view of the measurement vehicle 200 when the second laser scanner 220 is pulled out by the slide mechanism 202. The first laser scanner 210 and the 360° camera 270 are omitted from FIG. 4(h). FIG. 4(i) is a front view of the measurement vehicle 200. The second laser scanner 220 is omitted from FIG. 4(i).

[0031] The laser scanner including the first laser scanner 210 is, for example, a LiDAR. The LiDAR may rotate a scanning mirror in a tilt direction and a pan direction, and scan with an irradiated laser within a two-dimensional plane. Alternatively, the LiDAR may rotate a scanning mirror 360°, and scan with an irradiated laser within a rotation plane (for example, within a plane tilted at an arbitrary tilt angle θ from the xy plane to the xz plane, with the contact surface of the laser being the xy plane).

[0032] The configuration of the point cloud joining device 100 will be described with reference to Fig. 5. Fig. 5 is a diagram showing the configuration of the point cloud joining device 100. The point cloud joining device 100 includes a processor 110, a memory 120, an auxiliary storage device 130, a communication device 140, and an input / output interface 150.

[0033] The processor 110 executes each operation of the point cloud joining device 100. The processor 110 is, for example, a CPU (Central Processing Unit).

[0034] 5, the processor 110 includes a data acquisition unit 111, a point cloud generation unit 112, a tie point acquisition unit 113, a deviation amount calculation unit 114, and a self-position and orientation correction unit 115. The data acquisition unit 111, the point cloud generation unit 112, the tie point acquisition unit 113, the deviation amount calculation unit 114, and the self-position and orientation correction unit 115 may be realized by a program.

[0035] The memory 120 reads information stored in the auxiliary storage device 130. The memory 120 is, for example, a RAM (Random Access Memory).

[0036] The auxiliary storage device 130 stores information used by the point cloud joining device 100. The auxiliary storage device 130 is, for example, a read only memory (ROM), a hard disk drive (HDD), or a flash memory.

[0037] The communication device 140 performs communication between the point cloud joining device 100 and external devices. The communication device 140 is, for example, a communication chip or a NIC (Network Interface Card).

[0038] An input device and an output device are connected to the input / output interface 150. The input device is, for example, a keyboard or a mouse. The output device is, for example, a display.

[0039] 6 is a diagram showing information stored in the auxiliary storage device 130. The auxiliary storage device 130 includes self-position and orientation information 132, first distance and orientation information 133, second distance and orientation information 134, first laser point cloud 135, second laser point cloud 136, self-position and orientation correction information 137, and measurement point cloud 138.

[0040] The self-position and attitude information 132 is information that indicates the self-position and attitude of the measurement vehicle 200 at each time. The information on the self-position and attitude included in the self-position and attitude information 132 is obtained from signals that the measurement vehicle 200 receives from positioning satellites at regular intervals and from angular velocity and acceleration measured at each time by an inertial measurement unit provided on the measurement vehicle 200. The self-position and attitude information 132 is stored in the auxiliary storage device 130 via the communication device 140.

[0041] The first distance and orientation information 133 is information indicating the orientation and distance of an object (i.e., a point constituting a point cloud) relative to the first laser scanner 210. The distance and orientation are recorded for each time. The first distance and orientation information 133 is also stored in the auxiliary storage device 130 via the communication device 140.

[0042] The second distance and orientation information 134 is information indicating the orientation and distance of the target object (i.e., the points constituting the point cloud) relative to the second laser scanner 220. The distance and orientation are recorded for each time. The second distance and orientation information 134 is also stored in the auxiliary storage device 130 via the communication device 140.

[0043] The first laser point cloud 135 is a three-dimensional point cloud generated by the self-position and orientation information 132 and the first distance and direction information 133 .

[0044] The second laser point cloud 136 is a three-dimensional point cloud generated by the self-position and orientation information 132 and the second distance and orientation information 134 .

[0045] The self-position and orientation correction information 137 is newly generated self-position and orientation information so as to match the common feature points of the first laser point cloud 135 and the second laser point cloud 136.

[0046] The corrected point group 138 is a three-dimensional point group generated using the second distance and direction information 134 and the self-position and orientation corrected information 137 .

[0047] The measurement point cloud 139 is a three-dimensional point cloud generated by overlapping the first laser point cloud 135 and the correction point cloud 138 .

[0048] 7 is an image diagram of a case where a pedestrian bridge is measured as the object 900. The measurement vehicle 200 moves relative to the object 900 in the order of FIG. 7(a), FIG. 7(b), and FIG. 7(c).

[0049] As shown in FIG. 7( a ), first, the second laser scanner 220 captures the target object 900 and generates the second laser point cloud 136 .

[0050] Subsequently, as shown in FIG. 7(b), when the first laser scanner 210 captures the target object 900, a first laser point cloud 135 is generated.

[0051] 7(c) shows a state in which the measurement vehicle 200 has passed the object 900. As shown in FIG. 7(c), in measuring the object 900 having a surface facing the traveling direction of the measurement vehicle 200, the first laser point cloud 135 is a point cloud of the front and bottom surfaces of the object 900, and the second laser point cloud 136 is a point cloud of the rear and bottom surfaces of the object 900.

[0052] 8 is an image diagram of a case where a suspension wire and a trolley wire are measured as the object 900. FIG. 8(b) is a view of FIG. 8(a) from the right side of the paper, that is, from behind the measurement vehicle 200.

[0053] 8(b), the first laser scanner 210 measures from the left side of the object 900, and the second laser scanner 220 measures from the right side of the object 900. As a result, the measurement surfaces of the first laser scanner 210 and the second laser scanner 220 are different.

[0054] ***Explanation of Operation*** The operation of the point cloud joining device 100 will be described with reference to Fig. 9. Fig. 9 is a flowchart showing the operation of the point cloud joining device 100. The operation of the point cloud joining device 100 corresponds to a point cloud joining method.

[0055] In step a1, the data acquisition unit 111 of the point cloud joining device 100 acquires measurement data via the communication device 140. The measurement data includes self-position and orientation information 132, first distance and orientation information 133, and second distance and orientation information 134. The acquired measurement data is stored in the auxiliary storage device 130. Step a1 is a measurement data acquisition step in the point cloud joining method.

[0056] In step a2, the point cloud generation unit 112 generates a first laser point cloud 135 and a second laser point cloud 136. First, using the self-position and attitude information 132 and the first distance and orientation information 133 acquired at the same time, the point cloud generation unit 112 generates the first laser point cloud 135 by assuming that an object is located at a distance indicated by the first distance and orientation information 133 in the orientation indicated by the first distance and orientation information 133 relative to the self-position and attitude of the measurement vehicle 200 at that time.

[0057] Similarly, the point cloud generation unit 112 generates a second laser point cloud 136 using the self-position and attitude information 132 and the second distance and orientation information 134 acquired at the same time. At this time, the point cloud generation unit 112 generates the second laser point cloud 136 by assuming that the target object is located at a distance indicated by the second distance and orientation information 134 in the orientation indicated by the second distance and orientation information 134 relative to the self-position and attitude of the measurement vehicle 200 at that time. Step a2 is a point cloud generation step in the point cloud joining method.

[0058] Here, the self-position included in the self-position and attitude information 132 is based on information received from a positioning satellite as described above. Therefore, depending on the communication conditions with the positioning satellite, the self-position corresponding to the specified time may not be obtained. In this case, the point cloud generation unit 112 predicts the self-position at the time between the time when the self-position was not obtained and the reception times and the self-position before and after the time when the self-position was not obtained. The predicted self-position is then used to generate the first laser point cloud 135 and the second laser point cloud 136.

[0059] In step a3, the tie point acquisition unit 113 acquires tie points. Fig. 10 is a flowchart showing the operation of the tie point acquisition unit 113 in step a3. First, in step t1 in Fig. 10, the tie point acquisition unit 113 displays the first laser point cloud 135 and the second laser point cloud 136 in a cross section or a longitudinal section in an overlapping manner on an output device connected via the input / output interface 150. Here, the cross section is a measurement surface parallel to the traveling direction of the measurement vehicle 200. The longitudinal section is a measurement surface perpendicular to the traveling direction of the measurement vehicle 200.

[0060] FIG. 11 is a diagram showing an example of a user input screen displayed on the output device. The input screen has an input panel and an operation panel. As shown in FIG. 11, the first laser point cloud 135 and the second laser point cloud 136 are displayed superimposed on each other on the input panel. The operation panel displays whether the two point clouds displayed on the input panel are cross-sectional or longitudinal, as well as the measurement time and position of the point selected on the input panel. Along with the display of the input panel, operation panel, and point clouds, as shown in FIG. 11, a message prompting the user to input tie points may be displayed on the input screen.

[0061] The point cloud displayed on the input panel may also be displayed in regions divided from the entire point cloud. In this case, the user can change the display region using the buttons labeled "Previous Position" or "Next Position" on the operation panel shown in Fig. 11. The point cloud to be corrected may also be specified by pressing the "Start Correction Region" button at the start point of the region where the point cloud is to be corrected and the "End Correction Region" button at the end point of the region where the point cloud is to be corrected.

[0062] Here, the time it takes for the laser scanner to make one rotation is typically 1 / 100 seconds or less. Because the time it takes to measure one object is sufficiently short compared to the time it takes for the laser scanner to make one rotation, errors in the laser point cloud and self-position and attitude correction information 137 due to changes in the distance traveled by the measurement vehicle 200 or changes in attitude while measuring one object 900 can be ignored. However, the first laser point cloud 135 and the second laser point cloud 136 measure the same part of the object 900 at different times. Furthermore, errors in the self-position and attitude correction information 137 caused by changes in attitude vary depending on the mounting angle of the laser scanner. Therefore, if an error occurs in the self-position and attitude information 132 when measuring at least one of the point clouds, the first laser point cloud 135 and the second laser point cloud 136 for the same object may not overlap.

[0063] Errors in the self-position and attitude information 132 can occur, for example, when the self-position and attitude information 132 cannot be obtained from a positioning satellite, when the attitude of the measurement vehicle 200 changes due to a step, or when distortion occurs in the body of the measurement vehicle 200 when turning right or left or in other cases.

[0064] Hereinafter, the error amount caused by an error in the self-position and attitude information 132 will be referred to as the deviation amount. The deviation amount is composed of a position deviation amount and an attitude deviation amount. The position deviation amount includes an error in the height direction on the cross section, an error in the lateral direction, and an error in the depth direction on the longitudinal section. The attitude deviation amount includes an error in the roll direction, pitch direction, and yaw direction on the cross section.

[0065] 12 shows examples of point clouds displayed on the output device when there is an error in the self-position of the self-position and orientation information 132. In FIG. 12(a), the second laser point cloud 136 has an error in the height direction in the cross section. In FIG. 12(b), the second laser point cloud 136 has an error in the lateral direction in the cross section. In FIG. 12(c), the second laser point cloud 136 has an error in the depth direction in the longitudinal section.

[0066] Figure 13 shows examples of point clouds displayed on the output device when there is an attitude error. In Figure 13(a), the second laser point cloud 136 has a roll error. In Figure 13(b), the second laser point cloud 136 has a pitch error. In Figure 13(c), the first laser point cloud 136 has a yaw error.

[0067] Returning to FIG. 10, in step t2, the tie point acquisition unit 113 receives input from the user using an input device connected via the input / output interface 150 regarding cross-sectional feature points that are common feature points in the cross-section of the object 900 and longitudinal-sectional feature points that are common feature points in the longitudinal section of the object.

[0068] The user inputs cross-sectional feature points for the cross section displayed on the output device, and inputs longitudinal-sectional feature points for the longitudinal section. Two or more cross-sectional feature points and two or more longitudinal-sectional feature points are input.

[0069] When inputting tie points, the user clicks cross-sectional feature points and longitudinal-sectional feature points for each region on the input panel. The input cross-sectional feature points and longitudinal-sectional feature points do not have to be on the point cloud, and may be points that are expected to be intersections of the point cloud, for example. When input of the cross-sectional feature points and longitudinal-sectional feature points is completed, the user presses the "Confirm" button on the operation panel to finish. When the "Confirm" button is pressed, in step t3 in FIG. 10, the tie point acquisition unit 113 acquires the input tie points.

[0070] Fig. 14 is another example of a screen displayed on the output device. In Fig. 14, a first laser point cloud 135 and a second laser point cloud 136 are point clouds on a cross section of the same object, but do not overlap due to an error in the self-position and orientation information 132. In this case, the user inputs, for example, points A and B in Fig. 14 as cross section feature points.

[0071] Fig. 15 is another example of a screen displayed on the output device. In Fig. 15, a first laser point cloud 135 and a second laser point cloud 136 are point clouds on a cross section of the same object, but do not overlap due to an error in the self-position and orientation information 132. The user inputs, for example, points A and B in Fig. 15 as cross section feature points.

[0072] Fig. 16 is another example of a screen displayed on the output device. In Fig. 16, a first laser point cloud 135 and a second laser point cloud 136 are point clouds on a cross section of the same object, but do not overlap due to an error in the self-position and orientation information 132. For example, the user inputs points A and B in Fig. 16 as cross section feature points. Step a3 is a tie point acquisition process in the point cloud joining method.

[0073] Returning to Fig. 9, in step a4, the deviation amount calculation unit 114 calculates the positional deviation amount and the orientation deviation amount of the tie points of the second laser point cloud 136 relative to the tie points of the first laser point cloud 135. Furthermore, from the deviation amount at the tie points, the deviation amount at each point of the second laser point cloud 136, which measured the same object as the first laser point cloud 135, is calculated. Step a4 is a deviation amount calculation step in the point cloud joining method.

[0074] In calculating the deviation amount, the deviation amount calculation unit 114 predicts the erroneous elements among the elements constituting the position deviation amount and the attitude deviation amount. As described above, the elements of the position deviation amount are the deviation amounts in the height direction or the lateral direction in the transverse cross section and the depth direction in the longitudinal cross section. The elements of the attitude deviation amount are the roll error, pitch error, or yaw error in the transverse cross section.

[0075] An example of deviation calculation is shown in Figure 17. Figure 17(a) shows the point cloud before deviation calculation. Figure 17(b) shows the point cloud when the roll error in Figure 17(a) has been removed. For example, in step a3, it is assumed that points A and C in the first laser point cloud 135 and points B and D in the second laser point cloud 136 in Figure 17(a) are acquired as tie points.

[0076] In this case, for example, in Figure 17(a), a roll error is predicted from the angle between the line connecting points A and C and the line connecting points B and D. Also, in Figure 17(b), a pitch error is predicted from the difference between the distance in the Y direction between points A and C and the distance in the Y direction between points B and D. Furthermore, in Figure 17(b), a yaw error is predicted from the difference between the distance in the x direction between points A and C and the distance in the x direction between points B and D.

[0077] In addition, in Figure 17(b), the difference in the x-direction positions of points A and B is predicted to result in a horizontal error in the cross section. Also, in Figure 17(b), the difference in the y-direction positions of points A and B is predicted to result in a height error in the cross section. Furthermore, if there is an error in the z-direction between points A and B, it is predicted that there will be a depth error in the longitudinal section.

[0078] Note that the number of tie points may be increased to derive solutions for all three-dimensional position, roll, pitch, and yaw to resolve discrepancies in the amount of deviation. However, if it is difficult to obtain the required number of tie points, roll and pitch may be given priority for solution. This is because errors in roll and pitch are easier to see than in other elements, and the accuracy of tie point input tends to be high.

[0079] 9 , in step a5, the self-position and orientation correction unit 115 corrects the self-position and orientation information of the second laser point cloud so as to eliminate the deviation amount at each point calculated in step a4, thereby generating self-position and orientation correction information 137. The self-position and orientation correction information 137 is stored in the auxiliary storage device 130.

[0080] Here, the self position and orientation correction unit 115 generates self position and orientation correction information 137 by adding or subtracting the amount of deviation to or from the self position and orientation information 132. That is, when a point cloud is generated using the self position and orientation correction information 137 and the second distance and orientation information 134, the self position and orientation correction unit 115 adds or subtracts the amount of position deviation to or from the self position information included in the self position and orientation information 132, or adds or subtracts the amount of attitude deviation to or from the attitude information included in the self position and orientation information 132, so that the point cloud coincides with the first laser point cloud 135 of the same object, thereby generating the self position and orientation correction information 137. Step a5 is a self position and orientation correction information generation step in the point cloud joining method.

[0081] In step a6, the point cloud generation unit 112 generates a corrected point cloud 138 using the second distance and orientation information 134 and the self-position and attitude correction information 137. Step a6 is a corrected point cloud generation step in the point cloud joining method.

[0082] In step a7, the point cloud generation unit 112 combines the first laser point cloud 135 and the corrected point cloud 138 to generate a measurement point cloud 139, which is a three-dimensional point cloud within the measurement range. Step a7 is a measurement point cloud generation step in the point cloud joining method.

[0083] After step a7, the point cloud joining device 100 may display a message indicating the completion of measurement point cloud generation on the output device. Together with the message display, or in response to a user operation after step a7, the point cloud joining device 100 displays the measurement point cloud 139 on the input panel. Fig. 18 is a diagram showing an example of an input screen displaying the measurement point cloud 139.

[0084] If the user checks the measurement point cloud 139 and wants to add or change tie points, the user inputs the tie points by clicking the cross-sectional feature points and longitudinal-sectional feature points for each region on the input panel. Thereafter, the operations from step a4 to step a7 are repeated. In this case, in step a4, the deviation amount calculation unit 114 uses the correction point cloud 138 instead of the second laser point cloud 136.

[0085] ***Modification of the First Embodiment*** When the third laser scanner 230 is attached to the measurement vehicle 200, the amount of deviation of the first laser point cloud 135 from the third laser point cloud and the amount of deviation of the second laser point cloud 136 from the third laser point cloud may be calculated using the third laser point cloud acquired by the third laser scanner 230 as a reference, and the self-position and attitude information may be corrected. Since the third laser scanner 230 captures images in a direction perpendicular to the traveling direction of the measurement vehicle 200, the amount of attitude deviation, particularly the pitch error, is small, and therefore the accuracy of the generated measurement point cloud is improved.

[0086] In this case, in step a1, the data acquisition unit 111 of the point cloud joining device 100 acquires, as measurement data, the self-position and orientation information 132, the first distance and orientation information 133, and the second distance and orientation information 134, as well as the third distance and orientation information acquired by the third laser scanner 230.

[0087] Furthermore, in step a2, the point cloud generation unit 112 generates a third laser point cloud in addition to the first laser point cloud 135 and the second laser point cloud 136. At this time, the point cloud generation unit 112 first generates the third laser point cloud by using the self-position and attitude information 132 and the third distance and orientation information acquired at the same time, assuming that the target object is located at a location away by the distance indicated by the third distance and orientation information, in the orientation indicated by the third distance and orientation information, relative to the self-position and attitude of the measurement vehicle 200 at that time.

[0088] In step a3, the tie point acquisition unit 113 displays the third laser point cloud and the first laser point cloud 135 in the cross section or the longitudinal section in an overlapping manner on the output device connected via the input / output interface 150. Next, the tie point acquisition unit 113 accepts input from the user using the input device, and acquires the cross section characteristic points and the longitudinal section characteristic points of the object 900 as tie points.

[0089] Furthermore, the tie point acquisition unit 113 displays on the output device the third laser point cloud on the cross section or the longitudinal section and the second laser point cloud 136 in an overlapping manner. Subsequently, the tie point acquisition unit 113 receives input from the user using the input device, and acquires cross section characteristic points and longitudinal section characteristic points of the object 900 as tie points.

[0090] The user inputs cross-sectional feature points for the displayed cross-sections and longitudinal-section feature points for the displayed longitudinal sections. Two or more cross-sectional feature points and two or more longitudinal-section feature points are input.

[0091] In step a4, the deviation amount calculation unit 114 calculates the positional deviation amount and the orientation deviation amount of the tie points of the first laser point cloud 135 relative to the tie points of the third laser point cloud. Also, the deviation amount is calculated for the tie points of the second laser point cloud 136 relative to the tie points of the third laser point cloud. Next, from the deviation amount at the tie points, the deviation amount at each point of the first laser point cloud 135 and the second laser point cloud 136, which measured the same object as the third laser point cloud, is calculated.

[0092] In step a5, the self-position and attitude correction unit 115 generates first self-position and attitude correction information by correcting the self-position and attitude information of the first laser point cloud so as to eliminate the deviation amount at each point calculated in step a4, and second self-position and attitude correction information by correcting the self-position and attitude information of the second laser point cloud.

[0093] In step a6, the point cloud generation unit 112 generates a corrected point cloud 138 using the first distance and orientation information 133, the second distance and orientation information 134, the first self-position and orientation correction information, and the second self-position and orientation correction information.

[0094] In step a7, the point cloud generation unit 112 combines the third laser point cloud and the corrected point cloud 138 to generate a measurement point cloud, which is a three-dimensional point cloud within the measurement range.

[0095] ***Effect Description*** A tie point acquisition unit 113 acquires, as tie points, cross-sectional feature points that are common feature points in a cross section of the object and longitudinal-sectional feature points that are common feature points in a longitudinal section of the object from a first laser point cloud 135 that is a three-dimensional point cloud obtained by a first laser scanner 210 and a second laser point cloud 136 that is a three-dimensional point cloud obtained by a second laser scanner 220 that measures the same object in a different posture from the first laser scanner 210; The point cloud joining device 100 includes a shift amount calculation unit 114 that calculates a shift amount between a longitudinal cross-sectional feature point in the first laser point cloud 135 and a longitudinal cross-sectional feature point in the second laser point cloud 136, and a self position and orientation correction unit 115 that generates self position and orientation correction information by correcting the self position and orientation information based on the shift amount, and acquires common feature points in the first laser point cloud 135 and the second laser point cloud 136 as tie points, and corrects the self position and orientation information to eliminate the shift amount. This makes it possible to correct the self position and orientation information of the point clouds so that point clouds corresponding to the same object are joined.

[0096] 19 is a schematic diagram showing the case where the measurement vehicle 200 measures the object 900 when the self-position and attitude information 132 is correct (a) and when there is an error in the self-position and attitude information 132 (b). In addition, in FIG. 19, the object 900 is measured in the same way as in FIG. 7. The first laser point cloud 135 is a point cloud of the front and bottom surfaces of the object 900, and the second laser point cloud 136 is a point cloud of the rear and bottom surfaces of the object 900.

[0097] 19(b) illustrates a case where, when the second laser point cloud 136 is measured, the posture has changed due to a step compared to when the first laser point cloud 135 was measured at the same location, and the signal from the positioning satellite cannot be acquired correctly. In this case, as shown in FIG. 19(b), an error occurs in the self-position and posture information 132 of the second laser point cloud 136, and the first laser point cloud 135 and the second laser point cloud 136 do not overlap accurately. As a result, the distance between the front and rear surfaces of the measurement object 900, i.e., the width of the object 900 in the cross section, cannot be measured correctly.

[0098] However, the point cloud joining device 100 can calculate the amount of attitude deviation in the second laser point cloud 136 and correct the self-position and attitude information 132 so that the amount of attitude deviation is eliminated. This has the effect of accurately overlapping the first laser point cloud 135 and the corrected point cloud 138, and correctly measuring the width of the object 900 in the cross section.

[0099] Fig. 20 is a diagram showing an example of point clouds acquired for the same object. Fig. 20 illustrates a first laser point cloud 135 and a second laser point cloud 136 for a case where a suspension wire and a trolley wire on a railway are the object. Fig. 21 is a diagram showing an example of joining point clouds for the same object when (a) there is an error in the self-position and orientation information 132 and (b) there is no error in the self-position and orientation information 132.

[0100] 20, the first laser point cloud 135 and the second laser point cloud 136 are measured on different surfaces of the object 900, and therefore, some points are missing due to dirt on the surface, overlapping, etc. Therefore, for example, as shown in FIG. 21(a), if only the first laser point cloud 135 is acquired for a position on a suspension wire and only the second laser point cloud 136 is acquired for a trolley wire located vertically below that position, the measured distance between the suspension wire and the trolley wire may be longer than it actually is.

[0101] However, the point cloud joining device 100 can correct its own position and posture information so that point clouds corresponding to the same object are joined, and therefore, as shown in Figure 21(b), it can complement the gap and accurately measure the distance between the suspension wire and the trolley wire.

[0102] Furthermore, the self-position and attitude correction unit 115 of the point cloud joining device 100 generates self-position and attitude correction information 137 by correcting the self-position and attitude information of the second laser point cloud so as to eliminate the deviation amount at each point calculated in step a4. That is, the self-position and attitude correction information 137 used to generate the corrected point cloud 138 to be joined with the first laser point cloud 135 is individually regenerated. As a result, even if a contradiction occurs in the self-position and attitude information 132 due to distortion of the car body, it is possible to join point clouds corresponding to the same object.

[0103] Various aspects of the present disclosure are summarized below as appendices.

[0104] (Appendix 1) a tie point acquisition unit that acquires, as tie points, cross-sectional feature points that are common feature points in cross-sections of the object and longitudinal-sectional feature points that are common feature points in longitudinal sections of the object from a first laser point cloud that is a three-dimensional point cloud obtained by a first laser scanner and a second laser point cloud that is a three-dimensional point cloud obtained by a second laser scanner that measures the same object in a different attitude from the first laser scanner; a deviation amount calculation unit that calculates a posture deviation amount between a cross-sectional feature point in the first laser point cloud and a cross-sectional feature point in the second laser point cloud, and calculates a posture deviation amount between a longitudinal cross-sectional feature point in the first laser point cloud and a longitudinal cross-sectional feature point in the second laser point cloud; a self-position and orientation correction unit that generates self-position and orientation correction information by correcting the self-position and orientation information based on the attitude deviation amount; A point cloud joining device comprising: (Appendix 2) a tie point acquisition unit that displays a first laser point cloud and a second laser point cloud obtained by measuring the same object from a different posture than that of the first laser point cloud on an output device connected via an input / output interface, and acquires tie points, which are common feature points between the first laser point cloud and the second laser point cloud, in response to a user's input operation via an input device connected by the input / output interface; a point cloud generation unit that outputs a measurement point cloud, which is a three-dimensional point cloud of the measurement object, by superimposing point clouds corrected to eliminate an amount of attitude deviation between the first laser point cloud and the second laser point cloud at the tie point; A point cloud joining device comprising: (Appendix 3) The point cloud joining device described in Appendix 1 further includes a point cloud generation unit that generates a measurement point cloud, which is a three-dimensional point cloud for the measurement object, by overlaying the first laser point cloud and a corrected point cloud, which is a three-dimensional point cloud generated using the self-position and attitude correction unit. (Appendix 4) The point cloud joining device according to any one of appendices 1 to 3, further comprising a data acquisition unit that acquires self-position and orientation information at any time, first distance and orientation information indicating the orientation and distance of the object relative to the first laser scanner obtained at the any time, and second distance and orientation information indicating the orientation and distance of the object relative to the second laser scanner obtained at the any time. (Appendix 5) further comprising an input / output interface connectable to an external output device; the tie point acquisition unit displays, on the output device connected via the input / output interface, an input panel that displays the first laser point cloud and the second laser point cloud in an overlapping manner, and an operation panel that displays whether the first laser point cloud and the second laser point cloud displayed on the input panel are cross sections or longitudinal sections, and the measurement time and position of a point selected on the input panel; 5. The point cloud joining device according to any one of appendices 1 to 4. (Appendix 6) a first laser scanner mounted on the measurement vehicle; a second laser scanner attached to the measurement vehicle and configured to measure the same object in a different orientation from that of the first laser scanner; a tie point acquisition unit that acquires, as tie points, cross-sectional feature points that are common feature points in a cross section of the object and longitudinal-sectional feature points that are common feature points in a longitudinal section of the object from a first laser point cloud that is a three-dimensional point cloud obtained by the first laser scanner and a second laser point cloud that is a three-dimensional point cloud obtained by the second laser scanner; a deviation amount calculation unit that calculates a posture deviation amount between a cross-sectional feature point in the first laser point cloud and a cross-sectional feature point in the second laser point cloud, and calculates a posture deviation amount between a longitudinal cross-sectional feature point in the first laser point cloud and a longitudinal cross-sectional feature point in the second laser point cloud; a self-position and orientation correction unit that generates self-position and orientation correction information by correcting the self-position and orientation information based on the attitude deviation amount; A point cloud joining system comprising: (Appendix 7) a tie point acquisition step of acquiring, as tie points, cross-sectional feature points that are common feature points in cross sections of the object and longitudinal-sectional feature points that are common feature points in longitudinal sections of the object from a first laser point cloud that is a three-dimensional point cloud obtained by a first laser scanner and a second laser point cloud that is a three-dimensional point cloud obtained by a second laser scanner that measures the same object in a different attitude from the first laser scanner; a deviation amount calculation step of calculating a posture deviation amount between a cross-sectional feature point in the first laser point cloud and a cross-sectional feature point in the second laser point cloud, and calculating a posture deviation amount between a longitudinal cross-sectional feature point in the first laser point cloud and a longitudinal cross-sectional feature point in the second laser point cloud; a self-position and attitude correcting step of generating self-position and attitude corrected information by correcting the self-position and attitude information so as to eliminate the attitude deviation amount; A point group joining method comprising: (Appendix 8) a tie point acquisition unit that acquires, as tie points, cross-sectional feature points that are common feature points in cross-sections of the object and longitudinal-sectional feature points that are common feature points in longitudinal sections of the object from a first laser point cloud that is a three-dimensional point cloud obtained by a first laser scanner and a second laser point cloud that is a three-dimensional point cloud obtained by a second laser scanner that measures the same object in a different attitude from the first laser scanner; a deviation amount calculation unit that calculates a posture deviation amount between a cross-sectional feature point in the first laser point cloud and a cross-sectional feature point in the second laser point cloud, and calculates a posture deviation amount between a longitudinal cross-sectional feature point in the first laser point cloud and a longitudinal cross-sectional feature point in the second laser point cloud; a self-position and orientation correction unit that generates self-position and orientation correction information by correcting the self-position and orientation information so as to eliminate the attitude deviation amount; A point cloud joining program that includes: (Appendix 9) The point cloud joining program according to claim 8, further comprising a point cloud generation unit that generates a measurement point cloud, which is a three-dimensional point cloud for the measurement object, by overlaying the first laser point cloud and a corrected point cloud, which is a three-dimensional point cloud generated using the self-position and attitude correction unit. (Appendix 10) 10. The point cloud joining program according to claim 8 or 9, further comprising a data acquisition unit that acquires self-position and orientation information at any time, first distance and orientation information indicating the orientation and distance of the object relative to the first laser scanner obtained at the any time, and second distance and orientation information indicating the orientation and distance of the object relative to the second laser scanner obtained at the any time. [Explanation of symbols]

[0105] 100 Point cloud joining device, 110 Processor, 111 Data acquisition unit, 112 Point cloud generation unit, 113 Tie point acquisition unit, 114 Displacement amount calculation unit, 115 Self-position and attitude correction unit, 120 Memory, 130 Auxiliary storage device, 131 Point cloud joining program, 132 Self-position and attitude information, 133 First distance and orientation information, 134 Second distance and orientation information, 135 First laser point cloud, 136 Second laser point cloud, 137 Self-position and attitude correction information, 138 Corrected point cloud, 139 Measurement point cloud, 140 Communication device, 150 Input / output interface, 200 Measurement vehicle, 2010 Measurement device, 201 Roof carrier, 202 Slide mechanism, 203 Rear carrier, 210 First laser scanner, 220 Second laser scanner, 230 Third laser scanner, 240 Fourth laser scanner, 250 Overhead line measurement camera, 260 antenna, 270 360 camera, 280 front camera, 290 rear camera, 800 road-rail vehicle, 900 object

Claims

1. a tie point acquisition unit that acquires, as tie points, cross-sectional feature points that are common feature points in cross-sections of the object and longitudinal-sectional feature points that are common feature points in longitudinal sections of the object from a first laser point cloud that is a three-dimensional point cloud obtained by a first laser scanner and a second laser point cloud that is a three-dimensional point cloud obtained by a second laser scanner that measures the same object in a different attitude from the first laser scanner; a deviation amount calculation unit that calculates a posture deviation amount between a cross-sectional feature point in the first laser point cloud and a cross-sectional feature point in the second laser point cloud, and calculates a posture deviation amount between a longitudinal cross-sectional feature point in the first laser point cloud and a longitudinal cross-sectional feature point in the second laser point cloud; a self-position and orientation correction unit that generates self-position and orientation correction information by correcting the self-position and orientation information based on the attitude deviation amount; A point cloud joining device comprising:

2. a tie point acquisition unit that displays a first laser point cloud and a second laser point cloud obtained by measuring the same object from a different posture than that of the first laser point cloud on an output device connected via an input / output interface, and acquires tie points, which are common feature points between the first laser point cloud and the second laser point cloud, in response to a user's input operation via an input device connected by the input / output interface; a point cloud generation unit that outputs a measurement point cloud, which is a three-dimensional point cloud of the measurement object, by superimposing point clouds corrected to eliminate an amount of attitude deviation between the first laser point cloud and the second laser point cloud at the tie point; A point cloud joining device comprising:

3. The point cloud joining device according to claim 1, further comprising a point cloud generation unit that generates a measurement point cloud, which is a three-dimensional point cloud for the measurement object, by overlaying the first laser point cloud and a corrected point cloud, which is a three-dimensional point cloud generated using the self-position and attitude correction unit.

4. 2. The point cloud joining device according to claim 1, further comprising a data acquisition unit that acquires self-position and orientation information at any time, first distance and orientation information indicating the orientation and distance of an object relative to the first laser scanner obtained at the any time, and second distance and orientation information indicating the orientation and distance of the object relative to the second laser scanner obtained at the any time.

5. further comprising an input / output interface connectable to an external output device; the tie point acquisition unit displays, on the output device connected via the input / output interface, an input panel that displays the first laser point cloud and the second laser point cloud in an overlapping manner, and an operation panel that displays whether the first laser point cloud and the second laser point cloud displayed on the input panel are cross sections or longitudinal sections, and the measurement time and position of a point selected on the input panel; The point cloud joining device according to claim 1 .

6. a first laser scanner mounted on the measurement vehicle; a second laser scanner attached to the measurement vehicle and configured to measure the same object in a different orientation from that of the first laser scanner; a tie point acquisition unit that acquires, as tie points, cross-sectional feature points that are common feature points in a cross section of the object and longitudinal-sectional feature points that are common feature points in a longitudinal section of the object from a first laser point cloud that is a three-dimensional point cloud obtained by the first laser scanner and a second laser point cloud that is a three-dimensional point cloud obtained by the second laser scanner; a deviation amount calculation unit that calculates a posture deviation amount between a cross-sectional feature point in the first laser point cloud and a cross-sectional feature point in the second laser point cloud, and calculates a posture deviation amount between a longitudinal cross-sectional feature point in the first laser point cloud and a longitudinal cross-sectional feature point in the second laser point cloud; a self-position and orientation correction unit that generates self-position and orientation correction information by correcting the self-position and orientation information based on the attitude deviation amount; A point cloud joining system comprising:

7. a tie point acquisition step of acquiring, as tie points, cross-sectional feature points that are common feature points in cross sections of the object and longitudinal-sectional feature points that are common feature points in longitudinal sections of the object from a first laser point cloud that is a three-dimensional point cloud obtained by a first laser scanner and a second laser point cloud that is a three-dimensional point cloud obtained by a second laser scanner that measures the same object in a different attitude from the first laser scanner; a deviation amount calculation step of calculating a posture deviation amount between a cross-sectional feature point in the first laser point cloud and a cross-sectional feature point in the second laser point cloud, and calculating a posture deviation amount between a longitudinal cross-sectional feature point in the first laser point cloud and a longitudinal cross-sectional feature point in the second laser point cloud; a self-position and attitude correcting step of generating self-position and attitude corrected information by correcting the self-position and attitude information so as to eliminate the attitude deviation amount; A point group joining method comprising:

8. a tie point acquisition unit that acquires, as tie points, cross-sectional feature points that are common feature points in cross-sections of the object and longitudinal-sectional feature points that are common feature points in longitudinal sections of the object from a first laser point cloud that is a three-dimensional point cloud obtained by a first laser scanner and a second laser point cloud that is a three-dimensional point cloud obtained by a second laser scanner that measures the same object in a different attitude from the first laser scanner; a deviation amount calculation unit that calculates a posture deviation amount between a cross-sectional feature point in the first laser point cloud and a cross-sectional feature point in the second laser point cloud, and calculates a posture deviation amount between a longitudinal cross-sectional feature point in the first laser point cloud and a longitudinal cross-sectional feature point in the second laser point cloud; a self-position and orientation correction unit that generates self-position and orientation correction information by correcting the self-position and orientation information so as to eliminate the attitude deviation amount; A point cloud joining program that includes:

9. 9. The point cloud joining program according to claim 8, further comprising a point cloud generation unit that generates a measurement point cloud that is a three-dimensional point cloud for the measurement object by superimposing the first laser point cloud and a corrected point cloud that is a three-dimensional point cloud generated using the self-position and attitude correction unit.

10. 9. The point cloud joining program according to claim 8, further comprising a data acquisition unit that acquires self-position and orientation information at an arbitrary time, first distance and orientation information indicating an orientation and distance of an object relative to the first laser scanner obtained at the arbitrary time, and second distance and orientation information indicating an orientation and distance of an object relative to the second laser scanner obtained at the arbitrary time.

Images