Point cloud joining device, point cloud joining method, point cloud joining program, and measurement vehicle

Abstract

A tie point extraction unit (112) extracts tie points from each of a reference laser point cloud included in the reference measurement data and a target laser point cloud included in the target measurement data. A self-position and orientation correction unit (113) corrects self-position and orientation data included in the target measurement data so that the positions of the tie points match. A three-dimensionalization unit (114) generates a corrected laser point cloud using the self-position and orientation data after correction. At least one of the reference laser point cloud and the target laser point cloud includes an oblique laser point cloud obtained using an oblique laser scanner and a horizontal laser point cloud obtained using a horizontal laser scanner.

Description

[Technical field]

[0001] This disclosure relates to techniques for merging multiple point clouds obtained with a Mobile Mapping System (MMS). [Background technology]

[0002] In the world of MMS, the trend is for technology to consistently combine multiple point clouds obtained from multiple runs. On roads with wide lanes or interchanges, it is not possible to obtain a point cloud of the entire area in one drive. In such cases, using the above-mentioned automatic joining technology makes it possible to use point clouds of a wider range. For example, automatic joining technology is necessary to obtain a point cloud of the entire area of ​​a highway.

[0003] Auto-joining is a key technology for MMS as demand for highway and local road point clouds increases.

[0004] Patent Document 1 discloses acquiring measurement data using a measurement vehicle, generating a three-dimensional point cloud using the measurement data, and joining multiple three-dimensional point clouds. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2021-135380 A Summary of the Invention [Problem to be solved by the invention]

[0006] Conventional automatic joining software extracts features (such as utility poles and white lines) from point clouds obtained by measuring diagonally with a laser, and matches the features between the point clouds. For this reason, if the feature cannot be extracted, it will be impossible to join the point clouds together. Therefore, automatic joining will not work on roads with long distances between utility poles or roads with no breaks in the white lines.

[0007] The present disclosure aims to prevent automatic joining from failing due to an inability to extract features. [Means for solving the problem]

[0008] The point cloud joining device of the present disclosure is a tie point extraction unit that extracts common feature points as tie points from one or more feature points in a reference laser point cloud, which is a three-dimensional point cloud included in the reference measurement data, and one or more feature points in a target laser point cloud, which is a three-dimensional point cloud included in the target measurement data; a self-position and orientation correction unit that corrects self-position and orientation data included in the target measurement data so that positions of the tie points of the reference laser point cloud and positions of the tie points of the target laser point cloud coincide with each other; and a three-dimensional generation unit that generates a corrected laser point cloud, which is a three-dimensional point cloud, using the self-position and orientation data after the correction. At least one of the reference measurement data and the target measurement data is measurement data obtained using a special measurement vehicle equipped with an oblique laser scanner and a horizontal laser scanner. At least one of the reference laser point cloud and the target laser point cloud includes an oblique laser point cloud, which is a three-dimensional point cloud obtained using the oblique laser scanner, and a horizontal laser point cloud, which is a three-dimensional point cloud obtained using the horizontal laser scanner. Effect of the Invention

[0009] According to the present disclosure, in addition to the oblique laser point cloud obtained using the oblique laser scanner, the horizontal laser point cloud obtained using the horizontal laser scanner is used, which can prevent the automatic joining from failing due to the inability to extract features. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a configuration diagram of a point cloud joining device 100 according to a first embodiment. [Diagram 2] FIG. 2 is a configuration diagram of a storage unit 120 according to the first embodiment. [Diagram 3] FIG. 2 is a configuration diagram of a measurement vehicle 400 according to the first embodiment. [Figure 4] 3 is a flowchart of a point group joining method according to the first embodiment. [Diagram 5] FIG. 3 is a diagram showing a configuration of object measurement data 301 generated by the point cloud joining method according to the first embodiment. [Figure 6] A diagram showing the matching of two point clouds during automatic joining. [Figure 7] FIG. 4 is a diagram showing the effective range of an oblique laser scanner 421. [Figure 8] FIG. 4 is a diagram showing the effective ranges of an oblique laser scanner 421 and a horizontal laser scanner 422. [Figure 9] A diagram showing the relationship between laser measurement and the central reservation. [Figure 10] FIG. 13 is a diagram showing an example of a horizontal laser point cloud. [Figure 11] FIG. 13 is a diagram showing an example of a horizontal laser point cloud. [Figure 12] FIG. 13 is a diagram showing an example of a horizontal laser point cloud. [Figure 13] FIG. 13 is a diagram showing an example of a horizontal laser point cloud. [Figure 14] A diagram showing the reference point cloud, the target point cloud, and the target point cloud (3 patterns) after joining. [Figure 15] FIG. 2 is a hardware configuration diagram of the point cloud joining device 100 according to the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] In the embodiments and drawings, the same or corresponding elements are denoted by the same reference numerals. Descriptions of elements denoted by the same reference numerals as those of the elements described above will be omitted or simplified as appropriate. Arrows in the drawings mainly indicate data flow or processing flow.

[0012] Embodiment 1 The point group joining apparatus 100 will be described with reference to FIGS.

[0013] ***Configuration Description*** The configuration of the point cloud joining device 100 will be described with reference to FIG. The point cloud joining device 100 is a computer including hardware such as a processor 101, a memory 102, an auxiliary storage device 103, a communication device 104, and an input / output interface 105. These pieces of hardware are connected to each other via signal lines.

[0014] The processor 101 is an IC that performs arithmetic processing and controls other hardware. For example, the processor 101 is a CPU. IC is an abbreviation for Integrated Circuit. CPU is an abbreviation for Central Processing Unit.

[0015] The memory 102 is a volatile or non-volatile storage device. The memory 102 is also called a primary storage device or a main memory. For example, the memory 102 is a RAM. Data stored in the memory 102 is saved in the secondary storage device 103 as necessary. RAM is an abbreviation for Random Access Memory.

[0016] The auxiliary storage device 103 is a non-volatile storage device. For example, the auxiliary storage device 103 is a ROM, a HDD, a flash memory, or a combination of these. Data stored in the auxiliary storage device 103 is loaded into the memory 102 as needed. ROM is an abbreviation for Read Only Memory. HDD is an abbreviation for Hard Disk Drive.

[0017] The communication device 104 is a receiver and a transmitter. For example, the communication device 104 is a communication chip or a NIC. The communication of the point cloud stitching device 100 is performed using the communication device 104. NIC is an abbreviation for Network Interface Card.

[0018] The input / output interface 105 is a port to which an input device and an output device are connected. For example, the input / output interface 105 is a USB terminal, the input device is a keyboard and a mouse, and the output device is a display. Input and output of the point cloud joining device 100 is performed using the input / output interface 105. USB is an abbreviation for Universal Serial Bus.

[0019] The point cloud joining device 100 includes elements such as a feature point extraction unit 111, a tie point extraction unit 112, a self position and orientation correction unit 113, a three-dimensionalization unit 114, a three-dimensionalization accuracy prediction unit 115, and a camera position and orientation calculation unit 116. These elements are realized by software.

[0020] The auxiliary storage device 103 stores a point cloud joining program for causing the computer to function as a feature point extraction unit 111, a tie point extraction unit 112, a self position and orientation correction unit 113, a three-dimensionalization unit 114, a three-dimensionalization accuracy prediction unit 115, and a camera position and orientation calculation unit 116. The point cloud joining program is loaded into the memory 102 and executed by the processor 101. The auxiliary storage device 103 further stores an OS. At least a part of the OS is loaded into the memory 102 and executed by the processor 101. The processor 101 executes the point group joining program while executing the OS. OS is an abbreviation for Operating System.

[0021] Input and output data of the point cloud joining program is stored in the storage unit 120. The auxiliary storage device 103 functions as the storage unit 120. However, a storage device such as the memory 102, a register in the processor 101, or a cache memory in the processor 101 may function as the storage unit 120 instead of the auxiliary storage device 103 or together with the auxiliary storage device 103.

[0022] The point group joining program can be recorded (stored) in a computer-readable manner on a non-volatile recording medium such as an optical disk or a flash memory.

[0023] The configuration of the storage unit 120 will be described with reference to FIG. The storage unit 120 stores reference measurement data 200 and object measurement data 300. The reference measurement data 200 includes self position and orientation data 210, distance and orientation information 220, a reference laser point cloud 230, image data 240, and camera position and orientation data 250. The reference laser point cloud 230 includes an oblique laser point cloud 231 and a horizontal laser point cloud 232. The object measurement data 300 includes self-position and orientation data 310, distance and orientation information 320, an object laser point cloud 330, and image data 340. The object laser point cloud 330 includes an oblique laser point cloud 331 and a horizontal laser point cloud 332.

[0024] The reference measurement data 200 is the measurement data that serves as a reference, and the target measurement data 300 is the measurement data that is to be processed. The measurement data is obtained using a measurement vehicle. A measurement vehicle is a vehicle equipped with various sensors and is used in a Mobile Mapping System (MMS).

[0025] The reference measurement data 200 and the target measurement data 300 are obtained using a measurement vehicle 400 . The measurement vehicle 400 is a measurement vehicle (special measurement vehicle) equipped with an oblique laser scanner 421 and a horizontal laser scanner 422. The measurement vehicle is also called a mobile measurement device.

[0026] The configuration of the measurement vehicle 400 will be described with reference to FIG. A position and orientation measurement device is installed in the measurement vehicle 400. Specifically, a receiver 411 and an IMU 412 are installed in the measurement vehicle 400. The receiver 411 receives positioning signals transmitted from each of a plurality of positioning satellites and performs satellite positioning. The IMU 412 is an inertial measurement unit that measures angular rate and acceleration. The position and attitude of the measurement vehicle 400 at each time are calculated using the reception data obtained by the receiver 411 and the inertial measurement data obtained by the IMU 412 . Data indicating the position and attitude of the measurement vehicle 400 at each time is referred to as self-position and attitude data. The self-position and orientation data included in the reference measurement data 200 is referred to as self-position and orientation data 210 . The self-position and orientation data included in the object measurement data 300 is referred to as self-position and orientation data 310 .

[0027] The measurement vehicle 400 is equipped with an oblique laser scanner 421 and a horizontal laser scanner 422 . The oblique laser scanner 421 is a laser scanner that is installed facing an oblique direction and measures the distance and direction to each measurement point located in the oblique direction. The horizontal laser scanner 422 is a laser scanner that is installed facing the horizontal direction and measures the distance and direction to each measurement point located in the horizontal direction. A measurement point is a point that is measured by a laser scanner. Each point on the surface of a feature that exists in the measurement direction is a measurement point. Examples of features are roads, utility poles, and walls. Data indicating the distance and direction from the laser scanner to each measurement point is referred to as distance and direction information.

[0028] Using the self-position and orientation data and distance and direction information, the three-dimensional coordinate values ​​of each measurement point are calculated. The data indicating the three-dimensional coordinate values ​​of each measurement point is called a three-dimensional point cloud. A three-dimensional point cloud is made up of multiple three-dimensional points. The three-dimensional points indicate the three-dimensional coordinate values ​​of the measurement points. The three-dimensional point cloud included in the reference measurement data 200 is referred to as a reference laser point cloud 230. The reference laser point cloud 230 is generated using the self-position and orientation data 210 and the distance and orientation information 220. The three-dimensional point cloud generated using the distance and orientation information 220 of the oblique laser scanner 421 is referred to as the oblique laser point cloud 231. The three-dimensional point cloud generated using the distance and orientation information 220 of the horizontal laser scanner 422 is referred to as the horizontal laser point cloud 232. The three-dimensional point cloud included in the object measurement data 300 is referred to as an object laser point cloud 330. The object laser point cloud 330 is generated using self-position and orientation data 310 and distance and orientation information 320. The three-dimensional point cloud generated using the distance and orientation information 320 of the oblique laser scanner 421 is referred to as the oblique laser point cloud 331. The three-dimensional point cloud generated using the distance and orientation information 320 of the horizontal laser scanner 422 is referred to as the horizontal laser point cloud 332.

[0029] A camera is installed on the measurement vehicle 400. For example, a camera 431 and an all-around camera 432 are installed on the measurement vehicle 400. The camera 431 is a camera that captures images in front. The all-around camera 432 is a camera that captures all the surroundings. Images are obtained by taking pictures using a camera. Data including the images at each time point is called image data. The image data included in the reference measurement data 200 is referred to as image data 240 . The image data included in the object measurement data 300 is referred to as image data 340 .

[0030] Using the self-position and orientation data, the position and orientation of the camera at each time are calculated. The data indicating the position and orientation of the camera at each time is referred to as camera position and orientation data. The camera position and orientation data included in the reference measurement data 200 is referred to as camera position and orientation data 250. The camera position and orientation data 250 is generated using the self position and orientation data 210.

[0031] ***Explanation of Operation*** The operation procedure of the point cloud joining device 100 corresponds to a point cloud joining method. Also, the operation procedure of the point cloud joining device 100 corresponds to a processing procedure by a point cloud joining program.

[0032] The point group joining method will be outlined with reference to FIG. The point cloud joining apparatus 100 uses the reference measurement data 200 and the object measurement data 300 to generate the object measurement data 301 . The object measurement data 301 is measurement data that corresponds to the object measurement data 300 .

[0033] The configuration of the target measurement data 301 will be described with reference to FIG. The target measurement data 301 includes self position and orientation data 311 , distance and orientation information 320 , target laser point cloud 390 , image data 340 , camera position and orientation data 350 , and prediction accuracy 360 . Self-position and orientation data 311 is the self-position and orientation data 310 that has been corrected. The target laser point cloud 390 is a three-dimensional point cloud generated using the self-position and orientation data 311 and the distance and direction information 320. The camera position and orientation data 350 is generated using the self-position and orientation data 311 . The prediction accuracy 360 is data indicating the prediction accuracy of the target laser point cloud 390 .

[0034] Returning to FIG. 4, the point group joining method will be described in detail. In step S110, the feature point extraction unit 111 extracts one or more feature points from the reference laser point cloud 230. Specifically, the feature point extraction unit 111 extracts one or more feature points from each of the oblique laser point cloud 231 and the horizontal laser point cloud 232.

[0035] Furthermore, the feature point extraction unit 111 extracts one or more feature points from the target laser point cloud 330. Specifically, the feature point extraction unit 111 extracts one or more feature points from each of the oblique laser point cloud 331 and the horizontal laser point cloud 332.

[0036] For example, a laser point on a utility pole or a wall surface (especially a corner portion) becomes a feature point.

[0037] In step S120, the tie point extraction unit 112 extracts common feature points from one or more feature points in the reference laser point cloud 230 and the feature points in the target laser point cloud 330. The extracted feature points are called tie points.

[0038] In particular, the tie point extraction unit 112 extracts feature points that have high reliability as common feature points as tie points.

[0039] In step S130, the self-position and orientation correction unit 113 corrects the self-position and orientation data 310 so that the positions of the tie points of the reference laser point cloud 230 and the positions of the tie points of the target laser point cloud 330 coincide with each other.

[0040] The self-position and orientation data 310 is corrected as follows. First, the self-position and attitude correction unit 113 calculates the difference between the positions of the tie points of the reference laser point cloud 230 and the positions of the tie points of the target laser point cloud 330 as an error. Then, the self position and orientation correction unit 113 corrects each position and orientation indicated in the self position and orientation data 310 in accordance with the error.

[0041] In step S130, the self-position and orientation data 310 is corrected and the self-position and orientation data 311 is generated.

[0042] In step S140, the three-dimensional rendering unit 114 generates a target laser point cloud 390 using the self position and orientation data 311 and the distance and direction information 320. The target laser point cloud 390 shows a plurality of three-dimensional points corresponding to the plurality of measurement points. The target laser point cloud 390 is the target laser point cloud 330 after correction. The three-dimensional point indicates the three-dimensional coordinate value of the measurement point. The measurement point is specified as a point away from the self-position and orientation at the time of measurement in the distance direction indicated by the target laser point.

[0043] In step S150, the three-dimensional processing accuracy prediction unit 115 predicts the accuracy of the target laser point cloud 390 and calculates the predicted accuracy 360.

[0044] The prediction accuracy 360 is calculated as follows: First, the feature point extraction unit 111 extracts one or more feature points from each of the reference laser point cloud 230 and the target laser point cloud 390. Next, the tie point extractor 112 extracts tie points from each of the one or more feature points in the reference laser point cloud 230 and the one or more feature points in the target laser point cloud 390 . Next, the three-dimensional accuracy prediction unit 115 calculates the difference between the positions of the tie points of the reference laser point cloud 230 and the positions of the tie points of the target laser point cloud 390 as an error. Then, the three-dimensional representation accuracy prediction unit 115 uses the error to calculate the prediction accuracy 360. The larger the error, the lower the prediction accuracy 360, and the smaller the error, the higher the prediction accuracy 360.

[0045] In step S160 , the camera position and orientation calculation unit 116 generates camera position and orientation data 350 using the self position and orientation data 311 . Specifically, the camera position and orientation calculation unit 116 shifts the relative position and orientation of the camera with respect to the measurement vehicle 400 from the self position and orientation at each time, to calculate the position and orientation of the camera at each time.

[0046] ***Advantages of the First Embodiment*** The first embodiment is characterized in that a horizontally installed laser is used to extract features and perform automatic joining. That is, in the first embodiment, the horizontal laser is installed to improve the performance of automatic joining. Because the horizontal laser is mounted horizontally, it can see features much farther away than an angled laser. This makes it possible to capture feature points such as distant utility poles that cannot be captured normally, improving the performance of automatic joining. Utility poles are particularly effective in automatic joining, and the ability to capture distant utility poles is an advantage. According to the first embodiment, it becomes possible to use distant features and distant walls, and the number of sections where point clouds cannot be joined because tie points cannot be measured is reduced. Furthermore, improvement in joining accuracy is expected.

[0047] ***Example of the first embodiment*** The horizontal laser scanner 422 is highly effective when used both during the measurement in which the reference measurement data 200 is obtained (reference measurement) and during the measurement in which the object measurement data 300 is obtained (object measurement). However, the horizontal laser scanner 422 can be effectively used for only either the reference measurement or the target measurement. That is, the reference measurement is performed using a measurement vehicle (normal measurement vehicle) that is not equipped with the horizontal laser scanner 422, and the reference laser point cloud 230 does not need to include the horizontal laser point cloud 232. In this case, the target measurement is performed using a measurement vehicle (special measurement vehicle) that is equipped with the horizontal laser scanner 422, and the target laser point cloud 330 includes the horizontal laser point cloud 332. Also, the target measurement may be performed using a normal measurement vehicle, and the target laser point cloud 330 may not include the horizontal laser point cloud 332. In this case, the reference measurement is performed using a special measurement vehicle, and the reference laser point cloud 230 includes the horizontal laser point cloud 232.

[0048] It is desirable to use a laser scanner with a long measurement distance (laser flight distance) as the horizontal laser scanner 422. However, the distance accuracy of the horizontal laser scanner 422 is not that important.

[0049] ***Addition to the first embodiment*** Figure 6 shows how two point clouds are aligned using automatic joining. Automatic stitching is a technique for combining two completely separate measurements taken at different locations into a single point cloud by aligning the point clouds obtained from each measurement. Auto-joining uses multiple point clouds generated using different position information to reshape one point cloud to match the other. Automatic joining is done by extracting feature points from each point cloud and performing tie point matching. For example, automatic joining is done when a vehicle is traveling in different lanes in a situation where the satellite is not visible and measurements are taken for each lane. Automatic joining is performed by extracting feature points from point clouds and performing tie-point matching. Therefore, automatic joining had the problem of being unable to match point clouds for roads with few feature points.

[0050] FIG. 7 shows how a utility pole is measured using an oblique laser scanner 421. In a typical MMS, a laser is installed at an angle, such as the oblique laser scanner 421. This is because the laser irradiates various places as the measurement vehicle 400 travels. Then, a space is constructed using the laser point cloud. Here, it is assumed that the oblique laser scanner 421 is installed at an angle of 45 degrees and that the distance range of the oblique laser scanner 421 is 100 meters. In this case, the laser light hits the ground behind, and only a few meters behind can be measured at most. It is assumed that a utility pole is present diagonally in front of the measurement vehicle 400. The height of the utility pole is only about 16 meters. Therefore, the effective range of the oblique laser scanner 421 in the horizontal direction is only about 16 meters. In contrast, when the horizontal laser scanner 422 is used, the distance range of the horizontal laser scanner 422 itself becomes the effective range in the horizontal direction. In other words, if the distance range of the horizontal laser scanner 422 is 100 meters, the effective range in the horizontal direction is a range of ±100 meters.

[0051] FIG. 8 shows the horizontal effective ranges of the oblique laser scanner 421 and the horizontal laser scanner 422. The dark shaded areas indicate the effective range of the diagonal laser scanner 421, and the entire shaded area indicates the effective range of the horizontal laser scanner 422. The white circles represent utility poles. The horizontal laser scanner 422 can capture targets (such as utility poles) over a much wider range than the diagonal laser scanner 421. In FIG. 8, the oblique laser scanner 421 captures two utility poles, whereas the horizontal laser scanner 422 captures eight utility poles. Of course, even if the horizontal laser scanner 422 is used, the effective range is not always ±100 meters, and the effective range is narrowed if there is an obstruction such as a building.

[0052] Figure 9 shows the relationship between the laser measurement and the median strip. The diagonal dashed line represents the laser light from the diagonal laser, and the horizontal dashed dotted line represents the laser light from the horizontal laser. When there is a median strip, the laser light from the diagonal laser is blocked, so it is not possible to use the diagonal laser to obtain a point cloud of the opposite lane through the median strip. On the other hand, even if a horizontal laser is installed at the same height as the diagonal laser, the laser light from the horizontal laser is not blocked, so it is possible to obtain a point cloud of a distant lane through the median strip using the horizontal laser. In this way, the effectiveness of the horizontal laser is high.

[0053] Since the laser light of the horizontal laser scanner 422 is not irradiated to the ground, the height of the measurement point is unknown even if the horizontal laser points are referred to. Therefore, the height of the self-position cannot be corrected using only the horizontal laser point cloud. However, the laser light of the oblique laser scanner 421 is irradiated to the ground. Since the ground serves as a reference for height correction, it is possible to correct the height of the self-position by using the oblique laser point cloud.

[0054] An example of a horizontal laser point cloud is shown in Fig. 10. The two parallel solid lines represent the travel path of the measurement vehicle 400. By using the horizontal laser scanner 422, a cross section at the height at which the horizontal laser scanner 422 is installed can be obtained. By using the horizontal laser scanner 422, data from a long distance can be obtained.

[0055] An example of a horizontal laser point cloud of a utility pole is shown in Figure 11. The cross marks indicate the detected pole centers. Point cloud (1) was obtained in a very favorable measurement environment. Point cloud (2) was obtained in a fairly good measurement environment.

[0056] An example of a horizontal laser point cloud of a wall surface is shown in Figure 12. The white straight lines represent the detected straight wall surfaces. An example of a horizontal laser point cloud of a wall is shown in Figure 13. The crosses indicate the detected wall angles.

[0057] FIG. 14 shows five types of point clouds. Point cloud (1) is the reference point cloud. Point cloud (2) is the target point cloud before joining. Point cloud (3) is the target point cloud after joining using an oblique laser. Point cloud (4) is the target point cloud after joining using a horizontal laser. Point cloud (5) is the target point cloud after joining using an oblique and horizontal laser. Point cloud (5) and point cloud (4) are closer to point cloud (1) than point cloud (3). Therefore, it can be said that the use of a horizontal laser is effective.

[0058] The hardware configuration of the point cloud joining device 100 will be described with reference to FIG. The point cloud joining apparatus 100 includes a processing circuit 109 . The processing circuit 109 is hardware that realizes a feature point extraction unit 111, a tie point extraction unit 112, a self position and orientation correction unit 113, a three-dimensionalization unit 114, a three-dimensionalization accuracy prediction unit 115, and a camera position and orientation calculation unit . The processing circuitry 109 may be dedicated hardware, or may be a processor 101 that executes a program stored in the memory 102 .

[0059] When the processing circuitry 109 is dedicated hardware, the processing circuitry 109 may be, for example, a single circuit, a multiple circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof. ASIC is an abbreviation for Application Specific Integrated Circuit. FPGA is an abbreviation for Field Programmable Gate Array.

[0060] The point cloud joining apparatus 100 may include a plurality of processing circuits instead of the processing circuit 109.

[0061] In the processing circuit 109, some functions may be realized by dedicated hardware, and the remaining functions may be realized by software or firmware.

[0062] Thus, the functions of the point cloud joining device 100 can be realized by hardware, software, firmware, or a combination of these.

[0063] The first embodiment is an example of a preferred embodiment, and is not intended to limit the technical scope of the present disclosure. The first embodiment may be implemented partially or in combination with other embodiments. The procedure described using the flow chart, etc. may be modified as appropriate.

[0064] The "part" of each element of the point cloud joining device 100 may be read as a "process," a "step," a "circuit," or a "circuitry." [Explanation of symbols]

[0065] 100 Point cloud joining device, 101 Processor, 102 Memory, 103 Auxiliary storage device, 104 Communication device, 105 Input / output interface, 109 Processing circuit, 111 Feature point extraction unit, 112 Tie point extraction unit, 113 Self-position and orientation correction unit, 114 Three-dimensionalization unit, 115 Three-dimensionalization accuracy prediction unit, 116 Camera position and orientation calculation unit, 120 Memory unit, 200 Reference measurement data, 210 Self-position and orientation data, 220 Distance and orientation information, 230 Reference laser point cloud, 231 Oblique laser point cloud, 232 Horizontal laser point cloud, 240 Image data, 250 Camera position and orientation data, 300 Target measurement data, 301 Target measurement data, 310 Self-position and orientation data, 311 Self-position and orientation data, 320 Distance and orientation information, 330 Target laser point cloud, 331 Oblique laser point cloud, 332 Horizontal laser point cloud, 340 image data, 350 camera position and orientation data, 360 prediction accuracy, 390 target laser point cloud, 391 oblique laser point cloud, 392 horizontal laser point cloud, 400 measurement vehicle, 411 receiver, 412 IMU, 421 oblique laser scanner, 422 horizontal laser scanner, 431 camera, 432 all-around camera.

Claims

1. a tie point extraction unit that extracts common feature points as tie points from one or more feature points in a reference laser point cloud, which is a three-dimensional point cloud included in the reference measurement data, and one or more feature points in a target laser point cloud, which is a three-dimensional point cloud included in the target measurement data; a self-position and orientation correction unit that corrects self-position and orientation data included in the target measurement data so that positions of the tie points of the reference laser point cloud and positions of the tie points of the target laser point cloud coincide with each other; a three-dimensionalization unit that generates a corrected laser point cloud, which is a three-dimensional point cloud to be joined with a three-dimensional point cloud included in the reference measurement data, using the corrected self-position and orientation data and distance and orientation information included in the target measurement data; Equipped with At least one of the reference measurement data and the target measurement data is measurement data obtained using a special measurement vehicle equipped with an oblique laser scanner and a horizontal laser scanner, At least one of the reference laser point cloud and the target laser point cloud includes an oblique laser point cloud that is a three-dimensional point cloud obtained using the oblique laser scanner, and a horizontal laser point cloud that is a three-dimensional point cloud obtained using the horizontal laser scanner, The horizontal laser scanner is installed at the same height as the oblique laser scanner or at a higher height than the oblique laser scanner, The horizontal laser point cloud includes three-dimensional points at locations that are not measured by the oblique laser scanner but are measured by the horizontal laser scanner. Point cloud joining device.

2. The point cloud joining device includes a feature point extraction unit that extracts the one or more feature points from the reference laser point cloud and extracts the one or more feature points from the target laser point cloud; The tie point extraction unit extracts the tie points from each of the one or more feature points extracted from the reference laser point cloud and the one or more feature points extracted from the target laser point cloud. The point group joining apparatus according to claim 1 .

3. Both the reference measurement data and the target measurement data are data obtained by using the special measurement vehicle, Both the reference laser point cloud and the target laser point cloud include the oblique laser point cloud and the horizontal laser point cloud. The point group joining apparatus according to claim 1 or 2.

4. A normal measurement vehicle is a measurement vehicle in which the oblique laser scanner is installed and the horizontal laser scanner is not installed, the reference measurement data is data obtained by using the normal measurement vehicle, the reference laser point cloud includes the oblique laser point cloud and does not include the horizontal laser point cloud; The target measurement data is data obtained by using the special measurement vehicle, The target laser point cloud includes the oblique laser point cloud and the horizontal laser point cloud. The point group joining apparatus according to claim 1 or 2.

5. A normal measurement vehicle is a measurement vehicle in which the oblique laser scanner is installed and the horizontal laser scanner is not installed, The reference measurement data is data obtained by using the special measurement vehicle, the reference laser point cloud includes the oblique laser point cloud and the horizontal laser point cloud; The target measurement data is data obtained by using the normal measurement vehicle, The target laser point cloud includes the oblique laser point cloud and does not include the horizontal laser point cloud. The point group joining apparatus according to claim 1 or 2.

6. Extracting common feature points as tie points from one or more feature points in a reference laser point cloud, which is a three-dimensional point cloud included in the reference measurement data, and one or more feature points in a target laser point cloud, which is a three-dimensional point cloud included in the target measurement data; correcting self-position and orientation data included in the target measurement data so that positions of the tie points of the reference laser point cloud and positions of the tie points of the target laser point cloud coincide with each other; Using the corrected self-position and orientation data and distance and orientation information included in the target measurement data, a corrected laser point cloud is generated, which is a three-dimensional point cloud to be joined with the three-dimensional point cloud included in the reference measurement data. A point cloud joining method, At least one of the reference measurement data and the target measurement data is measurement data obtained using a special measurement vehicle equipped with an oblique laser scanner and a horizontal laser scanner, At least one of the reference laser point cloud and the target laser point cloud includes an oblique laser point cloud that is a three-dimensional point cloud obtained using the oblique laser scanner, and a horizontal laser point cloud that is a three-dimensional point cloud obtained using the horizontal laser scanner, The horizontal laser scanner is installed at the same height as the oblique laser scanner or at a higher height than the oblique laser scanner, The horizontal laser point cloud includes three-dimensional points at locations that are not measured by the oblique laser scanner but are measured by the horizontal laser scanner. Point cloud joining method.

7. a tie point extraction process for extracting common feature points as tie points from one or more feature points in a reference laser point cloud, which is a three-dimensional point cloud included in the reference measurement data, and one or more feature points in a target laser point cloud, which is a three-dimensional point cloud included in the target measurement data; a self-position and orientation correction process for correcting self-position and orientation data included in the target measurement data so that positions of the tie points of the reference laser point cloud and positions of the tie points of the target laser point cloud coincide with each other; a three-dimensional processing for generating a corrected laser point cloud, which is a three-dimensional point cloud to be joined with a three-dimensional point cloud included in the reference measurement data, using the corrected self-position and orientation data and distance and orientation information included in the target measurement data; A point cloud joining program for causing a computer to execute the above. At least one of the reference measurement data and the target measurement data is measurement data obtained using a special measurement vehicle equipped with an oblique laser scanner and a horizontal laser scanner, At least one of the reference laser point cloud and the target laser point cloud includes an oblique laser point cloud that is three-dimensional point cloud data obtained using the oblique laser scanner, and a horizontal laser point cloud that is three-dimensional point cloud data obtained using the horizontal laser scanner, The horizontal laser scanner is installed at the same height as the oblique laser scanner or at a higher height than the oblique laser scanner, The horizontal laser point cloud includes three-dimensional points at locations that are not measured by the oblique laser scanner but are measured by the horizontal laser scanner. Point cloud joining program.

8. A measurement vehicle for obtaining measurement data used to generate a three-dimensional point cloud, an oblique laser scanner for measuring distances and directions to each measurement point located in an oblique direction; a horizontal laser scanner for measuring distances and directions to each measurement point located in the horizontal direction; Equipped with The horizontal laser scanner is installed at the same height as the oblique laser scanner or at a higher height than the oblique laser scanner. Measurement vehicle.

Images