Laser scanning method, laser scanning system, and laser scanning program

Abstract

To efficiently manage constructions in a civil engineering site and manage a heavy machine.SOLUTION: The laser scan technique of the present invention includes performing a first laser scan on the tip of a blade 201 of a heavy machine 200 by a laser scanner 300 for the tip of blade 201 and performing a second laser scan on a work terminated area on which the heavy machine 200 did a work by a second laser scanner 200, the first laser scan being performed more frequently than the second laser scan.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to laser scanning technology.

Background Art

[0002] For example, there is a technology that uses laser scanning as a means of construction management at a civil engineering construction site (see, for example, Patent Document 1). Also, a technology that uses GNSS for position management of heavy equipment at a civil engineering construction site is also known (see, for example, Patent Document 2). In addition, a technology for controlling the cutting edge (such as the tip of a shovel) of a heavy machine is also known (see, for example, Patent Document 3).

Prior Art Documents

Patent Documents

[0003] [[ID=2,}}

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, the control of the cutting edge of a heavy machine and the measurement of the formed shape at a civil engineering construction site were performed by different technical means. For example, the acquisition of current data by laser scanning before the operation of the heavy machine, the control of the cutting edge by detecting the cutting edge with a surveying instrument during the operation of the heavy machine, and the measurement of the formed shape by laser scanning after the operation were carried out separately for each purpose using a surveying instrument.

[0005] This conventional method requires individual measurements, and improvements are demanded in terms of working time and working cost. Against this background, an object of the present invention is to efficiently perform construction management at a civil engineering construction site. [Means for solving the problem]

[0006] The present invention is a laser scanning method that performs a first laser scan of the cutting edge of heavy machinery using a laser scanning device, and a second laser scan of the area where work has been performed by the heavy machinery using the same laser scanning device, wherein the first laser scan is performed at a higher frequency than the second laser scan.

[0007] In the present invention, the first laser scan is performed to control the cutting edge. In the present invention, the second laser scan is performed to include at least a portion of the heavy machinery within its scanning range. In the present invention, a transition from the first laser scan to the second laser scan is performed based on the position of the cutting edge detected by the first laser scan.

[0008] In the present invention, one possible configuration is to transition from the first laser scan to the second laser scan when the distance between the position of the cutting edge detected by the first laser scan and the object being worked on by the cutting edge becomes greater than a predetermined value.

[0009] The present invention is a laser scanning system that includes a laser scanning device that performs a first laser scan on the cutting edge of a heavy machine and a second laser scan on the area where work has been performed by the heavy machine, and a control unit that performs the first laser scan at a higher frequency than the second laser scan.

[0010] The present invention is a program that can be read and executed by a computer, which causes the computer to perform a first laser scan of the cutting edge of a heavy machine using a laser scanning device, and a second laser scan of the area where work has been performed by the heavy machine using the same laser scanning device, wherein the first laser scan is performed at a higher frequency than the second laser scan. [Effects of the Invention]

[0011] According to the present invention, construction management and heavy equipment management at civil engineering construction sites can be performed efficiently. [Brief explanation of the drawing]

[0012] [Figure 1] This is a schematic diagram of an embodiment. [Figure 2] This is a schematic diagram of an embodiment. [Figure 3] This is a block diagram of a data processing unit. [Figure 4] This chart shows the timing of the scan. [Figure 5] This is a flowchart showing an example of the processing procedure. [Modes for carrying out the invention]

[0013] 1. First Embodiment (overview) Figure 1 shows a heavy machine 200 equipped with a cutting edge 201. In this example, the heavy machine 200 is a bulldozer, and the cutting edge 201 is a blade (dozer blade). The heavy machine 200 is not limited to this example; it may be a power shovel, wheel loader, or any other machine used for civil engineering work. For example, if the heavy machine 200 is a power shovel, the cutting edge 201 will be a shovel. Figure 1 shows an example of the heavy machine 200 performing leveling work, moving from right to left in the drawing to flatten the terrain.

[0014] Figure 1 shows a laser scanner 300. The laser scanner 300 includes a horizontal rotation unit and a vertical rotation unit disposed on the horizontal rotation unit, and the vertical rotation unit is provided with an optical system for emitting and receiving laser light (scanning light) for distance measurement. While horizontally rotating the horizontal rotation unit and vertically rotating the vertical rotation unit, laser scanning is performed by emitting the scanning light from the optical system in pulsed form in that state.

[0015] The laser scanning device 300 performs laser scanning on a specific range by adjusting the rotation range of the rotation unit and the range of pulsed light emission (or the reception range). The form and method of the laser scanning device are not particularly limited. For example, a laser scanner in a form where the optical axis of the optical system is mechanically reciprocated within a specific range for scanning or in a form where the scanning light is electronically scanned can also be used. Also, a laser scanning device in a form where a large number of measurement lights are simultaneously irradiated in a planar shape and point cloud data of a specific range is obtained with one light emission is also possible. Further, as the laser scanning device 300, a model equipped with a camera whose external calibration elements (position and orientation) in the laser scanning device 300 are known can also be used.

[0016] In the example of FIG. 1, using the laser scanning device 200, (1) a wide-range laser scan (wide-range scan) around the heavy machine 200 including the heavy machine 200, (2) a laser scan of a limited range centered on the cutting edge 201 (cutting-edge scan), and (3) a laser scan of the range where the work by the heavy machine 200 has been performed (work completion range scan) are performed.

[0017] Here, based on the wide-range scan of (1), the ranges of the cutting-edge scan of (2) and the work completion range scan of (3) are determined. The cutting-edge scan of (2) is performed at a relatively high rate, and the work completion scan of (3) is performed at a relatively low rate. Thereby, the cutting edge 201 and the change in the terrain are captured.

[0018] The data processing device 100 performs the processing of the scan data of (1) to (3) above, the processing related to the control of the cutting edge 201 based on the scan data, and the processing related to the measurement of the formed shape. Although FIG. 1 shows an example of civil engineering work targeting terrain, buildings or structures may also be the target.

[0019] By appropriately distributing the cutting edge scan of (2) and the work completion scan of (3) at appropriate timings by the laser scan device 300, the control of the cutting edge 201 and the calculation of the formed shape can be carried out by a series of laser scans by one laser scan device.

[0020] (Regarding the data processing device) FIG. 3 is a block diagram of the data processing device 300. The data processing device 100 is composed of a computer. The computer includes a CPU, a storage device, a communication interface, an interface for handling the control signal of the heavy machine 200, and a user interface. The computer to be used may be a general-purpose computer or a dedicated computer. For example, the data processing device 100 can be configured using a PC (Personal Computer) or a data processing server.

[0021] The data processing device 100 includes a wide-range scan control unit 101, an initial data acquisition unit 102, a heavy machine and cutting edge detection unit 103, a scan range setting unit 104, a scan range selection unit 105, a cutting edge scan control unit 106, a cutting edge control unit 1, a work completion range scan control unit 108, a work completion range data acquisition unit 109, a formed shape data calculation unit 110, and a height difference calculation unit 111.

[0022] These functional units are configured software-wise by an operation program installed in the computer that constitutes the data processing device 100. It is also possible to configure some or all of these functional units with dedicated hardware. [[ID=A]] [[ID=B]]

[0023] [[ID=C]] The wide-area scan control unit 101 controls the laser scan for the wide-area scan range shown in Figure 2. The wide-area scan is performed to determine the scan range for the cutting edge scan and the work completion range scan. The wide-area scan includes the area where work has been completed and the area where work will be performed.

[0024] The initial data acquisition unit 102 acquires scan data obtained by wide-area scanning as initial data.

[0025] The heavy machinery and blade tip detection unit 103 detects the heavy machinery 200 and the blade tip 201 based on the results of the wide-area scan. Detection can be performed using a reflective target, detecting the shape of the object from point cloud data, or a combination of both methods. It is also possible to capture the blade tip 201 using a camera mounted on the laser scanning device 300 and detect the blade tip 201 from the image. Of course, it is also possible to detect the blade tip 201 by using both the laser scan and the captured image.

[0026] For example, a reflective target (such as a reflective prism or retroreflective target) is placed on the heavy machinery 200 so that the vertices of a specific polygon are in a specific positional relationship. Because the reflection from the reflective target is strong, it becomes a bright spot in the laser scan point cloud. By detecting the polygon formed by this bright spot, the heavy machinery 200 can be identified from the laser scan point cloud.

[0027] Alternatively, for example, a reflective target can be placed at the tip of the cutting edge 201. By detecting this reflective target within the laser scan point cloud, the cutting edge 201 can be identified from the laser scan point cloud. Of course, it is also possible to identify the cutting edge 201 using multiple reflective targets. For example, multiple reflective targets can be placed on the cutting edge 201 so that its shape can be identified by multiple reflective points. In this case, the cutting edge 201 can be identified from the shape composed of multiple high-brightness reflective points. In this case, the orientation of the cutting edge 201 can be identified from the orientation of the polygonal shape.

[0028] The scan range setting unit 104 sets the range of the work completion range scan and the range of the blade tip scan based on the position information of the heavy machinery 200 and the blade tip 201 obtained from the wide-range scan.

[0029] The range of the work completion range scan is set as the area behind the heavy machinery 200 in the direction of travel. Here, the work completion range scan is set to include at least a portion of the heavy machinery 200. This is for the following reasons: A cutting edge scan is performed after the work completion range scan. At this time, by capturing a portion of the heavy machinery 200 in the work completion range scan, the position of the cutting edge 201 can be determined when performing the cutting edge scan, enabling a smooth transition to the cutting edge scan. In addition, by capturing a portion of the heavy machinery 200 in the work completion range scan, the position of the heavy machinery 200 can be determined, and the movement of the heavy machinery can be detected.

[0030] The scan range selection unit 105 determines the timing of the tip scan and the end-of-work range scan shown in Figure 2. Figure 4 shows an example of a timing chart for selecting the end-of-work range scan and the tip scan. In this example, the tip scan and the end-of-work range scan are set to the same range (same scan time), and the end-of-work range scan is performed once for every five tip scans. This allocation control is performed by the scan range selection unit 105.

[0031] As illustrated in Figure 4, the number of times the cutting edge scan is repeated per unit time is greater than the number of times the work completion range scan is repeated. This corresponds to the fact that the movement of the cutting edge 201 is faster than the movement of the heavy machine 200.

[0032] Instead of pre-determining the timing of the cutting edge scan and the work completion range scan, the timing of the scans can be selected according to the movement of the cutting edge 201. For example, the timing when the movement of the cutting edge 201 momentarily stops or slows down can be detected from the results of the cutting edge scan, and the work completion range scan can be performed at that timing.

[0033] For example, if the cutting edge 201 is displaced above the ground, the movement of the cutting edge 201 slows down at the highest position, and no cutting or other operations occur. When this condition is detected based on the cutting edge scan, the system transitions from the cutting edge scan to the work completion range scan.

[0034] Furthermore, it is also possible to detect when no work is being performed by the cutting edge 201 and use that as a trigger to switch from a cutting edge scan to a work completion range scan. For example, the distance between the cutting edge 201 and the terrain surface being worked on can be obtained from the cutting edge scan results, and if this distance exceeds a predetermined threshold, the system can switch from a cutting edge scan to a work completion range scan.

[0035] For example, it is also possible to detect the position and orientation of the cutting edge 201 from the cutting edge scan and transition from the cutting edge scan to the work completion range scan depending on that position and orientation. Furthermore, for example, it is also possible to transition from the cutting edge scan to the work completion range scan when the cutting edge 201 moves backward (the heavy machine moves backward). Of course, it is also possible to perform a determination by combining multiple of these modes and transition from the cutting edge scan to the work completion range scan accordingly.

[0036] Furthermore, it is possible to combine the above-mentioned predetermined schedule for selecting scans with a situation-based approach. For example, the default setting could be to select the blade tip scan based on the situation, and then select the work completion range scan if it has not been selected for a certain period of time. For instance, the work completion range scan could be selected at least once every 5 seconds.

[0037] The cutting edge scan control unit 106 controls the cutting edge scan. The cutting edge control unit 107 controls the movement of the cutting edge 201 based on the results of the cutting edge scan.

[0038] The following describes an example of the control performed by the cutting edge control unit 107. First, assume that a 3D model of the terrain (pre-work 3D model) has been obtained through prior surveying (e.g., laser scanning). On the other hand, assume that a 3D model of the terrain after work (post-work 3D model) has been created (designed) on the drawing (data).

[0039] In this case, the first step is to identify the correspondence between the results of the wide-area scan and the pre-process 3D model described above. Since the wide-area scan includes the pre-process portion, the correspondence between the results of the wide-area scan and the pre-process 3D model can be identified by exploring the correspondence between this pre-process portion and the pre-process 3D model.

[0040] Here, since the blade tip scan is performed based on the results of the wide-area scan, the correspondence between the blade tip scan and the wide-area scan is already known. Therefore, by identifying the correspondence between the results of the wide-area scan and the pre-work 3D model, the relationship between the blade tip scan and the pre-work 3D model can be determined.

[0041] Here, the movement of the cutting edge 201 is controlled so that the work corresponding to the difference between the pre-work 3D model and the post-work 3D model is performed. This control is performed by the cutting edge control unit 107.

[0042] For example, the pre-work 3D model and the post-work 3D model are compared to calculate the area to be excavated. Next, the movement trajectory of the cutting edge 201 required to excavate the soil in the calculated area is calculated. If the area cannot be excavated in one pass, the trajectory of the cutting edge 201 is set in a process that involves excavating in multiple passes.

[0043] Once the trajectory of the cutting edge 201 is set, the cutting edge scan detects any changes in the position of the cutting edge 201, and a control signal is sent to the heavy machine 200 to adjust the position of the cutting edge 201 so that it does not deviate from the set trajectory, thereby enabling real-time control of the position of the cutting edge 201.

[0044] Furthermore, it is also possible to send information about the difference between the set trajectory of the cutting edge 201 and the actual trajectory (trajectory measured by laser scanning) to the heavy machinery 200, and for the operator of the heavy machinery 200 to refer to this information and operate the cutting edge 201, or to assist the operator's operation based on this information. The above processing is performed in the cutting edge control unit 107.

[0045] The work completion range scan control unit 108 controls the work completion range scan. The work completion range data acquisition unit 109 acquires the point cloud data obtained from the work completion range scan.

[0046] The completed work data calculation unit 110 calculates the completed work based on the point cloud data obtained from the work completion range scan. An example of the calculation of the completed work is described below.

[0047] As described above, the correspondence between the results of the wide-area scan and the pre-work 3D model can be identified. Furthermore, since the final work area scan is performed based on the results of the wide-area scan, the correspondence between the results of the wide-area scan and the final work area scan can also be identified. Therefore, the correspondence between the results of the final work area scan and the pre-work 3D model can be identified.

[0048] Here, the finished shape is calculated by determining the difference between the results of the scan of the work completion area and the 3D model before work. For example, the amount of displacement of the terrain that was cut or raised by the cutting edge 201 is calculated. This amount of displacement becomes the finished shape.

[0049] The output calculation unit 111 calculates the output based on the completed work. For example, if 1 m of soil is prepared in advance... 3 The cost per unit is set as the unit cost. Then, the output is calculated by multiplying the unit cost by the amount of work completed.

[0050] (An example of processing) Figure 5 is a flowchart showing an example of processing performed by the data processing device 100. The program that executes this flowchart is stored in the storage device provided by the data processing device 100 and executed by the CPU of the data processing device 100. It is also possible to store the flowchart in a suitable storage medium and download it from there.

[0051] First, a wide-area scan is performed as shown in Figure 2 (step S101). The wide-area scan is controlled by the wide-area scan control unit 101. Next, the point cloud data obtained from the wide-area scan is acquired as initial data (step S102). The initial data is acquired by the initial data acquisition unit 102. Once the initial data is acquired, the heavy machinery 200 and the cutting edge 201 are detected from the initial data (step S103). This process is performed by the heavy machinery and cutting edge detection unit 103.

[0052] Once the heavy machinery 200 and the cutting edge 201 are detected, the scan range is set (step S104). Here, the work completion range scan range is set based on the position of the heavy machinery 200 as shown in Figure 2, and the cutting edge scan range is set based on the position of the cutting edge 201. This process is performed by the scan range setting unit 104.

[0053] Next, it is determined whether or not it is time for a cutting edge scan. This process is performed by the scan range selection unit 105. If it is time for a cutting edge scan, a cutting edge scan is performed (step S106); otherwise, a work completion range scan is performed (step S108).

[0054] After performing a tip scan (step S106) and obtaining point cloud data around the tip 201, the tip 201 is controlled based on this data (step S107). This process is performed by the tip control unit 107.

[0055] After performing the work completion range scan in step S108, the work completion range data, which is point cloud data obtained from the scan, is acquired (step S109), and the completed work data is calculated (step S110). The completed work data is calculated by the completed work data calculation unit 110. After calculating the completed work data, the amount of work completed is calculated (step S111). The amount of work completed is calculated by the amount of work completed calculation unit 111. It is also possible to calculate only one of the completed work or the amount of work completed.

[0056] If movement of the heavy machinery 200 exceeding a threshold is detected, or if a specified time has elapsed, the process from step S101 onwards is repeated. For example, if the heavy machinery 200 moves beyond a predetermined threshold, the process from step S101 onwards is repeated. The movement of the heavy machinery 200 is obtained from scan data and measurements from the GNSS position measuring device installed on the heavy machinery 200. By repeating the process from step S101 onwards, the point cloud data of the unworked area is updated. In this case, since wide-area scans are repeatedly performed as the heavy machinery moves, the scan range of the wide-area scan can be narrowed. This is useful when it is desired to increase the scan density of the wide-area scan. Note that if a wide-area scan is redone, the trajectory of the cutting edge 201 is reset each time.

[0057] 2. Second Embodiment In addition to the laser scanning device 200, a total station is used. The total station locks onto and tracks the cutting edge 201. If the total station detects movement of the cutting edge (for example, movement toward the ground) at a time when the cutting edge is not being scanned, the system switches to cutting edge scanning. This avoids problems caused by the cutting edge control being unable to keep up. Furthermore, since the total station continuously tracks the position of the cutting edge 201, the transition to cutting edge scanning can be made smoothly. The cutting edge 201 can also be controlled based on the position information of the cutting edge 201 detected by the total station.

[0058] A surveying device that integrates a laser scanning device and a total station, possessing the functions of both, is known. Using this surveying device, it is also possible to perform laser scanning while simultaneously measuring the position of the blade tip 201 using the functions of a total station.

[0059] 3. Third Embodiment Multiple laser scanning devices are used. In this case, three or more omnidirectional reflective prisms with different horizontal and vertical positions are placed on the heavy machinery 200. The multiple laser scanning devices operate synchronously.

[0060] First, each laser scanning device simultaneously performs a wide-area scan, and in each scan, three or more omnidirectional prisms of the heavy machinery are detected as common points in the scan data. Using these three or more common points, the correspondence between the point cloud data obtained by each laser scanning device is identified. This is the same as the first embodiment except that there are multiple laser scanning devices.

[0061] In this example, areas that are blind spots from the perspective of one laser scanning device can be scanned by other laser scanning devices, thus reducing blind spots in laser scanning and enabling more accurate measurements. It is preferable to use as many laser scanning devices as cost allows.

[0062] Another method involves mechanically installing multiple laser scanning devices in the same coordinate system. In this case, the external orientation elements of each laser scanning device become known, and the point cloud data of the heavy machinery 200 obtained by each laser scanning device 300 can be integrated without using targets such as reflective prisms.

[0063] 4. Fourth Embodiment In the third embodiment, the timing of the blade tip scan and the work completion range scan by the first laser scanning device are shifted from the timing of the blade tip scan and the work completion range scan by the second laser scanning device.

[0064] For example, at the timing of the blade tip scan by the first laser scanning device, the work completion range scan is performed by the second laser scanning device. Also, at the timing of the blade tip scan by the second laser scanning device, the work completion range scan is performed by the first laser scanning device.

[0065] Furthermore, when using three or more laser scanning devices, it is possible to ensure that each scan is performed by at least one laser scanning device at a given time.

[0066] In this example, a first laser scan is performed on the cutting edge of the heavy machinery using a first laser scanning device, and a second laser scan is performed on the area where the heavy machinery has performed work using the same first laser scanning device. At the same time that the first laser scan is being performed by the first laser scanning device, a second laser scanning device, different from the first laser scanning device, is performed on the area where the heavy machinery has performed work.

[0067] Alternatively, at the same time that the first laser scanning device is performing a laser scan on the area where the heavy machinery has performed work, a second laser scanning device, different from the first laser scanning device, is performed on the cutting edge of the heavy machinery. According to the method of this embodiment, the waiting period between each scan can be reduced. [Explanation of symbols]

[0068] 100...Data processing device, 200...Heavy machinery, 201...Blade tip, 300-ten...Laser scanning device.

Claims

1. A laser scanning method for a heavy machine, which is a bulldozer, power shovel, or wheel loader, in which the movement of the cutting edge is faster than the movement of the heavy machine, A first laser scan of the cutting edge of the heavy machinery by a laser scanning device, A second laser scan by the laser scanning device over the area where the heavy machinery work was performed, and Perform The first laser scan is a laser scanning method that is performed more frequently than the second laser scan.

2. The laser scanning method according to claim 1, wherein the first laser scan is performed for controlling the cutting edge.

3. The laser scanning method according to claim 1 or 2, wherein the second laser scan includes at least a portion of the heavy machinery within its scanning range.

4. A laser scanning method according to any one of claims 1 to 3, wherein a transition from a first laser scan to a second laser scan is performed based on the position of the cutting edge detected by the first laser scan.

5. The laser scanning method according to claim 4, wherein a transition from the first laser scan to the second laser scan is performed when the distance between the position of the cutting edge detected by the first laser scan and the object to be worked on by the cutting edge becomes greater than a predetermined value.

6. A laser scanning method for heavy machinery that performs leveling work to flatten terrain while moving, wherein the movement of the blade is faster than the movement of the heavy machinery, A first laser scan of the cutting edge of the heavy machinery by a laser scanning device, A second laser scan by the laser scanning device over the area where the heavy machinery work was performed, and Perform The first laser scan is a laser scanning method that is performed more frequently than the second laser scan.

7. The laser scanning method according to claim 6, wherein the first laser scan is performed for controlling the cutting edge.

8. The laser scanning method according to claim 6 or 7, wherein the second laser scan includes at least a portion of the heavy machinery within its scanning range.

9. The laser scanning method according to any one of claims 6 to 8, wherein the transition from the first laser scan to the second laser scan is performed based on the position of the cutting edge detected by the first laser scan.

10. The laser scanning method according to claim 9, wherein a transition from the first laser scan to the second laser scan is triggered when the distance between the position of the cutting edge detected by the first laser scan and the object to be worked on by the cutting edge becomes greater than a predetermined value.

11. A laser scanning system for a heavy machine, which is a bulldozer, power shovel, or wheel loader, in which the movement of the cutting edge is faster than the movement of the heavy machine, A first laser scan of the cutting edge of heavy machinery using a laser scanning device, A second laser scan by the laser scanning device over the area where the heavy machinery work was performed, and A laser scanning device that performs the following: A control unit that performs the first laser scan at a higher frequency than the second laser scan, and A laser scanning system including...

12. A laser scanning method for heavy machinery that performs leveling work to flatten terrain while moving, wherein the movement of the blade is faster than the movement of the heavy machinery, A first laser scan of the cutting edge of the heavy machinery by a laser scanning device, A second laser scan by the laser scanning device over the area where the heavy machinery work was performed, and A laser scanning device that performs the following: A control unit that performs the first laser scan at a higher frequency than the second laser scan, and A laser scanning system including...

Images