Tunnel excavation support system and tunnel excavation support method

Abstract

To provide a tunnel excavation support system that enables an operator to intuitively and accurately grasp a location to be excavated, thereby improving accuracy of a hit and shortening time.SOLUTION: A tunnel excavation support system of the invention displays an excavation state of a tunnel. It comprises a camera unit 1 acquiring point cloud data and RGB images of an excavation target while acquiring position and posture information, a processing PC section 13 comparing design data identified from the position and posture information with the point cloud data to manifest the point cloud data that should be focused on and that satisfies set conditions, and a monitor 51 for heavy machine overlaying the manifested point cloud data on an image based on the RGB image from the same viewpoint to display it.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a tunnel excavation support system and a tunnel excavation support method for displaying the excavation status of a tunnel. [Background technology]

[0002] In the construction of mountain tunnels, tunnel excavation progresses by repeating a series of cycles: excavation and blasting are followed by removal of excavated material, marking out areas where excavation is insufficient, then scaffolding is erected, and finally, sprayed concrete is applied.

[0003] Currently, after blasting, workers near the tunnel face use laser markers to illuminate areas where excavation is insufficient compared to the design cross-section, and heavy equipment operators then use breakers to knock down these areas (marking the target area). However, this work is dangerous because it takes place near the tunnel face. In addition, detailed observation of the tunnel face at the excavation site is performed in an unstable environment around the tunnel face, posing challenges in terms of safety and efficiency.

[0004] On the other hand, as disclosed in Patent Document 1 and others, systems have recently been developed that use 3D scanners to measure the tunnel cross-section after excavation, compare it with the design cross-section, and use the results to update the check and blasting patterns.

[0005] In detail, Patent Document 1 describes measuring the tunnel face and excavation cross-section using a 3D scanner mounted on heavy machinery equipped with a breaker for checks, comparing the measurement results with the design model, and coloring the design model to highlight areas that protrude or recede into the tunnel interior compared to the design, and displaying them in two dimensions on a tablet device or the like. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2019-158637 [Overview of the project]

Problems to be Solved by the Invention

[0007] However, on the tablet terminal viewed by the operator of the heavy machinery, only the difference between the design data and the measurement result by the 3D scanner is displayed on the two-dimensional screen, and there is no actual image. Therefore, it is difficult for the operator to recognize within a short time which position of the actual excavation target corresponds to the attention point that must be noted.

[0008] Therefore, an object of the present invention is to provide a tunnel excavation support system and a tunnel excavation support method that enable improvement of the accuracy and shortening of the time of hitting by intuitively and accurately grasping the location to be excavated.

Means for Solving the Problems

[0009] In order to achieve the above object, a tunnel excavation support system of the present invention is a tunnel excavation support system for displaying the excavation status of a tunnel, and includes a photographing unit that acquires its own position information and attitude information, and also acquires point cloud data and RGB images of the excavation target, an extraction processing unit that compares the design data specified from the position information and attitude information with the point cloud data, and makes the point cloud data to be noted that satisfies the set conditions apparent, and a display unit that superimposes and displays the apparent point cloud data on the image based on the RGB image from the same viewpoint.

[0010] Here, the point cloud data is preferably acquired by a distance image camera, and the relative positional relationship between the distance image camera and the color camera that acquires the RGB image is known.

[0011] Further, it can be configured to include a camera holder that fixes the distance image camera and a plurality of the color cameras, and a moving means unit on which the camera holder is mounted. Furthermore, the camera holder can also be configured such that three or more surveying targets are respectively attached via arms.

[0012] Further, it can be configured to include an overexcavation amount calculation unit that calculates the overexcavation amount based on the point cloud data and the design data. Furthermore, it is preferable that the display unit is a naked-eye stereoscopic display device configured such that parallax images are generated by a plurality of the color cameras. Also, the parallax images can be configured such that a plurality of parallax images with different shooting ranges are generated and joined together.

[0013] On the other hand, the camera holder may be configured such that a plurality of the distance image cameras with different shooting ranges, whose position and orientation information is known, are attached thereto, and the point cloud data acquired by each of the distance image cameras is converted into an absolute coordinate system and joined together.

[0014] Also, in the tunnel excavation support method of the present invention, it is a tunnel excavation support method for displaying the excavation status of a tunnel, and includes steps of acquiring the position information and orientation information of the imaging unit, and acquiring the point cloud data and RGB image of the excavation target by the imaging unit, comparing the design data specified from the position information and orientation information with the point cloud data, and visualizing the point cloud data to be noted that satisfies the set conditions, and a step of superimposing and displaying the visualized point cloud data on the image based on the RGB image at the same viewpoint.

Effects of the Invention

[0015] In the tunnel excavation support system of the present invention configured as described above, the point cloud data of the excavation target acquired by the imaging unit is compared with the design data to visualize the point cloud data to be noted, and it includes a display unit that superimposes and displays the visualized point cloud data on the image based on the RGB image at the same viewpoint.

[0016] Since the image based on the RGB image is a real image directly showing the actual excavation status, workers such as the operator of the heavy equipment can intuitively and accurately grasp the location to be excavated by looking at the visualized point cloud data superimposed on the image. As a result, the accuracy of hitting can be improved and the time can be shortened.

[0017] Furthermore, if point cloud data is acquired using a depth image camera, it becomes possible to acquire three-dimensional point cloud data of the excavation target in a much shorter time compared to acquiring point cloud data using a 3D scanner. As a result, the time until the excavation status is displayed is shortened, and the efficiency of the work can be improved.

[0018] Furthermore, by using multiple color cameras, it becomes possible to display three-dimensional real-world images and create a sense of depth, making it easier to identify areas that need to be excavated. In addition, if the camera holders for fixing the distance image camera and color camera are mounted on the moving mechanism, it becomes possible to quickly move to the optimal position and take pictures after blasting, and the effects of vibration (measurement error, durability) can be reduced compared to mounting the camera unit on heavy machinery that vibrates a lot.

[0019] Furthermore, if three or more surveying targets are attached to the camera holder via arms, the positional and orientation information of the imaging unit can be accurately measured based on the geometrically known positional relationship between the distance imaging camera and the color camera.

[0020] Furthermore, by having the over-excavation amount calculation unit compare three-dimensional point cloud data with design data to calculate the amount of over-excavation, the burden of construction management can be reduced. In addition, if the display unit is a glasses-free 3D display device, 3D viewing can be easily performed without wearing a heavy head-mounted display or special glasses, thus reducing fatigue and discomfort for heavy equipment operators. Moreover, since the operator's field of view is not narrowed, it is easier to detect surrounding hazards, making it safer.

[0021] Furthermore, by attaching multiple depth image cameras with different shooting ranges to the camera holder and combining the point cloud data converted to their respective absolute coordinate systems, it becomes possible to expand the shooting range, thus enabling the system to handle cases where the excavation target to be photographed is large.

[0022] Furthermore, in the tunnel excavation support method of the present invention, point cloud data of interest is made visible by comparing the point cloud data of the excavation target with design data, and the visible point cloud data is overlaid on an image based on an RGB image at the same viewpoint.

[0023] In this way, if point cloud data is overlaid onto real-world images that directly show the actual excavation situation, heavy equipment operators can intuitively and accurately grasp the areas to be excavated, thereby improving the accuracy of the initial assessment and reducing the time required. [Brief explanation of the drawing]

[0024] [Figure 1] This is an explanatory diagram illustrating the usage of the tunnel excavation support system of this embodiment. [Figure 2] This is an explanatory diagram showing the configuration of the tunnel excavation support system of this embodiment. [Figure 3] This is a perspective view illustrating the configuration of the camera unit. [Figure 4] This is an explanatory diagram showing how to obtain the position and orientation information of the camera unit. [Figure 5] This is an explanatory diagram showing the processing flow of the tunnel excavation support system of this embodiment. [Figure 6] This is an explanatory diagram illustrating a composite image in which manifested point cloud data is overlaid onto an image based on an RGB image. [Figure 7] This is an explanatory diagram showing the processing flow of a tunnel excavation support system that converts point cloud data into a surface. [Figure 8] This is a flowchart illustrating the tunnel excavation process performed using the tunnel excavation support system of this embodiment. [Modes for carrying out the invention]

[0025] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Figure 1 is an explanatory diagram illustrating the usage of the tunnel excavation support system of this embodiment. Figure 2 is an explanatory diagram showing the configuration of the tunnel excavation support system of this embodiment.

[0026] The tunnel excavation support system of this embodiment is used to display the tunnel excavation status. At tunnel excavation sites such as mountain tunnels, the excavation status of the excavation cross section, such as the tunnel face, is checked after excavation and blasting, and additional excavation (testing) is performed using an excavation machine 5 such as a breaker in areas where excavation is insufficient.

[0027] Therefore, the tunnel excavation support system of this embodiment displays the current tunnel excavation status compared with the design data, enabling safe and efficient excavation work such as initial marking.

[0028] As shown in Figures 1 and 2, the tunnel excavation support system of this embodiment includes a camera unit 1 which is an imaging unit that photographs the excavation target such as the tunnel face and adjacent tunnel walls, a computer (processing PC unit 13, management PC unit 6) that performs various calculations, and a display unit (heavy equipment monitor 51, naked-eye stereoscopic display device 61) that displays the results of the imaging and calculations.

[0029] Figure 3 is an explanatory diagram showing the configuration of camera unit 1. Camera unit 1 includes a ToF camera 11, which is a distance image camera for acquiring point cloud data of the excavation target, and a color camera 12 for acquiring RGB images.

[0030] The ToF camera 11 is a camera that can measure the distance to a subject by measuring the time-of-flight (Time-of-Flight) it takes for illuminating light, such as infrared light, to reflect off the subject and return. The ToF camera 11 diffuses the illumination range of the light, and can obtain not only brightness information for each pixel like a regular camera, but also distance information. In short, by taking images with the ToF camera 11, it is possible to obtain three-dimensional point cloud data of the area to be excavated.

[0031] The ToF camera 11 has the advantage of requiring less distance calculation load compared to a stereo camera and being able to capture (measure) images even in dark places. Furthermore, while obtaining point cloud data using a 3D scanner that rotates 360 degrees while emitting a laser takes tens of seconds to several minutes per measurement, the ToF camera 11 can obtain point cloud data in a shooting time of about 1 second.

[0032] Although a single ToF camera 11 can acquire three-dimensional point cloud data of an object, the camera unit 1 in this embodiment is equipped with two ToF cameras 11 to obtain a wide field of view. In short, by stitching together the point cloud data acquired by the two ToF cameras 11, it becomes possible to photograph even large-diameter tunnel faces and other excavation targets in a single shot.

[0033] On the other hand, in order to generate a stereoscopic image, multiple color cameras 12 are mounted on the camera unit 1. In this embodiment, the camera unit 1 is equipped with four color cameras 12.

[0034] By simultaneously capturing images with two spaced-apart color cameras 12 to obtain two RGB images and generate a parallax image, a three-dimensional image of the subject can be created. Furthermore, by capturing images with three or more color cameras 12, it becomes possible to create three or more parallax images, enabling the system to handle situations where the operator's or other worker's viewpoint is shifted from the display unit.

[0035] The point cloud data and disparity images acquired by camera unit 1 require the acquisition of each camera's viewpoint (where it is looking from) and orientation (where it is facing), i.e., position information (three-dimensional coordinates) and orientation information (three-dimensional vectors), in order to compare them with design data in later processing or to overlay the point cloud data and disparity images.

[0036] In this embodiment, the camera unit 1 is fixed to a camera holder 2 as shown in Figure 3, and the relative positional relationship between the ToF camera 11 and the color camera 12 is known information. Furthermore, the camera holder 2 has three or more arms 21, and a prism 41, which serves as a surveying target, is attached to the tip of each arm 21.

[0037] The prism 41 is attached to the camera holder 2 to detect the position and orientation information of the camera unit 1. Therefore, the relative positional relationship between the positions of the three prisms 41 and the ToF camera 11 and the color camera 12 is also known information.

[0038] The position and orientation information of the camera unit 1 is calculated by measuring the positions of three or more prisms 41 from an external total station 4. Automatic measurement is possible if the three or more prisms 41 each have shutters that open and close in conjunction with the total station 4 and measure sequentially. In this case, the greater the distance between the prisms 41, the higher the accuracy of the position and orientation measurement. Since the camera holder 2 is mounted on the shooting vehicle 3, which is the means of transport, it is desirable to ensure sufficient distance by attaching it to the end of an arm 21 that is the longest possible length that fits within the width of the vehicle.

[0039] Three or more prisms 41 are necessary to uniquely identify the position and orientation of camera unit 1 in three-dimensional space, and a larger number of prisms is better to reduce the variability of the measured values. On the other hand, a larger number of prisms will increase the measurement time for total station 4, so a balance must be considered.

[0040] The camera unit 1 is exposed to environmental conditions such as dust and water droplets from groundwater in tunnels, and is used mounted on a camera vehicle 3 that travels over uneven terrain. Therefore, it needs to be configured to be less susceptible to vibration, dust, and water droplets. The ToF camera 11 is selected to be dustproof and waterproof, and the color camera 12 is housed in a dustproof and waterproof case and can be attached to the camera holder 2 with vibration-damping rubber or the like interposed between them. Furthermore, the effects of vibration can be suppressed by attaching the camera holder 2, which secures the camera unit 1, to the vibration-damping tripod head 22.

[0041] The vibration-damping pan / tilt head 22 is mounted on a frame 23 provided on the cargo bed of the camera vehicle 3. By mounting it on the frame 23, the height of the camera unit 1 is adjusted to be approximately the same as the viewpoint of the operator of the excavator 5.

[0042] Since filming is performed with the filming vehicle 3 stopped in an uneven tunnel, depending on the vehicle's stopping position and angle, the required filming range may not fit within the field of view of the ToF camera 11 or the color camera 12. For this reason, it is desirable to configure the camera unit 1 so that its pan and tilt can be controlled remotely or automatically.

[0043] Figure 4 is an explanatory diagram of how the position and orientation information of camera unit 1 is obtained. Camera unit 1 is equipped with four color cameras 12, two ToF cameras 11, and three prisms 41 for measurement by total station 4. Since the geometric positional relationship between the ToF cameras 11, color cameras 12, and prisms 41 is known, the position and orientation of camera unit 1 are calculated using these known parameters and the measurements taken by total station 4.

[0044] Here, the vector from the centroid 42 between the three prisms 41 to each prism 41 is v ga ,v gb ,v gc This is defined as: v ga ,v gb ,v gcare three-dimensional vectors with xyz coordinates. Also, the vector from the center of gravity 42 of one camera of interest (here, the ToF camera 11) in the camera unit 1 is denoted as v gcam is defined. In a left-handed coordinate system where the positive directions of the x-axis and y-axis are northward and eastward, respectively, when the camera unit 1 is in the initial state with the positive direction of the x-axis facing in the horizontal plane, the initial values of the above vectors v ga0 , v gb0 , v gc0 , v gcam0 are determined only from known parameters.

[0045] And when the camera unit 1 is different from the initial state, each vector from the center of gravity 42 to each prism 41 can be expressed by the following equation using the rotation matrix Q. v * = Qv *0 Here, * includes ga, gb, gc, gcam.

[0046] Here, assuming q is a unit quaternion and q = q0 + q1i + q2j + q3k, the rotation matrix Q can be expressed by the following equation.

Equation

[0047] By surveying with the total station 4, the coordinates P a , P b , P c of each prism 41 are obtained, and the coordinate P g of the center of gravity 42 is P g =(P a + P b + P c ) / 3. Therefore, even in an unknown posture, the vectors v ga (= P a - P g ), v gb (= P b - P g ), v gc (= P c - P g ) can be obtained.

[0048] The three vectors v obtained in this way ga ,v gb ,v gc For each of the above equation v * =Qv *0 Since the above holds, Σ(v * -Qv *0 We find q0, q1, q2, and q3 that minimize ) using the least squares method.

[0049] Based on the above, the amount of rotation (q) of the camera unit 1 around the center of gravity 42 during measurement relative to the initial state is determined, and the coordinates of the camera position at that time are P. cam =P g +Qv gcam0 This can be expressed as follows. The point cloud data measured by the ToF camera 11 is obtained in a relative coordinate system with the position of the ToF camera 11 as the origin. Furthermore, in order to compare it with the design data expressed in an absolute coordinate system, the point cloud data is multiplied by a rotation matrix Q, and the coordinates P of the camera position obtained above are obtained. cam By translating it, the point cloud data is converted to an absolute coordinate system.

[0050] The calculations for acquiring the position and orientation information of the camera unit 1, as well as the calculations for coordinate transformation, are performed, for example, on a processing PC unit 13 mounted on the camera vehicle 3 (see Figure 2).

[0051] Furthermore, this processing PC unit 13 is also equipped with an extraction processing unit that compares design data and point cloud data to identify noteworthy point cloud data that meet the set conditions. In short, once the position and orientation information of the ToF camera 11 is determined, the absolute coordinates of the point cloud data are also revealed, so the extraction processing unit compares the point cloud data of the excavation target obtained by imaging with design data that is represented by the same absolute coordinates.

[0052] Furthermore, point cloud data that protrudes into the tunnel interior from the design data is highlighted by coloring according to the degree of protrusion in order to be targeted for excavation. In short, noteworthy point cloud data that meet the condition of exceeding the design line is colored in a way that is easily visible to heavy equipment operators, such as red or yellow. On the other hand, areas that are recessed from the design data are not to be excavated, so they are colored in a way that is less noticeable to heavy equipment operators, such as blue or green.

[0053] As shown in Figure 2, the processing PC unit 13 is connected to wireless transmission devices 15 such as a wireless router and a monitor 14 mounted on the camera vehicle 3, and can receive measurement results from the total station 4.

[0054] Furthermore, the calculation results from the processing PC unit 13 are sent via the wireless transmission device 15 to the wireless transmission device 52 on the excavator 5 and to the wireless transmission device 62 installed in the office where construction management is performed. Note that the wireless transmission device 62 in the office is an example, and communication between the tunnel excavation site and the office can also be done via wired connections through wireless relay devices inside the tunnel.

[0055] The excavator 5 is equipped with a heavy equipment monitor 51, which is a display unit that overlays the manifested point cloud data onto a parallax image generated based on an RGB image, at the same viewpoint. The heavy equipment monitor 51 may be an LCD monitor connected to a PC (personal computer) or a monitor of a tablet device.

[0056] Furthermore, by using a glasses-free stereoscopic monitor for the heavy equipment monitor 51, it becomes possible to easily view images in 3D with the naked eye by displaying multiple parallax images. Stereoscopic viewing allows the heavy equipment operator to gain a sense of depth, making it easier to identify protruding areas that need to be excavated.

[0057] Meanwhile, the calculation results from the processing PC unit 13 are also sent to the management PC unit 6 installed in the office. The management PC unit 6 can have the over-excavation amount calculation unit calculate the over-excavation amount based on the point cloud data and design data. For example, by multiplying the distance of each point in the point cloud data from the design plane by the unit area, it can calculate the volume of over-excavation for a specified range, such as one excavation cycle. The monitor connected to the management PC unit 6 can then display the calculation results from the processing PC unit 13 and the calculation results from the over-excavation amount calculation unit.

[0058] Furthermore, a glasses-free stereoscopic display device 61 can also be connected to the management PC unit 6. The glasses-free stereoscopic display device 61 is a device consisting of a glasses-free stereoscopic monitor, or a screen 611 and a projector 612, which makes it possible to easily view images in 3D with the naked eye without wearing a head-mounted display or special glasses. In addition, stereoscopic viewing is possible even if the observer moves their head from side to side. Further details are disclosed in Japanese Patent Application Publication No. 2022-59898.

[0059] Next, the processing flow of the tunnel excavation support system of this embodiment will be explained with reference to Figures 2 and 5. The position and orientation of camera unit 1 are measured after the camera vehicle 3 is stopped and camera unit 1 is pointed towards the subject. For example, three openable prisms 41 fixed to the camera holder 2 are automatically measured by a total station 4. As illustrated in Figure 1, the total station 4 is installed, for example, on the tunnel wall and can automatically measure the position of the prisms 41 that are the measurement targets.

[0060] The coordinates of the prism 41 in the absolute coordinate system, which have been automatically measured, are sent wirelessly to the processing PC unit 13, where the position and orientation of the camera unit 1 are calculated. Meanwhile, the ToF camera 11 and color camera 12 of the camera unit 1 photograph the excavation target area, such as the tunnel face and its surroundings.

[0061] Each shot taken by the ToF camera 11 is completed in a short time (for example, about 1 second). Furthermore, by using two ToF cameras 11, the entire width of the excavation face can be captured at once. Through these ToF camera 11 shots, point cloud data of the excavation target can be acquired. The point cloud data acquired through the shots is converted into absolute coordinates based on the position and orientation information of the camera unit 1.

[0062] The point cloud data, converted to absolute coordinates, is compared with three-dimensional design data such as CIM (Construction Information Modeling). Based on the direction and degree of the difference from the design data, the point cloud is color-coded. For example, points that protrude inward from the design line are colored with warm colors, while points that recede outward are colored with cool colors.

[0063] Meanwhile, multiple color cameras 12 (four in Figure 5) also capture images of the excavation target, and parallax images (real-world images) from multiple viewpoints are generated from the RGB images of each camera. Furthermore, based on the positional and orientation information of the multiple color cameras 12, parallax images (point cloud images) of the color-coded point cloud data as seen from each viewpoint are generated for the color-coded point cloud data.

[0064] The left side of Figure 6 shows an example of a point cloud image M1 and a real-world image M2 from the same viewpoint. This point cloud image M1 displays only one ring (length in the tunnel axis direction, for example, about 1m to 1.2m) corresponding to one excavation cycle. Then, by superimposing the point cloud image M1 and the real-world image M2 with a specified transparency, a composite image M3 like the one shown on the right side of Figure 6 is created. This composite image M3 is created according to the number of parallax images. In short, if there are four parallax images (M1, M2), four composite images M3 will be created.

[0065] The multiple composite images M3 created in this way are transmitted wirelessly and displayed on a heavy equipment monitor 51, which is positioned so that the operator of the excavator 5 can see them while operating the machine. The operator performs the excavation work while comparing the actual excavation face and other visible objects with the composite images M3 displayed on the heavy equipment monitor 51.

[0066] Meanwhile, the point cloud data, compared with the design data, is sent via wireless or wired transmission to a management PC unit 6 installed in the site office or elsewhere, and used for calculations such as over-excavation. In addition, the actual images obtained by the color camera 12 are sent to the management PC unit 6 and displayed three-dimensionally on a naked-eye stereoscopic display device 61 connected to it.

[0067] By the way, Figure 5 illustrates the processing flow for using point cloud data directly, but it is also possible to create a surface from the point cloud and then make that surface visible through color coding or other methods. Figure 7 is an explanatory diagram showing the processing flow of a tunnel excavation support system that converts point cloud data into a surface.

[0068] When creating a surface, surface data representing the excavation surface is generated from the coordinate-transformed point cloud data, and this is compared with the surface of the design model (inner tunnel surface) generated from the design data. The surface is then color-coded according to the degree of unevenness of the current conditions relative to the design line.

[0069] Next, each step of tunnel excavation using the tunnel excavation support system of this embodiment will be described with reference to Figure 8. The following explanation will use the NATM method, which utilizes blasting, as an example to describe the excavation process for mountain tunnels.

[0070] First, to begin the explanation of the blasting process, after blasting, heavy machinery such as wheel loaders and backhoes are used to move the excavated material. Next, the camera vehicle 3, equipped with camera unit 1, is moved to a position within a few meters of the excavation site (face) and parked there.

[0071] Then, with camera unit 1 pointed towards the excavation target (excavation area), the position and orientation of camera unit 1 are measured by total station 4. The excavation area is photographed by camera unit 1 simultaneously using ToF camera 11 and color camera 12, and point cloud data and actual images of the excavation area are acquired.

[0072] The point cloud data is transformed based on the position and orientation information of the ToF camera 11 and compared with the design data. Then, each point is color-coded according to the degree of unevenness in the current environment relative to the design line.

[0073] The live-action image M2 and the colored point cloud image M1 are superimposed at the same viewpoint, and the superimposed composite image M3 is wirelessly transmitted to the excavator 5 and displayed on the mounted heavy equipment monitor 51. In short, the heavy equipment monitor 51 displays the target locations as a point cloud with prominent coloring on top of the live-action image. The operator then performs the targeting using the breaker or other devices of the excavator 5 while looking at the heavy equipment monitor 51.

[0074] The live images are also transmitted to the office, allowing for remote stereoscopic observation of the tunnel face using the naked-eye stereoscopic display device 61. Furthermore, point cloud data is also transmitted to the office, enabling the calculation of excess excavation volume by comparing it with the design.

[0075] At the tunnel excavation site, after the initial marking, shoring is erected as needed. Subsequently, concrete is sprayed and rock bolts are installed at the excavation site and tunnel face. Meanwhile, in the office, the next blasting plan is formulated, taking into account the results of the tunnel face observation and the calculation of the excess excavation amount.

[0076] In short, the calculation results of the over-excavation amount by the over-excavation amount calculation unit, along with detailed remote observation of the tunnel face using parallax images from multiple color cameras 12, can be used to inform the next blasting plan. For example, the over-excavation amount (insufficient or excessive excavation) calculated based on the blasting results, and the condition of the rock face (quality, direction of grain, etc.) can be used to partially modify the initially planned next blasting plan (placement and amount of explosives, etc.) to enable more precise blasting. Based on the plan thus formulated, drilling and explosive charging for the next blasting are carried out, and the process from blasting onwards is repeated.

[0077] Next, the operation of the tunnel excavation support system and tunnel excavation support method of this embodiment will be described. In this tunnel excavation support system, the point cloud data of the excavation target acquired by the camera unit 1 is compared with the design data to make noteworthy point cloud data visible, and the system is equipped with a heavy equipment monitor 51 that displays the visible point cloud data superimposed on the actual image at the same viewpoint.

[0078] Since the parallax image based on the RGB image is a real-world image that directly shows the actual excavation situation, the operator of the excavator 5 can intuitively and accurately grasp the area to be excavated by looking at the colored point cloud data superimposed on the real-world image. As a result, the accuracy of the initial marking can be improved and the time required can be reduced.

[0079] Furthermore, if point cloud data is acquired using the ToF camera 11, it becomes possible to acquire three-dimensional point cloud data of the excavation target in a very short time (about 1 second) compared to acquiring point cloud data using a 3D scanner (tens of seconds to several minutes). As a result, the time until the excavation status is displayed is shortened, allowing excavation to start more quickly, and the tunnel excavation cycle time is shortened, thereby improving work efficiency.

[0080] Furthermore, by using multiple color cameras 12 to display three-dimensional real-world images, it becomes possible to perceive depth and understand the unevenness of the terrain more realistically, making it easier to identify areas that need to be excavated. In addition, by attaching multiple ToF cameras 11 to the camera holder 2, it becomes possible to handle cases where the area to be excavated is large.

[0081] Furthermore, if the camera holder 2 for fixing the ToF camera 11 and the color camera 12 is mounted on the shooting vehicle 3, it becomes possible to quickly move to the optimal position and take pictures after blasting, and the effects of vibration (measurement error, durability) can be reduced compared to mounting the shooting unit on the excavator 5, which experiences significant vibration.

[0082] Furthermore, if prisms 41 are attached to three or more arms 21 of the camera holder 2, the positional information and orientation information of the camera unit 1 can be measured accurately in a short time based on the geometrically known positional relationship with the ToF camera 11 and the color camera 12.

[0083] Furthermore, by having the over-excavation amount calculation unit compare three-dimensional point cloud data with design data to calculate the amount of over-excavation, the burden of construction management can be reduced. In addition, if the naked-eye stereoscopic display device 61 is connected to the management PC unit 6, real-world images can be easily viewed in 3D without having to wear a heavy head-mounted display or special glasses.

[0084] Furthermore, if the heavy equipment monitor 51 is also a glasses-free stereoscopic display device such as a glasses-free stereoscopic monitor, workers can easily view real-world images in 3D without experiencing fatigue or discomfort. In addition, since the worker's field of vision is not narrowed, they can more easily detect hazards in their surroundings, making it safer.

[0085] Furthermore, in the tunnel excavation support method of this embodiment, point cloud data of interest is highlighted by coloring by comparing the point cloud data of the excavation target with design data, and the highlighted point cloud data is overlaid on a real-world image from the same viewpoint.

[0086] In this way, if point cloud data is overlaid on real-world images that directly show the actual excavation situation, workers can intuitively and accurately grasp the areas that need to be excavated, thereby improving the accuracy of the initial assessment and reducing the time required.

[0087] While embodiments of the present invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments, and any design modifications that do not depart from the spirit of the present invention are included in the present invention.

[0088] For example, the above embodiment described a case in which a disparity image is generated by multiple color cameras 12 to display a stereoscopic image, but it is not limited to this, and if a stereoscopic image is not generated, there may be only one color camera 12 in the imaging unit.

[0089] Furthermore, although the above embodiment described the case in which point cloud data is acquired using the ToF camera 11, it is not limited to this, and point cloud data can also be acquired using, for example, a 3D scanner, a stereo camera, etc.

[0090] Furthermore, although the above embodiment described the case in which an existing vehicle is used as the filming vehicle 3, it is not limited to this, and a dedicated vehicle capable of remote control or automatic driving can also be used as the means of transport.

[0091] Furthermore, while the above embodiment described a case where the extraction processing unit colors the areas of interest with warm colors, it is not limited to this, and for example, processing such as hiding areas that do not require attention may also be used.

[0092] Furthermore, although the above embodiment was described using blasting for excavation work in mountain tunnels as an example, the present invention is not limited to this, and can also be applied to tunnels where excavation is carried out without the use of blasting. [Explanation of Symbols]

[0093] 1: Camera unit (shooting section) 11: ToF camera (distance imaging camera) 12: Color camera 13: Processing PC section 2: Camera holder 21: Arm 3: Filming vehicle (means of transportation) 41: Prism (surveying target) 51: Monitor (display unit) for heavy machinery 61: Autostereoscopic display device M1: Point cloud image M2: Live-action image M3: Composite image

Claims

1. A tunnel excavation support system that displays the tunnel excavation status, A camera unit acquires location and orientation information, as well as point cloud data and RGB images of the excavation target. An extraction processing unit compares design data identified from the position information and orientation information with the point cloud data to reveal noteworthy point cloud data that satisfy the set conditions, A display unit that overlays the revealed point cloud data onto the image based on the RGB image at the same viewpoint, A tunnel excavation support system characterized by comprising an over-excavation amount calculation unit that calculates the amount of over-excavation based on the point cloud data and the design data.

2. The tunnel excavation support system according to claim 1, characterized in that the point cloud data is acquired by a depth image camera, and the relative positional relationship between the depth image camera and the color camera that acquires the RGB image is known.

3. A camera holder for fixing the distance image camera and the multiple color cameras, The tunnel excavation support system according to claim 2, further comprising a moving means unit that mounts the aforementioned camera holder.

4. The tunnel excavation support system according to claim 3, characterized in that three or more survey targets are attached to the camera holder via arms.

5. The tunnel excavation support system according to claim 3 or 4, characterized in that the display unit is a naked-eye stereoscopic display device and a parallax image is generated by a plurality of color cameras.

6. The tunnel excavation support system according to claim 5, characterized in that multiple parallax images with different shooting ranges are generated and stitched together.

7. The tunnel excavation support system according to claim 3 or 4, characterized in that a plurality of distance image cameras with different shooting ranges, whose position and orientation information are captured, are attached to the camera holder, and the point cloud data acquired by each of the distance image cameras is converted to an absolute coordinate system and joined together.

8. A tunnel excavation support method that displays the excavation status of a tunnel, The steps include acquiring positional and orientation information of the imaging unit, and acquiring point cloud data and RGB images of the excavation target using the imaging unit, A step of comparing design data identified from the position information and orientation information with the point cloud data to make noteworthy point cloud data that satisfy the set conditions apparent, The steps include: displaying the revealed point cloud data overlaid on the image based on the RGB image from the same viewpoint; A tunnel excavation support method characterized by comprising the step of calculating the amount of excess excavation based on the point cloud data and the design data.

Images