Working face monitoring device, working face monitoring method and working face monitoring program

Abstract

To accurately predict the collapse of a working face.SOLUTION: A working face monitoring device comprises: an image acquisition unit that acquires images of a working face of a mountain tunnel where shotcrete has been applied; a working face image acquisition unit that extracts the working face from the images using a learned working face extraction model and acquires working face images; and a detection unit that detects cracks in the working face using the working face images and a learned crack detection model.SELECTED DRAWING: Figure 2A

Description

[Technical Field]

[0001] The present invention relates to a face monitoring device, a face monitoring method, and a face monitoring program. [Background technology]

[0002] In the above-mentioned technical field, Patent Document 1 discloses a monitoring method in which retroreflective paint is applied to a part of the tunnel face of a mountain tunnel, the distance to the painted part of the tunnel face is measured at any time using a laser rangefinder, and the collapse of the tunnel face is predicted based on changes in the relative positional relationship between the laser rangefinder and the tunnel face (see paragraphs

[0026] ,

[0030] , claim 1, etc. of the same document). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-008871 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in Patent Document 1 monitors the amount of displacement of the part of the face where retroreflective paint is applied, so it is only possible to monitor part of the face and is not possible to monitor the face sufficiently. [Means for solving the problem]

[0005] In order to achieve the above object, the face monitoring device according to the present invention comprises: an image acquisition unit that acquires an image of a tunnel face on which the shotcrete has been applied; a face image acquisition unit that extracts the face from the captured image using a learned face extraction model and acquires a face image; A detection unit that detects cracks on the face of the construction work using the face image and a learned crack detection model; Equipped with.

[0006] In order to achieve the above object, a method for monitoring a working face according to the present invention comprises: an image acquisition step of acquiring an image of a face of a mountain tunnel on which the sprayed concrete has been applied; a face image acquisition step of extracting the face from the captured image using a learned face extraction model and acquiring a face image; a detection step of detecting cracks on the face of the construction site using the face image and a trained crack detection model; Includes.

[0007] Furthermore, in order to achieve the above object, the face monitoring program according to the present invention comprises: an image acquisition step of acquiring an image of a face of a mountain tunnel on which the sprayed concrete has been applied; a face image acquisition step of extracting the face from the captured image using a learned face extraction model and acquiring a face image; a detection step of detecting cracks on the face of the construction site using the face image and a trained crack detection model; to be executed by the computer. [Effects of the Invention]

[0008] According to the present invention, the working face can be adequately monitored. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram for explaining an overview of monitoring by a face monitoring device according to a preferred embodiment of the present invention. FIG. [Figure 2A] 1 is a block diagram for explaining the configuration of a working face monitoring device according to a preferred embodiment of the present invention. [Figure 2B] 1 is a diagram schematically showing an image captured by a working face monitoring device according to a preferred embodiment of the present invention. FIG. [Figure 2C]1 is a diagram showing an example of a face image acquired by a face monitoring device according to a preferred embodiment of the present invention. FIG. [Figure 2D] FIG. 10 is a diagram showing an example of the results of a face monitoring device according to a preferred embodiment of the present invention identifying cracks. [Figure 2E] FIG. 10 is a diagram showing an example of the results of the face monitoring device according to the preferred embodiment of the present invention identifying a foot line. [Figure 3A] FIG. 2 is a diagram for explaining an example of a risk level table included in the face monitoring device according to the preferred embodiment of the present invention. [Figure 3B] FIG. 2 is a diagram for explaining an example of a warning table included in the face monitoring device according to the preferred embodiment of the present invention. [Figure 4] 1 is a diagram for explaining the hardware configuration of a face monitoring device according to a preferred embodiment of the present invention. FIG. [Figure 5] 1 is a flowchart for explaining a processing procedure of a working face monitoring device according to a preferred embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described in detail by way of example with reference to the drawings. However, the configurations, numerical values, processing flows, functional elements, etc. described in the following embodiments are merely examples, and are open to modification and alteration, and are not intended to limit the technical scope of the present invention to the following description.

[0011] A face monitoring device 100 according to a preferred embodiment of the present invention will be described with reference to FIGS. 1 to 5. FIG. 1 is a diagram for explaining an overview of monitoring a face 111 by the face monitoring device 100 according to this embodiment. When constructing a mountain tunnel 110, the mountain tunnel 110 is excavated by repeatedly blasting the face. After blasting the face, and before the next blasting, sprayed concrete is applied, shoring (not shown) is installed, and a detonator 113 is placed on the face 111. The face monitoring device 100 is used to monitor the face 111 while preparations for the next blasting are being made, such as applying the shotcrete, installing the shoring, and installing the detonator 113. Specifically, the face monitoring device 100 is used to detect cracks on the face 111 of the mountain tunnel 110.

[0012] 1 shows a schematic diagram of a detonator 113 being inserted into a borehole 112 formed in a working face 111 on which shotcrete has been applied. The formation of the borehole 112 and the insertion of the detonator 113 are carried out by a work machine 130 such as a drill jumbo. The detonator 113 is connected to a blasting device 114 via a leg wire 113a.

[0013] In this embodiment, the tunnel face 111 is imaged by a camera 120 installed in the mountain tunnel 110. The tunnel face monitoring device 100 detects cracks on the tunnel face 111 based on the images captured by the camera 120. The camera 120 captures images of the tunnel face 111 in real time. In this embodiment, the camera 120 is disposed closer to the tunnel entrance than the work machine 130, and the images captured by the camera 120 include not only the tunnel face 111 but also the work machine 130. The position of the camera 120 may be adjusted so that the images captured by the camera 120 include only the tunnel face 111. Multiple cameras 120 may also be disposed. When multiple cameras 120 are disposed, it is preferable to dispose the cameras 120 so that the entire tunnel face 111 is evenly imaged.

[0014] 2A, the configuration of the working face monitoring device 100 will be described. The working face monitoring device 100 includes a captured image acquisition unit 201, a working face image acquisition unit 202, and a detection unit 203.

[0015] The captured image acquisition unit 201 acquires images captured by the camera 120 of the face 111 of the mountain tunnel 110 on which shotcrete has been applied. The camera 120 and the face monitoring device 100 are connected by wire or wirelessly, and the images captured by the camera 120 are transmitted to the face monitoring device 100 by wired communication or wireless communication. The captured image acquisition unit 201 acquires the images transmitted to the face monitoring device 100. Note that instead of transmitting images by wired communication or wireless communication, the images captured by the camera 120 may be stored in a storage medium such as a hard disk drive (HDD), a solid state drive (SSD), or a universal serial bus (USB) memory, and the captured image acquisition unit 201 may acquire the images from the storage medium.

[0016] As shown in FIG. 2B , the captured image 210 captured by the camera 120 may include the inner circumferential surface 211 of the mountain tunnel 110 in addition to the tunnel face 111. The captured image 210 may also include only the tunnel face 111. The captured image 210 may further include, for example, at least one of a worker, a mancage 140, or a work machine 130. In other words, the captured image 210 may be an image that captures an object or person present inside the mountain tunnel 110.

[0017] The face image acquisition unit 202 extracts the face 111 from the captured image 210 using the trained face extraction model, and acquires a face image. The trained face extraction model can be created by deep learning using a convolutional neural network with a large number of captured images of the face 111 as training data. As the training data, for example, a data set in which the inner circumferential surface 211 of the mountain tunnel 110, the worker, the man cage 140, the work machine 130, etc. are color-coded by pixel is used.

[0018] It is also possible to create a trained face extraction model by using two types of convolutional neural networks in stages. Specifically, in the first stage, an image of the face 111 is divided into 20 × 20 patch images. Next, each patch is assigned a label: 1 if it is the face 111, and 0 if it is not the face 111. A convolutional neural network, such as a Resnet18 model, is then used to make predictions. The convolutional neural network is then trained to minimize the discrepancy between the part predicted by the convolutional neural network as the face 111 and the part annotated by the operator as the face 111. In the second stage, the model trained in the first stage is used to extract feature vectors for an 8 × 8 part of the 20 × 20 patch image. The extracted feature vectors are then collectively predicted by another convolutional neural network, which is trained in the same way as in the first stage.

[0019] Using the trained face extraction model created in the two stages described above, the face can be extracted from test images of the face and the face image can be obtained. The face image can be extracted without mistaking the erector boom, head bolts, etc. for the face. In this way, the face image can be obtained. The match rate between the face extracted by the trained face extraction model created in the two stages described above and the part annotated as the face by the operator was 79% to 95%. Thus, the trained face extraction model demonstrates a high accuracy rate. Figure 2C shows an example of the results of face extraction using the trained face extraction model. The shaded area in Figure 2C is the extracted face 213. In addition to the face 111, the captured image shown in Figure 2C also shows the erector boom 214, the drill jumbo 215, and other elements. As shown in Figure 2C, by using a learned face extraction model, the face image acquisition unit 202 can accurately and reliably extract only the face 111 from the captured image 210, even if something other than the face 111 is captured in the captured image 210.

[0020] The detection unit 203 detects cracks 212 on the face 111 using a face image and a trained crack detection model. The trained crack detection model can be created by deep learning using a convolutional neural network with a large number of face images as training data. For example, a data set in which cracks 212 are color-coded for each pixel is used as the training data. While the trained crack detection model can detect cracks 212, the face image may contain both the crack 212 and the leg line 113a of the detonator 113. Typically, the crack 212 and the leg line 113a are both linear, and their appearances may be similar in the face image. Therefore, there is a risk that the cracks detected by the detection unit 203 using the trained crack detection model may include not only the crack 212 but also the leg line 113a.

[0021] Therefore, the detection unit 203 uses the trained crack detection model to identify the crack 212 and the foot line 113a of the detonator 113 of the explosive loaded on the working face 111, and extracts and detects the crack 212 from the identified crack 212 and foot line 113a. A trained crack detection model that can also identify the foot line 113a can be created by training the trained crack detection model that can detect the crack 212 described above with test specimen images and working face images as training data. Here, the test specimen image refers to an image of a concrete test specimen. Specifically, it is an image of a crack generated by loading the test specimen with a hydraulic jack. The face image is created by dividing the image of the face into 20 x 20 patch images, and labeling each patch as follows: crack 212 in red, leg line 113a in green, and face 111 without crack 212 or leg line 113a in black. Using these specimen images and face images, a trained crack detection model capable of detecting the above-mentioned crack 212 is made to predict cracks in the image. Then, the trained crack detection model is trained to minimize the discrepancy between the part predicted by the trained crack detection model to be crack 212 and the part annotated by the operator as crack 212. Similarly, the trained crack detection model is trained to minimize the discrepancy between the part predicted by the trained crack detection model to be leg line 113a and the part annotated by the operator as leg line 113a.

[0022] When the trained crack detection model created in this way was used to detect cracks 212 and leg lines 113a from test face images, the match rate with the parts annotated by the operator as cracks 212 was 81% to 97%. Furthermore, the match rate with the parts annotated by the operator as leg lines 113a was 86% to 100%. Thus, the trained crack detection model exhibits a high accuracy rate for both the detection of cracks 212 and leg lines 113a. Figure 2D shows an example of the results of identifying crack 212 using the trained crack detection model. In Figure 2D, line 212 is the extracted crack 212. Figure 2E shows an example of the results of identifying leg lines 113a using the trained detection model. In Figure 2E, line 113a is the extracted leg line 113a. Thus, the trained crack detection model can accurately identify cracks 212 and leg lines 113a. Therefore, the trained crack detection model can accurately detect the crack 212 without mistaking the leg line 113a for the crack 212.

[0023] The tunnel face monitoring device 100 may have a determination unit 204 that determines the degree of danger of the crack 212 detected by the detection unit 203. The determination unit 204 determines the degree of danger of the crack 212, for example, according to the location of the detected crack 212. Specifically, the determination unit 204 determines that the higher the height of the location of the crack 212, that is, the closer the location of the crack 212 is to the ceiling of the mountain tunnel 110, the higher the degree of danger. If the location of the crack 212 is high, there is a high possibility that workers, vehicles, etc. below the crack 212 will be caught in a skin fall, which is dangerous. Furthermore, the higher the location of the crack 212, the greater the gravity acting on the tunnel face 111, and therefore the higher the possibility of a skin fall occurring.

[0024] The determining unit 204 may also determine the risk of the cracks 212 by further taking into account the distribution of the cracks 212. For example, even if the height of the cracks 212 at the locations where the cracks 212 occur is the same, if the distribution of the cracks 212 is different, the risk is determined to be different. Specifically, if multiple cracks 212 are densely distributed with close intervals between them or if multiple cracks 212 intersect with each other, the risk is determined to be higher.

[0025] The determination unit 204 may also determine the risk of the crack 212 by further taking into account the length, width, growth rate, number, etc. of the crack 212. For example, the longer the length of the crack 212, the higher the risk is determined to be. Furthermore, the wider the crack 212, the higher the risk is determined to be. Furthermore, the growth rate of the crack 212 can be calculated, for example, as the amount of change in the length of the crack 212 over elapsed time. The faster the growth rate of the crack 212, the higher the risk is determined to be. Furthermore, the greater the number of cracks 212, the higher the risk is determined to be.

[0026] The working face monitoring device 100 may also have a warning unit 205 that issues a warning based on the degree of danger of the crack 212 determined by the determination unit 204. The warning issued by the warning unit 205 is issued, for example, from a speaker or a warning light attached to the inside of the mountain tunnel 110, to the worker, to the management building, etc. The warning may be a warning sound, a voice instruction, a warning light, vibration, etc. When vibration is used as a warning, for example, a mobile terminal such as a smartphone carried by the worker is vibrated. Furthermore, the warning unit 205 may display warning information on a display unit such as a liquid crystal display. Furthermore,

[0027] FIG. 3A is a diagram illustrating an example of a risk level table 301 possessed by the face monitoring device 100. The risk level table 301 is a table that stores risk levels 312 in association with crack levels 311. The "length" in the crack level 311 is the length of the crack 212. The "angle" in the crack level 311 is the angle between the extension direction of the crack 212 and the horizontal direction. The "shortest distance from the face center" in the crack level 311 is the shortest distance from the face center to the crack 212. The "growth rate" in the crack level 311 is the amount of change in the length of the crack 212 over time. The risk level 312 is a value determined according to the likelihood of collapse or spalling of the face 111. The determination unit 204 refers to the risk level table 301 to determine the risk level of the crack 212. Specifically, the determination unit 204 calculates the total value of the risk 312 for each item of the crack degree 311, and sets the calculated value as the risk of the crack 212. If there are multiple cracks 212, the total value of the risk of each crack 212 is set as the risk of the multiple cracks. By using the total value of the risk of each of the multiple cracks 212, it is possible to determine that the overall risk is high even when there are multiple cracks with low risk levels.

[0028] FIG. 3B is a diagram illustrating an example of the warning table 302 possessed by the working face monitoring device 100. The warning table 302 stores warning contents 322 in association with crack risk levels 321. The crack risk level 321 is the sum of the risk levels for each crack level. The warning contents 322 are warning contents corresponding to the risk level 321. Specifically, the warning contents 322 are "urgent evacuation" when the risk level is 100 or higher. When the risk level is 60 or higher but less than 100, they are "investigation of the tunnel required." When the risk level is 40 or higher but less than 60, they are "caution required." Here, "caution required" means that work is possible, but caution is required. When the risk level is less than 40, they are "none." The warning unit 205 refers to the warning table 302 and issues a warning corresponding to the risk level of the crack 212 calculated by the determination unit 204.

[0029] The hardware configuration of the face monitoring device 100 will be described with reference to FIG. 4. The CPU (Central Processing Unit) 410 is a processor for arithmetic and control, and executes programs to realize the various functional components of the face monitoring device 100 shown in FIG. 2A. The CPU 410 may have multiple processors and execute different programs, modules, tasks, threads, etc. in parallel. The ROM (Read Only Memory) 420 stores fixed data such as initial data and programs, as well as other programs. The network interface 430 communicates with other devices via a network. The CPU 410 is not limited to a single CPU, and may include multiple CPUs or a GPU (Graphics Processing Unit) for image processing. The network interface 430 preferably has a CPU independent of the CPU 410 and writes and reads transmission / reception data 445 to and from an area of ​​the RAM (Random Access Memory) 440. It is also preferable to provide a DMAC (Direct Memory Access Controller) (not shown) for transferring data between the RAM 440 and storage 450. Furthermore, the CPU 410 processes the data after recognizing that the data has been received or transferred to the RAM 440. The CPU 410 also prepares the processing results in the RAM 440, and leaves the subsequent transmission or transfer to the network interface 430 or DMAC.

[0030] The RAM 440 is a random access memory used by the CPU 410 as a work area for temporary storage. The RAM 440 has a memory area reserved for storing data necessary for realizing this embodiment. The captured image data 441 is data of the captured image 210 acquired by the captured image acquisition unit 201. The face image data 442 is data of the face image acquired by the face image acquisition unit 202 by extracting the face 111 from the captured image 210. The risk data 443 is data of the risk calculated by the determination unit. The warning content data 444 is data of the content of the warning to be given depending on the risk of the crack 212.

[0031] The transmitted / received data 445 is data that is transmitted and received via the network interface 430. The RAM 440 also has an application execution area 446 for executing various application modules.

[0032] The storage 450 stores a database, various parameters, or the following data or programs required to implement this embodiment. The storage 450 stores a risk table 301 and a warning table 302. The risk table 301 is a table that manages the relationship between the degree 311 of the crack 212 and the risk 312 shown in FIG. 3. The warning table 302 is a table that manages the relationship between the risk 321 and warning content 322 shown in FIG. 3.

[0033] The storage 450 further stores a captured image acquisition module 451, a face image acquisition module 452, and a detection module 453. The captured image acquisition module 451 is a module that acquires a captured image 210 of the face 111. The face image acquisition module 452 is a module that extracts the face 111 from the captured image 210 and acquires the face image. The detection module 453 is a module that detects cracks 212 on the face 111. The determination module 454 is a module that determines the risk of the crack 212. The warning module 455 is a module that issues a warning according to the risk of the crack 212. These modules 451 to 455 are read into the application execution area 446 of the RAM 440 by the CPU 410 and executed. The control program 456 is a program for controlling the entire face monitoring device 100.

[0034] The input / output interface 460 interfaces input / output data with input / output devices. A display unit 461 and an operation unit 462 are connected to the input / output interface 460. A storage medium 464 may also be connected to the input / output interface 460. A speaker 463 serving as an audio output unit, a microphone (not shown) serving as an audio input unit, or a GPS position determination unit may also be connected. Note that the RAM 440 and storage 450 shown in FIG. 4 do not include programs or data related to the general-purpose functions of the face monitoring device 100 or other feasible functions.

[0035] Next, the processing procedure of the working face monitoring device 100 will be described with reference to the flowchart shown in Fig. 5. This flowchart is executed by the CPU 410 in Fig. 4 using the RAM 440, and realizes each functional configuration of the working face monitoring device 100 in Fig. 2A.

[0036] In step S501, the captured image acquisition unit 201 acquires the captured image 210 of the face 111. In step S503, the face image acquisition unit 202 extracts the face 111 from the captured image 210 using the trained face detection model, and acquires the face image. In step S505, the detection unit 203 detects a crack 212 on the face 111 from the face image using the trained crack detection model. If a crack 212 is detected (step S507: YES), the face monitoring device 100 proceeds to step S509. In step S509, the determination unit determines the degree of risk according to the extent of the detected crack 212. In step S511, the warning unit issues a warning according to the degree of risk of the crack 212. Specifically, if the degree of risk of the crack 212 is equal to or greater than a predetermined value and a warning is necessary (step S511: YES), a warning is issued in step S513.

[0037] If the crack 212 is not detected (step S507: NO), the working face monitoring device 100 returns to step S501. Also, if the risk of the crack 212 is less than the predetermined value and a warning is not necessary (step S511: NO), the working face monitoring device 100 returns to step S501.

[0038] According to this embodiment, the cracks 212 on the tunnel face 111 are detected based on the captured image 210 of the tunnel face 111, allowing for adequate monitoring of the tunnel face 111. Therefore, workers performing excavation work on the mountain tunnel 110 can be notified of signs of a collapse or a skin break on the tunnel face 111. Knowing signs of a collapse or a skin break on the tunnel face 111 allows for measures to be taken in advance to prevent a collapse or a skin break, thereby preventing a disaster at the tunnel face of the mountain tunnel 110. It is also possible to urge workers inside the mountain tunnel 110 to evacuate. In this way, the tunnel face monitoring device 100 allows for safe excavation work on the mountain tunnel 110. Furthermore, since the captured image 210 is captured in real time, it is also possible to monitor changes in the cracks 212. Furthermore, since the cracks 212 and the leg lines 113a are identified in the tunnel face image using a trained crack detection model, it is possible to accurately detect only the cracks 212.

[0039] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments and can be modified as appropriate. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. Furthermore, systems or devices that combine separate features included in each embodiment in any manner are also included in the scope of the present invention. The present invention may also be applied to a system consisting of multiple devices or to a single device. Furthermore, the present invention may also be applied when an information processing program that realizes the functions of the embodiments is supplied to a system or device and executed by a built-in processor. Therefore, the technical scope of the present invention also includes a program installed on a computer to realize the functions of the present invention, a medium storing the program, a WWW (World Wide Web) server from which the program is downloaded, and a processor that executes the program. In particular, the technical scope of the present invention also includes a non-transitory computer-readable medium storing a program that causes a computer to execute at least the processing steps included in the above-described embodiments.

Claims

1. an image acquisition unit that acquires an image of a tunnel face on which the shotcrete has been applied; a face image acquisition unit that extracts the face from the captured image using a learned face extraction model and acquires a face image; A detection unit that detects cracks on the face of the construction work using the face image and a learned crack detection model; A face monitoring device equipped with

2. The face monitoring device of claim 1, wherein the detection unit uses the trained crack detection model to identify the crack and the leg line of the detonator of the explosive loaded on the face, and extracts and detects the crack from the identified crack and leg line.

3. The working face monitoring device according to claim 1 or 2, wherein the captured image further includes at least one of an operator, a man cage, or a work machine.

4. The working face monitoring device according to claim 1 or 2, wherein the captured images are images captured in real time.

5. an image acquisition step of acquiring an image of a face of a mountain tunnel on which the sprayed concrete has been applied; a face image acquisition step of extracting the face from the captured image using a learned face extraction model and acquiring a face image; a detection step of detecting cracks on the face of the construction site using the face image and a trained crack detection model; A face monitoring method comprising:

6. an image acquisition step of acquiring an image of a face of a mountain tunnel on which the sprayed concrete has been applied; a face image acquisition step of extracting the face from the captured image using a learned face extraction model and acquiring a face image; a detection step of detecting cracks on the face of the construction site using the face image and a trained crack detection model; A face monitoring program that causes a computer to execute the following.

Images