Safety inspection system and safety inspection method

Abstract

To improve safety in inspection work after blasting.SOLUTION: The safety inspection system 1 according to the embodiment includes a flying object mounted with an imaging unit and capable of unmanned flight, an acquisition unit 240 that causes the flying object to fly and acquires an image of a construction site after blasting captured by the imaging unit, a determination unit 250 that determines whether or not a target object is present at the construction site based on the image acquired by the acquisition unit 240, and a provision unit 260 that provides a user with an image indicating a position of the target object when the determination unit 250 determines that the target object is present.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a safety inspection system and a safety inspection method. [Background technology]

[0002] In the past, at construction sites such as tunnel construction, ore mining, and excavation, work-related accidents have frequently occurred due to falling rocks and other debris from the walls (including the tunnel face) of tunnels and other structures being excavated. In particular, manual inspection work after blasting involves approaching the tunnel face after blasting to remove unexploded residual explosives, which can result in rocks falling from the wall during work and leading to fatal accidents. Therefore, technologies have been proposed to prevent falling rocks from the tunnel face during inspections after blasting.

[0003] For example, Patent Document 1 proposes a technology that includes a mancage attached to the tip of the boom of a heavy earth excavation machine and a face ground support body consisting of a flexible bag placed at the tip of the mancage, in which the mancage is brought closer to an appropriate position on the face according to the extension and angle adjustment of the boom, and then fluid is pumped into the face ground support body, causing the face ground support body to expand and press against the uneven shape of the face, thereby supporting the face directly above the worker. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 10-219690 Summary of the Invention [Problem to be solved by the invention]

[0005] However, because conventional ground excavation heavy machinery is a vehicle, it may not be able to travel on ground that is littered with rubble after blasting. Furthermore, there is a possibility that unexploded residual explosives may remain at the site, making it impossible to carry out safe inspection work after blasting. Therefore, a system that allows for safer inspection work after blasting has been desired.

[0006] The present invention has been made in consideration of the above circumstances, and aims to provide a safety inspection system and a safety inspection method that can further improve safety in inspection work after blasting. [Means for solving the problem]

[0007] The present invention has been made to solve the above-mentioned problems, and one aspect of the present invention is a safety inspection system comprising an aircraft equipped with an imaging unit, an acquisition unit that flies the aircraft and acquires images of a construction site after blasting that are captured by the imaging unit, a determination unit that determines whether a target object is present at the construction site based on the images acquired by the acquisition unit, and a provision unit that, when the determination unit determines that the target object is present, provides a user with an image showing the position of the target object.

[0008] Another aspect of the present invention is a safety inspection system that includes an aircraft equipped with an object detection unit that detects surrounding objects, an acquisition unit that flies the aircraft and acquires the object detection results of a construction site after blasting detected by the object detection unit, a determination unit that determines whether or not a target portion exists at the construction site based on the object detection results acquired by the acquisition unit, and a provision unit that, when the determination unit determines that the target portion exists, provides a user with an image showing the position of the target portion.

[0009] Another aspect of the present invention is an aircraft equipped with an imaging unit and an object detection unit that detects surrounding objects; This safety inspection system includes an acquisition unit that flies the aircraft and acquires images of the construction site after blasting captured by the imaging unit and object detection results of the construction site detected by the object detection unit; a determination unit that determines whether at least one of a target object and a target part is present at the construction site based on the image acquired by the acquisition unit and the object detection results; and a provision unit that, when the determination unit determines that at least one of the target object and the target part is present, provides a user with an image showing the position of the present target object or target part.

[0010] Another aspect of the present invention is a safety inspection method in which a computer flies an aircraft equipped with an imaging unit, acquires images of a construction site after blasting captured by the imaging unit, determines whether or not a target object is present at the construction site based on the acquired images, and, if it is determined that the target object is present, provides a user with an image showing the position of the target object.

[0011] Another aspect of the present invention is a safety inspection method in which a computer flies an aircraft equipped with an object detection unit that detects surrounding objects, obtains object detection results for a construction site after blasting detected by the object detection unit, determines whether or not a target area exists at the construction site based on the obtained object detection results, and if it is determined that the target area exists, provides a user with an image showing the position of the target area. [Effects of the Invention]

[0012] According to the present invention, safety during inspection work after blasting can be further improved. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram illustrating an example of a schematic configuration of a safety inspection system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the functional configuration of the drone device 100 according to the embodiment. [Figure 3]FIG. 2 illustrates an example of a functional configuration of a user terminal 200 according to an embodiment. [Figure 4] FIG. 10 is a diagram for explaining how a safety inspection is carried out in the embodiment. [Figure 5] 10A and 10B are diagrams for explaining determination of falling off of a wall portion. [Figure 6] FIG. 10 is a diagram showing a first example of a provided image. [Figure 7] FIG. 10 is a diagram showing a second example of a provided image. [Figure 8] FIG. 10 is a diagram showing a third example of a provided image. [Figure 9] FIG. 10 is a diagram showing a fourth example of a provided image. [Figure 10] 3 is a sequence diagram showing an example of processing executed by the safety inspection system 1. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments of the safety inspection system and safety inspection method of the present invention will be described with reference to the drawings. In the following description, as an example of the safety inspection system, a safety inspection system for carrying out safety inspection work after blasting at a specific construction site will be described. A specific construction site is, for example, a site that has a space such as a tunnel or building that is at least partially surrounded by walls (including not only side walls but also the ceiling and face, etc.) and where mining, excavation, demolition, etc. are carried out in that space. In the following, as an example of a construction site, a safety inspection system will be described for a construction site (tunnel construction site) where tunnel blasting excavation is carried out.

[0015] <Outline of the safety inspection system> Fig. 1 is a diagram illustrating an example of a schematic configuration of a safety inspection system according to an embodiment. The safety inspection system 1 illustrated in Fig. 1 includes, for example, a drone device 100 and a user terminal 200. The drone device 100 and the user terminal 200 are communicably connected via, for example, a network NW. The network NW is, for example, a Wi-Fi network, Bluetooth (registered trademark), the Internet, a cellular network, a LAN (Local Area Network), a WAN (Wide Area Network), or the like.

[0016] The drone device 100 is an unmanned aerial vehicle (UAV) that can be remotely controlled by operating instructions from the user terminal 200 or automatically controlled along a preset flight route. In the embodiment, the drone device 100 may be replaced by a helicopter, a balloon, an airplane, an airship, or other flying object.

[0017] 1 includes, for example, a LiDAR (Light Detection And Ranging) 110 and a camera 120, and transmits information acquired by the LiDAR 110 and the camera 120 to the user terminal 200. The LiDAR 110 is an example of an "object detection unit." The camera 120 is an example of an "imaging unit."

[0018] The user terminal 200 may be, for example, a tablet terminal or a smartphone, or may be a dedicated controller equipped with another display unit and capable of remotely controlling the drone device 100. The user terminal 200 is used, for example, by a user U who performs safety inspection work at a tunnel construction site. The user terminal 200 transmits operation instructions from the user U to the drone device 100 and receives information acquired by the LiDAR 110 and camera 120 of the drone device 100. The functional configurations of the drone device 100 and the user terminal 200 will be specifically described below.

[0019] <Drone Device 100> 2 is a diagram illustrating an example of the functional configuration of the drone 100 according to the embodiment. The drone 100 includes, for example, a LiDAR 110, a camera 120, a light emitting unit 130, a flight unit 140, a communication unit 150, a control unit 160, a battery 170, and a storage unit 180.

[0020] The LiDAR 110 irradiates the periphery of the drone device 100 with light (or electromagnetic waves with a wavelength similar to that of light) and measures the scattered light. The irradiated light is, for example, pulsed laser light. The LiDAR 110 detects the distance to a surrounding object (including a wall, the ground, etc.) based on the time between light emission and light reception. Furthermore, the LiDAR 110 detects point cloud data (three-dimensional data) indicating the surface of the surrounding object as an object detection result based on the detected distance. The LiDAR 110 is attached to any location on the drone device 100 (for example, on the front of the drone device 100 (in the +X-axis direction shown in FIG. 1)). Note that in the embodiment, an object detection device other than the LiDAR 110 may be used as long as it is a device that can detect the shape of a surrounding object.

[0021] The camera 120 includes, for example, a first camera 122 and a second camera 124. The first camera 122 and the second camera 124 are high-resolution digital cameras that use solid-state imaging elements such as a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The first camera 122 is installed, for example, on the right side (the Y-axis direction shown in FIG. 1) or the left side (the −Y-axis direction shown in FIG. 1) of the front of the drone device 100. Alternatively, the first camera 122 may be installed to the side of the LiDAR 110. The first camera 122 captures an image of a predetermined area (first area) including a side of the drone device 100 (for example, the direction of arrow A1 shown in FIG. 1). The first area includes at least an area below the drone device 100 (the −Z-axis direction shown in FIG. 1).

[0022] The second camera 124 is installed above the drone 100 (in other words, above the LiDAR 110) and captures an image of a predetermined area (second area) including a diagonally upward direction of the drone 100 (for example, the direction of arrow A2 shown in FIG. 1). The second area includes at least the area above the drone 100 (the Z-axis direction shown in FIG. 1). In other words, by superimposing an image of the first area and an image of the second area, it is possible to obtain an image of at least an angle of 180° or more to the right (or left) side of the drone 100. The first camera 122 and the second camera 124 may be installed anywhere on the drone 100 as long as they can capture images of the first area and the second area. The camera 120 may have three or more cameras, and a fisheye lens camera (omnidirectional camera) may be installed to capture images of the surroundings (360°) of the camera. The camera 120, for example, periodically and repeatedly captures images of the surroundings of the drone device 100 (first area and second area).

[0023] The light emitting unit 130 emits light (light source) in the shooting direction of the camera 120 mounted on the drone device 100. This makes it possible to acquire an image in a situation where the illuminance is equal to or greater than a predetermined level even in a dark environment.

[0024] The flight unit 140 flies within a tunnel at a tunnel construction site under the control of the control unit 160 based on operation instructions from the user U and a flight route for automatic control, etc., acquired from the user terminal 200. Here, the flight unit 140 includes, for example, a rotor, a motor, and an ESC (Electric Speed ​​Controller). For example, multiple rotors are installed on the drone device 100, and a motor rotor is connected to each rotor. The motor is, for example, a brushless motor. The ESC adjusts the power supplied to the motor in response to instructions from the control unit 160. This individually adjusts the rotation speed of each rotor, allowing the drone device 100 to fly in a desired direction and with a desired attitude. The flight unit 140 enables the drone device 100 to move up and down, forward and backward, left and right, turn, and perform a combination of these movements.

[0025] The communication unit 150 communicates with the user terminal 200, for example, via the network NW. For example, under the control of the control unit 160, the communication unit 150 receives instruction information (for example, steering instructions and instructions to operate the LiDAR 110 and the camera 120) from the user terminal 200, and transmits information detected by the LiDAR 110 and image information captured by the camera 120 to the user terminal 200.

[0026] The control unit 160 controls all of the components of the drone device 100. The control unit 160 includes, for example, a communication control unit 162, a flight control unit 164, and an execution control unit 166. The control unit 160 is realized, for example, by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). The control unit 160 may also be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device with a non-transitory storage medium) such as an HDD (Hard Disk Drive) or flash memory of the drone device 100, or may be stored in a removable storage medium such as a memory card or USB (Universal Serial Bus) memory, and installed in the drone device 100 by attaching the storage medium (non-transitory storage medium) to the drone device 100.

[0027] The communication control unit 162 performs control for communicating with the user terminal 200 via the communication unit 150. For example, if the information received by the communication unit 150 is information related to the flight (pilot instructions) of the drone device 100, the communication control unit 162 outputs the information to the flight control unit 164, and if the information is information related to the operation (on / off) of the LiDAR 110 or the camera 120, the communication control unit 162 outputs the information to the execution control unit 166.

[0028] The flight control unit 164 controls the ESC so that the drone 100 flies in accordance with, for example, a control command acquired from the user terminal 200. Furthermore, the flight control unit 164 may perform contact avoidance flight to avoid contact with a surrounding object when the distance between the drone 100 and a surrounding object becomes within a predetermined distance based on the object detection result detected by the LiDAR 110.

[0029] The execution control unit 166 controls detection by the LiDAR 110 based on control information such as the start (ON) and end (OFF) of detection by the LiDAR 110 acquired from the user terminal 200. The execution control unit 166 also controls image capture by the first camera 122 and the second camera 124 based on control information such as the start (ON) and end (OFF) of image capture by the camera 120 acquired from the user terminal 200. The execution control unit 166 may also control the light emitting unit 130 to emit light (turn on the light source) during image capture. The execution control unit 166 may also control the LiDAR 110 and the camera 120 to start operation when it detects that the drone device 100 has started flight, and to end operation of the LiDAR 110 and the camera 120 when it detects that the drone device 100 has ended flight.

[0030] The battery 170 is, for example, a secondary battery such as a lithium-ion battery. The battery 170 is a power supply unit that supplies power to each unit of the drone device 100. The battery 170 can be charged by connecting an adapter and a commercial power source to a terminal (not shown). The battery 170 is detachable from the drone device 100 and can be replaced with another battery 170.

[0031] The storage unit 180 may be realized by the various storage devices described above, or a solid state drive (SSD), an electrically erasable programmable read only memory (EEPROM), a read only memory (ROM), or a random access memory (RAM). The storage unit 180 stores, for example, programs and various other information. The storage unit 180 may also temporarily store information acquired by the LiDAR 110 and the camera 120, and may store a flight route for automatic control.

[0032] In addition to the above-described configuration, the drone 100 may include, for example, an internal positioning unit such as a Global Navigation Satellite System (GNSS) receiver. The GNSS receiver identifies the location of the drone 100 based on, for example, signals received from GNSS satellites. For example, the drone 100 may fly autonomously according to a flight route received from the user terminal 200 based on the location information.

[0033] <User terminal 200> 3 is a diagram illustrating an example of the functional configuration of a user terminal 200 according to an embodiment. The user terminal 200 includes, for example, a communication unit 210, an input unit 220, an output unit 230, an acquisition unit 240, a determination unit 250, a provision unit 260, a control unit 270, an application execution unit 280, and a storage unit 290. The acquisition unit 240, the determination unit 250, the provision unit 260, the control unit 270, and the application execution unit 280 are realized by a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI, ASIC, FPGA, or GPU, or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device with a non-transitory storage medium) such as an HDD or flash memory of the user terminal 200, or may be stored in a removable storage medium such as a memory card or USB memory, and installed in the user terminal 200 by inserting the storage medium (non-transitory storage medium) into the user terminal 200.

[0034] The communication unit 210 communicates with the drone 100 via the network NW. The communication unit 210 may also communicate with other external devices (for example, server devices) that can be connected via the network NW.

[0035] The input unit 220 receives input from the user U, for example, by operating various keys, buttons, etc. The input unit 220 may also include a microphone that receives voice input from the user U. The input unit 220 receives input, for example, instructions for operating the drone device 100, flight route, and instructions for operating the LiDAR 110 and the camera 120.

[0036] The output unit 230 includes, for example, a display unit and a speaker, and outputs predetermined information to the display unit and the speaker. The display unit is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) display. The input unit 220 may be configured integrally with the display unit as a touch panel. The display unit displays various types of information in the embodiments. The speaker outputs predetermined sounds. For example, the output unit 230 outputs images and sounds corresponding to information received by the communication unit 210, information processed by the user terminal 200, and information input by the input unit 220.

[0037] The acquisition unit 240 flies the drone device 100 by remote control from the user terminal 200 via the communication unit 210, and acquires the object detection results of the tunnel construction site after blasting detected by the LiDAR 110. The acquisition unit 240 also flies the drone device 100 and acquires images of the tunnel construction site after blasting taken by the camera 120 (hereinafter referred to as camera images).

[0038] The determination unit 250 determines whether or not a predetermined target portion exists in the tunnel construction site based on the object detection results acquired by the acquisition unit 240. The target portion is, for example, a portion of the wall (including not only the side walls but also the ceiling and face) of the tunnel construction site where there is a loss (a fallen surface, or fallen rock blocks, earth, etc.). The determination unit 250 also determines whether or not a target object exists in the tunnel construction site based on the camera image acquired by the acquisition unit 240. The target object is, for example, at least one of a crack in the wall of the tunnel construction site, gunpowder (unexploded gunpowder residue), and a detonator (including a fuse). The target object may also include rubble (rubble, broken rock), etc. The function of the determination unit 250 will be described in detail below.

[0039] The providing unit 260 generates information to be provided, such as images and sounds, for providing the result determined by the determining unit 250 to the user U, and transmits the generated information to the user terminal 200 to provide it to the user U. Details of the information provided by the providing unit 260 will be described later. In addition, the providing unit 260 may store information acquired from the drone device 100, the determination result by the determining unit 250, etc. in the storage unit 290, or transmit them to an external device via the communication unit 210.

[0040] The control unit 270 controls all of the functions of the user terminal 200. For example, the control unit 270 performs communication control by the communication unit 210, input / output control by the input unit 220 and the output unit 230, acquisition control by the acquisition unit 240, determination control by the determination unit 250, provision control by the provision unit 260, and execution control of the safety inspection application 292 by the application execution unit 280.

[0041] The application execution unit 280 is realized by executing a safety inspection application 292 stored in the storage unit 290. The safety inspection application 292 is, for example, downloaded from an external device via the network NW and installed in the user terminal 200. The safety inspection application 292 is application software that communicates with the drone device 100, transmits control instructions to the drone device 100, transmits operation instructions to the LiDAR 110 and the camera 120, and receives information acquired by the LiDAR 110 and the camera 120. The safety inspection application 292 also causes the acquisition unit 240 to execute an acquisition process, the determination unit 250 to execute a determination process, and the provision unit 260 to execute a provision process. Note that the transmitted information may be assigned identification information (e.g., a terminal ID) that identifies the user terminal 200, identification information (e.g., a user ID) that identifies the user U, or the like.

[0042] The storage unit 290 may be realized by the various storage devices described above, or an SSD, an EEPROM, a ROM, a RAM, or the like. The storage unit 290 stores, for example, a safety inspection application 292, object feature information 294, a trained model 296, a program, and various other information. The object feature information 294 and the trained model 296 are used in the determination process by the determination unit 250, and details will be described later. The storage unit 290 may also store information acquired by the LiDAR 110 and the camera 120.

[0043] <Specific examples of safety inspection work> Next, a specific example of a safety inspection operation in an embodiment will be described. FIG. 4 is a diagram for explaining how a safety inspection is performed in an embodiment. The example in FIG. 4 shows the state inside a tunnel 300, which is a construction site. In the example in FIG. 4, explosives placed near a tunnel face 310 of the tunnel 300 are detonated, followed by ventilation, and after ventilation is completed, a user U remotely flies the drone 100 using the user terminal 200 from a position at least a predetermined distance away from the tunnel face 310. The user terminal 200 also receives information acquired from the LiDAR 110 and camera 120 mounted on the drone 100 during flight, and inspects the safety inside the tunnel 300, including the vicinity of the tunnel face 310, based on the received information.

[0044] For example, the user terminal 200 remotely controls the drone device 100 to fly inside the tunnel 300, captures images of the post-blasting situation with the camera 120, and acquires detection information (e.g., point cloud data) of surrounding objects with the LiDAR 110. While flying, the drone device 100 captures images of the side walls (side surfaces) 320 and ground 330 of the tunnel 300 with the first camera 122, and captures images of the side walls 320 and ceiling of the tunnel 300 with the second camera 124.

[0045] For example, the user U remotely controls the drone 100 to move from the user U side (a position away from the tunnel face 310) to near the tunnel face 310 (within a predetermined distance) along the flight route FR shown in Fig. 4, and then makes a U-turn (for example, a 180° turn) to return to the user U side. By flying this flight route FR, for example, if the first camera 122 and the second camera 124 are installed in the positions shown in Fig. 1, when the drone 100 moves from the user U side to near the tunnel face 310, it can capture images of the ground 330, side wall 320, and top surface (ceiling) on ​​the right side of the tunnel 300 when looking in the direction of the tunnel face 310 from the position of the user U shown in Fig. 4, and can capture images of the area near the tunnel face 310 during the U-turn (turning), and can capture images of the ground 330, side wall 320, and top surface (ceiling) on ​​the left side of the tunnel 300 when making a U-turn and returning to the position of the user U. Furthermore, since the LiDAR 110 is installed in front of the drone device 100 (in other words, in the direction of travel), the shape of objects around the tunnel 300 can be recognized by flying based on the flight route FR described above.

[0046] Here, the providing unit 260 outputs (displays) on the output unit 230 an image captured by the camera 120 of the drone device 100 during flight, thereby making it easier for the user U to recognize the state of the rubble 340 in the tunnel 300. The providing unit 260 also outputs (displays) on the output unit 230 an image including target objects such as cracks C1 and C2, gunpowder (unexploded residual gunpowder) E1, and detonator (including fuse F1) D1 determined by the determining unit 250. The providing unit 260 also outputs (displays) on the output unit 230 an image showing the result of the determination of whether the wall portion (including the face 310, side wall 320, ceiling, etc.) has fallen out, determined by the determining unit 250.

[0047] <Judgment section 250> Next, a specific example of the processing of the determination unit 250 of the embodiment will be described. For example, the determination unit 250 performs existing image analysis processing on camera images captured by the first camera 122 and the second camera 124 to derive feature information for each object included in the image. The feature information includes, for example, color information, contour information, and shape information. For example, the determination unit 250 extracts color information (RGB information) for each pixel of the camera image and extracts objects based on color difference information from surrounding pixels. The determination unit 250 also extracts edge points from the camera image using existing image analysis processing and derives the contour of each object by connecting edge points that exist within a predetermined distance. The determination unit 250 may also acquire color information for each edge point and derive the contour by connecting edge points with similar acquired colors. The determination unit 250 also derives the shape, size, and the like of each object surrounded by the contour.

[0048] Next, the determination unit 250 uses the feature information obtained by these processes to compare it with object feature information 294 previously stored in the storage unit 290, and extracts a target object according to the feature information by pattern matching or the like. Here, the object feature information 294 has feature information associated with each target object (e.g., crack, gunpowder, detonator). If a target object that matches the derived feature information (has a similarity equal to or greater than a threshold) exists, the determination unit 250 determines that the target object exists.

[0049] Furthermore, instead of (or in addition to) the above-described determination process, the determination unit 250 may determine whether or not a target object exists in a captured camera image using a trained model 296 that has been trained in advance by machine learning using AI (Artificial Intelligence) technology, using an image of the target object (ground truth data) as training data. This trained model 296 is, for example, a learning model that receives a camera image as input and outputs whether or not a pre-specified target object exists and, if so, its position (image area).

[0050] The object feature information 294 and the trained model 296 described above are, for example, acquired in advance from an external device via the network NW. The object feature information 294 and the trained model 296 may also be prepared for each construction site. In particular, the trained model 296 may be trained for each construction site. For example, the shape and structure of the tunnel 300 and the like differ depending on the construction site, as do the ambient brightness and the types of explosives E1 and detonators D1 used at the site. Furthermore, the materials, qualities, strength, etc. contained in the walls and the like also differ. Therefore, by using the trained model 296 trained using correct answer data from the construction site where the actual judgment is performed as training data, the presence or absence of a target object can be estimated with greater accuracy.

[0051] Furthermore, the determination unit 250 determines whether or not a rock mass or the like has fallen out of the wall portion based on the object detection result detected by the LiDAR 110. For example, the determination unit 250 derives the average cross section of the tunnel 300 based on the object detection result in the tunnel (space) 300 by the LiDAR 110, and determines whether or not the wall portion has fallen out based on the degree of expansion of the portion where the space expands outward from the derived average cross section.

[0052] FIG. 5 is a diagram for explaining the determination of wall section dropout. For example, the determination unit 250 derives the average of the cross sections from an arbitrarily set start point to an end point in the space inside the tunnel 300, with the tunnel face 310 in front. That is, the determination unit 250 derives the average (i.e., average cross section CS) of the cross section (YZ plane) of the tunnel 300 in a predetermined section in the X-axis direction shown in FIG. 4. The average cross section CS may be derived from a camera image in addition to (or instead of) the point cloud data. Next, when the derived average cross section CS is extended toward the tunnel face 310, the determination unit 250 detects a portion (area AR1) that extends further toward the natural ground than the average cross section CS (outside the contour shape of the average cross section CS as viewed from inside the tunnel 300). The determination unit 250 then derives the degree of expansion of the area AR1, and if the derived degree of expansion is equal to or greater than a threshold, determines that rock masses or the like have fallen off (fell off) from the wall (for example, the excavation surface of the working face 310, etc.). The degree of expansion is, for example, the size [m 3 ], or may be a value that increases in accordance with the size of the area of ​​a two-dimensional region AR1 as shown in FIG.

[0053] The determination unit 250 may derive the amount, position, etc. of the shear 340 based on feature information of the object obtained from the camera image, the object detection result, etc.

[0054] <Example of provided image> Next, an example of a provided image provided to the user U by the providing unit 260 will be described with reference to the drawings. FIG. 6 is a diagram showing a first example of a provided image. Note that the content, type, size, position, etc. displayed in the provided image are not limited to the example in FIG. 6. The same applies to other examples of provided images described later.

[0055] Image IM10 shown in FIG. 6 includes an image of explosive E1. When the determination unit 250 determines that explosive E1 is present, the providing unit 260 displays a range image IM12 indicating the range (position) of explosive E1 superimposed on image IM10 so as to surround the image area (outline) of explosive E1. The shape (frame shape) of range image IM12 is, for example, rectangular (quadrilateral). By providing this image to user U via user terminal 200, it is possible to make it easier for user U to more clearly understand that determination unit 250 has detected the presence of explosive E1 and the position of explosive E1.

[0056] Fig. 7 is a diagram showing a second example of a provided image. Image IM20 shown in Fig. 7 includes a detonator D1. When the determination unit 250 determines that the detonator D1 is present in image IM20, the providing unit 260 displays a range image IM22 superimposed on image IM20 so as to surround the detonator D1 portion. In the example of Fig. 7, since a fuse F1 is connected to the detonator D1, if it is determined that the fuse F1 is also present, the providing unit 260 displays a range image IM22 superimposed on image IM20 so as to surround the image region where the detonator D1 and fuse F1 are present.

[0057] This makes it easier for the user U to grasp the approximate position of the detonator D1. In particular, in the embodiment, the presence or absence of a target object (explosive powder E1 or detonator D1) is determined from real-time images (moving images) obtained from the drone device 100 during flight, and the presence or absence is notified to the user U, so the range images IM12 and IM22 are generated in simple shapes such as rectangles, rather than complex shapes. This reduces the image processing load, and enables image information to be provided more quickly.

[0058] Furthermore, the providing unit 260 displays the range images IM12 and IM22 in a color different from the background color of the images IM10 and IM20. This makes it easier for the user U to recognize the position of the target object included in the camera image from the range images IM12 and IM22. Furthermore, the providing unit 260 may display the range images IM12 and IM22 in different display modes for each type of target object. Different display modes include, for example, different frame line colors, different line types, or different line patterns. This allows the user to easily understand the type of enclosed target object depending on the display mode of the range images.

[0059] In the embodiment, the shape of the range image may be an ellipse, a circle, or another polygon (including a concave-convex shape) instead of a rectangle. In this case, the providing unit 260 varies the shape of the range image when displaying the range image in a different display mode depending on, for example, the type of target object or the state of the image.

[0060] Fig. 8 is a diagram showing a third example of a provided image. Image IM30 shown in Fig. 8 includes three detonators D1 to D3, and corresponding fuses F1 to F3 are connected to the detonators D1 to D3, respectively. When the determination unit 250 determines that the three detonators D1 to D3 and the fuses F1 to F3 are present in image IM30, the providing unit 260 generates elliptical area images IM31 to IM33 so as to surround the detonator D1 and the fuse F1, the detonator D2 and the fuse F2, and the detonator D3 and the fuse F3, respectively, and displays them superimposed on image IM30.

[0061] FIG. 9 is a diagram showing a fourth example of a provided image. Image IM40 shown in FIG. 9 includes two cracks C1 and C2. When the determination unit 250 determines that cracks C1 and C2 are present in image IM40, the providing unit 260 generates range images IM41 and IM42 to surround the cracks C1 and C2, respectively, and displays them superimposed on image IM40. Note that the range images IM41 and IM42 shown in FIG. 9 have polygonal shapes corresponding to the shapes of the cracks C1 and C2. These range images IM41 and IM42 are, for example, within a processing distance from the images (line segments) showing the cracks C1 and C2, and have shapes that abstract the shapes of the cracks C1 and C2 so that the processing load for generating the range images IM41 and IM42 is less than a predetermined value.

[0062] For example, as shown in Figures 8 and 9, by generating a range image in a shape other than a rectangle, it is possible to make the range image easier for the user U to recognize, for example, in a situation where the camera image contains many vertical or horizontal line segments.

[0063] Furthermore, when the determination unit 250 estimates that a rock mass has fallen from the wall (or ceiling) based on the object detection result of the LiDAR 110 as shown in FIG. 5, the provision unit 260 may provide the user U with an area image surrounding the fallen area by superimposing it on the image shown in FIG. 5 or on the camera image.

[0064] Furthermore, the providing unit 260 may output audio information (warning sounds) in addition to the provided image according to the level of urgency at the construction site obtained from the camera image and object detection results, or may highlight the image according to the level of urgency. Highlighting may involve, for example, flashing the range image, displaying it in a highly visible color, displaying the interior area of ​​the range image in a predetermined color, or displaying text information such as "Warning." For example, if the length (size) of a crack in the image exceeds the estimated limit value for the possibility of wall collapse, or if the area (or extent) determined to be missing exceeds the limit value, the level of urgency may be deemed to be above a threshold, and an alarm sound may be output or the image may be highlighted. This allows the user U to sense danger early and take safer action, such as evacuation.

[0065] According to the above-described embodiment, for example, during post-blasting inspection work in blast excavation, images captured by the drone device 100 can be viewed on the user terminal 200, allowing for safer inspection of the surrounding area from outside of restricted areas such as near the excavation face. Furthermore, according to the embodiment, the stability of the ground surrounding the excavation face can be determined based on the amount of rock mass that has fallen from the wall. Furthermore, according to the present invention, for example, the presence or absence of unexploded residual explosives on the surface of the rubble pile can be determined, and the presence or absence of unexploded holes and detonators on the excavation face can also be determined from the images. Based on the above, this technology can be used as a core technology for efforts to unmann and automate each construction cycle, and its widespread use in construction sites such as mountain tunnels can prevent serious accidents. Therefore, safety during post-blasting inspection work can be further improved.

[0066] Furthermore, according to the embodiment, a high-definition camera and LiDAR can be used to obtain more accurate determination results, and the surrounding conditions can be confirmed in a short time by flying the drone device 100, thereby further improving productivity.

[0067] In addition, during safety inspection work using the safety inspection system 1, if it is determined that a target object is present, an inspection is carried out at the location specified in the range image, and if it is not determined that a target object is present, safety within the tunnel 300 is confirmed, and the work (construction) for the next process is carried out.

[0068] <Processing sequence> Next, an example of processing executed by the safety inspection system 1 in this embodiment will be described. Fig. 10 is a sequence diagram showing an example of processing executed by the safety inspection system 1. In the following description, of the various processes executed by the safety inspection system 1, a processing sequence using the drone device 100 and the user terminal 200 will be mainly described.

[0069] In the example of Figure 10, the user terminal 200 receives instruction information (e.g., a control instruction, an operation instruction, etc.) from the user U (step S100) and transmits the received information to the drone device 100 (step S110).

[0070] The drone 100 receives instruction information from the user terminal 200 and performs flight according to the received instruction information (pilot instruction) (step S120). Next, the drone 100 captures an image of a predetermined construction site using the camera 120 during flight (step S130). The drone 100 also performs object detection around the construction site using the LiDAR 110 during flight (step S140). Note that at least one of the processes of step S130 and step S140 may be executed. Next, the drone 100 transmits the image capture results from the camera 120 or the detection results (point cloud data) from the LiDAR 110 to the user terminal 200 (step S150).

[0071] The user terminal 200 determines whether or not a target object exists in the image based on the image obtained from the drone device 100 (step S160). Next, the user terminal 200 determines whether a rock mass or the like has fallen from the wall of the construction site based on the object detection result by the LiDAR 110 (step S170). Next, the user terminal 200 generates an image based on the processing results of steps S160 and S170 (step S180), and outputs the generated image to the output unit 230 (step S190). This ends the processing of this sequence.

[0072] <Modification> In the embodiment, the drone device 100 may be configured to be equipped with only either the LiDAR 110 or the camera 120. For example, when the drone device 100 is equipped with only the camera 120 (i.e., when the drone device 100 is not equipped with the LiDAR 110), the acquisition unit 240 acquires an image of the construction site after blasting captured by the camera 120, the determination unit 250 determines whether or not a target object exists at the construction site based on the acquired image, and the provision unit 260 provides the user U with an image indicating the location of the target object if it is determined that the target object exists. Also, when the drone device 100 is equipped with only the LiDAR 110 (i.e., when the drone device 100 is not equipped with the camera 120), the acquisition unit 240 acquires an object detection result of the construction site after blasting detected by the LiDAR 110, the determination unit 250 determines whether or not a target portion exists at the construction site based on the acquired object detection result, and the provision unit 260 provides the user U with an image indicating the location of the target portion if it is determined that the target portion exists. This further improves the safety of inspection work after blasting.

[0073] Furthermore, in the embodiment, at least some of the functions of the user terminal 200 may be provided in the drone device 100. For example, by providing the functions of the determination unit 250 and the provision unit 260 of the user terminal 200 in the drone device 100, the drone device 100 can determine whether or not a target object exists, and if a target object exists, generate information indicating the position of the target object included in the image (for example, a range image) and provide it to the user terminal 200. This reduces the processing load on the user terminal 200.

[0074] As described above, according to the embodiment, the safety inspection system 1 comprises a drone device (an example of an aircraft) capable of unmanned flight and equipped with a camera (an example of an imaging unit) 120 and a LiDAR (an example of an object detection unit) 110 that detects surrounding objects, an acquisition unit 240 that flies the drone device 100 and acquires images of the construction site after blasting captured by the camera 120 and object detection results of the construction site detected by the LiDAR 110, a determination unit 250 that determines whether at least one of a target object and a target part exists at the construction site based on the images and object detection results acquired by the acquisition unit 240, and a provision unit 260 that provides the user with an image showing the position of the existing target object or target part when the determination unit 250 determines that at least one of a target object and a target part exists, thereby further improving safety in inspection work after blasting.

[0075] For example, in the past, when inspecting the face, debris, etc., there was a concern that the natural ground might fall off due to blasting excavation, or that unexploded explosives might remain or be left behind, causing a disaster near the face. However, according to the present embodiment, an aircraft equipped with a camera 120, a LiDAR 110, or the like can be used to acquire information about the construction site, making it possible to easily and safely check for damage to support due to blasting, the presence of residual explosives on the surface of the debris, etc. Furthermore, according to the present embodiment, the instability of the natural ground around the face can be determined in a short time from the amount of fallen rock mass. Furthermore, according to the present embodiment, by using an unmanned aircraft, the aircraft can be moved without considering unevenness of the ground, and the aircraft's posture will not be affected by debris, etc. Furthermore, by using an aircraft, the user U does not need to go near the face, so the safety of the user U can be ensured even if a collapse or other incident occurs near the face.

[0076] The above describes the form for carrying out the present invention using an embodiment, but the specific configuration is not limited to this embodiment, and also includes designs within the scope that do not deviate from the gist of the present invention.

[0077] The Sustainable Development Goals (SDGs) are 17 international goals adopted at the United Nations Summit in September 2015. The safety inspection system and safety inspection method according to this embodiment can contribute to achieving one of the 17 SDGs, for example, goal 9, "Build resilient infrastructure, promote inclusive and sustainable industrialization, and foster innovation." [Explanation of symbols]

[0078] 1...Safety inspection system, 100...Drone device, 110...LiDAR, 120...Camera, 130...Light emitting unit, 140...Flight unit, 150, 210...Communication unit, 160, 270...Control unit, 170...Battery, 180, 290...Memory unit, 200...User terminal, 220...Input unit, 230...Output unit, 240...Acquisition unit, 250...Determination unit, 260...Provision unit, 280...Application execution unit, 300...Tunnel, 310...Face, 320...Side wall, 330...Ground, 340...Muddle

Claims

1. An unmanned flying vehicle equipped with an imaging unit, an acquisition unit that flies the aircraft and acquires images of the construction site after blasting captured by the imaging unit; a determination unit that determines whether or not a target object exists at the construction site based on the image acquired by the acquisition unit; a providing unit that provides a user with an image indicating the position of the target object when the determining unit determines that the target object exists; A safety inspection system equipped with:

2. the determination unit determines whether the target object is present in the image by comparing feature information for each object included in the image captured by the imaging unit with feature information previously related to the target object; The safety inspection system according to claim 1 .

3. When the determination unit determines that a target object exists at the construction site, the providing unit superimposes a range image indicating the range of the target object on the image captured by the imaging unit and provides the image to the user. The safety inspection system according to claim 1 .

4. The range image is an image that surrounds the area of ​​the target object included in the image captured by the imaging unit, and is displayed in a different display mode depending on the type of the target object. The safety inspection system according to claim 3.

5. The target object includes at least one of a crack in a wall of the construction site, gunpowder, and a detonator. The safety inspection system according to claim 1 .

6. an unmanned flying vehicle equipped with an object detection unit that detects surrounding objects; an acquisition unit that flies the flying object and acquires the object detection results of the construction site after blasting detected by the object detection unit; a determination unit that determines whether or not a target portion exists in the construction site based on the object detection result acquired by the acquisition unit; a providing unit that provides a user with an image indicating the position of the target portion when the determining unit determines that the target portion exists; A safety inspection system equipped with:

7. The construction site has a space at least partly surrounded by a wall, the object detection unit is a LiDAR (Light Detection And Ranging) the target portion is a missing portion of the wall portion, The determination unit derives an average cross section of the space based on the object detection result in the space by the LiDAR, and determines whether or not there is a missing wall portion based on the degree of expansion of a portion where the space expands outside the derived average cross section. The safety inspection system according to claim 6.

8. the providing unit provides the user with an image showing an area where the wall portion is missing. The safety inspection system according to claim 7.

9. an unmanned flying vehicle equipped with an imaging unit and an object detection unit that detects surrounding objects; an acquisition unit that flies the aircraft and acquires an image of the construction site after blasting captured by the imaging unit and an object detection result of the construction site detected by the object detection unit; a determination unit that determines whether at least one of a target object and a target portion is present at the construction site based on the image acquired by the acquisition unit and the object detection result; a providing unit that, when the determining unit determines that at least one of the target object and the target portion exists, provides a user with an image indicating the position of the target object or the target portion that exists; A safety inspection system equipped with:

10. The computer Flying an unmanned flying vehicle equipped with an imaging unit, and acquiring images of the construction site after blasting captured by the imaging unit; determining whether or not a target object is present at the construction site based on the acquired image; When it is determined that the target object exists, an image indicating the position of the target object is provided to a user. Safety inspection methods.

11. The computer Flying an unmanned flying vehicle equipped with an object detection unit that detects surrounding objects, and acquiring the object detection results of the construction site after blasting detected by the object detection unit; determining whether or not a target portion exists in the construction site based on the obtained object detection result; If it is determined that the target portion exists, an image indicating the location of the target portion is provided to the user. Safety inspection methods.

Images