Aircraft control system

Abstract

To provide an aircraft control device that allows the aircraft to photograph the tunnel face without having to install a marker on the wall at the tunnel face photography location. [Solution] Based on the point cloud data acquired by the scanner 105, it is determined whether or not face estimation is possible. If it is possible, face K is estimated based on the point cloud data, and the positional relationship between face K and the aircraft 1 is obtained. Based on the image data acquired by the camera 106, it is determined whether or not the flight distance of the aircraft 1 can be obtained. If it is possible, the flight distance of the aircraft 1 is obtained from the shortest face marker ML based on the image data. The output control unit 137 flies the aircraft 1 to the face shooting position based on the positional relationship between face K and the aircraft 1 obtained from the point cloud data in priority 1, and flies the aircraft 1 to the shooting position based on the flight distance of the aircraft 1 from the shortest face marker ML obtained from the image data in priority 2.

Description

Technical Field

[0001] The present invention relates to an aircraft control device for controlling an aircraft in a tunnel.

Background Art

[0002] In mountain tunnels, in order to evaluate the ground ahead during excavation, visual observation of the face situation (hereinafter referred to as "face observation") is carried out. Face observation requires skilled technicians, and furthermore, it is a dangerous operation because it needs to be carried out in an unstable situation where the ground is exposed. Due to the shortage of skilled technicians and problems in terms of safety, there is a demand in the field for alternative technologies to the face observation method.

[0003] In response to such a demand, Patent Document 1 describes a system for evaluating a face based on image data in which a target area including the face is imaged and point cloud data of a measured three-dimensional shape, and it is suggested that a device for acquiring the image data can be attached to an aircraft such as a drone, an unmanned aerial vehicle (UAV), or a balloon.

[0004] As for autonomously flying an aircraft such as a drone in a tunnel, for example, Patent Document 2 discloses a device that obtains flight-related information by reading markers provided on the tunnel wall and autonomously flies.

[0005] In the control of a moving object, there are cases where it is required to recognize the position of the aircraft itself even where signals from satellite positioning systems such as GPS (Global Positioning System) and GNSS (Global Navigation Satellite System) do not reach. In response to this requirement, Patent Document 3 discloses that an unmanned aircraft captures AR markers installed at key points on the flight path with a camera and recognizes its own reference position from the position information of the AR markers.

Prior Art Documents

Patent Documents

[0006] [Patent Document 1] Japanese Patent Publication No. 2023-34740 [Patent Document 2] Japanese Patent Publication No. 2022-55524 [Patent Document 3] Japanese Patent Publication No. 2022-15978 [Overview of the project] [Problems that the invention aims to solve]

[0007] To clearly photograph the conditions of the tunnel face with a camera mounted on an aircraft, it is desirable to stop the aircraft at a fixed distance (e.g., 15m) from the tunnel face. To accurately stop the aircraft at a fixed position from the tunnel face for photography, it was effective to place a marker at the stopping point to issue a stop command to the aircraft. However, the stopping position was near the face of the tunnel, making it dangerous to place markers there. Moreover, the position of the tunnel face moved with each blast, requiring the placement of new markers each time it moved, which was time-consuming.

[0008] To solve these problems, the inventors aimed to provide an aircraft control device that can photograph the tunnel face from an aircraft without installing a marker on the wall surface at the shooting location of the tunnel face. [Means for solving the problem]

[0009] Means 1 is an aircraft control device that controls an aircraft from a marker installed inside a tunnel to a face photography position for photographing the tunnel face, wherein the aircraft has a scanner that emits a laser to acquire point cloud data, a camera and an internal sensor, and the aircraft control device includes a point cloud data acquisition unit that acquires point cloud data acquired by the scanner, a face estimation feasibility determination unit that determines whether or not it is possible to estimate the tunnel face based on the point cloud data acquired by the point cloud data acquisition unit, a face aircraft position relationship acquisition unit that, when the determination of the face estimation feasibility determination unit is possible, estimates the tunnel face based on the point cloud data acquired by the point cloud data acquisition unit and acquires the position relationship between the estimated tunnel face and the aircraft, an image data acquisition unit that acquires image data acquired by the camera, an image flight distance acquisition feasibility determination unit that determines whether or not it is possible to acquire the flight distance traveled by the aircraft from the marker based on the image data acquired by the image data acquisition unit, and an image flight distance acquisition unit that, when the determination of the image flight distance acquisition feasibility determination unit is possible, acquires the flight distance traveled by the aircraft from the marker based on the image data acquired by the image data acquisition unit. The system comprises: an internal measurement data acquisition unit that acquires internal measurement data measured by the internal sensor; an internal measurement flight distance acquisition feasibility determination unit that determines whether it is possible to acquire the flight distance from the marker to the aircraft based on the internal measurement data acquired by the internal measurement data acquisition unit; an internal measurement flight distance acquisition unit that, when the determination by the internal measurement data acquisition feasibility determination unit is possible, acquires the flight distance from the marker to the aircraft based on the internal measurement data acquired by the internal measurement data acquisition unit; and an output control unit that controls the flight of the aircraft, wherein when it is determined that the face has been estimated based on the point cloud data, the output control unit controls the flight of the aircraft to the face shooting position based on the positional relationship between the estimated face and the aircraft acquired by the face-to-aircraft position relationship acquisition unit; and when it is not determined that the face has been estimated based on the point cloud data, and it is determined that the flight distance from the marker to the aircraft has been acquired based on the image data, the output control unit controls the flight of the aircraft to the face shooting position based on the flight distance from the marker to the aircraft acquired by the image flight distance acquisition unit.This aircraft control device is characterized by controlling the flight of the aircraft to the face photography position based on the flight distance acquired by the internal measurement flight distance acquisition unit, when it is not determined that the face has been estimated based on the point cloud data, and when it is not determined that the flight distance traveled by the aircraft from the marker has been acquired based on the image data, and when it is determined that the flight distance traveled by the aircraft from the marker has been acquired based on the internal measurement data.

[0010] Means 2 is an aircraft control device that controls an aircraft from a marker installed inside the tunnel to a face photography position for photographing the tunnel face, wherein the aircraft has a scanner that emits a laser to acquire point cloud data and a camera, and the aircraft control device includes a point cloud data acquisition unit that acquires the point cloud data acquired by the scanner, a face estimation feasibility determination unit that determines whether or not it is possible to estimate the tunnel face based on the point cloud data acquired by the point cloud data acquisition unit, a face aircraft position relationship acquisition unit that, when the face estimation feasibility determination unit determines that it is possible, estimates the tunnel face based on the point cloud data acquired by the point cloud data acquisition unit and acquires the position relationship between the estimated tunnel face and the aircraft, an image data acquisition unit that acquires image data acquired by the camera, and an image flight distance acquisition unit that determines whether or not it is possible to acquire the flight distance the aircraft travels from the marker based on the image data acquired by the image data acquisition unit. The aircraft control device comprises: a unit for determining whether or not to acquire image flight distance; an image flight distance acquisition unit that, when the determination of whether or not to acquire image flight distance is possible, acquires the flight distance traveled by the aircraft from the marker based on the image data acquired by the image data acquisition unit; and an output control unit for controlling the flight of the aircraft, wherein the output control unit controls the flight of the aircraft to the face shooting position based on the positional relationship between the estimated face and the aircraft acquired by the face-to-aircraft position relationship acquisition unit when it is determined that the face has been estimated based on the point cloud data; and when it is determined that the face has been estimated based on the point cloud data, and the flight distance traveled by the aircraft from the marker based on the image data, the output control unit controls the flight of the aircraft to the face shooting position based on the flight distance traveled by the aircraft from the marker acquired by the image flight distance acquisition unit.

[0011] Means 3 is an aircraft control device that controls an aircraft from a marker installed inside the tunnel to a face photography position for photographing the tunnel face, wherein the aircraft has a scanner that emits a laser to acquire point cloud data and an internal sensor, and the aircraft control device includes a point cloud data acquisition unit that acquires the point cloud data acquired by the scanner, a face estimation feasibility determination unit that determines whether or not it is possible to estimate the tunnel face based on the point cloud data acquired by the point cloud data acquisition unit, a face aircraft position relationship acquisition unit that, when the face estimation feasibility determination unit determines that it is possible, estimates the tunnel face based on the point cloud data acquired by the point cloud data acquisition unit and acquires the position relationship between the estimated tunnel face and the aircraft, an internal measurement data acquisition unit that acquires internal measurement data measured by the internal sensor, and an internal measurement flight unit that determines whether or not it is possible to acquire the flight distance the aircraft flies from the marker based on the internal measurement data acquired by the internal measurement data acquisition unit. The aircraft control device comprises: a distance acquisition feasibility determination unit; an internal measurement flight distance acquisition unit that, when the internal measurement flight distance acquisition feasibility determination unit determines that it is possible, acquires the flight distance the aircraft has flown from the marker based on the internal measurement data acquired by the internal measurement data acquisition unit; and an output control unit that controls the flight of the aircraft, wherein when the output control unit determines that the face has been estimated based on the point cloud data, it controls the flight of the aircraft to the face photography position based on the estimated positional relationship between the face and the aircraft acquired by the face-aircraft position relationship acquisition unit; and when it is determined that the face has not been estimated based on the point cloud data, and when it is determined that the flight distance the aircraft has flown from the marker based on the internal measurement data, it controls the flight of the aircraft to the face photography position based on the flight distance the aircraft has flown from the marker acquired by the internal measurement flight distance acquisition unit.

[0012] Means 4 is an aircraft control device that controls an aircraft from a marker installed inside a tunnel to a face photography position for photographing the tunnel face, wherein the aircraft has a camera and an internal sensor, and the aircraft control device includes an image data acquisition unit that acquires image data acquired by the camera, an image flight distance acquisition feasibility determination unit that determines whether it is possible to acquire the flight distance the aircraft has flown from the marker based on the image data acquired by the image data acquisition unit, an image flight distance acquisition unit that, when the determination by the image flight distance acquisition feasibility determination unit is possible, acquires the flight distance the aircraft has flown from the marker based on the image data acquired by the image data acquisition unit, an internal measurement data acquisition unit that acquires internal measurement data measured by the internal sensor, and an internal measurement flight distance acquisition feasibility determination unit that determines whether it is possible to acquire the flight distance the aircraft has flown from the marker based on the internal measurement data acquired by the internal measurement data acquisition unit, and the internal measurement flight distance The aircraft control device comprises: an internal measurement flight distance acquisition unit that acquires the flight distance traveled by the aircraft from the marker based on the internal measurement data acquired by the internal measurement data acquisition unit when the acquisition feasibility determination unit determines that it is possible; and an output control unit that controls the flight of the aircraft, wherein when the output control unit determines that the flight distance traveled by the aircraft from the marker has been acquired based on the image data, it controls the flight of the aircraft to the face shooting position based on the flight distance traveled by the aircraft from the marker acquired by the image flight distance acquisition unit; and when it is not determined that the flight distance traveled by the aircraft from the marker has been acquired based on the image data, but when it is determined that the flight distance traveled by the aircraft from the marker has been acquired based on the internal measurement data, it controls the flight of the aircraft to the face shooting position based on the flight distance traveled by the aircraft from the marker acquired by the internal measurement flight distance acquisition unit.

[0013] Means 5 is the aircraft control device according to means 1, means 3, or means 4, characterized in that the internal sensor is an inertial sensor. [Effects of the Invention]

[0014] According to the present invention, a marker is provided on the shaft side rather than the face shooting position, and the flying object autonomously flies from this marker to the face shooting position. Therefore, there is no need to install a marker at a face shooting position close to an unstable face, which is highly safe, and there is no need to reinstall the shortest face marker every time the face advances, and the labor required for marker installation work can be reduced. In addition, since the self-position of the flying object is determined by at least two of the estimation methods including estimation based on point cloud data, estimation based on image data, and estimation based on measurement by an internal sensor, the flying object can be reliably controlled to the face shooting position.

Brief Description of the Drawings

[0015] [Figure 1] It is a diagram for explaining the flight path of the flying object according to the first embodiment of the present invention. [Figure 2] It is a diagram for explaining the relationship between the position of the last marker and the stop position of the flying object according to the first embodiment of the present invention. [Figure 3] It is a diagram for explaining the acquisition of point cloud data according to the first embodiment of the present invention. [Figure 4] It is a diagram for explaining the relationship between the coordinates of the face and the marker coordinates and the flight distance to the face shooting position according to the first embodiment of the present invention. [Figure 5] It is a block diagram showing an example of the hardware configuration of the flying object according to the first embodiment of the present invention. [Figure 6] It is a block diagram showing an example of the hardware configuration of the user terminal according to the first embodiment of the present invention. [Figure 7] It is a block diagram of the flying object according to the first embodiment of the present invention. [Figure 8] It is a block diagram of the user terminal according to the first embodiment of the present invention. [Figure 9] It is a sequence diagram showing the flow of information between the flying object and the user terminal according to the first embodiment of the present invention. [Figure 10] It is a flowchart showing an example of the flow of processing executed by the flight control device according to the first embodiment of the present invention. [Figure 11] This is a flowchart showing an example of the processing flow executed by the flight control device according to the second embodiment of the present invention. [Figure 12] This is a flowchart showing an example of the processing flow executed by the flight control device according to the third embodiment of the present invention. [Figure 13] This is a flowchart showing an example of the processing flow executed by the flight control device according to the fourth embodiment of the present invention.

Mode for Carrying Out the Invention

[0016] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Needless to say, the present invention is not limited to the embodiments.

[0017] 〔First Embodiment〕 Hereinafter, the first embodiment of the present invention will be described with reference to FIGS. 1 to 10.

[0018] FIG. 1 is a diagram for explaining the flight path of the flying object, FIG. 2 is a diagram for explaining the relationship between the last marker position and the stop position of the flying object, FIG. 3 is a diagram for explaining the acquisition of point cloud data, FIG. 4 is a diagram for explaining the relationship between the coordinates of the runway and the marker coordinates and the flight distance to the runway photographing position, FIG. 5 is a diagram showing an example of the hardware configuration of the flying object, FIG. 6 is a diagram showing an example of the hardware configuration of the user terminal, FIG. 7 is a block diagram of the flying object, FIG. 8 is a block diagram of the user terminal, FIG. 9 is a sequence diagram showing the flow of information between the flying object and the user terminal, and FIG. 10 is a flowchart showing an example of the processing flow executed by the flight control device.

[0019] As shown in FIG. 1, the flight system 1000 according to the present embodiment includes a flying object 1, a user terminal 2, and a marker M.

[0020] As shown in FIG. 1, after various setting information is set in the flying object 1 by the user terminal 2, the flying object 1 flies, that is, autonomously flies, under internal control from the marker M installed in the tunnel T to the runway photographing position S via the marker M in order.

[0021] In this embodiment, the aircraft 1 is an unmanned aerial vehicle, a so-called drone, capable of autonomously flying along a desired flight path. The aircraft 1 photographs the tunnel face K inside the tunnel T, and the condition of the tunnel face K is confirmed.

[0022] In the following explanation, the tunnel coordinate system is defined as follows: the depth direction of tunnel T (tunnel axis direction) is the X-axis, the horizontal axis perpendicular to the X-axis is the Y-axis, and the axis perpendicular to both the X and Y axes is the Z-axis.

[0023] The shooting position S at the tunnel face is set closer to the tunnel face K than the shortest distance marker ML at the tunnel face K.

[0024] A marker M is installed on the wall of tunnel T, on the side of the tunnel entrance from the face photography position S. Marker M is a so-called AR marker, and it displays a mark that allows the aircraft 1 to determine identification information.

[0025] Marker M is placed at appropriate intervals from the initial marker MS located at the takeoff point to the shortest distance marker ML, which is the marker with the shortest distance to the tunnel face K.

[0026] The aircraft 1 begins its flight from the position of the initial marker MS according to the flight plan instructed by the user terminal 2, and flies sequentially along the markers M to the position of the shortest face marker ML.

[0027] Then, controlled by the aircraft control device described later, the aircraft 1 flies to the face photography position S, as shown in Figure 2, stops at face photography position S, photographs the face K, and then returns to the position of the shortest face marker ML. After that, it sequentially follows the markers M back to the initial takeoff marker MS and lands (Figure 1).

[0028] The tunnel face photography position S is the position where the aircraft 1 stops in order to photograph the tunnel face K, and is set at a positional relationship where it is separated by a predetermined distance in the X-axis direction, Y-axis direction, and Z-axis direction from any point on the tunnel face K (for example, X=74m (tunnel face No. 70), Y=0m (corresponding to the tunnel central axis), Z=0m (corresponding to the floor height)).

[0029] The predetermined distances in the X, Y, and Z axes from an arbitrary point used to define the tunnel face K to the tunnel face photography position S are called the positional relationship information KSI from the tunnel face K to the tunnel face photography position S. Furthermore, information regarding an arbitrary point on the tunnel face K used to define the tunnel face K is called the tunnel face position information KI.

[0030] The predetermined distance in the X-axis direction (for example, 15m) of the positional relationship information KSI from the tunnel face K to the tunnel face photography position S is set from the tunnel face K towards the tunnel entrance. Based on the face position information KI and the positional relationship information KSI from face K to face photography position S, the X, Y, and Z coordinates (for example, the X coordinate being 15m away from face K towards the tunnel entrance in the X-axis direction, the Y coordinate corresponding to the tunnel's central axis, and the Z coordinate corresponding to a position 3m above the floor height) are calculated and set as the face photography position S.

[0031] The positional relationship information from the tunnel face K to the tunnel face photography position S is set appropriately based on the size of the tunnel face K to be photographed and the field of view of the imaging device of the aircraft 1.

[0032] The closest face marker ML is installed at a sufficient distance from face K toward the tunnel entrance, and further at a sufficient distance toward the tunnel entrance from the face photography position S, because attempting to install it close to face K would pose a risk to the installation process due to surface damage from face K.

[0033] In order to fly from the closest marker ML on the tunnel face and stop at the tunnel face photography position S, aircraft 1 needs to determine that it has reached the tunnel face photography position S. There are two possible ways to do this.

[0034] The first method involves recognizing the tunnel face K based on measurements, determining the positional relationship between the tunnel face K and the aircraft 1, and determining that the aircraft 1 has reached the tunnel face photography position S when the determined positional relationship between the tunnel face K and the aircraft 1 becomes the pre-set positional relationship information KSI from the tunnel face K to the tunnel face photography position S.

[0035] The second method involves calculating the flight distance from the shortest face marker ML to the shooting position S using the face position information KI set as the target for shooting, the positional relationship information KSI from face K to the shooting position S, and the marker position information MI of the shortest face marker ML. The aircraft 1 then determines the distance it has flown from the shortest face marker ML, and when this flight distance matches the calculated flight distance from the shortest face marker ML to the shooting position S, it determines that the aircraft 1 has reached the shooting position S.

[0036] Here, "flight distance" refers not only to a simple distance but also to the relative positions of the two endpoints related to that flight distance. For example, the flight distance from the shortest face marker ML to the face photography position S indicates the relative positions of the shortest face marker ML and the face photography position S, and is represented, for example, by a vector in the X, Y, and Z coordinate system.

[0037] Regarding the first method, the following means are available for the aircraft 1 to recognize the tunnel face K based on measurements and to understand the positional relationship between the tunnel face K and the aircraft 1.

[0038] This is a so-called LiDAR SLAM (Light Detection And Ranging Simultaneous Localization and Mapping) technique, in which a laser is shone from the aircraft 1 towards multiple points on the tunnel face K using LiDAR (Light Detection And Ranging), and the time it takes for the reflected light to be received is measured to determine the distance to the points from which the laser was shone. From the resulting point cloud data, the distance and shape to the tunnel face K are estimated. As a result, the aircraft 1 recognizes the tunnel face K based on the measurements and understands the positional relationship between the tunnel face K and the aircraft 1; in other words, it can recognize its own position in relation to the tunnel face K.

[0039] To recognize the tunnel face K using LiDAR SLAM, as shown in Figure 3, the tunnel face K can be recognized based on whether or not the shape in front can be recognized as a closed surface. For example, point cloud data obtained from the side to the front of the aircraft 1 is used to determine, using a predetermined threshold, that the side walls W1 and W2 of the tunnel T exist parallel to each other to the side, and that a perpendicular wall exists between them in front. The average of the point cloud data forming the wall in front is then recognized as the tunnel face K.

[0040] By recognizing the position of the tunnel face K through measurement, the positional relationship between the tunnel face K and the aircraft 1 can be determined. Then, by making the positional relationship between the tunnel face K and the aircraft 1 correspond to the positional relationship information KSI from the predetermined tunnel face K to the tunnel face photography position S, it can be determined that the aircraft 1 has reached the tunnel face photography position S. Based on this, the aircraft 1 will fly autonomously.

[0041] This LiDAR SLAM method recognizes the tunnel face K through actual measurement, allowing for relatively accurate recognition of the position of tunnel face K in accordance with the current conditions and understanding its positional relationship with the aircraft 1.

[0042] On the other hand, reliability decreases if sufficient point cloud data cannot be obtained due to reasons such as diffuse reflection caused by dust generation, many irregularities on the tunnel face, or if the data is highly variable or does not represent a closed shape on the tunnel face.

[0043] The second method of determination is explained below.

[0044] As shown in Figure 4(a), the tunnel face K is newly formed by blasting every 1m in the direction of tunnel T's travel, based on a pre-set plan. That is, the position of the tunnel face K is predetermined by the construction plan, and the set tunnel face position information KI is known. For example, the X coordinate of tunnel face K No. 70 is set to 74.0m. In addition to the X coordinate, the tunnel face position information KI may also include the Y coordinate (for example, the coordinate corresponding to the tunnel's central axis) and Z coordinate (for example, the coordinate corresponding to the floor height) at any point on the tunnel face K surface.

[0045] As shown in Figure 4(b), each marker M is placed in a predetermined position, and its position is determined by surveying or other means. Therefore, the X, Y, and Z coordinates of the placement position of each marker M are known as marker position information MI.

[0046] To calculate the flight distance from the shortest marker ML to the shooting position S, using the face position information KI set as the target for shooting, the positional relationship information KSI from face K to shooting position S, and the marker position information MI of the shortest marker ML, the positional relationship between the face position information KI and the marker position information MI of the shortest marker ML is calculated, and the positional relationship information KSI from face K to shooting position S is subtracted from the calculated positional relationship.

[0047] As a concrete calculation example, to explain the X-axis direction, for example, for face K set as No. 70, the closest face marker ML is marker M of No. 4, and if the positional relationship information KSI from face K to face photography position S is 15m away towards the tunnel entrance in the X-axis direction, then the flight distance from the closest face marker ML to face photography position S is: 74m (X-coordinate of the position information of face K) - 42.8m (X-coordinate of the position information of the shortest face marker ML) - 15m (X-axis value of the positional relationship information KSI from face K to face photography position S) = 16.2m This is the result.

[0048] If the Y and Z coordinates are set as the face K position and the face photography position S, the difference between these coordinates and the Y and Z coordinates of the shortest face marker ML is calculated to determine the flight distance of the aircraft 1 in the Y and Z directions.

[0049] To determine the flight distance traveled by aircraft 1 from the shortest face marker ML, aircraft 1 can use the following methods:

[0050] By using so-called Visual SLAM (Visual Simultaneous Localization and Mapping), which estimates the distance to multiple feature points using captured images (video) to determine the self-position, the flight distance of aircraft 1 from the shortest face marker ML can be determined.

[0051] Visual SLAM allows us to determine that aircraft 1 has reached the designated face photography position S when its flight distance from the shortest face marker ML reaches the face photography position S (for example, 16.2m in the X direction as mentioned earlier). Based on this, aircraft 1 will fly autonomously.

[0052] A system that uses Visual SLAM to fly aircraft 1 to the tunnel face photography position S does not need to use the actual tunnel face K as a reference, so it can fly to the tunnel face photography position S even when the tunnel face K surface cannot be identified.

[0053] On the other hand, reliability decreases if the required illuminance cannot be obtained or if there is interference from ambient light.

[0054] Other means by which aircraft 1 can determine the flight distance it has traveled from the shortest face marker ML include the following:

[0055] The aircraft 1 is equipped with an internal sensor, an inertial measurement sensor (IMU), and the movement of the aircraft 1 is determined by the output of the inertial sensor, thereby determining the flight distance of the aircraft 1 from the shortest face marker ML.

[0056] Furthermore, the internal sensors installed inside the aircraft 1 may include, in addition to the IMU, at least one of other sensors such as an altimeter, airspeed indicator, accelerometer, or gyro sensor. The aircraft control device estimates and calculates the position (distance, bearing, etc.) in real time based on various data (e.g., altitude, speed, acceleration, angular velocity, etc.) output from the internal sensors during flight to determine the flight distance from the shortest face marker ML of the aircraft 1. The internal sensors may also include means for detecting control information such as the propeller rotation speed.

[0057] Based on information from its internal sensors, when the flight distance from the shortest face marker ML of the aircraft 1 reaches the set face photography position S (for example, 16.2m in the X direction as mentioned earlier), the aircraft 1 can determine that it has reached the face photography position S. Based on this, the aircraft 1 will fly autonomously.

[0058] A system that flies the aircraft 1 to the tunnel face photography position S using information from internal sensors can fly to the tunnel face photography position S in a situation less affected by the external environment because it relies solely on information from the aircraft 1, but it is relatively unreliable.

[0059] Figure 5 is a block diagram showing an example of the hardware configuration of aircraft 1. The aircraft 1 comprises a bus 100, a battery 101, a control device 102, an input / output device 103, a motor control circuit 104, a scanner 105, a camera 106, an inertial sensor 107, a communication device 108, and a motor 112.

[0060] The control device 102 includes a hardware processor 110 and memory 111. Motor 112 is connected to motor control circuit 104.

[0061] Bus 100 is a communication channel that connects each piece of hardware within the aircraft 1 to each other, and each piece of hardware within the aircraft 1 exchanges data via bus 100.

[0062] Battery 101 is, for example, a lithium-ion battery, and supplies power to various parts of the aircraft 1.

[0063] The hardware processor 110 of the control device 102 is, for example, a CPU, and controls the entire aircraft 1 according to the control program. The hardware processor 110 functions as a flight controller.

[0064] The memory 111 of the control device 102 is a storage device that stores various programs, data, etc., and is, for example, ROM, RAM, or SSD. The memory 111 stores, for example, a control program for controlling the entire aircraft 1.

[0065] The input / output device 103 is a device for inputting and outputting various types of information to the aircraft 1, for example, when the aircraft is not flying. Note that the input / output device 103 is not required.

[0066] The motor control circuit 104 is an electronic circuit for controlling the motor 112. The motor control circuit 104 receives control commands from the hardware processor 110 and drives and controls the motor 112.

[0067] Motor 112 is controlled by motor control circuit 104. Motor 112 rotates a propeller (not shown) fixed to a rotating shaft to make the aircraft 1 fly. The aircraft 1 may have one motor 112 or multiple motors (for example, four).

[0068] The scanner 105 is a distance measuring sensor, such as LiDAR (Light Detection And Ranging), which acquires point cloud data by irradiating a two-dimensional or three-dimensional laser and receiving the reflected light with a light-receiving unit.

[0069] Camera 106 captures images of, for example, markers M placed on the surrounding wall of tunnel T and characteristic points within tunnel T. Camera 106 also captures images to check the condition of tunnel T and the tunnel face K. Camera 106 may capture still images or video, but it is preferable to use a stereo camera. In this embodiment, camera 106 is used for recognizing marker M, identifying characteristic points within tunnel T, and confirming the condition of tunnel T and tunnel face K. However, it is not limited to this, and separate cameras may be provided for recognizing marker M, identifying characteristic points within tunnel T, and confirming the condition of tunnel T and tunnel face K. In this case, cameras appropriate to their respective uses may be used; for example, a general-purpose camera for recognizing marker M, a stereo camera for identifying characteristic points within tunnel T, and a 4K camera for confirming the condition of tunnel T and tunnel face K.

[0070] The internal inertial sensor 107 is installed on the aircraft 1 and measures angular velocity and acceleration in three orthogonal axes to detect three-dimensional inertial motion. From its output, it attempts to calculate the flight distance and direction of the aircraft 1.

[0071] The distance and direction calculated from the measurement data of the inertial sensor 107 are applied to the reference position read from the marker M to estimate the self-position.

[0072] The communication device 108 communicates with the user terminal 2 via wireless communication. The communication device 108 communicates with the user terminal 2 using, for example, a wireless LAN, Wi-Fi, Bluetooth, or network.

[0073] Figure 6 is a block diagram showing an example of the hardware configuration of user terminal 2. User terminal 2 could be, for example, a smartphone, tablet, or laptop computer.

[0074] User terminal 2 includes a bus 200, a battery 201, a control device 202, an input / output device 203, and a communication device 204. The control device 202 includes a hardware processor 205 and a memory 206.

[0075] Bus 200 is a communication path that connects each piece of hardware within the user terminal 2 to each other, and each piece of hardware within the user terminal 2 exchanges data via bus 200.

[0076] Battery 201 is, for example, a lithium-ion battery, and supplies power to various parts of the user terminal 2.

[0077] The hardware processor 205 of the control device 202 is, for example, a CPU or an electronic circuit, and controls the entire user terminal 2 according to the system program of the user terminal 2. The hardware processor 205 reads system programs and other data stored in memory 206.

[0078] The memory 206 of the control device 202 is a storage device that stores various programs and data, such as the system program of the user terminal 2, and is, for example, ROM, RAM, or SSD.

[0079] The input / output device 203 is a device for inputting and outputting various types of information. For example, the input / output device 203 is a touch panel. Based on input operations by an operator or other user, the input / output device 203 inputs various types of data into the user terminal 2. Furthermore, the input / output device 203 displays the results of various information processing within the user terminal 2 on a display screen.

[0080] The communication device 204 has the function of communicating with the aircraft 1, and the user terminal 2 sends and receives various data to and from the aircraft 1 using the communication device 204.

[0081] Figure 7 is a block diagram showing an example of the function of the aircraft 1. The control device 102 of the aircraft 1 has functions to control the aircraft 1, such as flying according to the aircraft 1's marker M, taking off and landing, and taking pictures with the camera 106.

[0082] Furthermore, the control device 102 has the function of controlling the aircraft 1 from the shortest face marker ML to the face photography position S. The control device 102 corresponds to the aircraft control device of the present invention.

[0083] The control device 102 includes, for example, an input / output unit 120, a communication unit 121, a storage unit 122, a setting information acquisition unit 123, a marker arrival determination unit 124, a point cloud data acquisition unit 125, a face estimation feasibility determination unit 126, a face aircraft position relationship acquisition unit 127, an image data acquisition unit 129, an image flight distance acquisition feasibility determination unit 130, an image flight distance acquisition unit 131, an inertial measurement data acquisition unit 133, an inertial measurement flight distance acquisition feasibility determination unit 134, an inertial measurement flight distance acquisition unit 135, and an output control unit 137.

[0084] The input / output unit 120, communication unit 121, setting information acquisition unit 123, marker arrival determination unit 124, point cloud data acquisition unit 125, face estimation feasibility determination unit 126, face aircraft position relationship acquisition unit 127, image data acquisition unit 129, image flight distance acquisition feasibility determination unit 130, image flight distance acquisition unit 131, inertial measurement data acquisition unit 133, inertial measurement flight distance acquisition feasibility determination unit 134, inertial measurement flight distance acquisition unit 135, and output control unit 137 are realized, for example, by the hardware processor 110 performing calculations using various programs and various data stored in memory 111. The storage unit 122 is realized, for example, by storing various programs and various data.

[0085] The input / output section 120 is a connection section to which output devices such as displays and input devices such as mice, keyboards, and touchpads are connected when the aircraft is not flying, for example during maintenance, and various types of data are input and output there. Note that the input / output section is not required.

[0086] The communication unit 121 connects the aircraft 1 to the network via wireless communication. This connects the aircraft 1 to the user terminal 2, enabling data transmission and reception.

[0087] The memory unit 122 stores information such as face position information KI, positional relationship information KSI from face K to face shooting position S, marker position information MI, setting information such as flight plan, data acquired by scanner 105, camera 106 and inertial sensor 107, and various programs. Alternatively, information acquired by each unit may be stored in the memory unit 122.

[0088] The setting information acquisition unit 123 acquires various setting information from the various information stored in the storage unit 122, as illustrated below.

[0089] The setting information acquisition unit 123 acquires positional relationship information KSI from the tunnel face K to the tunnel face shooting position S in order to understand the positional relationship between the tunnel face K and the aircraft 1, which is estimated based on the measurements.

[0090] The setting information acquisition unit 123 acquires the tunnel face position information KI for the tunnel face K to be photographed, which is specified from the user terminal 2, identifies the shortest marker ML on that tunnel face, and acquires the marker position information MI.

[0091] The setting information acquisition unit 123 calculates and acquires the flight distance that the aircraft 1 will fly from the shortest marker ML to the shooting position S, based on the shooting face position information KI for the shooting face K to be photographed, the positional relationship information KSI from the shooting face K to the shooting face photography position S, and the marker position information MI of the shortest marker ML of the shooting face.

[0092] The marker arrival determination unit 124 determines whether the aircraft 1 has reached the position of the shortest face marker ML based on the image of marker M acquired by camera 106 during flight and the information of the shortest face marker ML identified by setting information acquisition unit 123.

[0093] The point cloud data acquisition unit 125 acquires the point cloud data acquired by the scanner 105.

[0094] The tunnel face estimation feasibility determination unit 126 determines whether or not it is possible to estimate the tunnel face based on the point cloud data acquired by the point cloud data acquisition unit 125.

[0095] The tunnel face estimation feasibility determination unit 126 has predetermined thresholds set to address situations such as insufficient point cloud data being obtained due to reasons such as diffuse reflection caused by dust generation or many irregularities on the tunnel face, or when the data is too sparse or the tunnel face is not recognized as having a closed shape, and determines that the tunnel face cannot be estimated. In addition, the unit may also determine that the tunnel face cannot be estimated if it is determined from the acquired point cloud data that a sufficient flight route cannot be secured due to underground equipment or heavy machinery.

[0096] When the face estimation feasibility determination unit 126 determines that it is possible, the face K is estimated based on the point cloud data acquired by the point cloud data acquisition unit 125, and the positional relationship between the estimated face K and the aircraft 1 is obtained.

[0097] The tunnel face K is estimated by, for example, using point cloud data obtained from the side to the front of the aircraft 1, determining, with a predetermined threshold, that the side walls W1 and W2 of the tunnel T exist parallel to each other to the side, and that there is a wall perpendicular to them in front. The average of the point cloud data forming the wall in front is then estimated as the tunnel face K (Figure 3).

[0098] The face aircraft position relationship acquisition unit 127 can acquire its own position relative to the face K by acquiring the estimated position relationship between the face K and the aircraft 1.

[0099] The image data acquisition unit 129 acquires image data acquired by the camera 106.

[0100] The image flight distance acquisition feasibility determination unit 130 determines, based on the image data acquisition unit 129, whether or not it is possible to acquire the flight distance traveled by the aircraft 1 from the shortest face marker ML.

[0101] The image flight distance acquisition feasibility determination unit 130 has predetermined thresholds set to correspond to situations such as when the necessary illumination cannot be obtained, when there is an influence of ambient light, when there is a brightness standard for the image data, or when an image of feature points cannot be obtained, and determines that it is not possible to acquire the flight distance of the aircraft 1 from the shortest face marker ML.

[0102] When the image flight distance acquisition feasibility determination unit 130 determines that it is possible to acquire the image flight distance, the image data acquisition unit 129 determines that the flight distance traveled by the aircraft 1 from the shortest face marker ML.

[0103] The flight distance of aircraft 1 can be obtained, for example, by estimating the distance to multiple feature points using captured images (video).

[0104] The image flight distance acquisition unit 131 can acquire its own position relative to the shortest face marker ML by acquiring the flight distance traveled by the aircraft 1 from the shortest face marker ML.

[0105] The inertial measurement data acquisition unit 133 acquires inertial measurement data measured by the inertial sensor 107. The inertial measurement data acquisition unit 133 corresponds to the internal measurement data acquisition unit of the present invention.

[0106] The inertial measurement flight distance acquisition feasibility determination unit 134 determines, based on the inertial measurement data acquisition unit 133, whether or not it is possible to acquire the flight distance traveled by the aircraft 1 from the shortest face marker ML. The inertial measurement flight distance acquisition feasibility determination unit 134 corresponds to the internal measurement flight distance acquisition feasibility determination unit of the present invention.

[0107] The inertial measurement flight distance acquisition feasibility determination unit 134 has a predetermined threshold set to correspond to cases such as when the measured inertial measurement data is abnormal, and determines that it is not possible to acquire the flight distance of the aircraft 1 from the shortest face marker ML.

[0108] The inertial measurement flight distance acquisition unit 135 acquires the flight distance traveled by the aircraft 1 from the shortest face marker ML based on the inertial measurement data acquisition unit 133, when the determination of whether or not the inertial measurement flight distance can be acquired by the determination unit 134 is deemed acceptable. The inertial measurement flight distance acquisition unit 135 corresponds to the internal measurement flight distance acquisition unit of the present invention.

[0109] The flight distance of aircraft 1 is obtained, for example, by knowing the movement of aircraft 1 from the output of the inertial sensor 107, and thereby obtaining the flight distance of aircraft 1 from the shortest face marker ML.

[0110] The inertial measurement flight distance acquisition unit 135 can acquire its own position relative to the shortest face marker ML by acquiring the flight distance traveled by the aircraft 1 from the shortest face marker ML.

[0111] The output control unit 137 controls the motor 112 via the motor control circuit 104 based on predetermined information such as the flight plan, thereby controlling the flight of the aircraft 1 between the initial marker MS and the face-shortest marker ML. This control is a known control method, so its explanation is omitted.

[0112] The output control unit 137 controls the flight of the aircraft 1 between the shortest face marker ML and the set face shooting position S, based on the acquired self-position.

[0113] In other words, during the outward flight control from the shortest face marker ML to the face photography position S, the acquired self-position is controlled to be the set face photography position S, and during the return flight control from the face photography position S to the shortest face marker ML, the acquired self-position is controlled to be the position of the shortest face marker ML (for example, one that takes into account a predetermined separation distance).

[0114] Specifically, the flight control of the outward journey from the shortest face marker ML to the face photography position S by the output control unit 137 employs the following methods: controlling the positional relationship between the face K and the aircraft 1 acquired by the face aircraft position relationship acquisition unit 127 so that it becomes the pre-set positional relationship information KSI from face K to face photography position S (face aircraft position relationship control); controlling the flight distance traveled by the aircraft 1 from the shortest face marker ML acquired by the image flight distance acquisition unit 131 so that it becomes the pre-set flight distance from the shortest face marker ML to face photography position S (image marker flight distance control); and controlling the flight distance traveled by the aircraft 1 from the shortest face marker ML acquired by the inertial measurement flight distance acquisition unit 135 so that it becomes the pre-set flight distance from the shortest face marker ML to face photography position S (inertial measurement marker flight distance control).

[0115] The output control unit 137 controls the flight of the aircraft 1 with priority 1 for controlling the relative position of the aircraft face, priority 2 for controlling the flight distance of the image marker, and priority 3 for controlling the flight distance of the inertial measurement marker.

[0116] Figure 8 is a block diagram showing an example of the functions of user terminal 2. The user terminal 2 comprises an input / output unit 210 and a communication unit 211. The input / output unit 210 and the communication unit 211 are realized, for example, by a hardware processor 205 performing calculations using system programs and various data stored in memory 206.

[0117] The input / output section 210 is a connection section to which output devices such as displays and input devices such as mice, keyboards, and touchpads are connected, and various setting values ​​and signals are input and output.

[0118] The communication unit 211 connects the user terminal 2 to the network via wired or wireless communication. This connects the user terminal 2 to the aircraft 1, enabling data transmission and reception.

[0119] Hereafter, we will describe an example of the processing flow performed by the flight system 1000 according to this embodiment.

[0120] Figure 9 is a sequence diagram showing an example of the flow of information and processing between the aircraft 1 and the user terminal 2 in the flight system 1000 according to this embodiment.

[0121] The aircraft 1 acquires a flight plan corresponding to the marker M installed inside the tunnel T. The flight plan can be acquired by input from the user terminal 2, or by any other method. The flight plan is stored in the aircraft 1, for example, in the memory unit 122.

[0122] The user inputs the face position information KI, which is set on user terminal 2 by the user (step SA1). The set face position information KI is updated in advance according to the construction plan as tunnel excavation progresses in the X direction, and for example, the X coordinate is specified as shown in Figure 4(a). In addition to the X coordinate, the face position information KI may also include the Y coordinate (for example, the coordinate corresponding to the tunnel's central axis) and the Z coordinate (for example, the coordinate corresponding to the floor height) as arbitrary points on the face K surface.

[0123] The user inputs the marker location information MI installed on user terminal 2 (step SA2). The installed marker location information (MI) is the position obtained by surveying or other means for each installed marker. For example, as shown in Figure 4(b), the predetermined position of each marker is specified by X, Y, and Z coordinates.

[0124] The user inputs positional relationship information KSI from the tunnel face K to the tunnel face shooting position S, which is set by the user on the user terminal 2 (step SA3). The positional relationship information KSI from the set tunnel face K to the tunnel face photography position S is, for example, information indicating that it is 15m away from the tunnel entrance in the X-axis direction, as shown in Figures 1 and 2, and positional relationship information in the Y-axis and Z-axis directions may also be set.

[0125] The face position information KI, marker position information MI, and positional relationship information KSI from face K to face photography position S, which are entered into user terminal 2, are transmitted to aircraft 1.

[0126] The tunnel face position information KI transmitted from user terminal 2 is received and acquired by aircraft 1 (step SA4). The tunnel face position information KI is stored in, for example, the memory unit 122 of aircraft 1.

[0127] The marker position information MI transmitted from user terminal 2 is received and acquired by aircraft 1 (step SA5). The marker position information MI is stored in, for example, the memory unit 122 of aircraft 1.

[0128] The positional relationship information KSI from the tunnel face K to the tunnel face photography position S, transmitted from the user terminal 2, is acquired by the aircraft 1 (step SA6). The positional relationship information KSI from the tunnel face K to the tunnel face photography position S is stored in the aircraft 1, for example, in the memory unit 122.

[0129] Once aircraft 1 has finished acquiring the face position information KI, marker position information MI, and positional relationship information KSI from face K to face shooting position S, it transmits a signal indicating completion of information acquisition to user terminal 2. Upon receiving the signal indicating completion of information acquisition, user terminal 2 becomes able to input values ​​for information to specify the flight plan and information to specify the face position to be photographed.

[0130] The user specifies the flight plan to be used for this flight on user terminal 2 (step SA7). The flight plan is set to fly along marker M, and the flight plan is specified, for example, by a flight plan number.

[0131] The user specifies the tunnel face location to be photographed on user terminal 2 (step SA8). The specified tunnel face location is the location of the tunnel face location information KI, and the value of the tunnel face location information KI is specified, for example, by the tunnel face number.

[0132] The flight plan and the target site location specified on user terminal 2 are transmitted to aircraft 1, and aircraft 1 obtains the specified values ​​for the flight plan and the target site location.

[0133] Based on the specified value of the tunnel face position of the target to be photographed, the aircraft 1 identifies the shortest tunnel face marker ML for the specified tunnel face position from the acquired marker position information MI (step SA9).

[0134] The aircraft 1 calculates the positional relationship (flight distance) between the shortest face marker ML and the face photography position S from the marker position information MI of the identified shortest face marker ML, the face position information KI of the specified face position, and the positional relationship information KSI from face K to face photography position S (step SA10).

[0135] Based on the specified flight plan values, aircraft 1 identifies the designated flight plan from among the acquired flight plans, and updates the flight plan to reflect the acquired marker position information MI as needed, thereby identifying the flight plan to be used for the current flight (step SA11).

[0136] Steps SA4, SA5, SA6, SA9, and SA11 are performed by, for example, the setting information acquisition unit 123 of the aircraft 1.

[0137] After identifying the shortest face marker ML, calculating the positional relationship between the shortest face marker ML and the face photography position S, and updating the flight plan, the aircraft 1 sends a signal indicating that all information settings are complete to the user terminal 2. Upon receiving the signal, the user terminal 2 becomes able to input a command to the aircraft 1 to begin flight.

[0138] The user inputs a flight commencement command into user terminal 2 (step SA12). A flight commencement signal is transmitted from user terminal 2 to aircraft 1.

[0139] When aircraft 1 receives a flight start signal from user terminal 2, flight control is performed based on the identified flight plan, for example, flying from the initial marker MS's installation position (takeoff position) to the shortest face marker ML by following marker M (step SA13).

[0140] When the aircraft 1 flies to the position of the shortest face marker ML, the aircraft 1 performs the outward flight from the position of the shortest face marker ML to the face photography position S based on the flight control described later (step SA14).

[0141] Aircraft 1 stops at the face photography position S and photographs the face K with camera 106 (step SA15). Image data of the tunnel face K, captured by camera 106, is stored in memory unit 122.

[0142] Aircraft 1 performs a return flight to the initial marker MS's installation position (takeoff position) (step SA16). The return flight is performed based on the reverse flight control and flight plan of the outward flight from the position of the shortest face marker ML (described later) to the face photography position S.

[0143] When the aircraft 1 returns to its takeoff position, it lands and outputs the image data of the tunnel face K stored in the memory unit 122 to, for example, the user terminal 2 (step SA17). The user terminal 2, upon receiving the image data of the tunnel face K, may store the image data of the tunnel face K in the memory 206. Furthermore, the image data of face K may be output to user terminal 2 simultaneously with its acquisition.

[0144] Furthermore, the aircraft 1 may transmit a signal to the user terminal 2 indicating that the flight has ended.

[0145] Figure 10 is a flowchart showing an example of the processing flow performed in this embodiment. The flowchart shown in Figure 10 illustrates the process flow when the aircraft 1 flies from the position of the shortest face marker ML to the face photography position S, under the control of the control device 102.

[0146] When the control by the control device 102 starts, it first determines whether the aircraft 1 has reached the shortest face marker ML (step SB1).

[0147] The decision in step SB1 is made by the marker arrival determination unit 124. If the decision in step SB1 is NO, return to the step before step SB1.

[0148] If the decision in step SB1 is YES, the following processes are performed in parallel: acquisition of point cloud data by scanner 105 (step SB2) to acquisition of self-position of tunnel face (step SB7), acquisition of image data by camera 106 (step SB3) to acquisition of image marker flight distance (step SB9), and acquisition of inertial measurement data by inertial sensor 107 (step SB4) to acquisition of inertial measurement marker flight distance (step SB12).

[0149] In step SB2, point cloud data is acquired by the point cloud data acquisition unit 125.

[0150] Following the acquisition of point cloud data (step SB2), it is determined whether or not it is possible to estimate the tunnel face K based on the acquired point cloud data (step SB5).

[0151] The decision in step SB5 is made by the face estimation feasibility determination unit 126. If the decision in step SB5 is NO, return to the step before step SB2.

[0152] If the decision in step SB5 is YES, the face K is estimated based on the acquired point cloud data (step SB6). The estimation of the face K in step SB6 is performed by the face aircraft position relationship acquisition unit 127.

[0153] Obtain the estimated positional relationship between the projectile face K and the aircraft 1 (step SB7). The positional relationship between the face K and the aircraft 1 in step SB7 is acquired by the face-aircraft positional relationship acquisition unit 127.

[0154] Image data acquisition in step SB3 is performed by the image data acquisition unit 129.

[0155] Following the acquisition of image data (step SB3), it is determined whether or not it is possible to obtain the flight distance of aircraft 1 from the shortest face marker ML based on the acquired image data (step SB8).

[0156] The decision in step SB8 is made by the image flight distance acquisition feasibility determination unit 130. If the decision in step SB8 is NO, return to the step before step SB3.

[0157] If the determination in step SB8 is YES, the flight distance traveled by aircraft 1 from the shortest face marker ML is obtained based on the acquired image data (step SB9). In step SB9, the flight distance traveled by the aircraft 1 from the shortest face marker ML is acquired by the image flight distance acquisition unit 131.

[0158] The acquisition of inertial measurement data for step SB4 is performed by the inertial measurement data acquisition unit 133.

[0159] Following the acquisition of inertial measurement data (step SB4), it is determined whether or not it is possible to obtain the flight distance of aircraft 1 from the shortest face marker ML based on the acquired inertial measurement data (step SB11).

[0160] The decision in step SB11 is made by the inertial measurement flight distance acquisition feasibility determination unit 134. If the decision in step SB11 is NO, return to the step before step SB4.

[0161] If the decision in step SB11 is YES, the flight distance traveled by aircraft 1 from the shortest face marker ML is obtained based on the acquired inertial measurement data (step SB12). The flight distance traveled by the aircraft 1 from the shortest face marker ML is acquired by the inertial measurement flight distance acquisition unit 135.

[0162] The decisions and processing in steps SB14 to SB21 are performed by the output control unit 137.

[0163] Following steps SB7, SB9, and SB12, it is determined whether or not the tunnel face K has been estimated based on the point cloud data (step SB14).

[0164] The output control unit 137 determines that the tunnel face K has been estimated based on the acquired point cloud data if the tunnel face estimation feasibility determination unit 126 determines that it is possible (step SB5 is YES), and determines that the tunnel face K has not been estimated based on the acquired point cloud data if the tunnel face estimation feasibility determination unit 126 determines that it is not possible (step SB5 is NO).

[0165] Alternatively, if step SB5 is YES, the face aircraft position relationship acquisition unit 127 has acquired the position relationship between the face K and the aircraft 1. If step SB5 is NO, the face aircraft position relationship acquisition unit 127 has not acquired the position relationship between the face K and the aircraft 1. Therefore, in step SB14, the output control unit 137 may determine whether the face estimation feasibility determination unit 126 can make a decision based on whether the face aircraft position relationship acquisition unit 127 has acquired the position relationship between the face K and the aircraft 1, and then determine whether the face K has been estimated based on the acquired point cloud data.

[0166] If the determination in step SB14 is YES, the output control unit 137 controls the motor 112 via the motor control circuit 104 so that the positional relationship between the tunnel face K and the aircraft 1 acquired by the tunnel face aircraft position relationship acquisition unit 127 becomes the pre-set positional relationship information KSI from the tunnel face K to the tunnel face shooting position S, that is, so that its own position becomes the tunnel face shooting position S based on the positional relationship between the tunnel face K and the aircraft 1 acquired by the tunnel face aircraft position relationship acquisition unit 127, and flies the aircraft 1 to the tunnel face shooting position S (step SB15).

[0167] If the decision in step SB14 is NO, then it is determined whether or not the flight distance traveled by aircraft 1 from the shortest face marker ML has been obtained based on the image data (step SB16).

[0168] If the image flight distance acquisition feasibility determination unit 130 determines that it is possible (step SB8 is YES), the output control unit 131 determines that the flight distance traveled by the aircraft 1 from the shortest face marker ML has been acquired based on the acquired image data. If the image flight distance acquisition feasibility determination unit 130 determines that it is not possible (step SB8 is NO), the output control unit 131 determines that the flight distance traveled by the aircraft 1 from the shortest face marker ML has been acquired based on the acquired image data.

[0169] Alternatively, if step SB8 is YES, the image flight distance acquisition unit 131 has acquired the flight distance traveled by the aircraft 1 from the shortest face marker ML. If step SB8 is NO, the image flight distance acquisition unit 131 has not acquired the flight distance traveled by the aircraft 1 from the shortest face marker ML. Therefore, in step SB16, the output control unit 137 may determine whether the image flight distance acquisition feasibility determination unit 130 can make a decision based on whether the image flight distance acquisition unit 131 has acquired the flight distance traveled by the aircraft 1 from the shortest face marker ML, and then determine whether the flight distance traveled by the aircraft 1 from the shortest face marker ML has been acquired based on the acquired image data.

[0170] If the determination in step SB16 is YES, the output control unit 137 controls the motor 112 via the motor control circuit 104 so that the flight distance traveled by the aircraft 1 from the shortest face marker ML acquired by the image flight distance acquisition unit 131 is equal to the flight distance from the shortest face marker ML to the face shooting position S, that is, based on the flight distance traveled by the aircraft 1 from the shortest face marker ML acquired by the image flight distance acquisition unit 131, its own position becomes the face shooting position S, and flies the aircraft 1 to the face shooting position S (step SB17).

[0171] If the decision in step SB16 is NO, then it is determined whether or not the flight distance traveled by aircraft 1 from the shortest face marker ML has been obtained based on the inertial measurement data (step SB18).

[0172] If the Inertial Measurement Flight Distance Acquisition Feasibility Determination Unit 134 determines that it is possible (step SB11 is YES), the output control unit 137 determines that the flight distance traveled by the aircraft 1 from the shortest face marker ML has been acquired based on the acquired inertial measurement data. If the Inertial Measurement Flight Distance Acquisition Feasibility Determination Unit 134 determines that it is not possible (step SB11 is NO), the output control unit 137 determines that the flight distance traveled by the aircraft 1 from the shortest face marker ML has not been acquired based on the acquired inertial measurement data.

[0173] Alternatively, if step SB11 is YES, the inertial measurement flight distance acquisition unit 135 has acquired the flight distance traveled by the aircraft 1 from the shortest face marker ML. If step SB11 is NO, the inertial measurement flight distance acquisition unit 135 has not acquired the flight distance traveled by the aircraft 1 from the shortest face marker ML. Therefore, in step SB18, the output control unit 137 may determine whether the inertial measurement flight distance acquisition feasibility determination unit 134 can make a decision based on whether the inertial measurement flight distance acquisition unit 135 has acquired the flight distance traveled by the aircraft 1 from the shortest face marker ML, and then determine whether the flight distance traveled by the aircraft 1 from the shortest face marker ML has been acquired based on the acquired inertial measurement data.

[0174] If the determination in step SB18 is YES, the output control unit 137 controls the motor 112 via the motor control circuit 104 so that the flight distance traveled by the aircraft 1 from the shortest face marker ML acquired by the inertial measurement flight distance acquisition unit 135 is equal to the flight distance from the shortest face marker ML to the face photography position S, that is, based on the flight distance traveled by the aircraft 1 from the shortest face marker ML acquired by the inertial measurement flight distance acquisition unit 135, the aircraft 1's own position becomes the face photography position S, and flies the aircraft 1 to the face photography position S (step SB19).

[0175] If the result in step SB18 is NO, error handling is performed (step SB20), and the process terminates. Error handling includes at least one of the following processes: notifying user terminal 2 that an error has occurred, landing on the spot, or detecting the shortest face marker ML and returning to the shortest face marker ML.

[0176] After controlling the aircraft 1 to fly to the face photography position S by step SB15, step SB17, or step SB19, it is determined whether or not the aircraft 1 has reached the face photography position S (step SB21).

[0177] If the decision in step SB21 is YES, the process of flying from the position of the shortest face marker ML to the face photography position S, controlled by the control device 102, is terminated. If the decision in step SB21 is NO, the process returns to the point between step SB1 and the parallel steps SB2, SB3, and SB4.

[0178] According to this embodiment, the shortest face marker ML is placed on the tunnel entrance side of the face photography position S, and the aircraft 1 autonomously flies from this shortest face marker ML to the face photography position S. Therefore, there is no need to install a marker at the face photography position S, which is close to the unstable face K, thus increasing safety. Furthermore, there is no need to re-install the shortest face marker ML each time the face K progresses, thus reducing the effort required for marker installation work. Furthermore, since the aircraft's position is determined by its own position, estimation from point cloud data, estimation from image data, and estimation from measurements by internal sensors, the aircraft can be reliably controlled to the face shooting position S.

[0179] [Second Embodiment] A second embodiment of the present invention will be described below with reference to Figure 11. Note that we will omit explanations of parts that are the same as in the first embodiment and will mainly explain the parts that are different.

[0180] Figure 11 is a flowchart illustrating an example of the processing flow performed by the aircraft control system.

[0181] The second embodiment differs from the first embodiment in that steps SB4, SB11, SB12, SB18, and SB19 are not performed.

[0182] The flight control by the output control unit 137 for the outward journey from the shortest face marker ML to the face shooting position S employs two methods: controlling the position relationship between the face K and the aircraft 1 acquired by the face aircraft position relationship acquisition unit 127 so that it becomes the pre-set position relationship information KSI from the face K to the face shooting position S (face aircraft position relationship control); and controlling the flight distance traveled by the aircraft 1 from the shortest face marker ML acquired by the image flight distance acquisition unit 131 so that it becomes the pre-set flight distance from the shortest face marker ML to the face shooting position S (image marker flight distance control).

[0183] The output control unit 137 controls the flight of the aircraft 1 with priority 1 for controlling the positional relationship of the aircraft face and priority 2 for controlling the flight distance of the image marker.

[0184] When the output control unit 137 determines that the tunnel face K has been estimated based on the point cloud data (when step SB14 is YES), it controls the flight of the aircraft 1 to the tunnel face photography position S based on the estimated positional relationship between the tunnel face K and the aircraft 1 acquired by the tunnel face aircraft positional relationship acquisition unit 127 (step SB15).

[0185] When the output control unit 137 determines that the tunnel face K has not been estimated based on the point cloud data (when step SB14 is NO), and determines that the flight distance traveled by the aircraft 1 from the shortest tunnel face marker ML has been obtained based on the image data (when step SB16 is YES), it controls the flight of the aircraft 1 to the tunnel face shooting position S based on the flight distance traveled by the aircraft 1 from the shortest tunnel face marker ML obtained by the image flight distance acquisition unit 131 (step SB17).

[0186] If the result in step SB16 is NO, error handling is performed (step SB20), and the process terminates.

[0187] After controlling the aircraft 1 to fly to the face photography position S by either step SB15 or step SB17, it is determined whether or not the aircraft 1 has reached the face photography position S (step SB21).

[0188] If the decision in step SB21 is YES, the process of flying from the position of the shortest face marker ML to the face photography position S, controlled by the control device 102, is terminated. If the decision in step SB21 is NO, the process returns to the point between step SB1 and the parallel steps SB2 and SB3.

[0189] [Third Embodiment] A third embodiment of the present invention will be described below with reference to Figure 12. Note that we will omit explanations of parts that are the same as those in the first and second embodiments, and will mainly describe the parts that are different.

[0190] Figure 12 is a flowchart showing an example of the processing flow performed by the aircraft control system.

[0191] The third embodiment differs from the first embodiment in that steps SB3, SB8, SB9, SB16, and SB17 are not performed.

[0192] The flight control by the output control unit 137 for the outward journey from the shortest face marker ML to the face photography position S employs the following methods: controlling the position relationship between the face K and the aircraft 1 acquired by the face aircraft position relationship acquisition unit 127 so that it becomes the pre-set position relationship information KSI from the face K to the face photography position S (face aircraft position relationship control); and controlling the flight distance traveled by the aircraft 1 from the shortest face marker ML acquired by the inertial measurement flight distance acquisition unit 135 so that it becomes the pre-set flight distance from the shortest face marker ML to the face photography position S (inertial measurement marker flight distance control).

[0193] The output control unit 137 controls the flight of the aircraft 1 with priority 1 for controlling the relative position of the aircraft face and priority 2 for controlling the flight distance of the inertial measurement marker.

[0194] When the output control unit 137 determines that the tunnel face K has been estimated based on the point cloud data (when step SB14 is YES), it controls the flight of the aircraft 1 to the tunnel face photography position S based on the estimated positional relationship between the tunnel face K and the aircraft 1 acquired by the tunnel face aircraft positional relationship acquisition unit 127 (step SB15).

[0195] When the output control unit 137 determines that the tunnel face K has not been estimated based on the point cloud data (when step SB14 is NO), and determines that the flight distance traveled by the aircraft 1 from the shortest tunnel face marker ML has been obtained based on the inertial measurement data (when step SB18 is YES), it controls the flight of the aircraft 1 to the tunnel face photography position S based on the flight distance traveled by the aircraft 1 from the shortest tunnel face marker ML obtained by the inertial measurement flight distance acquisition unit 135 (step SB19).

[0196] If the result in step SB18 is NO, error handling is performed (step SB20), and the process terminates.

[0197] After controlling the aircraft 1 to fly to the face photography position S by either step SB15 or step SB19, it is determined whether or not the aircraft 1 has reached the face photography position S (step SB21).

[0198] If the decision in step SB21 is YES, the process of flying from the position of the shortest face marker ML to the face photography position S, controlled by the control device 102, is terminated. If the decision in step SB21 is NO, the process returns to the point between step SB1 and the parallel steps SB2 and SB4.

[0199] [Fourth Embodiment] A fourth embodiment of the present invention will be described below with reference to Figure 13. Note that the same parts as in the first to third embodiments will be omitted from the explanation, and the differences will be explained primarily.

[0200] Figure 13 is a flowchart showing an example of the processing flow performed by the aircraft control system.

[0201] The fourth embodiment differs from the first embodiment in that steps SB2, SB5, SB6, SB7, SB14, and SB15 are not performed.

[0202] The flight control of the outward journey from the shortest face marker ML to the face photography position S by the output control unit 137 employs two methods: one that controls the flight distance traveled by the aircraft 1 from the shortest face marker ML acquired by the image flight distance acquisition unit 131 so that it becomes the pre-set flight distance from the shortest face marker ML to the face photography position S (image marker flight distance control); and another that controls the flight distance traveled by the aircraft 1 from the shortest face marker ML acquired by the inertial measurement flight distance acquisition unit 135 so that it becomes the pre-set flight distance from the shortest face marker ML to the face photography position S (inertial measurement marker flight distance control).

[0203] The output control unit 137 controls the flight of the aircraft 1 with priority 1 for image marker flight distance control and priority 2 for inertial measurement marker flight distance control.

[0204] When the output control unit 137 determines that the flight distance traveled by the aircraft 1 from the shortest face marker MK has been obtained based on the image data (when step SB16 is YES), it controls the flight of the aircraft 1 to the face shooting position S based on the flight distance traveled by the aircraft 1 from the shortest face marker ML obtained by the image flight distance acquisition unit 131 (step SB17).

[0205] When the output control unit 137 determines that the flight distance traveled by the aircraft 1 from the shortest face marker ML has been acquired based on the image data (when step SB16 is NO), and determines that the flight distance traveled by the aircraft 1 from the shortest face marker ML has been acquired based on the inertial measurement data (when step SB18 is YES), the output control unit 137 controls the flight of the aircraft 1 to the face shooting position S based on the flight distance traveled by the aircraft 1 from the shortest face marker ML acquired by the inertial measurement flight distance acquisition unit 135 (step SB19).

[0206] If the result in step SB18 is NO, error handling is performed (step SB20), and the process terminates.

[0207] After controlling the aircraft 1 to fly to the face photography position S by either step SB17 or step SB19, it is determined whether or not the aircraft 1 has reached the face photography position S (step SB21).

[0208] If the decision in step SB21 is YES, the process of flying from the position of the shortest face marker ML to the face photography position S, controlled by the control device 102, is terminated. If the decision in step SB21 is NO, the process returns to the point between step SB1 and the parallel steps SB3 and SB4.

[0209] [Other variations] The present invention is not limited to the embodiments described above. For example, the following are also included.

[0210] In this embodiment, control was performed to stop the aircraft 1 at the face photography position S in order to photograph the face K. However, it is also possible to photograph the face K while the aircraft is flying without stopping, upon reaching the face photography position S. Furthermore, even when recording video, it is possible to use the video data corresponding to the point in time when the aircraft reached the face photography position S.

[0211] In this embodiment, an inertial sensor is used as the internal sensor, but other measuring sensors, such as altimeters, speedometers, accelerometers, and gyroscopes, can also be used. Furthermore, the internal measurement data may include control information such as the rotation speed of the aircraft's propellers. This allows the internal measurement flight distance acquisition unit to acquire the flight distance traveled by the aircraft 1 from the shortest face marker ML.

[0212] In this embodiment, the identification of the shortest face marker and the calculation of the distance between the shortest face marker and the face photography position are performed by the aircraft, but it is also possible to perform these operations on a user terminal and transmit the results to the aircraft.

[0213] In this embodiment, the autonomous flight of the aircraft 1 was controlled between the shortest face marker ML and the face photography position S. However, it is not limited to this, and it may also be applied to the control of the autonomous flight of the aircraft between a marker M located on the tunnel entrance side of the shortest face marker ML (for example, a marker M one tunnel entrance side of the shortest face marker ML) and the face photography position S. In this case, the marker M located on the tunnel entrance side is treated as the shortest face marker ML in the embodiment, and the control is configured to be performed using the same settings.

[0214] The technical aspects of each embodiment, including the modified examples, may be combined and applied to other embodiments to form examples. [Explanation of symbols]

[0215] 1000 Flight Systems 1. Flying object 100 bus 101 Battery 102 Control device 103 Input / Output Devices 104 Motor control circuit 105 Scanner 106 Camera 107 Inertial Sensor 108 Communication equipment 110 Hardware Processors 111 memory 112 Motor 120 Input / output section 121 Communications Department 122 Storage section 123 Configuration Information Acquisition Unit 124 Marker arrival determination unit 125 Point cloud data acquisition unit 126. Section for determining whether or not to estimate the working face. 127 Sectional Aircraft Position Relationship Acquisition Unit 129 Image Data Acquisition Unit 130 Image flight distance acquisition availability judgment unit 131 Image Flight Distance Acquisition Unit 133 Inertial measurement data acquisition unit 134 Inertial measurement flight distance acquisition feasibility determination unit 135 Inertial measurement flight distance acquisition unit 137 Output Control Unit 2 User terminals 200 buses 201 Battery 202 Control device 203 Input / Output Devices 204 Communication equipment 205 Hardware Processors 206 memory 210 Input / output section 211 Communications Department

Claims

1. An aircraft control device that controls an aircraft from a marker installed inside the tunnel to a position for photographing the tunnel face, The aforementioned flying object has a scanner that emits a laser to acquire point cloud data, a camera, and an internal sensor. The aforementioned aircraft control device is A point cloud data acquisition unit that acquires point cloud data acquired by the scanner, A face estimation feasibility determination unit that determines whether or not it is possible to estimate the face based on the point cloud data acquired by the point cloud data acquisition unit, When the face estimation feasibility determination unit determines that it is possible, the face-to-flying object position relationship acquisition unit estimates the face based on the point cloud data acquired by the point cloud data acquisition unit and acquires the position relationship between the estimated face and the flying object. An image data acquisition unit that acquires image data acquired by the aforementioned camera, An image flight distance acquisition feasibility determination unit determines whether it is possible to acquire the flight distance traveled by the aircraft from the marker based on the image data acquired by the image data acquisition unit, When the image flight distance acquisition feasibility determination unit determines that it is possible, the image flight distance acquisition unit acquires the flight distance traveled by the aircraft from the marker based on the image data acquired by the image data acquisition unit, An internal measurement data acquisition unit that acquires internal measurement data measured by the internal sensor, An internal measurement flight distance acquisition feasibility determination unit determines whether it is possible to acquire the flight distance traveled by the aircraft from the marker based on the internal measurement data acquired by the internal measurement data acquisition unit, When the Internal Measurement Flight Distance Acquisition Feasibility Determination Unit determines that acquisition is possible, the Internal Measurement Flight Distance Acquisition Unit acquires the flight distance traveled by the aircraft from the marker based on the Internal Measurement Data Acquisition Unit acquired, The aircraft comprises an output control unit for controlling the flight of the aircraft, The output control unit, When it is determined that the tunnel face has been estimated based on the point cloud data, the tunnel face aircraft position relationship acquisition unit controls the flight of the aircraft to the tunnel face photography position based on the estimated position relationship between the tunnel face and the aircraft acquired by the unit. When it is determined that the tunnel face has not been estimated based on the point cloud data, and when it is determined that the flight distance traveled by the aircraft from the marker has been obtained based on the image data, the image flight distance acquisition unit controls the flight of the aircraft to the tunnel face shooting position based on the flight distance traveled by the aircraft from the marker obtained by the unit. When it is determined that the tunnel face has not been estimated based on the point cloud data, and when it is determined that the flight distance traveled by the aircraft from the marker has not been obtained based on the image data, and when it is determined that the flight distance traveled by the aircraft from the marker has been obtained based on the internal measurement data, the internal measurement flight distance acquisition unit controls the flight of the aircraft to the tunnel face shooting position based on the flight distance traveled by the aircraft from the marker that it has acquired. A flight control device characterized by the following features.

2. An aircraft control device that controls an aircraft from a marker installed inside the tunnel to a position for photographing the tunnel face, The aforementioned flying object has a scanner that emits a laser to acquire point cloud data, and a camera. The aforementioned aircraft control device is A point cloud data acquisition unit that acquires point cloud data acquired by the scanner, A face estimation feasibility determination unit that determines whether or not it is possible to estimate the face based on the point cloud data acquired by the point cloud data acquisition unit, When the face estimation feasibility determination unit determines that it is possible, the face-to-flying object position relationship acquisition unit estimates the face based on the point cloud data acquired by the point cloud data acquisition unit and acquires the position relationship between the estimated face and the flying object. An image data acquisition unit that acquires image data acquired by the aforementioned camera, An image flight distance acquisition feasibility determination unit determines whether it is possible to acquire the flight distance traveled by the aircraft from the marker based on the image data acquired by the image data acquisition unit, When the image flight distance acquisition feasibility determination unit determines that it is possible, the image flight distance acquisition unit acquires the flight distance traveled by the aircraft from the marker based on the image data acquired by the image data acquisition unit, The aircraft comprises an output control unit for controlling the flight of the aircraft, The output control unit, When it is determined that the tunnel face has been estimated based on the point cloud data, the tunnel face aircraft position relationship acquisition unit controls the flight of the aircraft to the tunnel face photography position based on the estimated position relationship between the tunnel face and the aircraft acquired by the unit. When it is determined that the tunnel face has not been estimated based on the point cloud data, and when it is determined that the flight distance traveled by the aircraft from the marker has been obtained based on the image data, the image flight distance acquisition unit controls the aircraft's flight to the tunnel face shooting position based on the flight distance traveled by the aircraft from the marker obtained. A flight control device characterized by the following features.

3. An aircraft control device that controls an aircraft from a marker installed inside the tunnel to a position for photographing the tunnel face, The aforementioned flying object has a scanner that emits a laser to acquire point cloud data, and an internal sensor. The aforementioned aircraft control device is A point cloud data acquisition unit that acquires point cloud data acquired by the scanner, A face estimation feasibility determination unit that determines whether or not it is possible to estimate the face based on the point cloud data acquired by the point cloud data acquisition unit, When the face estimation feasibility determination unit determines that it is possible, the face-to-flying object position relationship acquisition unit estimates the face based on the point cloud data acquired by the point cloud data acquisition unit and acquires the position relationship between the estimated face and the flying object. An internal measurement data acquisition unit that acquires internal measurement data measured by the internal sensor, An internal measurement flight distance acquisition feasibility determination unit determines whether it is possible to acquire the flight distance traveled by the aircraft from the marker based on the internal measurement data acquired by the internal measurement data acquisition unit, When the Internal Measurement Flight Distance Acquisition Feasibility Determination Unit determines that acquisition is possible, the Internal Measurement Flight Distance Acquisition Unit acquires the flight distance traveled by the aircraft from the marker based on the Internal Measurement Data Acquisition Unit acquired, The aircraft comprises an output control unit for controlling the flight of the aircraft, The output control unit, When it is determined that the tunnel face has been estimated based on the point cloud data, the tunnel face aircraft position relationship acquisition unit controls the flight of the aircraft to the tunnel face photography position based on the estimated position relationship between the tunnel face and the aircraft acquired by the unit. When it is determined that the tunnel face has not been estimated based on the point cloud data, and when it is determined that the flight distance traveled by the aircraft from the marker has been obtained based on the internal measurement data, the internal measurement flight distance acquisition unit controls the aircraft's flight to the tunnel face photography position based on the flight distance traveled by the aircraft from the marker that it has obtained. A flight control device characterized by the following features.

4. An aircraft control device that controls an aircraft from a marker installed inside the tunnel to a position for photographing the tunnel face, The aforementioned flying object has a camera and an internal sensor, The aforementioned aircraft control device is An image data acquisition unit that acquires image data acquired by the aforementioned camera, An image flight distance acquisition feasibility determination unit determines whether it is possible to acquire the flight distance traveled by the aircraft from the marker based on the image data acquired by the image data acquisition unit, When the image flight distance acquisition feasibility determination unit determines that it is possible, the image flight distance acquisition unit acquires the flight distance traveled by the aircraft from the marker based on the image data acquired by the image data acquisition unit, An internal measurement data acquisition unit that acquires internal measurement data measured by the internal sensor, An internal measurement flight distance acquisition feasibility determination unit determines whether it is possible to acquire the flight distance traveled by the aircraft from the marker based on the internal measurement data acquired by the internal measurement data acquisition unit, When the Internal Measurement Flight Distance Acquisition Feasibility Determination Unit determines that acquisition is possible, the Internal Measurement Flight Distance Acquisition Unit acquires the flight distance traveled by the aircraft from the marker based on the Internal Measurement Data Acquisition Unit acquired, The aircraft comprises an output control unit for controlling the flight of the aircraft, The output control unit, When it is determined that the flight distance traveled by the aircraft from the marker has been obtained based on the image data, the image flight distance acquisition unit controls the flight of the aircraft to the face shooting position based on the flight distance traveled by the aircraft from the marker obtained. When it is determined that the flight distance traveled by the aircraft from the marker has not been obtained based on the image data, and when it is determined that the flight distance traveled by the aircraft from the marker has been obtained based on the internal measurement data, the internal measurement flight distance acquisition unit controls the flight of the aircraft to the face photography position based on the flight distance traveled by the aircraft from the marker that it has acquired. A flight control device characterized by the following features.

5. The internal sensor is an inertial sensor. The aircraft control device according to claim 1, claim 3, or claim 4.

Images