Self-location estimation system and method for unmanned aerial vehicle, unmanned aerial vehicle, program, and storage medium

Abstract

Provided is a system in which an uncrewed aircraft performs autonomous flight so as to follow the outer wall of a structure such as a tunnel.　This self-position estimation system includes: a vertical-section measurement data acquisition unit 138 that acquires vertical-section measurement data measured using a vertical-section measurement device when flying along the outer wall surface of a structure extending along a prescribed route in the lateral direction; and a vertical-section interior position estimation unit 132 that matches a circle corresponding to an outer wall surface of the tunnel to the acquired vertical-section measurement data, and estimates the vertical-section interior position of the uncrewed aircraft within the vertical section on the basis of the self position in relation to the matched circle.

Description

[Technical field]

[0001] The present invention relates to a system and method for estimating a self-position of an unmanned aerial vehicle, an unmanned aerial vehicle, a program, and a storage medium. [Background technology]

[0002] Recently, attempts have been made to use drones to inspect the exterior walls of horizontally extending structures such as tunnels. Drones can estimate their own horizontal position using GPS satellite positioning information, as well as their own vertical position using an altimeter or the like, and fly autonomously along a flight plan route (Non-Patent Document 1 below). Then, an image of the tunnel exterior wall surface is captured by an imaging device such as a camera mounted on the unmanned aerial vehicle, and inspection is performed based on the captured image. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Kenzo Nonami, "All about drone industrial applications", Ohmsha, February 2018, p.73 Summary of the Invention [Problem to be solved by the invention]

[0004] In such an inspection using an imaging device mounted on an unmanned aerial vehicle, it is necessary to capture an image of the outer wall surface of a structure such as a tunnel from a predetermined position. However, even if a flight plan route is set so as to fly a certain distance from the outer wall surface of the tunnel and the flight is performed according to the flight plan route set based on GPS measurement data, there may be an error between the set position and the actual position of the unmanned aerial vehicle relative to the structure.

[0005] Therefore, one of the objectives of the present invention is to provide a system and method that enables an unmanned aerial vehicle to fly autonomously at a predetermined position relative to a structure such as a tunnel when flying outside the structure. [Means for solving the problem]

[0006] One aspect of the present invention provides a self-position estimation system for an unmanned aerial vehicle flying along an outer wall surface of a structure extending laterally along a predetermined path, wherein the unmanned aerial vehicle includes a longitudinal section measurement device that measures the boundary shape within a longitudinal section in a direction intersecting the flight path of the unmanned aerial vehicle and generates longitudinal section measurement data, and the self-position estimation system includes a longitudinal section measurement data acquisition unit that acquires the longitudinal section measurement data measured by the longitudinal section measurement device when the unmanned aerial vehicle flies along the outer wall surface of the structure extending laterally along a predetermined path, and a longitudinal section position estimation unit that matches the acquired longitudinal section measurement data with a first predetermined shape corresponding to the outer wall surface of the structure, and estimates the position of the unmanned aerial vehicle within the longitudinal section based on the self-position relative to the matched predetermined shape.

[0007] In one aspect of the present invention, the first predetermined shape corresponds at least in part to a geometric shape, and the longitudinal section position estimation unit matches the geometric shape to the longitudinal section measurement data and estimates the position within the longitudinal section based on the self-position relative to the matched geometric shape.

[0008] In one aspect of the present invention, the first predetermined shape is all or part of a circle or an ellipse, and the longitudinal section position estimation unit estimates the longitudinal section position of the unmanned aerial vehicle within the longitudinal section as a distance based on a center point of the first predetermined shape and an angle relative to a reference axis.

[0009] In one aspect of the present invention, the longitudinal section measurement data is discrete data, and the intra-longitudinal section position estimating unit removes outliers in the discrete data and performs matching.

[0010] In one aspect of the present invention, the system further includes a flight control unit that controls the flight of the unmanned aircraft so that it flies along the flight plan route based on the position of the unmanned aircraft within the longitudinal section estimated by the longitudinal section position estimation unit.

[0011] In one aspect of the present invention, a flight plan route is set as a plurality of waypoints, and the waypoints are set including an angle relative to a reference axis passing through a reference point of the structure corresponding to a center point of a first predetermined shape in a longitudinal section, and a distance from an outer wall surface of the structure at the angle.

[0012] In one aspect of the present invention, the unmanned aerial vehicle includes an imaging device, an attitude adjustment device that adjusts the attitude of the imaging device, and a longitudinal section measurement device that measures the boundary shape within a longitudinal section in a direction transverse to the flight path of the unmanned aerial vehicle and generates longitudinal section measurement data, and the flight control system further includes an attitude control unit that controls the attitude adjustment device so that the imaging direction of the imaging device is directed toward the outer wall surface of the structure, in accordance with the positional relationship between the unmanned aerial vehicle's own position estimated by the longitudinal section position estimation unit and the structure.

[0013] In one aspect of the present invention, the first predetermined shape is all or part of a circle or an ellipse, and the attitude control unit controls the attitude adjustment device so that the imaging direction of the imaging device faces the center or focus of the matching circle or ellipse.

[0014] One aspect of the present invention provides an unmanned aerial vehicle having the above-described self-location estimation system.

[0015] One aspect of the present invention provides a self-position estimation method for an unmanned aerial vehicle executed by a computer, wherein the unmanned aerial vehicle includes a longitudinal section measuring device that measures a boundary shape within a longitudinal section in a direction intersecting the flight path of the unmanned aerial vehicle and generates longitudinal section measurement data, the self-position estimation method including: a longitudinal section acquisition step of acquiring the longitudinal section measurement data measured by the longitudinal section measuring device when the unmanned aerial vehicle flies along an outer wall surface of a structure extending laterally along a predetermined path; and a longitudinal section position estimation step of matching the acquired longitudinal section measurement data with a first predetermined shape corresponding to the outer wall surface of the structure, and estimating the position of the unmanned aerial vehicle within the longitudinal section based on the self-position relative to the matched predetermined shape.

[0016] One aspect of the present invention provides a program for causing a computer to execute the above-described self-location estimation method.

[0017] One aspect of the present invention provides a computer-readable recording medium having the above-mentioned program recorded thereon. Effect of the Invention

[0018] SUMMARY OF THE PRESENT EMBODIMENTS In accordance with the present invention, a system and method are provided for enabling an unmanned aerial vehicle to fly autonomously to a predetermined position relative to a structure, such as a tunnel, when flying outside the structure. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 is an external view of a multicopter, which is an example of an unmanned aerial vehicle (multicopter) according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram illustrating a flight control system for the unmanned aerial vehicle shown in FIG. [Diagram 3] 2 is a block diagram showing the configuration of an information processing unit of the unmanned aerial vehicle shown in FIG. 1. [Figure 4] 2 is a diagram illustrating a hardware configuration of an information processing unit of the unmanned aerial vehicle shown in FIG. 1. [Figure 5A] FIG. 2 is a vertical cross-section of a tunnel illustrating the tunnel and flight plan path in one embodiment of the present invention. [Figure 5B] FIG. 2 is a top view illustrating a tunnel and a flight plan path in one embodiment of the present invention. [Figure 6] 13 is a diagram for explaining a method for estimating the position of an unmanned aerial vehicle within a longitudinal section. FIG. [Figure 7] 4 is a flowchart showing the flow of autonomously flying the unmanned aerial vehicle shown in FIG. 1 inside a tunnel and photographing the inner wall surface of the tunnel. [Figure 8] 2 is a flowchart showing in detail the flow of autonomous flight by the unmanned aerial vehicle shown in FIG. 1. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. However, the present invention is not limited to the specific embodiment described below, and various embodiments can be taken within the scope of the technical concept of the present invention. For example, the unmanned aerial vehicle of the present invention is not limited to the multicopter shown in FIG. 1, but may be any unmanned aerial vehicle such as a rotorcraft or a fixed-wing aircraft. In addition, the system configuration of the unmanned aerial vehicle 1 is not limited to that shown in the figure, and may be any configuration as long as the same operation is possible. For example, the operation performed by multiple components may be performed by a single component, such as integrating the function of a communication circuit into a flight control unit, or the operation performed by a single component may be performed by multiple components, such as distributing the function of a main calculation unit to multiple calculation units. In addition, various data stored in the memory of the unmanned aerial vehicle 1 may be stored in a different location from the memory, and information recorded in various memories may be stored by distributing one type of information to multiple types, or multiple types of information may be stored together in one type.

[0021] In the following description, an unmanned aerial vehicle autonomously flies outside a horizontally extending tunnel and captures images of the tunnel's outer wall surface. The shape of the tunnel's outer wall surface is known from blueprints and the like, and in this embodiment, the shape of the tunnel's outer wall surface in a vertical cross section (longitudinal cross section) is semicircular, with the central semicircular portion extending horizontally (laterally) in the same cross section. Note that protrusions are provided at intervals on the outer wall surface in the circumferential and depth directions.

[0022] FIG. 1 is an external view of a multicopter, which is an example of an unmanned aerial vehicle (multicopter) 1 according to an embodiment of the present invention. The unmanned aerial vehicle 1 includes, in terms of appearance, a control unit 101, six motors 102 driven by control signals from the control unit 101, six rotors (rotating wings) 103 that rotate by driving the respective motors 102 to generate lift, six arms 104 connecting the control unit 101 and the respective motors 102, and landing legs 105 that support the unmanned aerial vehicle when landing. The number of motors 102, rotors 103, and arms 104 may each be three or more, such as three or four. The six motors 102 are rotated by control signals from the control unit 101, and the rotation speed of each of the six rotors 103 is thereby controlled, thereby controlling the flight of the unmanned aerial vehicle 1, such as ascent, descent, forward / backward / left / right flight, and turning.

[0023] In addition, a pedestal 106 is attached above the control unit 101. A camera 108 for photographing an object at high resolution is attached to the pedestal 106 via an attitude adjustment device 107 that supports the camera 108 so that it can rotate horizontally and vertically.

[0024] In addition, a stand 110 is attached below the control unit 101. In front of the stand 110, a vertical section measuring device (vertical section LIDAR) 109 is provided for measuring the shape of the surrounding space in a vertical section perpendicular to the central axis. The unmanned aerial vehicle 1 also has a flight position sensor 111 and an antenna 117.

[0025] Fig. 2 is a diagram showing a flight control system of the unmanned aerial vehicle shown in Fig. 1. The flight control system 150 of the unmanned aerial vehicle 1 includes a control unit 101, a motor 102 electrically connected to the control unit 101, a rotor 103 mechanically connected to the motor 102, a longitudinal section measuring device 109, a flight position sensor 111, a compass 112, an IMU 113, and an altimeter 114.

[0026] The control unit 101 is configured to process information for controlling the flight of the unmanned aerial vehicle 1 and to control the electrical signals for that purpose, and is typically a unit configured with circuits required to realize such functions by arranging and wiring various electronic components on a board. The control unit 101 further comprises an information processing unit 120, a communication circuit 121, a control signal generating unit 122, a speed controller 123, and an interface 125.

[0027] Camera 108 is a camera for capturing images of an object (the outer wall surface of a tunnel in this embodiment) at high resolution. Camera 108 is attached to base 106 via attitude adjustment device 107 that supports camera 108 so that it can rotate horizontally and vertically. Camera 108 acquires image data of its shooting range while unmanned aerial vehicle 1 is flying, and the acquired images are stored in a storage device such as a memory described below. The images are typically video images consisting of a series of still images.

[0028] The attitude adjustment device 107 is configured to change the attitude of the base 106 to which the camera 108 is attached in response to a control signal from the control unit 101, thereby making it possible to change the imaging direction of the camera 108.

[0029] In this embodiment, the longitudinal section measuring device 109 is a vertical section LIDAR, and while the unmanned aerial vehicle 1 is on the ground or hovering, a pulsed laser is irradiated at a predetermined angular interval around a horizontal axis extending in the longitudinal direction in a vertical cross section of the unmanned aerial vehicle 1, and the laser reflected by an object in the surrounding space is received, thereby measuring the distance to the object in the surrounding space based on the time from irradiation to reception. By driving the longitudinal section measuring device 109 while the unmanned aerial vehicle 1 is flying in the tunnel, the longitudinal section measuring device 109 generates longitudinal section measurement data related to the vertical cross-sectional shape (longitudinal cross-sectional shape) of the inner wall surface of the tunnel. As the longitudinal section measurement data, for example, discrete data in polar coordinates is obtained. In this embodiment, a vertical section LIDAR is used as the longitudinal section measuring device 109, but the present invention is not limited thereto. For example, by capturing images of the entire circumference of a vertical section centered on an unmanned aerial vehicle using a visible light camera, integrating the captured images to create a 3D model, it is possible to measure the shape of the inner wall of the tunnel in the vertical section and estimate the vehicle's own position in the vertical section. SfM is a known algorithm for creating such a 3D model, and Dense Visual SLAM is a known algorithm for estimating the vehicle's own position using SfM.

[0030] The flight position sensor 111 is a navigation sensor, such as a GPS (Global Positioning System) sensor, that senses the coordinates of the flight position of the unmanned aerial vehicle 1. The flight position sensor 111 preferably senses three-dimensional coordinates. The coordinates acquired by the flight position sensor 111 include latitude, longitude, and altitude. Compass 112 is a so-called compass needle, and detects the angle ahead of unmanned aerial vehicle 1 relative to north.

[0031] The IMU 113 is an inertial measurement unit that detects translational motion using an acceleration sensor and rotational motion using an angular velocity sensor (gyro). Furthermore, the IMU 113 can calculate a velocity by integrating the translational motion (acceleration) detected by the acceleration sensor, and can calculate a moving distance (position) by integrating the velocity. Similarly, the IMU 113 can calculate an angle (attitude) by integrating the rotational motion (angular velocity) detected by the angular velocity sensor.

[0032] The altimeter 114 is, for example, a barometer, and estimates the altitude of the unmanned aerial vehicle based on air pressure. Antenna 117 is an aerial for receiving radio signals including information and various data for piloting and controlling unmanned aerial vehicle 1, and for transmitting radio signals including telemetry signals from unmanned aerial vehicle 1.

[0033] The communication circuit 121 is an electronic circuit, typically a wireless signal processing IC, for demodulating the wireless signals received through the antenna 117 to receive control signals, various data, and the like for the unmanned aerial vehicle 1, and inputting the control signals and various data to the information processing unit 120, and for generating wireless signals for carrying telemetry signals and the like output from the unmanned aerial vehicle 1. Note that, for example, the communication of control signals and the communication of control signals and various data may be performed by different communication circuits in different frequency bands. For example, it is possible to adopt a configuration in which communication with a transmitter of a controller (transmitter) for manual operation is performed at a frequency of 950 MHz, and data communication is performed at a frequency of 2 GHz, 1.7 GHz, 1.5 GHz, or 800 MHz.

[0034] Control signal generating unit 122 is configured to convert control command value data obtained by calculation by information processing unit 120 into a pulse signal (such as a PWM signal) representing a voltage, and is typically an IC including an oscillator circuit and a switching circuit. Speed ​​controller 123 is configured to convert the pulse signal from control signal generating unit 122 into a drive voltage that drives motor 102, and is typically a smoothing circuit and an analog amplifier. Although not shown, unmanned aerial vehicle 1 is equipped with a power supply system including a battery device such as a lithium polymer battery or a lithium ion battery, and a power distribution system to each element.

[0035] The interface 125 is a configuration that electrically connects the information processing unit 120 and functional elements such as the camera 108, the attitude adjustment device 107, the longitudinal section measurement device 109, the flight position sensor 111, the compass 112, the IMU 113, and the altimeter 114 by converting the form of signals so that signals can be transmitted and received between them. For convenience of explanation, the interface is shown as one configuration in the drawings, but it is normal to use a different interface depending on the type of functional element to be connected. Also, depending on the type of signal input / output of the functional element to be connected, the interface 125 may not be necessary. Even if the information processing unit 120 in FIG. 2 is connected without the interface 125, depending on the type of signal input / output of the functional element to be connected, an interface may be necessary.

[0036] Fig. 3 is a block diagram showing the configuration of the information processing unit of the unmanned aerial vehicle shown in Fig. 1. As shown in Fig. 3, the information processing unit 120 includes a cross-sectional shape data storage unit 131, an optical vertical section position estimation unit 132, a self-position data generation unit 133, a flight path data storage unit 134, a flight control unit 135, a camera attitude calculation unit 136, an attitude control unit 137, a vertical section measurement data acquisition unit 138, and a sensor measurement data acquisition unit 139. In this embodiment, the cross-sectional shape data storage unit 131, the vertical section position estimation unit 132, the self-position data generation unit 133, the flight path data storage unit 134, the flight control unit 135, the camera attitude calculation unit 136, the attitude control unit 137, the vertical section measurement data acquisition unit 138, and the sensor measurement data acquisition unit 139 constitute a self-position estimation system 140 that generates self-position data regarding the self-position of the unmanned aerial vehicle.

[0037] Fig. 4 is a diagram showing a hardware configuration of the information processing unit of the unmanned aerial vehicle shown in Fig. 1. Information processing unit 120 includes CPU 120a, RAM 120b, ROM 120c, external memory 120d, input unit 120e, output unit 120f, and communication unit 120g. RAM 120b, ROM 120c, external memory 120d, input unit 120e, output unit 120f, and communication unit 120g are connected to CPU 120a via bus 120h.

[0038] The CPU 120a comprehensively controls each device connected to the system bus 120h. The ROM 120c and the external memory store the BIOS and OS, which are control programs for the CPU 120a, as well as various programs and data required to realize the functions executed by the computer.

[0039] The RAM 120b functions as a main memory, a working area, etc. for the CPU 120a. The CPU 120a loads programs and the like required for executing processes from the ROM 120c or the external memory 120d into the RAM 120b, and executes the loaded programs to realize various operations. The external memory 120d is composed of, for example, a flash memory, a hard disk, a DVD-RAM, a USB memory, or the like.

[0040] The input unit 120e receives operation instructions, etc. from a user, etc. The input unit 120e is configured with input devices, such as, for example, an input button, a keyboard, a pointing device, a wireless remote control, a microphone, and a camera.

[0041] The output unit 120f outputs data processed by the CPU 120a and data stored in the RAM 120b, the ROM 120c, and the external memory 120d. The output unit 120f is composed of output devices such as a CRT display, an LCD, an organic EL panel, a printer, and a speaker.

[0042] The communication unit 120g is an interface for connecting and communicating with an external device directly or via a network, and is configured with an interface such as a serial interface or a LAN interface.

[0043] Each unit 121, 122, 125, 131 to 139 of the self-location estimation system 140 shown in Figures 2 and 3 is realized by various programs stored in the ROM 120c and the external memory 120d using the CPU 120a, the RAM 120b, the ROM 120c, the external memory 120d, the input unit 120e, the output unit 120f, the communication unit 120g, etc. as resources.

[0044] The cross-sectional shape data storage unit 131 stores cross-sectional shape data relating to the cross-sectional shape of the inner wall surface of the tunnel. Furthermore, the flight route data storage unit 134 stores flight plan route data. The cross-sectional shape data storage unit 131 and the flight route data storage unit 134 are realized by reading data recorded in a recording medium such as the ROM 120c or the external memory 120d, and storing the data in the memory. Note that the cross-sectional shape data storage unit 131 and the flight route data storage unit 134 can also be omitted by incorporating the cross-sectional shape data and the flight plan route data into the program.

[0045] In this embodiment, the information processing unit 120 functions as a self-location estimation system, but these systems may be installed separately from the information processing unit 120 and configured to be provided in the unmanned aerial vehicle. The self-location estimation system or its components do not need to be configured as one physical device, and may be configured as multiple physical devices. The self-location estimation system may be configured as any appropriate device such as a computer at a ground station separate from the unmanned aerial vehicle, a PC, a smartphone, a tablet terminal, a cloud computing system, or a combination thereof. The functions of each part of the self-location estimation system may be configured to be executed in a distributed manner in one or more of one or more devices provided in the unmanned aerial vehicle and one or more devices separate from the unmanned aerial vehicle.

[0046] Figures 5A and 5B show a tunnel and a flight plan route in one embodiment of the present invention, where Figure 5A is a vertical cross-sectional view of the tunnel and Figure 5B is a top view. As shown in Figures 5A and 5B, in this embodiment, the vertical cross-section of the outer wall surface of the tunnel is approximately semicircular, and a protrusion that protrudes outward is formed on the outer wall surface. Also, the tunnel extends linearly with the same cross-sectional shape.

[0047] For this reason, the cross-sectional shape data is, for example, X 2 +Z 2 =R 2 (center coordinates (X=0, Z=0) and radius R). Note that the shape of the tunnel's inner wall is constant regardless of the depth direction (Y direction), so Y does not exist in the above formula.

[0048] In this embodiment, the cross-sectional shape data includes data indicating a circle (semicircle) as the geometric shape corresponding to the outer wall surface of the tunnel, but is not limited to this and may be a shape formed by a convex curve such as an ellipse depending on the cross-sectional shape of the tunnel. In addition, the data representing the semicircle includes the radius R and the center coordinate, but may only include information that it is a circle. Note that the geometric shape referred to here refers to a shape that can be specified by a mathematical formula related to the X coordinate and the Z coordinate, for example.

[0049] The flight plan route data is data that represents the flight plan route of the unmanned aerial vehicle 1, and typically includes a series of multiple waypoints P1 to P2 that exist on the flight plan route. 4n In this embodiment, each of the waypoints P1 to P 4n is set as a set (Y, α, d) of the Y coordinate in the depth direction of the tunnel, the counterclockwise angle α based on the upper side of the center O of the tunnel outer wall surface in the longitudinal section, and the distance d from the tunnel outer wall surface in the longitudinal section. The flight plan route is typically a set of straight lines connecting the multiple waypoints in order, but may also be a curve with a predetermined curvature within a predetermined range of the waypoints. The flight plan route data may include data that determines the flight speed at the multiple waypoints. The flight plan route data is typically used to determine the flight plan route in an autonomous flight, but may also be used as a guide during flight in a non-autonomous flight. The flight plan route data is typically input to and stored in the unmanned aerial vehicle 1 before flight.

[0050] In this embodiment, the Y coordinate is not necessarily limited to a coordinate on a straight line. In other words, if the tunnel is curved, the Y coordinate may be a distance along a path of the tunnel from the starting point toward the depth of the tunnel.

[0051] In this embodiment, as shown in FIGS. 5A and 5B, the flight plan route is set as follows. - At a given position (α and d are constant) in the longitudinal section (XZ section), move a given distance toward the back of the tunnel in the direction of extension (Y direction) (waypoints P1 to P n ), - In the longitudinal section (XZ section), move counterclockwise on the concentric circle of the tunnel by a certain angle in Figure 5A (Z and d are constant, waypoint P n ~P n+1 ), - At a given position (α and d are constant) in the longitudinal section (XZ section), move a given distance toward the hand in the direction of tunnel extension (Y direction) (Z and d are constant, waypoint P n+1 ~P 2n ), - Move counterclockwise for a given angle on the concentric circle of the tunnel in the vertical section (XZ section) in Figure 5A (waypoint P 2n ~P 2n+1 ), - At a given position (α and d are constant) in the longitudinal section (XZ section), move a given distance toward the back of the tunnel in the direction of extension (Y direction) (waypoint P 2n+1 ~P 3n ), - Move the tunnel along the concentric circle in the longitudinal section (XZ section) in a counterclockwise direction by a certain angle in Figure 5A (Z and d are constant, waypoint P 3n ~P 3n+1 ), - At a given position (α and d are constant) in the longitudinal section (XZ section), move a given distance toward the front in the direction of tunnel extension (Y direction) (waypoint P 3n+1 ~P 4n ) Each waypoint P1~P 4n Each waypoint P1 to P2 is a position where the camera 108 captures an image of the outer wall of the tunnel. 4n Each waypoint P1~P 4n The heading in the above may also be set in combination.

[0052] The longitudinal section measurement data acquisition unit 138 can control the longitudinal section measurement device 109 and acquires the longitudinal section measurement data acquired by the longitudinal section measurement device 109 . The sensor measurement data acquisition unit 139 is capable of communicating with the flight position sensor 111, the compass 112, the IMU 113, and the altimeter 114, and acquires measurement values ​​from each sensor.

[0053] The longitudinal section position estimation unit 132 estimates the longitudinal section position (α, d) of the unmanned aerial vehicle in the longitudinal section based on the longitudinal section measurement data acquired from the longitudinal section measurement device 109 and the cross-sectional shape data recorded in the cross-sectional shape data storage unit 131. FIG. 6 is a diagram for explaining a method for estimating the longitudinal section position of the unmanned aerial vehicle in the longitudinal section. As shown in FIG. 6, the longitudinal section measurement data consists of points Q1 to Qn (n=12 in the example shown in FIG. 6) represented by polar coordinates with respect to the position of the longitudinal section measurement device 109 of the unmanned aerial vehicle. In this embodiment, the longitudinal section position estimation unit 132 matches a first predetermined shape corresponding to the outer wall surface of the tunnel, which is a circle in this embodiment. Note that matching a circle includes not only matching a part of a circle, but also naturally matching a part of a circle such as a semicircle or an arc.

[0054] Specifically, a circle is matched to points Q1 to Qn. Although the least squares method, RANSAC, etc. can be used as an algorithm for circle matching, RANSAC, which performs matching while excluding outliers, is suitable. When matching circles using RANSAC, the procedure is as follows. Step 1. Randomly select three points from the discrete points Q1 to Q12. Step 2. Derive a tentative circular model passing through the three points determined in step 1. Step 3: Calculate the error between the obtained provisional circular model and the discrete points Q1 to Q12. If the error is smaller than a predetermined value, add it to the "correct model candidates." Step 4. Repeat steps 1 to 3. Step 5. The correct model candidate that best matches the data is taken as the result of circle matching.

[0055] In step 3, if the radius of the obtained provisional circular model is different from the radius R of the tunnel outer wall surface in the cross-sectional shape data by more than a predetermined value (i.e., if the difference between the radius of the provisional circular model and the radius R of the tunnel outer wall surface in the cross-sectional shape data is equal to or greater than a threshold value), it may be excluded from the "correct model candidates".

[0056] For example, in the example shown in FIG. 6, Q2 and Q7 are measurement points on the protrusion 204 of the outer wall surface 202 of the tunnel 200, and are far from the trends of other discrete points among the discrete points corresponding to the outer wall surface 202 of the tunnel 200. In the example shown in FIG. 6, when Q2, Q7, and Q11 are selected in step 1, the circle model becomes C1 shown by the dashed line. In contrast, when Q1, Q5, and Q10 are selected in step 1, the circle model becomes C2 shown by the dashed line. When the error between such circle models C1 and C2 and Q1 to Q12 is calculated, the error of the circle model C1 is larger than the error of the circle model C2. For this reason, in step 5, values ​​that deviate from the trends of other data such as Q2 and Q7 used when setting the circle model C1 are not used as data for determining the circle, and a circle from which the influence of the outliers such as the circle model C2 has been removed is matched.

[0057] In this embodiment, RANSAC is used as an algorithm for performing matching while excluding outliers, but it is also possible to use Markov Localization, Monte-Carlo Localization, Particle Filter, Hough Transform, etc.

[0058] The longitudinal section position estimation unit 132 estimates the position (angle α and distance d) of the unmanned aerial vehicle in the longitudinal section based on the circle corresponding to the matched outer wall surface of the tunnel in this manner. The angle α is the angle with respect to the vertical direction based on the center O of the circle C2. Furthermore, d is the distance between the drone's position A and point B on the circle C2 that is closest to point A, and the straight line connecting points A and B passes through the center of the circle C2. For this reason, for example, if the X coordinate of the position A of the unmanned aerial vehicle based on the origin O of the circle matched by the longitudinal section position estimation unit 132 is XA and the Z coordinate is ZA, Angle α = arctan(XA / -ZA) Distance d = (XA 2 +ZA 2 ) 1 / 2 -R can be converted as:

[0059] The self-position data generating unit 133 generates self-position data including a three-dimensional position and attitude (heading θ) relative to a reference position. Specifically, the self-position data generating unit 133 acquires the angle α and distance d estimated by the longitudinal section position estimating unit 132. The self-position data generating unit 133 also estimates the Y coordinate position Y in the direction in which the tunnel 200 extends, based on the coordinates of the flight position of the unmanned aerial vehicle 1 measured by the flight position sensor 111. In this embodiment, the self-position data generating unit 133 estimates the Y coordinate position Y based on the flight position sensor 111, but if the tunnel 200 is curved, this Y coordinate position may be the distance in the direction in which the tunnel extends at the center of the tunnel 200 in the width direction.

[0060] Furthermore, the self-position data generating unit 133 generates data on the heading θ based on the angle measured by the compass 112. Then, based on these data, the self-position data generating unit 133 generates self-position data including three-dimensional position (Y, α, d) information, which is a set of the Y coordinate in the depth direction of the tunnel, the counterclockwise angle α with respect to a reference axis that passes through the center point O of the tunnel outer wall surface in the vertical section and extends upward, and the distance d from the tunnel outer wall surface in the vertical section, and information on the heading θ. Note that the reference axis only needs to pass through the center point O, and may be set to extend in the horizontal direction, for example.

[0061] In this embodiment, the self-position data generating unit 133 calculates the reference X-coordinate position X r , reference Y coordinate position Y r , and the reference heading θ r In addition, the self-position data generating unit 133 can estimate the reference Z coordinate position Z r The reference coordinates X r , Y r , Z r The self-position data generating unit 133 generates a reference three-dimensional position (Y r , α r , d r ) and the reference heading θ r It is also used to detect the accuracy of self-location estimation, which will be described later.

[0062] The flight control unit 135 controls the flight of the unmanned aerial vehicle to follow the flight plan route of the flight plan route data stored in the flight route data storage unit 134 based on the self-position data generated by the self-position data generation unit 133. Specifically, the flight control unit 135 determines the attitude, speed, etc. of the unmanned aerial vehicle 1 using various sensors, determines the current flight position and heading of the unmanned aerial vehicle 1 based on the self-position data generated by the self-position data generation unit 133, calculates a control command value for each rotor 103 by comparing it with target values ​​such as a control signal, a flight plan route (target), a speed limit, and an altitude limit, and outputs data indicating the control command value to the control signal generation unit 122. The control signal generation unit 122 converts the control command value into a pulse signal representing a voltage and transmits it to each speed controller 123. Each speed controller 123 converts the pulse signal into a drive voltage and applies it to each motor 102, thereby controlling the drive of each motor 102 to control the rotation speed of each rotor 103, thereby controlling the flight of the unmanned aerial vehicle 1.

[0063] Based on the self-position data of the unmanned aerial vehicle 1 estimated by the self-position data generation unit 133, the camera attitude calculation unit 136 estimates the shooting direction of the camera 108 so that the camera 108 can shoot a predetermined area of ​​the outer wall surface 202 of the tunnel 200. Note that the shooting direction of the camera 108 here refers to the direction of the center of the angle of view of the camera 108. Specifically, in this embodiment, since the outer wall surface of the tunnel 200 is circular, the camera attitude calculation unit 136 calculates the shooting angle such that the shooting direction of the camera 108 is perpendicular to the outer wall surface 202 of the tunnel 200, and that the shooting direction of the camera 108 is along the measurement surface of the longitudinal section measurement device 109 and faces the center point of the outer wall surface 202 of the tunnel 200. In this embodiment, since the cross-sectional shape of the outer wall surface 202 of the tunnel 200 is approximately circular, the camera attitude calculation unit 136 calculates an angle such that the shooting direction faces the center of the circle. However, this is not limited to this, and if the outer wall surface 202 of the tunnel 200 is elliptical, the shooting angle may be calculated to face the center (the intersection of the major axis and minor axis) or the focal point.

[0064] The attitude control unit 137 calculates the rotation angle of the attitude adjustment device 107 based on the shooting angle of the camera 108 calculated by the camera attitude calculation unit 136, and transmits a control signal according to the calculated rotation angle to the attitude adjustment device 107.

[0065] The flow of how the unmanned aerial vehicle described above autonomously flies inside a tunnel and photographs the inner wall surface of the tunnel will be described below. Fig. 7 is a flow chart showing a flow of autonomously flying inside a tunnel and photographing the inner wall surface of the tunnel by the unmanned aerial vehicle shown in Fig. 1. As shown in Fig. 7, first, the information processing unit 120 refers to the flight plan route data stored in the flight route data storage unit 134, and the flight control unit 135 sets the first waypoint as a target (S100).

[0066] Next, the information processing unit 120 estimates the self-position and attitude of the unmanned aerial vehicle 1 using the self-position data generation unit 133, while the flight control unit 135 controls the number of rotations of the motor 102 based on the estimated self-position and attitude, and the unmanned aerial vehicle 1 performs autonomous flight toward the target (S110). Figure 8 is a flowchart showing in detail the flow of autonomous flight by the unmanned aerial vehicle shown in Figure 1.

[0067] As shown in FIG. 8, first, the sensor measurement data acquisition unit 139 acquires the coordinates of the flight position of the unmanned aerial vehicle 1 from the flight position sensor 111, and estimates the Y-coordinate position Y based on the distance from the initial position (S111).

[0068] Next, the self-position data generation unit 133 acquires data on the heading θ of the unmanned aerial vehicle 1 from the compass 112 (S112). At this time, if the acquired heading θ deviates from the set direction, for example, does not follow the direction in which the tunnel extends, the heading may be adjusted so that the unmanned aerial vehicle 1 faces the set direction.

[0069] Next, the longitudinal section measurement data acquisition unit 138 drives the longitudinal section measurement device 109 to generate longitudinal section measurement data relating to the longitudinal section shape of the outer wall surface of the tunnel, and acquires this generated longitudinal section measurement data (S113).

[0070] Next, the longitudinal section position estimation unit 132 refers to the cross-sectional shape data recorded in the cross-sectional shape data memory unit 131, matches a circle to the longitudinal section measurement data, and estimates the position of the unmanned aerial vehicle in the longitudinal section (angle α and distance d) based on the matched circle (S114).

[0071] Next, the self-position data generation unit 133 generates self-position data including a three-dimensional position (Y, α, d) and information regarding the heading θ based on the Y coordinate position Y estimated in S111, the position in the longitudinal section (angle α and distance d) estimated in S112, and the heading θ acquired in S114 (S115).

[0072] Next, the sensor measurement data acquisition unit 139 acquires the measurement data of the IMU 113, and the self-position data generation unit 133 calculates the reference X-coordinate position X r , reference Y coordinate position Y r , and the reference heading θ r is estimated (S116).

[0073] Next, the sensor measurement data acquisition unit 139 acquires measurement data related to altitude from the altimeter 114, and the self-position data generation unit 133 calculates the reference Z coordinate position Z r is estimated (S117).

[0074] Next, the self-position data generating unit 133 calculates the reference X-coordinate position X r , reference Y coordinate position Y r , and the reference Z coordinate position Z obtained in S117 r For reference, the 3D position (Y r , α r , d r ) (S118).

[0075] Then, the self-location data generating unit 133 examines the stability of the self-location data. Specifically, the three-dimensional position (Y, α, d) and the heading θ included in the self-location data generated in S115 and the reference heading θ estimated in S116 are r and the reference 3D position (Y r , α r , d r In this embodiment, the heading θ and the three-dimensional position (Y, α, d) are compared with the reference heading θ r and the reference 3D position (Y r , α r , d r ) individually. If the difference between any of the values ​​of the self-position data and the reference value is equal to or greater than a predetermined value, the self-position data generation unit 133 determines that the self-position data is unstable. If it is determined in S119 that the self-position data is unstable (NO in S119), the flight control unit 135 hovers or flies, for example, in a direction away from the center of the tunnel based on the GPS position information acquired from the flight position sensor 111, and repeats S111 to S119 again.

[0076] Furthermore, if it is determined in S119 that the self-position data is stable (YES in S119), the flight control unit 135 determines whether the unmanned aerial vehicle 1 has reached the target based on the self-position data generated by the self-position data generating unit 133 (S120).

[0077] When the flight control unit 135 determines that the unmanned aerial vehicle 1 has not reached the target (NO in S120), the flight control unit 135 calculates a control command value for each rotor 103 so as to fly toward the target based on the self-position data generated by the self-position data generation unit 133, and outputs data indicating the control command value to the control signal generation unit 122 (S121). The control signal generation unit 122 converts the control command value into a pulse signal representing a voltage and transmits it to each speed controller 123. Each speed controller 123 converts the pulse signal into a drive voltage and applies it to each motor 102, thereby controlling the drive of each motor 102 to control the rotation speed of each rotor 103, thereby controlling the flight of the unmanned aerial vehicle 1.

[0078] In this embodiment, the waypoint is set by a set (Y, α, d) of the Y coordinate, the counterclockwise angle α based on the above the center O of the tunnel outer wall surface in the longitudinal section, and the distance d from the tunnel outer wall surface in the longitudinal section. Therefore, even if the tunnel has a shape that curves laterally or expands in the Y coordinate direction, it is possible to fly at a specified position (for example, at a constant distance) relative to the tunnel wall surface.

[0079] By repeating steps S111 to S121 at predetermined short time intervals, the unmanned aerial vehicle 1 autonomously flies toward the target. Note that, in this embodiment, the estimation of the Y coordinate position in S111, the estimation of the position in the longitudinal section in S112 and S113, and the acquisition of information on the heading θ from the compass in S114 are performed in this order, but the order of these estimation steps may be changed, or they may be performed simultaneously.

[0080] In this way, when the unmanned aerial vehicle 1 reaches the target, and in S120 the flight control unit 135 determines that the unmanned aerial vehicle 1 has reached the target (YES in S120), the camera attitude calculation unit 136 calculates the rotation angle of the attitude adjustment device 107 so that the direction of shooting by the camera 108 is perpendicular to the outer wall surface 202 of the tunnel 200. Specifically, based on the self-position data generated by the self-position data generation unit 133, the camera attitude calculation unit 136 calculates the shooting angle along the measurement surface of the longitudinal section measurement device 109 and toward the center of the tunnel 200 relative to the unmanned aerial vehicle 1 within the longitudinal section. Then, based on the shooting angle calculated by the attitude control unit 137, the rotation angle of the attitude adjustment device 107 is calculated, and a control signal according to the calculated rotation angle is transmitted to the attitude adjustment device 107 to control the attitude of the attitude adjustment device 107 (S140).

[0081] Next, the information processing unit 120 drives the camera 108 to capture an image of the inner wall surface of the tunnel (S150). Image data of the captured image is stored in the memory of the information processing unit 120, or is recorded in an appropriate terminal via the antenna 117.

[0082] And the target reached by unmanned aerial vehicle 1 is the final waypoint P 4n If not (NO in S160), the next waypoint P is set as the target (S170), and S110 to S160 are repeated.

[0083] In addition, the target reached by unmanned aerial vehicle 1 is the final waypoint P 4n If this is the case (NO at S160), the unmanned aerial vehicle 1 determines that all necessary steps have been completed, and ends the operation by, for example, flying to return to the initial position, via the flight control unit 135.

[0084] According to this embodiment, the following effects are achieved. In this embodiment, the longitudinal section position estimation unit 132 matches a circle to the longitudinal section measurement data, and estimates the position of the unmanned aerial vehicle in the longitudinal section based on its own position relative to the matched circle. This allows the accurate position of the unmanned aerial vehicle relative to the outer wall surface 202 of the tunnel 200 to be estimated, and based on this, the flight control unit 135 can control the unmanned aerial vehicle 1 to fly at a specified position relative to the tunnel without being affected by protrusions.

[0085] In this embodiment, the intra-longitudinal cross-sectional position estimating unit 132 performs matching using a geometric shape, that is, a circle. This reduces the amount of calculation required for matching, and allows easy matching with limited memory.

[0086] In this embodiment, the longitudinal section position estimation unit 132 performs matching using a circle, and estimates the distance and angle based on the center of the circle. If the outer wall surface 202 of the tunnel 200 is approximately circular, the unmanned aerial vehicle 1 can be easily flown at a predetermined distance from the outer wall surface 202 of the tunnel 200 by estimating the distance and angle based on the center of the circle.

[0087] In this embodiment, the waypoints indicating the flight plan route are set including an angle based on the center point in the longitudinal section of the tunnel and a distance from the outer wall surface. This allows the polar coordinates estimated by the longitudinal section position estimation unit 132 to be used for flight control as is, and also allows the unmanned aircraft 1 to fly a desired distance from the outer wall surface reliably. Furthermore, even if there is no information about the radius of the tunnel outer wall surface, the unmanned aircraft can be flown a predetermined distance from the outer wall surface of the tunnel.

[0088] In this embodiment, the longitudinal section data is discrete data, and the longitudinal section position estimation unit 132 removes outliers in the discrete data by the RANSAC algorithm and performs matching. As a result, even if a protrusion 204 is formed on the outer wall surface 202 of the tunnel 200, the position of the tunnel outer wall surface can be accurately identified without being affected by the protrusion 204.

[0089] In this embodiment, the attitude control unit 137 controls the attitude adjustment device 107 so that the imaging direction of the camera 108 faces the center of the tunnel based on the self-position data generated by the self-position data generation unit 133. This allows the camera 108 to reliably capture images in a direction facing the center of the tunnel.

[0090] In this embodiment, the cross-sectional shape of the tunnel is described as semicircular, but the present invention is not limited to this, and the cross-sectional shape may be circular or elliptical, or a part of a circle or ellipse (arc-shaped), or even a geometric shape such as rectangular.

[0091] In the present embodiment, the tunnel extends in a straight line in the horizontal direction, but the tunnel may extend in a curved shape. Furthermore, in the present embodiment, the tunnel extends with the same cross-sectional shape, but the present invention can be applied to the tunnel expanding or contracting in diameter. [Explanation of symbols]

[0092] 1 unmanned aerial vehicle 101 Control unit 102 Motor 103 Rotor 104 Arm 105 Landing leg 106 Pedestal 107 Posture adjustment device 108 Camera 109 Longitudinal section measuring device 110 Mount 111 Flight Position Sensor 112 Compass 114 Altimeter 117 Antenna 120 Information Processing Unit 120a CPU 120b RAM 120c ROM 120d External Memory 120e Input section 120f Output section 120g Communication section 120h system bus 121 Communication Circuit 122 Control signal generator 123 Speed ​​Controller 125 Interface 131 Cross-sectional shape data storage unit 132 Estimation of position in longitudinal section 133 Self-location data generation unit 134 Flight route data storage unit 135 Flight Control Unit 136 Camera Attitude Calculation Unit 137 Attitude Control Unit 140 Self-location estimation system 200 Tunnels 202 Exterior wall 204 Protrusion

Claims

1. A self-position estimation system for an unmanned aerial vehicle that flies along an outer wall surface of a structure extending laterally along a predetermined path, comprising: The unmanned aerial vehicle includes a longitudinal section measurement device that measures a boundary shape within a longitudinal section in a direction transverse to a flight path of the unmanned aerial vehicle and generates longitudinal section measurement data; a longitudinal section measurement data acquisition unit that acquires the longitudinal section measurement data measured by the longitudinal section measurement device when the unmanned aerial vehicle flies along an outer wall surface of a structure extending along a predetermined path in a lateral direction; a longitudinal section position estimation unit that matches a first predetermined shape corresponding to an outer wall surface of the structure with the acquired longitudinal section measurement data, and estimates a position of the unmanned aerial vehicle within the longitudinal section based on its own position with respect to the matched predetermined shape; A self-position data generation unit that generates self-position data based on the position in the longitudinal section estimated by the longitudinal section position estimation unit and the coordinates of the flight position acquired by the flight position sensor; Including, The self-location data generating unit compares the three-dimensional position of the generated self-location data with a reference three-dimensional position obtained from an IMU and an altimeter, and determines that the self-location data is unstable when a difference between the three-dimensional position included in the self-location data and the reference three-dimensional position is equal to or greater than a predetermined value. Self-location estimation system.

2. the first predetermined shape corresponds at least in part to a geometric shape; the longitudinal section position estimation unit matches the geometric shape to the longitudinal section measurement data, and estimates the position in the longitudinal section based on a self-position with respect to the matched geometric shape; The self-location estimation system according to claim 1 .

3. the first predetermined shape being all or part of a circle or an ellipse; The longitudinal section position estimation unit estimates a distance based on a center point of the first predetermined shape and an angle with respect to a reference axis as a longitudinal section position of the unmanned aerial vehicle in the longitudinal section. The self-location estimation system according to claim 2 .

4. The longitudinal section measurement data is discrete data; The longitudinal section position estimation unit removes outliers in the discrete data and performs matching. The self-location estimation system according to any one of claims 1 to 3.

5. A self-positioning estimation system as described in any one of claims 1 to 4, further comprising a flight control unit that controls the flight of the unmanned aircraft so that it flies along a flight plan route based on the position of the unmanned aircraft within the longitudinal section estimated by the longitudinal section position estimation unit.

6. The flight plan route is set as a plurality of waypoints, The waypoint is set by including an angle with respect to a reference axis passing through a reference point of the structure corresponding to a center point of the first predetermined shape in the longitudinal section, and a distance from an outer wall surface of the structure at the angle. The self-location estimation system according to claim 5 .

7. The unmanned aerial vehicle includes: An imaging device; an attitude adjustment device for adjusting the attitude of the imaging device; a longitudinal section measuring device that measures a boundary shape within a longitudinal section in a direction transverse to a flight path of the unmanned aerial vehicle and generates longitudinal section measurement data; a flight control system; The flight control system includes: The self-position estimation system according to claim 1, further comprising an attitude control unit that controls the attitude adjustment device so that the imaging direction of the imaging device is directed toward an outer wall surface of the structure, in accordance with a positional relationship between the self-position estimated by the longitudinal section position estimation unit and the structure.

8. the first predetermined shape being all or part of a circle or an ellipse; The attitude control unit controls the attitude adjustment device so that the imaging direction of the imaging device faces the center or focus of the matching circle or ellipse. The self-location estimation system according to claim 7 .

9. A self-location estimation system according to any one of claims 1 to 8, unmanned aircraft.

10. 1. A computer-implemented method for estimating a self-location of an unmanned aerial vehicle, comprising: The unmanned aerial vehicle includes a longitudinal section measurement device that measures a boundary shape within a longitudinal section in a direction transverse to a flight path of the unmanned aerial vehicle and generates longitudinal section measurement data; a longitudinal section acquisition step of acquiring the longitudinal section measurement data measured by the longitudinal section measurement device when the unmanned aerial vehicle flies along an outer wall surface of a structure extending laterally along a predetermined path; a longitudinal section position estimation step of matching a first predetermined shape corresponding to an outer wall surface of the structure to the acquired longitudinal section measurement data, and estimating a position of the unmanned aerial vehicle within the longitudinal section based on its own position relative to the matched predetermined shape; a self-position data generating step of generating self-position data based on the position in the longitudinal section estimated in the position in the longitudinal section estimating step and the coordinates of the flight position acquired by the flight position sensor; a comparison step of comparing a three-dimensional position of the self-location data generated in the self-location data generation step with a reference three-dimensional position obtained from an IMU and an altimeter, and determining that the self-location data is unstable when a difference between the three-dimensional position included in the self-location data and the reference three-dimensional position is equal to or greater than a predetermined value; A self-location estimation method comprising:

11. A program for causing a computer to execute the self-location estimation method according to claim 10.

12. A computer-readable recording medium having the program according to claim 11 recorded thereon.

Images