Flying Body

Abstract

The present invention can estimate its own position with high accuracy. A flying object comprises: two cameras with different focal lengths; a stabilizer equipped with at least the camera with the shorter focal length of the two cameras; and a flight controller that calculates the moving speed of the flying object based on images captured by the two cameras.

Description

Technical Field

[0001] The invention relates to a flying object. Background Art

[0002] A technology for estimating the flight position of a flying object without relying on GPS is known. For example, Patent Document 1 discloses an optical flow method using a multi-resolution method, a feature point detector algorithm, and a method combining these as a method for calculating the horizontal translation speed of a drone using an ultrasonic altimeter and a camera.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent No. 5854655 Summary of the invention

[0006] Problems to be solved by the invention

[0007] However, in the method described in Patent Document 1, when the height increases, the resolution of the image decreases, and the accuracy decreases.

[0008] The present invention has been made in view of such a background, and an object of the present invention is to provide a technology capable of estimating one's own position with high accuracy.

[0009] Solutions for solving problems

[0010] In order to solve the above-mentioned problems, the main invention of the present invention is a flying object, which comprises: two cameras with different focal lengths; a stabilizer, which carries at least the camera with a shorter focal length among the two cameras; and a flight controller, which calculates the moving speed of the flying object based on the images taken by the two cameras.

[0011] In addition, the problems disclosed in the present application and solutions thereto will be clarified by the embodiments of the invention and the accompanying drawings.

[0012] Effects of the Invention

[0013] According to the present invention, one's own position can be estimated with high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a diagram showing a configuration example of a flying object 1 according to an embodiment of the present invention.

[0015] Figure 2 It is a diagram for explaining the outline of the flying object 1 according to the present embodiment.

[0016] Figure 3 1 is a diagram showing an example of a flying object 1 equipped with two cameras 3 and 6 having different focal lengths.

[0017] Figure 4 1 is a diagram showing a second example of the flying object 1 equipped with two cameras 3 and 6 having different focal lengths.

[0018] Figure 5 1 is a diagram showing a third example of the flying object 1 equipped with two cameras 3 and 6 having different focal lengths.

[0019] Figure 6 This is a diagram showing an example of an air vehicle 1 on which cameras 3 and 6 and a rangefinder 4 are mounted in a horizontal direction.

[0020] Figure 7 This is a diagram showing an example of the flying object 1 in which a posture control mechanism 51 for stabilizing the carrying unit 7 is provided instead of the stabilizer 5 .

[0021] Figure 8 The figure shows an example of the flying object 1 provided with a plurality of carrying sections 7 on which the cameras 3 and 6 and the range finder 4 are mounted. DETAILED DESCRIPTION

[0022] The contents of the embodiments of the present invention will be listed and described. The flying object according to the embodiments of the present invention has the following structure.

[0023] [Item 1] A flying object, characterized in that it comprises:

[0024] Two cameras with different focal lengths;

[0025] a stabilizer carrying at least the camera with the shorter focal length among the two cameras; and

[0026] A flight controller calculates the moving speed of the flying object based on the images captured by the two cameras.

[0027] [Project 2]

[0028] The flying object according to item 1, characterized in that:

[0029] A distance calculation unit is provided for calculating the distance to the photographed location based on the two photographed images respectively photographed by the two cameras.

[0030] [Item 3]

[0031] The flying object according to item 1, characterized in that:

[0032] It also has a rangefinder.

[0033] The stabilizer is equipped with the camera with a shorter focal length and the rangefinder.

[0034] Figure 1It is a diagram showing a configuration example of a flying object 1 according to an embodiment of the present invention.

[0035] The flight controller 11 may include one or more processors such as a programmable processor (eg, a central processing unit (CPU)).

[0036] The flight controller 11 has a memory 102 and can access the memory 102. The memory 102 stores logic, codes and / or program instructions that can be executed by the flight controller 11 to perform one or more steps.

[0037] The memory 102 may also include, for example, a detachable medium such as an SD card, a random access memory (RAM), or an external storage device. The data obtained from the camera 3, the rangefinder 4, and the sensor type 103 may also be directly transmitted and stored in the memory 102. For example, the still image / moving image data taken by the camera 3 is recorded in a built-in memory or an external memory. The rangefinder 4 can measure the distance to the object and store the measured distance in the memory 102. The rangefinder 4 can, for example, measure the distance (height) to the ground, or measure the distance to the ceiling. The camera 3 and the rangefinder 4 are arranged on the flying body 1 via the stabilizer 5. The stabilizer 5 is preferably configured so that the intersection of the universal joint axes is located at the center of gravity of the flying body 1.

[0038] The flight controller 11 includes a control module configured to control the state of the flying object 1. For example, the control module controls the propulsion mechanism (motor 106, etc.) of the flying object 1 via the ESC 105 to adjust the six degrees of freedom (translation x, y and z and rotation θ x ,θ y and θ z ) of the flying object. The propeller 107 is rotated by the motor 106 to generate lift for the flying object 1. The control module can control one or more of the states of the sensor class 103.

[0039] The flight controller 11 can communicate with a transceiver 104, which is configured to send and / or receive data from one or more external devices (e.g., a transceiver (proportional controller), a terminal, a display device, or other remote controller). The transceiver can use any appropriate communication means such as wired communication or wireless communication.

[0040] For example, the transceiver 104 can utilize one or more of a local area network (LAN), a wide area network (WAN), infrared, wireless, WiFi, a peer-to-peer (P2P) network, a telecommunications network, cloud communications, and the like.

[0041] The transceiver unit 104 is capable of sending and / or receiving one or more of the data acquired by the sensor type 103, the processing results generated by the flight controller 11, the specified control data, the user command from the terminal or the remote controller, etc.

[0042] The sensor class 103 of this embodiment may include an inertial sensor (acceleration sensor, gyroscope sensor), a GPS sensor, a proximity sensor (eg, radar), or a visual / image sensor (eg, camera).

[0043] Figure 2 1 is a diagram for explaining the outline of the flying object 1 of the present embodiment. As described above, the flying object 1 carries the camera 3 and the rangefinder 4 on the main body 2 via the stabilizer 5. By providing the stabilizer 5, Figure 2 As shown in (b), even if the main body 2 of the flying object 1 is tilted, the camera 3 and the rangefinder 4 can be kept substantially horizontal.

[0044] The distance meter 4 can measure the distance (height) to the ground, for example. The distance meter 4 can obtain the distance by measuring the reflection time of ultrasonic waves, for example. In addition, the distance meter 4 can also use optics, infrared rays, lasers, etc., not limited to ultrasonic waves, and any type of distance meter can be used.

[0045] The flight controller 11 can calculate the horizontal moving speed of the flying object 1 based on the distance measured by the rangefinder 4 and the continuous images taken by the camera 3. For example, the flight controller 11 can calculate the distance of each pixel based on the height measured by the rangefinder 4 and the viewing angle of the camera 3, and calculate the speed based on the moving distance per unit time. In addition, the method of calculating the horizontal moving speed of the flying object 1 based on the multiple images taken by the camera 3 can use a known method.

[0046] Figure 3 1 is a diagram showing an example of a flying object 1 equipped with two cameras 3 and 6 having different focal lengths. Figure 3 As shown in (a), a (telephoto) camera 3 with a longer focal length and a rangefinder 4 are mounted on a stabilizer 5, and a (wide-angle) camera 6 with a shorter focal length is mounted on the main body 2 without passing through the stabilizer 5. The flight controller 11 can calculate the horizontal moving speed of the flying body 1 based on the two images with different resolutions taken by the two cameras 3 and 6. Figure 3 As shown in (b), by mounting the telephoto camera 3 on the stabilizer 5, the camera 3 can be stabilized even at a high altitude to capture high-definition images. In addition, the rangefinder 4 can be stabilized and accurately measure the distance (altitude). Therefore, the flight controller 11 can estimate the horizontal movement speed with high accuracy.

[0047] The wide-angle camera 6 may also be mounted on the stabilizer 5 . Figure 41 is a diagram showing a second example of a flying object 1 equipped with two cameras 3 and 6 having different focal lengths. Figure 4 As shown in (a), two cameras 3 and 6 with different focal lengths are mounted on the main body 2 via a stabilizer 5. Figure 4 As shown in (b), even when the flying body 1 is tilted, the cameras 3, 6 and the rangefinder 4 are directed downward in the vertical direction through the stabilizer 5. As a result, the cameras 3 and 6 can stably and clearly capture images with different resolutions. The rangefinder 4 can also stably and accurately measure the distance (height). Therefore, the flight controller 11 can estimate the horizontal movement speed with high accuracy.

[0048] The cameras 3 , 6 and the rangefinder 4 may also be oriented upward in the vertical direction. Figure 5 2 is a diagram showing a third example of a flying object 1 equipped with two cameras 3 and 6 having different focal lengths. Figure 5 As shown in (a), the cameras 3, 6 and the rangefinder 4 are arranged on the upper surface of the main body 2. Thus, the cameras 3 and 6 take images in the vertical direction, and the rangefinder 4 measures the distance in the vertical direction. For example, in a large exhibition hall, even if it is difficult to extract feature points through concrete or tiles on the ground, it is possible to take a picture of the ceiling and intercept it, measure the distance to the ceiling, and calculate the horizontal movement distance based on them. The cameras 3, 6 and the rangefinder 4 can also be oriented in the horizontal direction. Figure 6 FIG. 1 is a diagram showing an example of an aircraft 1 equipped with cameras 3, 6 and a rangefinder 4 facing the horizontal direction. Figure 6 In the example of , cameras 3, 6 and a rangefinder 4 are mounted on each side of the stabilizer 5. The flight controller 11 can estimate the moving speed on a plane parallel to each side surface based on the images captured by the two cameras 3 and 6 and the distance to the object existing in the horizontal direction measured by the rangefinder 4. Thus, for example, even on the water surface of an indoor swimming pool, the horizontal moving speed of the flying object 1 can be estimated by the cameras 3, 6 and the rangefinder 4 facing the lateral wall, and autonomous flight can be achieved on the water surface.

[0049] The stabilizer 5 may be any mechanism as long as it stabilizes the posture of the mounted object (for example, the camera 3 and the rangefinder 4 ). Figure 71 is a diagram showing an example of an aircraft 1 provided with a posture control mechanism 51 for stabilizing the carrying portion 7 instead of the stabilizer 5. The posture control mechanism 51 includes a support member 511 disposed on the upper surface of the main body 2 of the aircraft 1 and an arm 512 connected to the support member 511. The carrying portion 7 is provided at the front end portion of the arm 512. The arm 512 and the support member 511 are rotatably connected via a hinge 513. The hinge 513 is provided at the center of gravity of the aircraft 1. In addition, the hinge 513 is configured to be rotatable relative to an orthogonal axis. Thus, the arm 512 can be rotated 360 degrees around the center of gravity of the aircraft 1. Therefore, even if the main body 2 of the aircraft 1 is tilted, as shown in FIG. Figure 7 As shown in (b), the arm 512 is also stable in the vertical direction, and the cameras 3, 6 and the rangefinder 4 mounted on the loading unit 7 are kept facing downward in the vertical direction. Therefore, the cameras 3 and 6 can capture high-definition images, and the rangefinder 4 can measure stable distances (heights).

[0050] A plurality of mounting sections 7 on which the cameras 3 and 6 are mounted may be provided. Figure 8 1 is a diagram showing an example of an air vehicle 1 provided with a plurality of loading sections 7 on which cameras 3, 6 and a rangefinder 4 are mounted. Figure 8 In the example of FIG. 1 , arrow F indicates the forward direction of the flying object 1, and up and down are the vertical directions. Figure 8 As shown, two arms 512U and 512D are rotatably connected to the support member 511 in the up-down direction. Loading parts 7U and 7D are provided at the front ends of the arms 512U and 512D so that the surface (hereinafter referred to as the loading surface) on which the cameras 3, 6 and the rangefinder 4 are mounted faces the up-down direction. In addition, loading parts 7LU and 7LD are provided on the arms 512U and 512D so that the loading surfaces face the left side of the flying body 1, and loading parts (not shown) are provided on the opposite sides of the loading parts 7LU and 7LD so that the loading surfaces face the right side of the flying body 1. Thus, for each of the up-down, up-down, left-right and right-left directions of the flying body 1, the cameras 3 and 6 can capture images with different resolutions, and the rangefinder 4 can measure the distance of the flying body 1 relative to the surrounding objects. Therefore, the flight controller 11 can estimate the moving speed of the flying body 1 in the up-down, up-down, left-right and right-forward and backward directions by using arbitrary images and distances in the up-down, up-down, left-right and right-left directions. And, as Figure 8 As shown, an arm 514 tilted relative to the arm 512 may be connected, and a loading unit 7 may be provided at the front end of the arm 514 to measure an image and a distance in a tilted direction with an angle relative to the vertical direction. Figure 8In the example, an arm 514 inclined at 45 degrees is connected to the front and rear of the arm 512. The arm 512U on the upper side of the flying body 1 is connected with: an arm 514FU, which is provided with a loading portion 7FU with a loading surface facing 45 degrees above the front side of the flying body 1; and an arm 514BU, which is provided with a loading portion 7BU with a loading surface facing 45 degrees above the rear side of the flying body 1. In addition, the arm 512D on the lower side of the flying body 1 is connected with: an arm 514FD, which is provided with a loading portion 7FD with a loading surface facing 45 degrees below the front side of the flying body 1; and an arm 514BD, which is provided with a loading portion 7BD with a loading surface facing 45 degrees below the rear side of the flying body 1. In this way, Figure 8 In the example of , it is possible to capture an image of approximately 360 degrees of the flying object 1. Thus, the flight controller 11 can estimate the moving speed in the up, down, left, right, front, and back directions.

[0051] As described above, according to the flying object 1 of the present embodiment, since the camera 3 and the rangefinder 4 are mounted on the flying object 1 via the stabilizer 5, the direction of shooting and measuring by the camera 3 and the rangefinder 4 can be stabilized. As a result, the estimation accuracy of the moving speed of the flying object 1 using the image and the distance can be improved. Such a structure of the flying object 1 of the present embodiment can be used in various flying objects for any purpose from personal hobbies to industrial uses. In addition, the use environment can be either outdoor or indoor. Based on the image captured by the camera and the distance measured by the rangefinder, the moving speed of the flying object 1 can be estimated, and the position of the flying object 1 can be estimated based on the moving speed. Therefore, it can replace GPS in places where GPS does not work, such as under bridges, valleys, caves, etc. In addition, it can serve as a backup means in the event that some failure occurs during the process of obtaining GPS signals or using GPS.

[0052] In addition, when a plurality of cameras with different focal lengths are mounted, if the optical axis of a camera with a longer focal length is offset, the error will increase when the moving distance is calculated based on the distance of each pixel. However, according to the flying object 1 of this embodiment, the camera 3 with a longer focal length can be stabilized by the stabilizer 5. Therefore, the moving speed can be estimated with high accuracy.

[0053] The present embodiment has been described above, but the above embodiment is for easy understanding of the present invention and is not intended to limit the present invention. The present invention can be changed and improved within the scope of the gist thereof, and the present invention includes equivalent forms thereof.

[0054] For example, in this embodiment, the distance is measured by the rangefinder 4, but the present invention is not limited thereto. The rangefinder 4 may be omitted, and the distance to the photographed object may be estimated using two images taken by the cameras 3 and 6. This may be achieved by using general stereo image processing. In this case, by stabilizing the cameras 3 and 6 with the stabilizer 5, the moving speed and distance (height) may be estimated with high accuracy. Figure 7 In the example, cameras 3, 6 and a rangefinder 4 are arranged on the loading section 7 provided at the front end of the arm 512, and the moving speed of the flying body 1 is estimated based on the images taken by the cameras 3 and 6 and the distance measured by the rangefinder 4, but the flight controller 11 preferably corrects the moving speed based on the length of the arm 12 (the distance from the hinge 513 to the mounting surface of the loading section 7).

[0055] In addition, Figure 8 In the example of FIG. 1 , the mounting surface of the mounting portion 7 is directed obliquely upward and downward in the front-rear direction, but may be directed in the front-rear direction as in the left-right direction.

[0056] In addition, in the present embodiment, the camera 3 and the rangefinder 4 are fixed to the stabilizer 5, but the rangefinder 4 may also be movable. In this case, the flight controller 11 may also obtain the point to be intercepted (i.e., the point from which the feature quantity is extracted, hereinafter referred to as the interception point) from the image captured by the camera 3, and rotate the rangefinder 4 to change the pointing direction of the rangefinder 4, so as to emit ultrasonic waves and lasers toward the interception point obtained. Thus, the interception point used when analyzing the image from the camera 3 to estimate the moving speed can be made roughly consistent with the point that becomes the object of measuring the distance from the rangefinder 4. Thus, when calculating the distance of each pixel in the image, the distance to the interception point used to calculate the distance (horizontal distance) of the interception point movement can be accurately obtained, and the estimation accuracy of the moving speed can be improved.

[0057] Explanation of symbols

[0058] 1 flying body; 2 main body; 3 camera; 4 rangefinder; 5 stabilizer; 6 camera; 7 loading part; 51 posture control mechanism; 511 supporting member; 512 arm; 513 hinge.

Claims

1. A flying object, characterized in that: have: Two cameras with different focal lengths, wherein the two cameras are a telephoto camera with a longer focal length and a wide-angle camera with a shorter focal length; a stabilizer, which is arranged at the center of gravity of the flying object, further comprising a rangefinder, and is used to carry at least one of the telephoto camera and the wide-angle camera; a flight controller, which calculates the moving speed of the flying object based on the two images with different resolutions taken by the two cameras; as well as A distance calculation unit calculates the distance to the photographed location based on two images with different resolutions photographed by the two cameras.

2. The flying object according to claim 1, characterized in that: The stabilizer is equipped with the camera with a shorter focal length and the rangefinder.

Images