Device, method, and program

The system improves small aircraft position estimation by combining inertial sensors with GPS and other sensors, using complementary filtering and adaptive method switching to maintain accurate positioning in various environments.

JP2024169467AActive Publication Date: 2024-12-05SONY GROUP CORP
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
JP2024158266
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2014-10-17
Filing Date
2024-09-12
Publication Date
2024-12-05
Estimated Expiration
2035-09-16

AI Technical Summary

Technical Problem

Small aircraft face challenges in accurately estimating their position due to low accuracy inertial sensors and difficulty in obtaining reliable GPS signals, leading to potential flight instability.

Method used

A system that combines inertial sensors with GPS receivers and additional sensors like barometers and imaging devices to estimate position, using complementary filtering and sensor fusion to improve accuracy, and dynamically switches estimation methods based on environmental conditions.

Benefits of technology

Enhances the accuracy of position estimation for small aircraft by leveraging multiple sensors and adaptive method switching, ensuring stable flight operations even in areas with poor GPS reception.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024169467000001_ABST
    Figure 2024169467000001_ABST
Patent Text Reader

Abstract

To more satisfactorily estimate a position of a flying object.SOLUTION: There is provided, a device that has a GPS signal receiver for receiving a GPS (Global Positioning System) signal, and a position estimation part for estimating a position of a flying object on the basis of an estimation part for estimating an altitude corresponding to an atmospheric pressure measured by a barometer of a flying object, a GPS signal received by the GPS signal receiver, and an altitude estimated by the estimation part. The position estimation part estimates a position of a flying object on the basis of any one position estimation method from among a first position estimation method based on a GPS signal received by the GPS signal receiver and a second position estimation method based on altitude estimated by the estimation part.SELECTED DRAWING: Figure 5
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to an apparatus, a method, and a program. [Background technology]

[0002] In passenger aircraft, functions such as automatic flight are realized by estimating attitude and position using a combination of a high-precision inertial sensor and a GPS (Global Positioning System) receiver. In recent years, inertial sensors have become smaller and cheaper, and as a result, even small aircraft are equipped with inertial sensors and GPS receivers to estimate position and attitude.

[0003] For example, Patent Document 1 discloses a technique for determining whether a moving object is stationary by using a combination of a GPS receiver and an inertial sensor. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2008-232869 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, unlike airliners, small aircraft may fly in places where it is difficult to estimate the position using a GPS receiver. In general, the accuracy of the inertial sensor of a small aircraft is low, so when it becomes difficult to estimate the position using a GPS receiver, the position of the aircraft cannot be estimated properly. As a result, it may not be possible to fly the aircraft as desired.

[0006] It would therefore be desirable to provide a mechanism that allows for a better estimation of the position of an air vehicle. [Means for solving the problem]

[0007] According to the present disclosure, there is provided an apparatus comprising: a GPS signal receiver that receives a Global Positioning System (GPS) signal; an estimation unit that estimates an altitude corresponding to an atmospheric pressure measured by a barometer of the flying object; and a position estimation unit that estimates a position of the flying object based on the GPS signal received by the GPS signal receiver and the altitude estimated by the estimation unit, wherein the position estimation unit estimates the position of the flying object based on either a first position estimation method based on the GPS signal received by the GPS signal receiver or a second position estimation method based on the altitude estimated by the estimation unit.

[0008] The present disclosure also provides a method including receiving, by a processor, a Global Positioning System (GPS) signal, estimating an altitude corresponding to an air pressure measured by a barometer of the aircraft, and estimating a position of the aircraft based on the received GPS signal and the estimated altitude, wherein the method estimates a position of the aircraft based on either a first position estimation method based on the received GPS signal or a second position estimation method based on the estimated altitude.

[0009] In addition, according to the present disclosure, there is provided a program for causing a processor to receive a Global Positioning System (GPS) signal, estimate an altitude corresponding to air pressure measured by a barometer of the aircraft, and estimate a position of the aircraft based on the received GPS signal and the estimated altitude, thereby estimating the position of the aircraft based on one of a first position estimation method based on the received GPS signal and a second position estimation method based on the estimated altitude. Effect of the Invention

[0010] As described above, according to the present disclosure, it is possible to better estimate the position of an aircraft. Note that the above effects are not necessarily limiting, and any of the effects shown in this specification or other effects that can be understood from this specification may be achieved in addition to or instead of the above effects. [Brief description of the drawings]

[0011] [Figure 1] FIG. 2 is an explanatory diagram for explaining a first example of attitude estimation using a combination of a gyro sensor and an acceleration sensor. [Diagram 2] FIG. 11 is an explanatory diagram for explaining a second example of attitude estimation using a combination of a gyro sensor and an acceleration sensor. [Diagram 3] FIG. 11 is an explanatory diagram for explaining an example of position estimation using a combination of an inertial sensor and another sensor. [Figure 4] FIG. 1 is an explanatory diagram illustrating an example of a schematic configuration of a system according to an embodiment of the present disclosure. [Diagram 5] FIG. 2 is a block diagram showing an example of the configuration of an aircraft according to the embodiment. [Figure 6] FIG. 1 is an explanatory diagram for explaining a first example of imaging by a monocular imaging device installed on an aircraft using a gimbal. [Figure 7] FIG. 11 is an explanatory diagram for explaining a second example of imaging by a monocular imaging device installed on an aircraft using a gimbal. [Figure 8] FIG. 11 is an explanatory diagram for explaining an example of the arrangement of markers. [Figure 9] FIG. 11 is an explanatory diagram for explaining an example of a case in which motion capture is used. [Figure 10] FIG. 2 is an explanatory diagram for explaining features of each position estimation method. [Figure 11] FIG. 11 is an explanatory diagram for explaining an example of position estimation according to a time delay parameter. [Figure 12] FIG. 11 is an explanatory diagram for explaining an example of a result of position estimation without a time delay parameter. [Figure 13]FIG. 11 is an explanatory diagram for explaining an example of a result of position estimation according to a time delay parameter. [Figure 14] FIG. 11 is an explanatory diagram for explaining a first example of selection (switching) of a position estimation method. [Figure 15] FIG. 11 is an explanatory diagram for explaining a second example of selection (switching) of a position estimation method. [Figure 16] FIG. 11 is an explanatory diagram for explaining a second example of selection (switching) of a position estimation method. [Figure 17] 10 is a flowchart showing an example of a schematic flow of a position estimation process according to the embodiment. [Figure 18] 10 is a flowchart showing an example of a schematic flow of a first selection process according to the embodiment. [Figure 19] 13 is a flowchart showing an example of a schematic flow of a second selection process according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are denoted by the same reference numerals, and redundant description will be omitted.

[0013] In addition, in this specification and drawings, elements having substantially the same functional configuration may be distinguished by adding different alphabets after the same reference numeral. For example, multiple elements having substantially the same functional configuration may be distinguished as motors 120A, 120B, and 120C as necessary. However, when there is no need to distinguish between multiple elements having substantially the same functional configuration, only the same reference numeral may be used. For example, when there is no need to distinguish between motors 120A, 120B, and 120C, they will simply be referred to as motor 120.

[0014] The explanation will be given in the following order. 1. Basic Pose and Position Estimation 2. System Overview 3. Aircraft configuration 4. Position Estimation According to This Embodiment 5. Processing flow 6. Summary

[0015] <1. Basic attitude and position estimation> First, an example of basic attitude estimation and position estimation will be described with reference to FIGS.

[0016] (1) Inertial Sensor For example, the inertial sensor of the flying object includes a gyro sensor (or an angular velocity sensor) that detects the angular velocity of the flying object, and an acceleration sensor that measures the acceleration of the flying object. The inertial sensor may further include a digital compass (or a geomagnetic sensor).

[0017] (2) Posture estimation (a) Estimation from angular velocity For example, the change in the attitude of the flying object is calculated by integrating the angular velocity detected by a gyro sensor, and therefore the attitude of the flying object is calculated from the initial value of the attitude of the flying object and the change in the attitude of the flying object.

[0018] The angular velocity detected by the gyro sensor has high reliability in the short term, but has a drift characteristic that changes gradually over time. Therefore, if the attitude is calculated only from the angular velocity (and the initial value) detected by the gyro sensor, errors will accumulate in the attitude over time.

[0019] (b) Estimation from acceleration For example, assuming that the flying object is hovering and not moving, the attitude of the flying object can be calculated from the direction of gravity detected by the acceleration sensor.

[0020] The acceleration detected by the acceleration sensor contains high-frequency noise due to the effects of vibration, etc. However, since the direction of gravity is always the same regardless of time, the acceleration sensor has high reliability over the long term.

[0021] (c) Sensor fusion By configuring a complementary filter using the above-mentioned characteristics of the gyro sensor and the characteristics of the acceleration sensor, it is possible to calculate a highly reliable attitude. Combining multiple sensors in this way can be called sensor fusion. An example will be described below with reference to Figures 1 and 2.

[0022] FIG. 1 is an explanatory diagram for explaining a first example of attitude estimation using a combination of a gyro sensor and an acceleration sensor. Referring to FIG. 1, an attitude y1(t) is calculated from the acceleration detected by the acceleration sensor, and an attitude y2(t) is calculated from the angular velocity detected by the gyro sensor. y1(t) is expressed as x(t)+n1(t), and y2(t) is expressed as x(t)+n2(t). x(t) is the actual attitude of the flying object, n1(t) is high-frequency noise caused by the acceleration sensor, and n2(t) is low-frequency noise caused by the gyro sensor. Since the attitude y1(t) includes high-frequency noise n1(t), the high-frequency noise n1(t) is cut from the attitude y1(t) by passing the attitude y1(t) through a low-pass filter 11. On the other hand, since the attitude y2(t) contains low-frequency noise n2(t), the low-frequency noise n2(t) is removed from the attitude y2(t) by passing the attitude y2(t) through a high-pass filter 13. For example, the low-pass filter 11 and the high-pass filter 13 are complementary filters. Therefore, the original signal without noise can be obtained by adding together the output value of the low-pass filter 11 and the output value of the high-pass filter 13 as follows:

[0023]

number

[0024] FIG. 2 is an explanatory diagram for explaining a second example of attitude estimation using a combination of a gyro sensor and an acceleration sensor. Referring to FIG. 2, similarly to the example of FIG. 1, an attitude y1(t) is calculated from the acceleration detected by the acceleration sensor, and an attitude y2(t) is calculated from the angular velocity detected by the gyro sensor. In particular, in this example, instead of using the high-pass filter 13, an attitude y2(t) is subtracted from the attitude y1(t), and the resulting difference (i.e., n1(t)-n2(t)) is passed through the low-pass filter 11. As a result, the negative value of the low-frequency noise n2(t) is output. Then, the low-frequency noise n2(t) is subtracted from the attitude y2(t) as follows, and as a result, the original signal without noise is obtained.

[0025]

number

[0026] By not using the high-pass filter 13 in this way, it is possible to eliminate the delay caused by the high-pass filter 13.

[0027] (3) Location estimation (a) Position estimation using inertial sensors Based on the attitude calculated as described above, the acceleration detected by the acceleration sensor (that is, the acceleration in the coordinate system of the acceleration sensor) is converted into acceleration in the ground coordinate system (for example, the acceleration from the ground).

[0028] Furthermore, the change in the velocity of the flying object is calculated by integrating the converted acceleration, so that the velocity of the flying object is calculated from the initial value of the velocity of the flying object and the change in the velocity of the flying object.

[0029] Furthermore, the change in position of the flying object is calculated by integrating the velocity, so that the position of the flying object is calculated from the initial value of the position of the flying object and the change in the position of the flying object.

[0030] The acceleration detected by the acceleration sensor contains noise, so the position of the flying object calculated from the acceleration contains errors that accumulate over time.

[0031] (b) Sensor fusion In order to avoid the accumulation of errors as described above, a more reliable position estimation is realized by using a combination of an inertial sensor and other sensors. For example, a position estimation is realized by using a combination of an inertial sensor and a GPS receiver. An example will be described below with reference to FIG. 3.

[0032] FIG. 3 is an explanatory diagram for explaining an example of position estimation using a combination of an inertial sensor and another sensor. Referring to FIG. 3, acceleration a in a ground coordinate system is obtained by converting acceleration detected by an acceleration sensor. The acceleration a is corrected in a portion 21 and integrated in a portion 23, and as a result, a change in velocity is calculated. The already calculated velocity v is corrected in a portion 21 and the above change is added in a portion 23. As a result, an updated velocity vU is obtained. In addition, the updated velocity vU is integrated in a portion 23, and as a result, a change in position is calculated. The already calculated position p is corrected in a portion 21 and the above change is added in a portion 23. As a result, an updated position pU is obtained. Furthermore, a difference (e.g., px-pU) between a position px generated using another sensor (e.g., a GPS receiver) and an updated position pU is calculated as an error e, and the error e is passed through a filter 25. As a result, an acceleration correction value Δa, a velocity correction value Δv, and a position correction value Δp are output. These correction values ​​are then used for the correction in part 21. In this way, the inertial sensor is complemented by other sensors (for example a GPS receiver).

[0033] The process in the filter 25 is expressed as follows.

[0034]

number

[0035] Δat, Δvt, and Δpt are the correction values ​​after updating, and Δat-1, Δvt-1, and Δpt-1 are the correction values ​​before updating. k1, k2, and k3 are expressed as follows using the time constant Tc of the filter 25.

[0036]

number

[0037] <2. System Overview> Next, a schematic configuration of a system 1 according to an embodiment of the present disclosure will be described with reference to Fig. 4. Fig. 4 is an explanatory diagram showing an example of a schematic configuration of a system 1 according to an embodiment of the present disclosure. Referring to Fig. 4, the system 1 includes an aircraft 100, a control device 200, and a pilot device 300.

[0038] (1) 100 flying vehicles The aircraft 100 is a device capable of flying. For example, the aircraft 100 is capable of flying using multiple rotors (for example, four rotors). For example, the aircraft 100 performs stationary hovering, movement (ascending, descending, moving horizontally, moving diagonally, etc.), rotation, etc., by controlling the rotation of each rotor. Note that the aircraft 100 may be a device capable of flying using a mechanism other than rotors.

[0039] For example, the flying object 100 can estimate the attitude and position of the flying object 100 by itself and control the flight based on the attitude and the position. For example, the flying object 100 flies automatically based on a specified flight path.

[0040] For example, the air vehicle 100 communicates with another device (e.g., the control device 200). The air vehicle 100 may directly communicate with the other device wirelessly, or may communicate with the other device via a relay node by wirelessly communicating with the relay node.

[0041] For example, the flying object 100 is equipped with an imaging device and captures images during flight using the imaging device. The flying object 100 may store the captured images generated by the imaging, or may transmit the captured images to another device such as the control device 200. The captured images may be still images or moving images.

[0042] (2) Control device 200 The control device 200 is a device that executes control related to the flight of the aircraft 100. For example, the control includes generating and providing flight information for the aircraft 100 (e.g., information indicating a flight path, etc.) and / or issuing instructions to the aircraft 100 (e.g., a takeoff instruction and / or a return instruction, etc.).

[0043] For example, the control device 200 communicates with the air vehicle 100. The control device 200 may communicate wirelessly directly with the air vehicle 100, or may communicate with the air vehicle 100 via a relay node.

[0044] For example, the control device 200 acquires captured images generated by the flying object 100 and displays them as necessary. The captured images may be moving images, and the control device 200 may acquire and display the moving images in a streaming manner.

[0045] As an example, the control device 200 is a portable device such as a notebook computer or a tablet terminal, etc. Of course, the control device 200 is not limited to this example and may be another type of device.

[0046] (3) Control device 300 The flight control device 300 is a device that allows a user to fly the air vehicle 100. As an example, the flight control device 300 is a proportional system (or propo).

[0047] For example, the control device 300 is connected to the control device 200. The control device 300 generates control information related to the operation of the flying object in response to an operation by a user, and transmits the control information to the control device 200. The control device 200 may transmit the control information to the flying object 100, or may generate other control information from the control information and transmit the other control information to the flying object 100.

[0048] <3. Configuration of the flying vehicle> Next, an example of the configuration of the flying object 100 according to an embodiment of the present disclosure will be described with reference to Fig. 5. Fig. 5 is a block diagram showing an example of the configuration of the flying object 100 according to an embodiment of the present disclosure. Referring to Fig. 5, the flying object 100 includes a rotor 110, a motor 120, a sensor unit 130, an imaging unit 140, a memory unit 150, a wireless communication unit 160, a processing unit 170, and a battery 190.

[0049] (1) Rotor 110 and Motor 120 The rotors 110A to 110D generate lift through rotation, thereby causing the flying object 100 to fly.

[0050] (2) Motor 120 The motors 120A to 120D rotate the rotors 110A to 110D in response to the control of the processing unit 170 (control unit 181). For example, the motor 120 changes the rotation speed of the rotor 110 in response to the control of the processing unit 170 (control unit 181).

[0051] (2) Sensor unit 130 The sensor unit 130 includes one or more sensors. For example, the sensor unit 130 includes an inertial sensor 131, a GPS receiver 133, a barometer 135, and an ultrasonic sensor 137.

[0052] For example, the inertial sensor 131 includes an acceleration sensor and a gyro sensor. The inertial sensor 131 may further include a geomagnetic sensor.

[0053] Note that the sensors included in the sensor unit 130 are not limited to these examples. The sensor unit 130 may not include one or more of the above-mentioned sensors (excluding the inertial sensor 131). The sensor unit 130 may also include other sensors.

[0054] (3) Imaging unit 140 The imaging unit 140 captures an image and generates a captured image. The captured image may be a still image or a moving image. The captured image may be stored in the storage unit 150 or may be transmitted to another device via the wireless communication unit 160.

[0055] The imaging unit 140 includes one or more imaging devices. Each of the one or more imaging devices includes a lens and an image sensor. The one or more imaging devices may include an infrared imaging device and / or an omnidirectional imaging device.

[0056] (4) Storage section 150 The storage unit 150 stores various information. The storage unit 150 stores programs for the operation of the flying object 100 and / or various data.

[0057] For example, the storage unit 150 includes a non-volatile memory (for example, a memory card, etc.) The storage unit 150 may include a magnetic storage device (for example, a hard disk drive) instead of or in addition to the non-volatile memory.

[0058] (5) Wireless communication unit 160 The wireless communication unit 160 performs wireless communication. The wireless communication unit 160 may perform wireless communication directly with another device (e.g., the control device 200), or may perform wireless communication with a relay node for communication with the other device.

[0059] For example, the wireless communication unit 160 includes an antenna, a radio frequency (RF) circuit, and / or a baseband processor.

[0060] (6) Processing section 170 The processing unit 170 performs various processes for the flying object 100. The processing unit 170 includes a selection unit 171, an information acquisition unit 173, an information generation unit 175, an attitude estimation unit 177, a position estimation unit 179, and a control unit 181. Note that the processing unit 170 may further include other components in addition to these components. That is, the processing unit 170 may also perform operations other than those of these components.

[0061] For example, the processing unit 170 includes a circuit. More specifically, for example, the processing unit 170 includes one or more integrated circuits. For example, the one or more integrated circuits hold a program for the operation of the processing unit 170. For example, the selection unit 171, the information acquisition unit 173, the information generation unit 175, the attitude estimation unit 177, the position estimation unit 179, and the control unit 181 may be implemented as the above-mentioned program. The one or more integrated circuits may include a System-on-a-Chip (SoC), a microcontroller, and / or another processor, etc.

[0062] (a) Posture estimation unit 177 The attitude estimation unit 177 estimates the attitude of the flying object 100.

[0063] For example, the attitude estimation unit 177 acquires information generated using the inertial sensor 131, and estimates the attitude of the flying object 100 from the information. For example, the information indicates information indicating an angular velocity detected by a gyro sensor included in the inertial sensor 131, and information indicating an acceleration detected by an acceleration sensor included in the inertial sensor 131. The attitude estimation of the flying object 100 is, for example, as described above as basic attitude estimation. Therefore, a detailed description will be omitted here.

[0064] (b) Position estimation section 179 The position estimation unit 179 estimates the position of the flying object 100. This will be explained in detail later.

[0065] (c) Control unit 181 The control unit 181 controls the flight of the flying object 100.

[0066] (c-1) Control of the rotor 110 For example, the control unit 181 controls the rotation of the rotor 110. Specifically, for example, the control unit 181 controls the operation of the motor 120 to control the rotation of the rotor 110.

[0067] For example, the control unit 181 adjusts the rotation speed of each of the rotors 110A, 110B, 110C, and 110D to cause the flying object 100 to perform stationary hovering, movement (ascending, descending, moving horizontally, and moving diagonally, etc.), rotation, and the like.

[0068] More specifically, for example, the attitude of the aircraft 100 (tilts on the roll, pitch, and yaw axes) can be changed by adjusting the rotation speed of each rotor. Therefore, for example, the aircraft 100 can be tilted on the roll and pitch axes, which generates a horizontal thrust force and causes the aircraft 100 to move horizontally. Also, for example, an increase or decrease in the rotation speed of the rotors changes the speed of movement in the vertical direction.

[0069] (c-2) Controlling the attitude of the flying object 100 For example, the control unit 181 controls the attitude of the flying object 100.

[0070] For example, the control unit 181 corrects the error between the attitude of the flying object 100 estimated by the attitude estimation unit 177 and the target attitude. More specifically, for example, the control unit 181 calculates the error, and calculates the rotation of the rotor 110 (or the operation of the motor 120) to correct the error. Then, the control unit 181 controls the rotation of the rotor 110 as described above.

[0071] (c-3) Controlling the position of the aircraft 100 For example, the control unit 181 controls the position of the flying object 100.

[0072] - Control examples For example, the control unit 181 moves the flying body 100 from the position of the flying body 100 estimated by the position estimation unit 179 to a target position. More specifically, for example, the control unit 181 calculates the movement of the flying body 100 from the above position to the target position, and calculates a target attitude that produces the movement. Then, as described above, the control unit 181 corrects the error between the attitude of the flying body 100 and the target attitude.

[0073] - Example of obtaining target position As an example, the control unit 181 acquires one or more target positions from flight path information held by the control unit 181. The flight path information may indicate one or more target positions, and the control unit 181 may calculate one or more target positions from the flight path information.

[0074] As another example, the control device 200 may transmit information indicating a target position to the flying object 100, and the control unit 181 may acquire the information indicating the target position.

[0075] (c-4)Other The operations of the selection unit 171, the information acquisition unit 173, and the information generation unit 175 will be described in detail later, similar to the operation of the position estimation unit 179.

[0076] (7) Battery 190 Battery 190 stores power for operating aircraft 100. Battery 190 may be a primary battery that is only capable of discharging, or a secondary battery that is also capable of charging.

[0077] 4. Position Estimation According to an Embodiment of the Present Disclosure Next, an example of position estimation according to an embodiment of the present disclosure will be described with reference to FIGS.

[0078] (1) Multiple location estimation methods In an embodiment of the present disclosure, a plurality of position estimation methods are provided for estimating the position of the flying object 100. For example, the plurality of position estimation methods include at least one of a method using a GPS receiver, a method using an imaging device installed in the flying object 100, a method using an imaging device that images the flying object 100, a method using a barometer, and a method using an ultrasonic sensor.

[0079] (a) Method using a GPS receiver For example, one position estimation method uses a GPS receiver.

[0080] The GPS receiver estimates the position in response to receiving signals from multiple GPS satellites. That is, the GPS receiver calculates the latitude, longitude, and altitude in response to receiving signals from multiple GPS satellites. Therefore, the GPS receiver 133 of the flying object 100 estimates the position of the flying object 100. That is, the GPS receiver 133 of the flying object 100 calculates the latitude, longitude, and altitude of the flying object 100.

[0081] (b) A method using an imaging device installed in the flying object 100 For example, one method for estimating the position is to use an imaging device installed in the flying object 100.

[0082] (b-1)SLAM For example, a method using an imaging device installed in the aircraft 100 includes SLAM (Simultaneous Localization and Mapping).

[0083] -Monocular SLAM For example, there is SLAM that uses a monocular imaging device (hereinafter referred to as "monocular SLAM"). In monocular SLAM, the 3D position of a feature point in a captured image that occurs due to movement is used to simultaneously estimate the position and orientation of the camera and the 3D position of the feature point.

[0084] In this method, since it is uncertain whether the feature point is a large one located far away or a small one located nearby, it is necessary to present a feature point of known size at the time of initialization. Also, in a case where the flying object 100 rotates on the spot, since no parallax of the feature points occurs, the distance of the feature points becomes uncertain, and it may be difficult to estimate the position and attitude using new feature points.

[0085] To compensate for these weaknesses, for example, a monocular imaging device is installed on the flying object 100 so that the optical axis of the monocular imaging device is the vertical axis of the flying object 100. Furthermore, the monocular imaging device is installed on the flying object 100 using a gimbal. That is, a SLAM using a monocular imaging device installed on the flying object 100 using a gimbal so that the optical axis is the vertical axis of the flying object 100 (hereinafter referred to as "monocular SLAM with gimbal") is provided as a position estimation method. For example, the optical axis is a vertical axis. A specific example of this point will be described below with reference to Figs. 6 and 7.

[0086] FIG. 6 is an explanatory diagram for explaining a first example of imaging by a monocular imaging device installed on the flying object 100 using a gimbal. Referring to FIG. 6, the flying object 100 is shown. In this example, the monocular imaging device is installed on the upper side of the flying object 100 using a gimbal so that the optical axis 41 is the vertical axis (axis in the vertical direction 43). By using the gimbal, the optical axis 41 remains the vertical axis even if the flying object 100 is tilted. Therefore, the monocular imaging device always images a range 45 in the direction opposite to the vertical direction 43.

[0087] FIG. 7 is an explanatory diagram for explaining a second example of imaging by a monocular imaging device installed on the flying object 100 using a gimbal. Referring to FIG. 7, the flying object 100 is shown. In this example, the monocular imaging device is installed on the underside of the flying object 100 using a gimbal so that the optical axis 41 is the vertical axis (axis in the vertical direction 43). By using the gimbal, the optical axis 41 remains the vertical axis even if the flying object 100 is tilted. Therefore, the monocular imaging device always captures an image in a range 47 in the vertical direction 43.

[0088] In this way, by installing a monocular imaging device on the flying object 100 so that the optical axis is in the vertical direction of the flying object 100, it is possible to, for example, make on-the-spot rotation that is unlikely to cause parallax correspond to the roll axis and pitch axis of the flying object 100. The flying object 100 tilts toward the roll axis and pitch axis in order to move, but the tilt is within a range of about ±15 degrees. Therefore, it is unlikely that feature points will be replaced by new ones due to rotation, so that position estimation is unlikely to become difficult. Furthermore, by installing using a gimbal, it is possible to always capture images of, for example, a range in the vertical direction or a range in the opposite direction to the vertical direction. Therefore, the robustness of position estimation can be improved.

[0089] For example, if a texture pattern of a takeoff and landing site (e.g., a heliport) of the aircraft 100 is stored in advance, the takeoff and landing site can be used as a feature point of known size when the aircraft 100 takes off. This can make position estimation more accurate.

[0090] -Stereo SLAM For example, there is SLAM that uses a stereo imaging device (hereinafter referred to as "stereo SLAM"). In stereo SLAM, the three-dimensional position of a feature point is estimated by utilizing the parallax of two cameras. Therefore, it is possible to estimate the position even if the flying object 100 does not move. In addition, it is possible to estimate the position even if there are no feature points of known size.

[0091] -SLAM Features In SLAM, errors accumulate in the estimated position depending on the amount of movement. Therefore, when the flying object 100 moves over a long distance, drift may occur as with an inertial sensor. However, SLAM can complement position estimation using an inertial sensor because the amount of error may be smaller than that of an inertial sensor and the error does not become large when the same feature points are used.

[0092] (b-2) Marker For example, a method using an imaging device installed on the flying object 100 includes a method of estimating the position of the flying object based on known positions where markers are located and an image of the marker generated by an imaging device installed on the flying object 100.

[0093] For example, a plurality of markers are installed within a certain area, and the position of each of the plurality of markers is stored in advance in the flying object 100. When the flying object 100 flies within the certain area, an imaging device installed in the flying object 100 captures an image, and the position of the flying object 100 is estimated from the position of each marker in the captured image generated by the imaging and the known position of each marker. As an example, the imaging device is an omnidirectional camera that enables imaging in all directions. As an example, each of the plurality of markers is a light-emitting marker (for example, a light-emitting LED (Light-Emitting Diode)), and appears as a glowing mass in the captured image. An example of the arrangement of the markers will be described below with reference to FIG. 8.

[0094] FIG. 8 is an explanatory diagram for explaining an example of marker placement. Referring to FIG. 8, an area 51 and an aircraft 100 flying in the area 51 are shown. Four markers 53 are placed in the area 51. An imaging device (e.g., an omnidirectional camera) installed in the aircraft 100 captures images, and as a result, captures images of the four markers 53. Then, the position of the aircraft 100 is estimated from the positions of each of the four markers 53 in the captured image and the known positions of the four markers 53.

[0095] For example, the direction from the center of the image captured by the omnidirectional camera to the marker coincides with the direction from the actual omnidirectional camera to the marker. Therefore, the position (Mix, Miy) and observation direction Theta (Greek letter) of the marker i, as well as the estimated position (x, y) and estimated direction alpha (Greek letter) of the flying object 100 are expressed as follows:

[0096]

number

[0097] Since there are three unknowns (i.e., x, y, and alpha (Greek letter)) and four observation points (four markers), the three unknowns are calculated using a nonlinear optimization problem.

[0098] Of course, the method of calculating the estimated position (x, y) and the estimated direction alpha (Greek letter) is not limited to the above example. As an example, a plurality of candidates for the estimated position (x, y) and the estimated direction alpha (Greek letter) may be prepared in advance, and the observation direction theta (Greek letter) may be calculated from a candidate included in the plurality of candidates according to the above formula. Then, the difference between the calculated observation direction and the actual observation direction obtained from the captured image may be calculated as an error, and further, the likelihood of the candidate may be calculated from the calculated error. Then, one of the plurality of candidates may be selected based on the likelihood of each of the plurality of candidates, and the one candidate may be estimated as the position and direction of the flying object 100. At this time, a particle filter may be used.

[0099] In addition, outdoors, the light-emitting marker may be buried in the captured image due to the reflection of sunlight, etc. Cutting off visible light with a filter may also make separation difficult. Therefore, for example, the light-emitting marker may have a specific light-emitting pattern. Then, the light-emitting marker having the specific light-emitting pattern may be detected from the captured image.

[0100] The above-described position estimation method (i.e., a method of estimating the position based on an image captured by a marker) does not accumulate errors proportional to time, and therefore can complement position estimation using an inertial sensor.

[0101] (c) A method using an imaging device to capture images of an aircraft For example, one method of estimating the position is to use an imaging device that captures an image of the flying object 100 .

[0102] For example, in motion capture, an image of the flying object 100 is captured by an imaging device, and the position and movement of the flying object 100 are calculated based on the captured image. Hereinafter, an example of a case in which motion capture is used will be described with reference to FIG.

[0103] FIG. 9 is an explanatory diagram for explaining an example of a case where motion capture is used. For example, the information processing device 400 calculates the position and movement of the flying object 100 based on the captured images generated by the imaging devices 410 and 420. Then, the information processing device 400 transmits information indicating the position (and the movement) to the flying object 100. As an example, the information processing device 400 transmits the above information to the flying object 100 via the control device 200. Note that the information processing device 400 may transmit the above information to the flying object 100 directly or via a relay node. Also, the information processing device 400 and the control device 200 may be the same device.

[0104] In the above-described position estimation method (i.e., the method using an imaging device that captures an image of the flying object 100), there is no accumulation of errors proportional to time. Therefore, this position estimation method can complement position estimation using an inertial sensor.

[0105] (d) Barometer method For example, one method for estimating position is to use a barometer.

[0106] For example, the altitude of the aircraft 100 is estimated from the air pressure measured by the barometer 135 of the aircraft 100. Specifically, the altitude corresponding to the air pressure measured by the barometer 135 of the aircraft 100 is estimated as the altitude of the aircraft 100.

[0107] The barometer-based method does not accumulate errors proportional to time, so it can complement the inertial sensor-based method.

[0108] (e) Method using ultrasonic sensors For example, one method for estimating a position is to use an ultrasonic sensor.

[0109] For example, the ultrasonic sensor 137 of the flying object 100 emits ultrasonic waves in the vertical direction and receives the reflected waves of the ultrasonic waves. Then, the altitude of the flying object 100 is estimated from the time from the emission of the ultrasonic waves to the reception of the reflected waves.

[0110] The method using ultrasonic sensors does not accumulate errors proportional to time, so this position estimation method can complement position estimation using inertial sensors.

[0111] (f) Characteristics of each method The above-mentioned position estimation methods each have different characteristics, which will be described below with reference to FIG.

[0112] FIG. 10 is an explanatory diagram for explaining the characteristics of each position estimation method. Referring to FIG. 10, the accuracy (resolution and / or position error), sampling period, time delay, filter time constant, use environment, and compatibility with the inertial sensor of each position estimation method are shown. The time delay is a time delay associated with the position estimation. The filter time constant is a time constant of a filter used to calculate a correction value, and is a value according to the noise characteristics of the position estimation. For example, in a method using a GPS receiver, the accuracy is 5 to 10 meters, the sampling period is 2 to 5 Hz, and the time delay associated with the position estimation is 400 to 600 milliseconds. In monocular SLAM, the accuracy is several centimeters, the sampling period is about 30 Hz, and the time delay associated with the position estimation is 30 to 100 milliseconds. Thus, the accuracy, period, time delay, filter time constant, use environment, and compatibility with the inertial sensor differ depending on the position estimation method.

[0113] (2) Selection of location estimation method and location estimation The information acquisition unit 173 acquires information indicating a position estimation method selected from a plurality of position estimation methods for estimating the position of the flying body 100. The position estimation unit 179 estimates the position of the flying body 100 from first information generated using the inertial sensor 131 of the flying body 100 and second information generated through the position estimation method in accordance with parameters for the position estimation method.

[0114] (a) Multiple location estimation methods For example, the multiple position estimation methods include at least one of a method using a GPS receiver, a method using an imaging device installed in the flying object 100, a method using an imaging device that images the flying object 100, a method using a barometer, and a method using an ultrasonic sensor. The description of these position estimation methods is as described above.

[0115] (b) Obtaining information indicating the location estimation method As a first example, as described later, the selection unit 171 dynamically selects a location estimation method from the plurality of location estimation methods, and the information acquisition unit 173 acquires information indicating the location estimation method.

[0116] As a second example, a user may statically select a location estimation method from among the multiple location estimation methods, and information indicating the location estimation method may be stored in the flying object 100. Then, the information acquisition unit 173 may acquire the information.

[0117] For example, the information indicating the location estimation method is identification information of the location estimation method.

[0118] (c) First information generated using the inertial sensor 131 For example, the inertial sensor 131 includes an acceleration sensor, and the first information generated using the inertial sensor 131 includes information indicating the acceleration of the flying object 100.

[0119] For example, the first information generated using the inertial sensor 131 further includes information indicating the attitude of the flying object 100. The attitude is estimated by the attitude estimation unit 177 as described above.

[0120] (d) Second information generated through a location estimation method For example, the second information generated through the position estimation method is information indicating the position of the flying object 100.

[0121] (d-1) First Example: Case of a method using a GPS receiver As a first example, the position estimation method is a method using a GPS receiver. In this case, for example, the information generating unit 175 acquires output information (information indicating the latitude, longitude, and altitude of the flying object 100) of the GPS receiver 133, and generates the second information indicating the position of the flying object 100 (for example, information indicating the position of the flying object 100 with a predetermined position as the origin) from the output information. Then, the position estimating unit 179 acquires the second information.

[0122] (d-2) Second Example: Case of a Method Using an Imaging Device Installed on the Aircraft 100 As a second example, the position estimation method is a method that uses an imaging device installed in the flying object 100 (for example, SLAM, or a position estimation method based on an image captured by a marker). In this case, for example, the information generation unit 175 acquires an image generated by an imaging device included in the imaging unit 140, and generates the second information indicating the position of the flying object 100 based on the acquired image. Then, the position estimation unit 179 acquires the second information.

[0123] (d-3) Third Example: Case of a method using an imaging device to capture an image of the flying object 100 As a third example, the position estimation method is a method using an imaging device that captures an image of the flying object 100. Referring again to Fig. 9, for example, the information processing device 400 generates the second information indicating the position of the flying object 100, and the second information is transmitted to the flying object 100. Then, the position estimation unit 179 acquires the second information.

[0124] (d-4) Fourth Example: Case of the Method Using a Barometer As a fourth example, the position estimation method is a method using a barometer. In this case, for example, the information generating unit 175 acquires output information (information indicating the atmospheric pressure) of the barometer 135, and generates the second information indicating the position of the flying object 100 (for example, information indicating the altitude of the flying object 100) from the output information. Then, the position estimating unit 179 acquires the second information.

[0125] (d-5) Fifth Example: Method using ultrasonic sensors As a fifth example, the position estimation method is a method using an ultrasonic sensor. In this case, for example, the position estimation unit 179 acquires output information (information indicating distance) of the ultrasonic sensor 137 as the second information indicating the position of the flying object 100 (for example, information indicating the altitude of the flying object 100).

[0126] (e) Estimation of position according to parameters (e-1) Parameter retention For example, parameters for each of the plurality of position estimation methods are held in the flying object 100. As an example, a table including parameters for each of the plurality of position estimation methods is held. Then, the position estimation unit 179 acquires the held parameters.

[0127] (e-2) Parameter example: Time delay parameters For example, the parameters for the position estimation method include a parameter related to a time delay associated with position estimation in the position estimation method (hereinafter referred to as a "time delay parameter").

[0128] -Example of a time delay parameter As an example, the time delay parameter is the time delay associated with the position estimation by the position estimation method. As another example, the time delay parameter may be a difference between the time delay associated with the position estimation by the position estimation method and the time delay associated with the position estimation using an inertial sensor. Examples of the time delay for each position estimation method are as described with reference to FIG. 10.

[0129] -Location estimation example For example, the position estimation unit 179 calculates a correction value from the second information generated through the position estimation method according to the time delay parameter, and estimates the position of the flying object 100 from the first information generated using the inertial sensor 131 and the correction value.

[0130] More specifically, for example, the position estimation unit 179 acquires third information indicating the already estimated position of the flying body 100 according to the time delay parameter. Then, the position estimation unit 179 calculates the correction value from the second information generated through the position estimation method and the third information. A specific example will be described below with reference to Figs. 3 and 11.

[0131] Referring again to FIG. 3, for example, the position estimation unit 179 acquires information indicating the acceleration a as the first information (information generated using the inertial sensor 131). Then, the position estimation unit 179 estimates the position pU of the flying object from the information indicating the acceleration a and the correction values ​​(acceleration correction value Δa, velocity correction value Δv, and position correction value Δ) output by the filter 25. The position estimation unit 179 also acquires information indicating the position px as second information generated through the position estimation method, and acquires information indicating the position pU as third information indicating the already estimated position of the flying object 100. Then, the position estimation unit 179 calculates the difference between the position px and the position pU as an error e from the information indicating the position px and the information indicating the position pU, and passes the error e through the filter 25 to calculate the correction values ​​(acceleration correction value Δa, velocity correction value Δv, and position correction value Δ). In particular, the position estimation unit 179 acquires information (third information) indicating the position pU according to a parameter related to a time delay (that is, a time delay parameter) associated with position estimation in the above-mentioned position estimation method.

[0132] FIG. 11 is an explanatory diagram for explaining an example of position estimation according to a time delay parameter. Referring to FIG. 11, the third information 61 indicating the position pU of the flying object 100 and the second information 63 (information indicating the position px) generated through a position estimation method are shown. For example, the time delay 65 (i.e., the time delay associated with the generation of the third information 61) associated with the position estimation using the inertial sensor 131 is much shorter than the time delay 67 (i.e., the time delay associated with the generation of the second information) associated with the position estimation by a position estimation method (e.g., a method using a GPS receiver). Therefore, for example, if the second information 63A obtained at time T3 and the third information 61F that is the latest at that time are simply acquired, and the difference between the position px indicated by the second information 63A and the position pU indicated by the third information 61F is calculated as the error e, the error e may become an inappropriate value. That is, the difference between the positions at different times is calculated as the error e. Therefore, for example, the position estimation unit 179 acquires the third information 61A and the second information 63A indicating the position per time T1 that is the time lag 67 before the time T3 according to a parameter indicating the time lag 67 (or a parameter indicating the difference between the time lag 67 and the time lag 65). Then, the position estimation unit 179 calculates the difference between the position px indicated by the second information 63A and the position pU indicated by the third information 61A as the error e. Similarly, the position estimation unit 179 acquires the third information 61F and the second information 63B indicating the position per time T2. Then, the position estimation unit 179 calculates the difference between the position px indicated by the second information 63B and the position pU indicated by the third information 61F as the error e. Note that the position pU (i.e., the third information 61) is held for at least a period equivalent to the time lag 67 in order to calculate the error e.

[0133] For example, in this manner, the position of the flying object 100 is estimated according to the time delay parameters for the position estimation method. This allows, for example, calculation of an appropriate correction value suited to the position estimation method. As a result, the position of the flying object 100 can be better estimated. Hereinafter, a specific example of this point will be described with reference to Figs. 12 and 13.

[0134] FIG. 12 is an explanatory diagram for explaining an example of a result of position estimation without a time delay parameter. In this example, a method using a GPS receiver is used as a position estimation method. A position 1010 estimated using an inertial sensor changes according to the movement of the flying object 100. On the other hand, a position 1020 estimated using a GPS receiver does not change immediately after the movement of the flying object 100. In this example, since position estimation is performed without a time delay parameter for the above position estimation method (i.e., a method using a GPS receiver), even though the position 1010 changes, the estimated position 1030 is pulled back to the original position due to the unchanged position 1020.

[0135] FIG. 13 is an explanatory diagram for explaining an example of a result of position estimation according to a time delay parameter. In this example, a method using a GPS receiver is also used as the position estimation method. A position 1010 estimated using an inertial sensor changes according to the movement of the flying object 100. On the other hand, a position 1020 estimated using a GPS receiver does not change immediately after the movement of the flying object 100. In this example, since the position estimation is performed according to the time delay parameter for the above position estimation method (i.e., the method using a GPS receiver), the pullback of the estimated position 1030 caused by the unchanged position 1020 is small, and a good estimated position 1030 can be obtained.

[0136] (e-3) Parameter examples: parameters related to noise characteristics For example, the parameters for the position estimation method include parameters relating to characteristics of noise accompanying position estimation in the position estimation method (hereinafter referred to as "noise characteristic parameters").

[0137] -Example of noise characteristic parameters As an example, the noise characteristic parameter is the time constant of a filter used to calculate the correction value. Referring again to Fig. 3, the noise characteristic parameter is, for example, the time constant of filter 25. Examples of the filter time constant for each position estimation method are as described with reference to Fig. 10.

[0138] As another example, the noise characteristic parameter may be a band of the noise, and a time constant of a filter may be calculated from the noise characteristic parameter (the band of the noise).

[0139] -Location estimation example For example, the position estimation unit 179 calculates a correction value from the second information generated through the position estimation method according to the noise characteristic parameters, and estimates the position of the flying object 100 from the first information generated using the inertial sensor 131 and the correction value.

[0140] Referring again to FIG. 3, for example, a time constant (i.e., a noise characteristic parameter) for a position estimation method selected from among a plurality of position estimation methods is set as the time constant Tc of the filter 25. The time constant is a time constant corresponding to the characteristics of noise accompanying position estimation in the position estimation method. The position estimation unit 179 acquires information indicating the position px as second information generated through the position estimation method, and calculates the difference between the position px and the position pU of the flying object 100 that has already been estimated as an error e. Then, the position estimation unit 179 calculates correction values ​​(acceleration correction value Δa, velocity correction value Δv, and position correction value Δ) by passing the error e through the filter 25 (time constant Tc=noise characteristic parameter). Then, the position estimation unit 179 estimates the position pU of the flying object 100 from the first information (e.g., information indicating the acceleration a) generated using the inertial sensor 131 and the above correction values.

[0141] For example, in this manner, the position of the flying object 100 is estimated according to the noise characteristic parameters for the position estimation method. This allows, for example, calculation of an appropriate correction value suited to the position estimation method. As a result, the position of the flying object can be better estimated.

[0142] As described above, the position estimation unit 179 estimates the position of the flying object 100 according to parameters for a position estimation method selected from among a plurality of position estimation methods. This makes it possible to apply other position estimation methods in addition to, for example, a position estimation method using a GPS receiver. Therefore, even if position estimation using a GPS receiver is difficult, it becomes possible to better estimate the position of the flying object 100. In addition, even if the position estimation method is switched among the plurality of position estimation methods, the characteristics of the position estimation method (e.g., time delay and / or noise characteristics, etc.) are reflected in the position estimation, so that the position of the flying object 100 can be better estimated. In addition, it becomes possible to more easily switch the position estimation method among the plurality of position estimation methods.

[0143] (3) Dynamic selection of location estimation method For example, the selection unit 171 dynamically selects a position estimation method from among the multiple position estimation methods while the flying object 100 is flying. That is, while the flying object 100 is flying, the position estimation method is switched among the multiple position estimation methods.

[0144] (a) Delivery of initial value For example, when a new location estimation scheme is selected, an initial value of the location estimate is provided for location estimation in the new location estimation scheme.

[0145] (a-1) Operation of each component For example, the selection unit 171 selects a first position estimation method from among the multiple position estimation methods. Then, the information acquisition unit 173 acquires information indicating the first position estimation method. Then, the position estimation unit 179 estimates the position of the flying object 100 from the first information generated by using the inertial sensor 131 and the second information generated through the first position estimation method, in accordance with the parameters for the first position estimation method.

[0146] Furthermore, after selecting the first location estimation method, the selection unit 171 selects a second location estimation method from among the multiple location estimation methods. In particular, the selection unit 171 provides the estimated location as an initial value for location estimation in the second location estimation method. For example, the information generation unit 175 performs location estimation in the second location estimation method, and the selection unit 171 provides the estimated location to the information generation unit 175 as the initial value.

[0147] Then, the information acquisition unit 173 acquires information indicating the second position estimation method. After that, the position estimation unit 179 estimates the position of the flying object 100 from the first information generated by using the inertial sensor 131 and the second information generated through the second position estimation method, in accordance with the parameters for the second position estimation method.

[0148] (a-2) Example of location estimation method For example, the second position estimation method is a method for estimating the relative position of the flying object 100. More specifically, for example, the second position estimation method is a method using an imaging device installed in the flying object 100 (for example, SLAM, or a method of position estimation based on an image captured by a marker).

[0149] For example, the first position estimation method is a method for estimating the absolute position of the flying object 100. More specifically, for example, the first position estimation method is a method using a GPS receiver, a method using an imaging device that images the flying object 100, a method using a barometer, or a method using an ultrasonic sensor. Note that the first position estimation method may also be a method for estimating the relative position of the flying object 100.

[0150] (a-3) Specific examples Referring again to FIG. 3, for example, the selection unit 171 selects a first position estimation method (for example, a method using a GPS receiver). Then, the position estimation unit 179 estimates a position pU of the flying object 100 from first information (for example, information indicating acceleration a) generated using the inertial sensor 131 and second information (information indicating the position px) generated through the first position estimation method. After that, for example, the selection unit 171 selects a second position estimation method (for example, SLAM) and provides the position pU to the information generation unit 175 as an initial value of position estimation in the second position estimation method. Then, the information generation unit 175 sets the position pU as an initial value of position estimation in the second position estimation method, and generates second information (information indicating the position px) through the second position estimation method. Then, the position estimation unit 179 estimates the position pU of the flying body 100 from the first information (e.g., information indicating the acceleration a) generated using the inertial sensor 131 and the second information (information indicating the position px) generated by the information generation unit 175 through the second position estimation method.

[0151] (a-4)Other For example, if the second position estimation method is a method for estimating the absolute position of the flying body 100 (e.g., a method using a GPS receiver), the selection unit 171 does not provide the estimated position as the initial value for position estimation in the second position estimation method.

[0152] In addition to the estimated position of the aircraft 100, the selection unit 171 may also provide the attitude of the aircraft 100 estimated by the attitude estimation unit 177 as an initial value of the attitude for position estimation using the second position estimation method.

[0153] For example, as described above, when a new location estimation method is selected, an initial value of location estimation is provided for location estimation in the new location estimation method. This makes it possible to perform seamless location estimation, for example. It also makes it possible to transfer the location estimation value in the method for estimating the absolute location to the method for estimating the relative location. In addition, since the location estimation value in the method for estimating the relative location is reset every time a selection is made, accumulation of errors can be reduced.

[0154] (b) Selection Trigger (b-1) Location conditions For example, when the position of the flying object 100 satisfies a predetermined position condition, the selection unit 171 newly selects a position estimation method from among the multiple position estimation methods.

[0155] More specifically, for example, when the position of the flying object 100 reaches a predetermined switching position, the selection unit 171 newly selects a position estimation method from among the plurality of position estimation methods. A specific example will be described below with reference to Figs. 14 to 16.

[0156] -First example FIG. 14 is an explanatory diagram for explaining a first example of the selection (switching) of the position estimation method. Referring to FIG. 14, a bridge 71 is shown. In this example, the flying object 100 flies along a route from Start to Goal in order to capture an image of the entire back side of the bridge 71. That is, the flying object 100 travels between an area 75 where radio waves from a GPS satellite 73 reach sufficiently and an area 77 where radio waves from the GPS satellite 73 reach poorly. In this case, when the flying object 100 enters the area 77 from the area 75, the selection unit 171 newly selects SLAM (for example, stereo SLAM) as the position estimation method. That is, when the position of the flying object 100 reaches the boundary position between the area 75 and the area 77 (or a position near the boundary position), the selection unit 171 newly selects SLAM (for example, stereo SLAM) as the position estimation method. Here, the position already estimated is taken over as the initial value of the position estimation in SLAM. Also, the attitude that has already been estimated may be taken over as the initial value of the attitude for position estimation in SLAM. Meanwhile, when the flying object 100 enters area 75 from area 77, the selection unit 171 newly selects the method using GPS as the position estimation method. That is, when the position of the flying object 100 reaches the boundary position between area 77 and area 75 (or a position near the boundary), the selection unit 171 newly selects the method using GPS as the position estimation method. In this manner, the position estimation method is switched (that is, switched between the method using GPS and SLAM).

[0157] In addition, the position estimation method may be selected in a similar manner when the flying object 100 travels between indoors and outdoors.

[0158] -Second example FIG. 15 is an explanatory diagram for explaining a second example of the selection (switching) of the position estimation method. Referring to FIG. 15, a high-rise building 81 is shown. In this example, the flying object 100 flies along a route including a path between the high-rise buildings 81. That is, the flying object 100 flies in an area 83 where radio waves from a GPS satellite reach sufficiently, and an area 85 where radio waves from a GPS satellite reach poorly (that is, an area between the high-rise buildings 81). In this case, when the flying object 100 enters the area 85 from the area 83, the selection unit 171 newly selects SLAM (e.g., stereo SLAM) as the position estimation method. That is, when the position of the flying object 100 becomes the boundary position between the area 83 and the area 85 (or a position near the boundary), the selection unit 171 newly selects SLAM (e.g., stereo SLAM) as the position estimation method. Here, the already estimated position is taken over as the initial value of the position estimation in SLAM. Also, the attitude that has already been estimated may be taken over as the initial value of the attitude for position estimation in SLAM. Meanwhile, when the flying object 100 enters area 83 from area 85, the selection unit 171 newly selects the method using GPS as the position estimation method. That is, when the position of the flying object 100 becomes the boundary position between area 85 and area 83 (or a position near the boundary), the selection unit 171 newly selects the method using GPS as the position estimation method. In this manner, the position estimation method is switched (that is, switched between the method using GPS and SLAM).

[0159] In addition, the position estimation method may be selected in a similar manner when the flying object 100 flies through a valley in a mountain.

[0160] -Third example FIG. 16 is an explanatory diagram for explaining a third example of the selection (switching) of the position estimation method. Referring to FIG. 16, a stadium 91 is shown. A marker 93 is placed in the stadium 91. In this example, the flying object 100 flies inside and outside the stadium 91. That is, the flying object 100 flies in an area 95 where the marker 93 cannot be imaged and an area 97 where the marker 93 can be imaged. In this case, when the flying object 100 enters the area 97 from the area 95, the selection unit 171 newly selects a position estimation method based on an image of the marker as the position estimation method when the flying object 100 is at the boundary between the area 95 and the area 97 (or a position near the boundary), the selection unit 171 newly selects a position estimation method based on an image of the marker as the position estimation method. Here, the already estimated position is taken over as the initial value of the position estimation based on the image of the marker. Also, the already estimated attitude may be taken over as the initial value of the attitude for position estimation based on the captured image of the marker. Meanwhile, when the flying object 100 enters area 95 from area 97, the selection unit 171 newly selects the method using GPS as the position estimation method. That is, when the position of the flying object 100 becomes the boundary position between area 97 and area 95 (or a position near the boundary), the selection unit 171 newly selects the method using GPS as the position estimation method. In this manner, the position estimation method is switched (that is, switched between the method using GPS and the method of position estimation based on the captured image of the marker).

[0161] In addition, the position estimation method may be selected in a similar manner when the flying object 100 travels between indoors, where markers are placed, and outdoors, where no markers are placed.

[0162] As described above, when the position of the flying object 100 satisfies the above-mentioned predetermined position condition, a new position estimation method is selected. This makes it possible to automatically switch the position estimation method at a desired position, for example. Therefore, an appropriate position estimation method can be applied depending on the area in which the flying object 100 is located.

[0163] (b-2) Reliability of location estimation method The selection unit 171 may select a location estimation method from among the plurality of location estimation methods based on a reliability that is dynamically calculated for each of the plurality of location estimation methods.

[0164] The reliability for each of the plurality of location estimation methods may be normalized (for example, to a numerical value from 0 to 100). Then, when the reliability calculated for the current location estimation method is lower than the reliability calculated for the other location estimation methods, the selection unit 171 may newly select the other location estimation method.

[0165] The reliability of a method using a GPS receiver may be calculated based on the capture status of GPS satellites (e.g., the number of GPS satellites from which signals are received and / or the reception sensitivity of signals transmitted by the GPS satellites). The reliability of SLAM may be calculated based on the number of feature points. The reliability of a method of position estimation based on captured images of markers may be calculated based on the detection status of the markers.

[0166] This may result in, for example, a more reliable location estimation scheme being applied, improving the accuracy of location estimation.

[0167] For example, as described above, a position estimation method is dynamically selected from the plurality of position estimation methods during the flight of the flying object 100. This makes it possible to better estimate the position of the flying object 100, for example, when the flying object 100 flies through various areas.

[0168] (4) Combination of location estimation methods The information acquisition unit 173 may acquire information indicating two or more position estimation methods selected from the plurality of position estimation methods. Then, the position estimation unit 179 may estimate the position of the flying object 100 from the first information generated by using the inertial sensor 131 and the second information generated through each of the two or more position estimation methods, according to parameters for each of the two or more position estimation methods. That is, two or more position estimation methods may be used in combination.

[0169] For example, the second information generated through each of the two or more position estimation methods is information indicating the position of the flying object 100.

[0170] As an example, the position estimation unit 179 may calculate an average (i.e., an average position) of the positions estimated through the two or more position estimation methods. Then, the position estimation unit 179 may estimate the position of the flying body 100 from the first information and information indicating the average (i.e., the average position).

[0171] As another example, the position estimation unit 179 may multiply the position estimated through the position estimation method by a weight corresponding to the position estimation method for each of the two or more position estimation methods. Then, the position estimation unit 179 may calculate the sum of the weighted positions for the two or more position estimation methods. Then, the position estimation unit 179 may estimate the position of the flying body 100 from the first information and information indicating the sum. Note that the weight corresponding to the position estimation method may be a reliability for the position estimation method.

[0172] This may, for example, improve the accuracy of position estimation.

[0173] <5. Processing flow> Next, an example of processing according to an embodiment of the present disclosure will be described with reference to FIGS.

[0174] (1) Location estimation process FIG. 17 is a flowchart showing an example of a schematic flow of a position estimation process according to an embodiment of the present disclosure.

[0175] The information acquisition unit 173 acquires information indicating a position estimation method selected from among a plurality of position estimation methods for estimating the position of the aircraft 100 (S501).

[0176] The position estimation unit 179 obtains and applies parameters for the position estimation method (S503).

[0177] The position estimation unit 179 estimates the position of the aircraft 100 from the first information generated using the inertial sensor 131 of the aircraft 100 and the second information generated through the position estimation method (S505). Then, the process repeats step S505.

[0178] (2) Selection process (a) First Example FIG. 18 is a flowchart showing an example of a schematic flow of the first selection process according to an embodiment of the present disclosure.

[0179] The selection unit 171 acquires information indicating the position of the flying object 100 (S521). For example, the position is a position estimated by the position estimation unit 179.

[0180] If the position of the flying object 100 satisfies a predetermined position condition (S523: YES), the selection unit 171 selects a new position estimation method from among a plurality of position estimation methods (S525). Then, the selection unit 171 provides the position of the flying object 100 (for example, the position estimated by the position estimation unit 179) as an initial value for position estimation in the position estimation method (S527). Then, the process returns to step S521.

[0181] If the position of the flying object 100 does not satisfy the predetermined position condition (S523: NO), the process returns to step S521.

[0182] (b) Second Example FIG. 19 is a flowchart showing an example of a schematic flow of the second selection process according to an embodiment of the present disclosure.

[0183] The selection unit 171 acquires information indicating the reliability of each of a plurality of position estimation methods (S541).

[0184] If the reliability of the other position estimation method is higher than the reliability of the current position estimation method (S543: YES), the selection unit 171 newly selects the other position estimation method (S545). Then, the selection unit 171 provides the position of the flying object 100 (for example, the position estimated by the position estimation unit 179) as an initial value for position estimation in the other position estimation method (S547). Then, the process returns to step S541.

[0185] If the reliability of the current position estimation method is higher than the reliability of the other position estimation methods (S543: NO), the process returns to step S541.

[0186] <6. Summary> Up to this point, the flying body 100 and each process according to the embodiment of the present disclosure have been described with reference to Figs. 1 to 19. According to the embodiment of the present disclosure, the flying body 100 includes an information acquisition unit 173 that acquires information indicating a position estimation method selected from a plurality of position estimation methods for estimating the position of the flying body 100, and a position estimation unit 179 that estimates the position of the flying body 100 from first information generated using the inertial sensor 131 of the flying body 100 and second information generated through the position estimation method according to parameters for the position estimation method. This makes it possible to, for example, estimate the position of the flying body 100 more accurately.

[0187] In addition, a module for the flying object 100 may include the information acquisition unit 173 and the position estimation unit 179 (as well as one or more other components included in the processing unit 170).

[0188] Although the preferred embodiments of the present disclosure have been described above with reference to the accompanying drawings, it goes without saying that the present disclosure is not limited to the examples. It is clear that a person skilled in the art can think of various modified or amended examples within the scope of the claims, and it is understood that these also naturally belong to the technical scope of the present disclosure.

[0189] For example, the processing steps in the processes described herein do not necessarily have to be performed in chronological order according to the order depicted in the flowcharts or sequence diagrams. For example, the processing steps in the processes may be performed in an order different from that depicted in the flowcharts or sequence diagrams, or may be performed in parallel.

[0190] Also, a computer program (in other words, a computer program for causing the processor to execute the operation of the components of the device) for causing the processor (e.g., a CPU, DSP, etc.) included in the device (e.g., an aircraft or a module for an aircraft) of the present specification to function as the components of the device (e.g., the selection unit 171, the information acquisition unit 173, the information generation unit 175, the attitude estimation unit 177, the position estimation unit 179, and / or the control unit 181) can be created. A recording medium on which the computer program is recorded may also be provided. Also, an device (e.g., an aircraft or a module for an aircraft) including a memory for storing the computer program and one or more processors capable of executing the computer program may also be provided. Also, a method including the operation of the components of the device (e.g., the selection unit 171, the information acquisition unit 173, the information generation unit 175, the attitude estimation unit 177, the position estimation unit 179, and / or the control unit 181) is included in the technology according to the present disclosure.

[0191] In addition, the effects described in this specification are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that are apparent to a person skilled in the art from the description of this specification, in addition to or in place of the above effects.

[0192] Note that the following configurations also fall within the technical scope of the present disclosure. (1) an acquisition unit that acquires information indicating a position estimation method selected from a plurality of position estimation methods for estimating a position of an aircraft; a position estimation unit that estimates a position of the flying object from first information generated by an inertial sensor of the flying object and second information generated through the position estimation method according to parameters for the position estimation method; An apparatus comprising: (2) The device described in (1), wherein the parameters include a parameter related to a time delay associated with position estimation using the position estimation method. (3) The device described in (2), wherein the position estimation unit calculates a correction value from the second information according to the parameter regarding the time delay, and estimates the position of the aircraft from the first information and the correction value. (4) The device described in (3), wherein the position estimation unit obtains third information indicating an already estimated position of the aircraft according to the parameter related to the time delay, and calculates the correction value from the second information and the third information. (5) The device according to any one of (1) to (4), wherein the parameters include a parameter related to characteristics of noise accompanying position estimation by the position estimation method. (6) The device described in (5), wherein the position estimation unit calculates a correction value from the second information according to the parameters related to the characteristics of the noise, and estimates the position of the aircraft from the first information and the correction value. (7) The apparatus according to (6), wherein the parameter related to the noise is a time constant of a filter used to calculate the correction value. (8) The device described in any one of (1) to (7), wherein the multiple position estimation methods include at least one of a method using a Global Positioning System (GPS) receiver, a method using an imaging device installed on the aircraft, a method using an imaging device that images the aircraft, a method using a barometer, and a method using an ultrasonic sensor. (9) The apparatus described in (8), wherein the method using an imaging device installed on the aircraft includes a method of estimating the position of the aircraft based on a known position where a marker is located and an image of the marker generated by an imaging device installed on the aircraft. (10) The method using an imaging device installed on the aircraft includes SLAM (Simultaneous Localization and Mapping) using a monocular imaging device installed on the aircraft using a gimbal so that the optical axis is the vertical axis of the aircraft. (11) the first information includes information indicating an acceleration of the flying object; The second information is information indicating the position of the aircraft. The device according to any one of (1) to (10) above. (12) The device according to any one of (1) to (11), further comprising a selection unit that dynamically selects a position estimation method from among the plurality of position estimation methods while the aircraft is flying. (13) the selection unit selects a first location estimation method from the plurality of location estimation methods; the acquisition unit acquires information indicating the first location estimation method; the position estimation unit estimates a position of the aircraft from first information generated by the inertial sensor and second information generated through the first position estimation method in accordance with parameters for the first position estimation method; the selection unit, after selecting the first location estimation method, selects a second location estimation method from among the plurality of location estimation methods, and provides the estimated location as an initial value of location estimation in the second location estimation method; the acquisition unit acquires information indicating the second location estimation method; the position estimation unit estimates a position of the aircraft from first information generated by the inertial sensor and second information generated through the second position estimation method in accordance with parameters for the second position estimation method; The apparatus described in (12). (14) The apparatus described in (13), wherein the second position estimation method is a method for estimating the relative position of the flying object. (15) The device according to any one of (12) to (14), wherein the selection unit newly selects a position estimation method from among the plurality of position estimation methods when the position of the aircraft satisfies a predetermined position condition. (16) The device according to any one of (12) to (15), wherein the selection unit selects a location estimation method from among the plurality of location estimation methods based on a reliability dynamically calculated for each of the plurality of location estimation methods. (17) the acquisition unit acquires information indicating two or more location estimation methods selected from the plurality of location estimation methods; the position estimation unit estimates a position of the aircraft from first information generated by using the inertial sensor and second information generated through each of the two or more position estimation methods in accordance with parameters for each of the two or more position estimation methods; The device according to any one of (1) to (16) above. (18) The device according to any one of (1) to (17), wherein the device is the aircraft or a module for the aircraft. (19) The processor: acquiring information indicating a position estimation method selected from a plurality of position estimation methods for estimating a position of the flying object; estimating a position of the aircraft from first information generated by an inertial sensor of the aircraft and second information generated through the position estimation method according to parameters for the position estimation method; The method includes: (20) acquiring information indicating a position estimation method selected from a plurality of position estimation methods for estimating a position of the flying object; estimating a position of the aircraft from first information generated by an inertial sensor of the aircraft and second information generated through the position estimation method according to parameters for the position estimation method; A program for causing a processor to execute the above. [Explanation of symbols]

[0193] 1 System 51, 93 Marker 100 Flying Objects 171 Selection Section 173 Information Acquisition Department 179 Position estimation part 200 Control device 300 Flight Controls 400 Information processing device 410, 420 Imaging device

Claims

1. A GPS signal receiver for receiving a GPS (Global Positioning System) signal; an estimation unit that estimates an altitude corresponding to an air pressure measured by a barometer of the flying object; a position estimation unit that estimates a position of the flying object based on a GPS signal received by the GPS signal receiver and an altitude estimated by the estimation unit; Equipped with The position estimation unit estimates the position of the flying object based on either a first position estimation method based on the GPS signal received by the GPS signal receiver or a second position estimation method based on the altitude estimated by the estimation unit. Device.

2. The position estimation unit is estimating a position of the aircraft based on an average of a position of the aircraft estimated based on a first position estimation method based on the GPS signal received by the GPS signal receiver and a position of the aircraft estimated based on a second position estimation method based on the altitude estimated by the estimation unit; 2. The apparatus of claim 1.

3. The position estimation unit is a position of the flying object is estimated based on the GPS signal received by the GPS signal receiver and a weighted position obtained by multiplying the position of the flying object estimated based on the GPS signal received by the GPS signal receiver by a weight corresponding to a first position estimation method based on the GPS signal received by the GPS signal receiver, and a weighted position obtained by multiplying the position of the flying object estimated based on the altitude estimated by the estimation unit by a weight corresponding to a second position estimation method based on the altitude estimated by the estimation unit; 2. The apparatus of claim 1.

4. a weight corresponding to the first location estimation method and a weight corresponding to the second location estimation method each having a value according to a reliability of the location estimation method; 4. The apparatus of claim 3.

5. The position estimation unit is When the position of the aircraft is at a predetermined switching position, estimating the position of the aircraft based on one of the first and second position estimation methods.

2. The apparatus of claim 1.

6. The position estimation unit is when the position of the aircraft reaches the predetermined switching position within a predetermined range from a border between an area where the capture status of GPS satellites satisfies a predetermined condition and an area where the capture status of GPS satellites does not satisfy the predetermined condition, the position of the aircraft is estimated based on one of the first and second position estimation methods.

6. The apparatus of claim 5.

7. The position estimation unit is When the position of the aircraft has moved from an area where a GPS satellite capture situation satisfies a predetermined condition to the predetermined switching position, the position of the aircraft is estimated based on the second position estimation method from the first and second position estimation methods; When the position of the aircraft moves from an area where the capture status of GPS satellites does not satisfy a predetermined condition to the predetermined switching position, the position of the aircraft is estimated based on the first position estimation method from the first and second position estimation methods.

6. The apparatus of claim 5.

8. The predetermined condition is that the number of GPS satellites from which GPS signals are received and / or the reception sensitivity of the signals transmitted by the GPS satellites exceeds a predetermined threshold.

7. The apparatus of claim 6.

9. The apparatus of claim 1 , wherein the apparatus is the air vehicle or a module for the air vehicle.

10. The processor: Receiving a Global Positioning System (GPS) signal; Estimating an altitude corresponding to an air pressure measured by a barometer of the aircraft; estimating a position of the air vehicle based on the received GPS signals and the estimated altitude; Including, estimating a position of the flying object based on one of a first position estimation method based on the received GPS signal and a second position estimation method based on the estimated altitude; method.

11. Receiving a Global Positioning System (GPS) signal; Estimating an altitude corresponding to an air pressure measured by a barometer of the aircraft; estimating a position of the air vehicle based on the received GPS signals and the estimated altitude; to a processor, estimating a position of the flying object based on one of a first position estimation method based on the received GPS signal and a second position estimation method based on the estimated altitude; program.

Citation Information

Patent Citations

  • Positioning device for mobile body

    JP2010019703A

  • Image processing device, image processing method and program

    JP2011095797A

  • Mobile body positioning method, and mobile body positioning device

    JP2012177564A

  • Positioning marker, and system for estimating position and posture of imaging apparatus

    JP2013205278A

  • Transmission origin and physical property condition (deterioration condition) simultaneous identification method in elastic wave tomography performed on measurement target with heterogeneous physical properties

    JP2014174040A