Sensing system and unmanned aerial vehicle

Abstract

This sensing system comprises a ranging sensor that outputs first sensor data indicating a distance between an unmanned aerial vehicle and the ground or a ground object located below the unmanned aerial vehicle, an altitude sensor that outputs second sensor data indicating the altitude of the unmanned aerial vehicle, an acquisition device that acquires map data containing information on the elevation of each point on a map, and a processing device that estimates the height of the ground object located below the unmanned aerial vehicle on the basis of the distance indicated by the first sensor data, the altitude indicated by the second sensor data, and an elevation of a point below the unmanned aerial vehicle, the elevation being indicated by the map data.

Description

Sensing Systems and Unmanned Aerial Vehicles

[0001] The present disclosure relates to a sensing system and an unmanned aerial vehicle.

[0002] An unmanned aerial vehicle (UAV) is an aircraft that cannot carry a person due to its structure and can fly by remote control or automatic pilot. Rotary-wing unmanned aerial vehicles are unmanned aerial vehicles that obtain lift using propellers, i.e., rotors, that rotate around an axis. Small unmanned aerial vehicles equipped with multiple rotors (multi-rotor UAVs) are also called "drones," "multirotors," or "multicopters," and are widely used for applications such as aerial photography, surveying, logistics, and pesticide spraying.

[0003] In order to safely land an unmanned aerial vehicle, it is necessary to measure the height of the unmanned aerial vehicle from the ground and determine whether there are any obstacles at the landing point. Conventionally, these processes have been mainly achieved using an imaging device or LiDAR (light detection and ranging). Patent Document 1 describes an unmanned aerial vehicle equipped with LiDAR and an imaging device.

[0004] Japanese Patent Application Laid-Open No. 2022-162125

[0005] Improved technology is desired for determining whether there are any features below an unmanned aerial vehicle that could obstruct landing.

[0006] The present disclosure provides a sensing system capable of estimating the height of a feature located below an unmanned aerial vehicle, and an unmanned aerial vehicle equipped with the sensing system.

[0007] In an exemplary and non-limiting embodiment, the sensing system of the present disclosure is a sensing system used for an unmanned aerial vehicle, and includes a ranging sensor that outputs first sensor data indicating the distance between the unmanned aerial vehicle and a feature or ground located below the unmanned aerial vehicle, an altitude sensor that outputs second sensor data indicating the altitude of the unmanned aerial vehicle, an acquisition device that acquires map data including information on the elevation of each point on a map, and a processing device that estimates the height of the feature located below the unmanned aerial vehicle based on the distance indicated by the first sensor data, the altitude indicated by the second sensor data, and the elevation of the point below the unmanned aerial vehicle indicated by the map data.

[0008] In an exemplary and non-limiting embodiment, the unmanned aerial vehicle of the present disclosure includes multiple rotors and the above-described sensing system.

[0009] According to an embodiment of the present disclosure, there is provided a sensing system capable of estimating the height of a feature located below an unmanned aerial vehicle, and an unmanned aerial vehicle equipped with the sensing system. The sensing system can be suitably used as a landing determination system.

[0010] 1 is a block diagram schematically showing several examples of rotary drive devices that rotate rotors in an unmanned aerial vehicle with multiple rotors. FIG. 2 is a plan view schematically showing one basic configuration example of an unmanned aerial vehicle with multiple rotors. FIG. 3 is a side view schematically showing one basic configuration example of an unmanned aerial vehicle with multiple rotors. FIG. 4 is a plan view schematically showing another basic configuration example of an unmanned aerial vehicle with multiple rotors. FIG. 5 is a block diagram showing an example basic configuration of a battery-powered multicopter. FIG. 6 is a block diagram showing an example basic configuration of a series hybrid drive multicopter. FIG. 7 is a block diagram showing an example basic configuration of a parallel hybrid drive multicopter. FIG. 8 is a block diagram showing an example configuration of a sensing system. FIG. 9 is a diagram schematically showing a multicopter landing on the ground. FIG. 10 is a diagram schematically showing a multicopter hovering above the ground. FIG. 11 is a diagram schematically showing another configuration of a multicopter hovering above the ground. FIG. 12 is a diagram schematically showing a multicopter hovering above trees. 1 is a diagram illustrating an example in which a multicopter with a load suspended therefrom is hovering above a tree. 2 is a diagram illustrating an example in which a multicopter, an agricultural machine, a server, and a terminal device are connected via a communication network.

[0011] An unmanned aerial vehicle with multiple rotors includes a rotary drive unit that rotates the rotors (hereinafter sometimes referred to as "propellers"). Hereinafter, such an unmanned aerial vehicle will be referred to as a "multicopters."

[0012] There are various configurations of the rotary drive device provided in a multicopter. Fig. 1A is a block diagram schematically illustrating four examples of the rotary drive device 3 in the present disclosure.

[0013] The first rotation drive device 3A shown in FIG. 1A has a plurality of electric motors (hereinafter referred to as "motors") 14 that rotate a plurality of rotors, and a battery 52 that stores power to be supplied to each motor 14. The battery 52 is, for example, a secondary battery such as a polymer lithium-ion battery. Each rotor 2 is connected to the output shaft of the corresponding motor 14 and is rotated by the motor 14. In order to increase the payload and / or flight time, it is necessary to increase the power storage capacity of the battery 52. ​​The power storage capacity of the battery 52 can be increased by increasing the size of the battery 52, but increasing the size of the battery 52 results in an increase in weight.

[0014] The second rotation drive device 3B shown in FIG. 1A includes a power transmission system 23 mechanically connected to the rotor 2 and an internal combustion engine 7a that provides driving force (torque) to the power transmission system 23. The power transmission system 23 includes mechanical components such as gears or belts, and transmits torque from the output shaft of the internal combustion engine 7a to the rotor 2. The internal combustion engine 7a can efficiently generate mechanical energy by burning fuel. Examples of the internal combustion engine 7a include a gasoline engine, a diesel engine, and a hydrogen engine. The number of internal combustion engines 7a included in the rotation drive device 3B is not limited to one.

[0015] The third rotary drive device 3C shown in FIG. 1A includes multiple motors 14, a power buffer 9 that stores power to be supplied to each motor 14, a power generator 8 such as an alternator that generates power, and an internal combustion engine 7a that provides mechanical energy for the power generator 8 to generate electricity. A typical example of the power buffer 9 is a battery such as a secondary battery, but it may also be a capacitor. In the third rotary drive device 3C, even if the power buffer 9 does not have a large storage capacity, the power generator 8 generates power using the driving force (mechanical energy) of the internal combustion engine 7a, thereby enabling an increase in payload and / or flight time. This type of drive is called a "series hybrid drive." The power generator 8 and internal combustion engine 7a in the series hybrid drive are called a "range extender" because they extend the flight distance of the multicopter.

[0016] 1A includes a plurality of motors 14, a power buffer 9 that stores power to be supplied to each motor 14, a power generator 8 such as an alternator that generates power, an internal combustion engine 7a that provides driving force for generating power to the power generator 8, and a power transmission system 23 that transmits the driving force generated by the internal combustion engine 7a to a rotor 2 to rotate the rotor 2. At least one rotor 2 of the plurality of rotors 2 is rotated by the internal combustion engine 7a, and the other rotors 2 are rotated by the motor 14. In the fourth rotary drive device 3D, the mechanical energy generated by the internal combustion engine 7a can also be used to rotate the rotor 2 without being converted into electric power, thereby improving energy utilization efficiency. This type of drive is called a "parallel hybrid drive."

[0017] Fig. 1B is a plan view schematically illustrating one basic configuration example of multicopter 10. The configuration example of Fig. 1B includes the first rotational drive device 3A shown in Fig. 1A as the rotational drive device 3. That is, the rotational drive device 3 (3A) in this example includes a motor 14 and a battery 52. ​​Fig. 1C is a side view schematically illustrating the multicopter.

[0018] 1B and 1C includes a plurality of rotors 2, an airframe 4, and an airframe frame 5 that supports the rotors 2 and the airframe 4. The airframe frame 5 supports the airframe 4 at its center and rotatably supports the plurality of rotors 2 with a plurality of arms 5A extending outward from the center. A motor 14 that rotates the rotor 2 is provided near the tip of each arm 5A.

[0019] 1B, the multicopter 10 is a quad-type multicopter (quadcopter) having four rotors 2. The rotors 2 located on one diagonal line rotate in the same direction (clockwise or counterclockwise), while the rotors 2 located on different diagonal lines rotate in opposite directions.

[0020] The main body 4 includes a control device 4a that controls the operation of devices and components mounted on the multicopter 10, a group of sensors 4b connected to the control device 4a, a communication device 4c connected to the control device 4a, and a battery 52.

[0021] The control device 4 a may include, for example, a flight control device such as a flight controller and a host computer (companion computer). The companion computer can perform advanced arithmetic processing such as image processing, obstacle detection, and obstacle avoidance based on the sensor data acquired by the sensor group 4 b.

[0022] The sensor group 4b may include an acceleration sensor, an angular velocity sensor, a geomagnetic sensor, a barometric pressure sensor, an altitude sensor, a temperature sensor, a flow rate sensor, an imaging device, a laser sensor, an ultrasonic sensor, an obstacle contact sensor, and a Global Navigation Satellite System (GNSS) receiver. The acceleration sensor and the angular velocity sensor may be mounted on the airframe main body 4 as components of an IMU (Inertial Measurement Unit). Examples of the laser sensor may include, for example, a laser range finder used to measure the distance to the ground, and a two-dimensional or three-dimensional LiDAR.

[0023] The communication device 4c may include a wireless communication module for transmitting and receiving signals via an antenna to a transmitter or ground station (Ground Control Station (GCS)) on the ground, a mobile communication module using a cellular communication network, etc. The communication device 4c may receive signals such as control commands transmitted from the ground and transmit sensor data such as image data acquired by the sensor group 4b as telemetry information. The communication device 4c may have a function for communicating between multicopters and a satellite communication function. The control device 4a can be connected to a computer on the cloud via the communication device 4c. Some or all of the functions of the companion computer may be performed by the computer on the cloud.

[0024] The battery 52 is a secondary battery that stores power by charging and supplies power to the motors 14 by discharging. The battery 52 and the multiple motors 14 operate to rotate the multiple rotors 2, making it possible to generate a desired thrust.

[0025] Each of the multiple rotors 2 generally has multiple blades with a fixed pitch angle and generates thrust by rotation. The pitch angle may be variable. The multiple rotors 2 do not all need to have the same diameter (propeller diameter); one or more rotors 2 may have a larger diameter than the other rotors 2. The thrust (static thrust) generated by a rotating rotor 2 is generally proportional to the cube of the rotor 2 diameter. Therefore, when rotors 2 with different diameters are included, the rotor 2 with a relatively larger diameter may be referred to as the "main rotor," and the rotor 2 with a relatively smaller diameter may be referred to as the "sub-rotor." Note that, regardless of the diameter, the configuration of the rotary drive device 3 may include a rotor 2 capable of generating a relatively larger thrust and a rotor 2 with a relatively smaller thrust. In this case, the rotor 2 capable of generating a relatively larger thrust may be referred to as the "main rotor," and the rotor 2 with a relatively smaller thrust may be referred to as the "sub-rotor." For example, the rotor 2 that generates a relatively large thrust per rotation may be referred to as the "main rotor," and the rotor 2 that generates a relatively small thrust per rotation may be referred to as the "sub-rotor." In one example, the main rotor may be positioned more inward than the sub-rotors. In other words, each rotor 2 may be positioned so that the distance from the center of the airframe to the rotation axis of each main rotor is shorter than the distance from the center of the airframe to the rotation axis of each sub-rotor.

[0026] In this example, the rotary drive device 3 includes a plurality of motors 14. As mentioned above, the rotary drive device 3 may include an internal combustion engine 7a.

[0027] 1D is a plan view schematically illustrating an example of the basic configuration of a multicopter 10 including a second rotational drive device 3B as the rotational drive device 3. In the example shown in FIG. 1D, an internal combustion engine 7a is supported by the airframe main body 4. In this example, the driving force generated by the internal combustion engine 7a is transmitted to multiple rotors 2 via multiple power transmission systems 23, causing each rotor 2 to rotate. The control device 4a can change the rotational speed of each rotor 2 by controlling each power transmission system 23.

[0028] In a "parallel hybrid drive" in which some of the multiple rotors 2 are rotated by the internal combustion engine 7a and the other rotors 2 are rotated by the motor 14, the internal combustion engine 7a and the battery 52 are supported on the aircraft body 4. At least one rotor 2 of the multiple rotors 2 is connected to the internal combustion engine 7a via the power transmission system 23, and the other rotors 2 are connected to the motor 14.

[0029] In such a parallel hybrid drive, the diameter of one or more rotors 2 rotated by the internal combustion engine 7a may be larger than the diameter of the other rotors 2 rotated by the motor 14. In other words, the internal combustion engine 7a may be used to rotate the main rotor, and the motor 14 may be used to rotate the sub-rotor. In such a case, the main rotor is mainly used to generate thrust, and the sub-rotor is used to generate thrust and for attitude control. The main rotor may also be called a "booster rotor," and the sub-rotor may also be called an "attitude control rotor."

[0030] In the case of parallel hybrid drive, the internal combustion engine 7a is used for both thrust generation and power generation. By selectively transmitting the driving force (torque) generated by the internal combustion engine to one or both of the rotor and the power generator, it is possible to achieve a good balance between thrust generation and power generation.

[0031] Equipping a multicopter with an internal combustion engine 7a and using the internal combustion engine 7a to generate thrust and / or electricity contributes to an increase in payload and flight time. It is desirable to control the attitude of a multicopter by rotating the propellers with a motor, which has better response characteristics than an internal combustion engine. Therefore, in applications where precise control of the attitude of a multicopter is required, it is desirable to employ a parallel hybrid drive or a series hybrid drive to increase the payload and flight time.

[0032] Increased payload and flight time may further expand the applications of multicopters. For example, in the agricultural field, multicopters are currently being used for spraying pesticides or monitoring crop growth conditions. However, by connecting various ground implements (hereinafter, sometimes simply referred to as "implements") to a multicopter, various agricultural tasks can be performed from the air. Agricultural implements are sometimes called "implements." Examples of implements include sprayers that spray pesticides on crops, mowers, seeders, spreaders, rakes, balers, harvesters, plows, harrows, or rotary tillers. Work vehicles such as tractors are not included in the "implements" of this disclosure.

[0033] In the example shown in FIG. 1C , a work implement 200 is coupled to the multicopter 10. The work implement 200 can spray, for example, pesticides or fertilizers on a field or crops within the field. Increasing the payload and flight time allows for a larger and / or more versatile work implement 200. For example, by changing the work implement 200 coupled to the multicopter 10, a variety of ground tasks (agricultural operations) can be performed, including liquid and granular application of pesticides, fertilization, thinning, weeding, transplanting, direct seeding, and harvesting. The work implement 200 may be equipped with a mechanism such as a robotic hand. In this case, a single work implement 200 can perform a variety of ground tasks. Furthermore, if the work implement 200 has a sufficient space to accommodate the materials, the work implement 200 can also transport agricultural materials or harvested products over a wide area.

[0034] 1C , the multicopter 10 includes a power supply device 76. The power supply device 76 is a device that supplies power to the work machine 200 from a drive energy source, such as the battery 52 or the power generation device 8, included in the multicopter 10. Various functions of the work machine 200 can be performed using this power. The work machine 200 includes actuators such as motors that operate using power obtained from the power supply device 76 of the multicopter 10. The work machine 200 preferably includes a battery that stores power.

[0035] 2A is a block diagram showing an example of the basic configuration of a battery-powered multicopter 10. The battery-powered multicopter 10 includes a plurality of rotors 12, a plurality of motors 14 that rotate the rotors 12, a plurality of ESCs (electric speed controllers) 16 each having a motor drive circuit that drives the motors 14, a battery 52 that supplies power to the corresponding motor 14 via each ESC 16, a control device 4a that controls the plurality of ESCs 16 to control attitude while flying, a sensor group 4b, a communication device 4c, and a power supply device 76 electrically connected to the battery 52. ​​The rotor 12 is an example of a rotor 2. The control device 4a, the sensor group 4b, the communication device 4c, and other devices are connected to each other so that they can communicate with each other, for example, via a controller area network (CAN) bus. 2A, for simplicity, the rotor 12, the motor 14, and the ESC 16 are each shown as a single block, but there are actually multiple rotors 12, motors 14, and ESCs 16. This also applies to FIGS. 2B and 2C.

[0036] The control device 4a can receive control commands wirelessly from, for example, a ground station 6 located on the ground via the communication device 4c. The number of ground stations 6 is not limited to one, and they may be distributed across multiple locations. The communication device 4c can also receive control commands wirelessly from a control device of a pilot on the ground. The control device 4a may have a function to automatically or autonomously perform takeoff, flight, obstacle avoidance, and landing operations based on sensor data obtained from the sensor group 4b.

[0037] The control device 4a may be configured to communicate with the work machine 200 connected to the power supply device 76 and to acquire a signal indicating the state of the work machine 200 from the work machine 200. The control device 4a may also provide a signal to the work machine 200 that controls the operation of the work machine 200. Furthermore, the work machine 200 may generate a signal instructing the operation of the multicopter 10 and transmit the signal to the control device 4a. Such communication between the control device 4a and the work machine 200 may be performed wired or wirelessly.

[0038] FIG. 2B is a block diagram showing an example of the basic configuration of a series hybrid drive multicopter 10. Similar to the battery-powered multicopter 10, the series hybrid drive multicopter 10 includes multiple rotors 12, multiple motors 14, multiple ESCs 16, a control device 4a, a sensor group 4b, and a communication device 4c. The illustrated series hybrid drive multicopter 10 further includes an internal combustion engine 7a, a fuel tank 7b for storing fuel for the internal combustion engine 7a, a power generation device 8 driven by the internal combustion engine 7a to generate electric power, a power buffer 9 for temporarily storing the electric power generated by the power generation device 8, and a power supply device 76 electrically connected to the power buffer 9. The power buffer 9 is, for example, a battery such as a secondary battery. The electric power generated by the power generation device 8 is supplied to the motor 14 via the power buffer 9 and the ESC 16. The electric power generated by the power generation device 8 may also be supplied to the work machine 200 via the power supply device 76.

[0039] 2C is a block diagram showing an example of the basic configuration of a parallel hybrid drive multicopter 10. Similar to the series hybrid drive multicopter 10, the parallel hybrid drive multicopter 10 includes multiple rotors 12, multiple motors 14, multiple ESCs 16, a control device 4a, a sensor group 4b, a communication device 4c, an internal combustion engine 7a, a fuel tank 7b, a power generator 8, a power buffer 9, and a power supply device 76. The parallel hybrid drive multicopter 10 further includes a drive train 27 that transmits the driving force of the internal combustion engine 7a, and a rotor 22 that rotates by receiving the driving force of the internal combustion engine 7a from the drive train 27. The rotor 22 is an example of a rotor 2. The number of rotors 22 connected to the drive train 27 and rotating may be one or more.

[0040] In the parallel hybrid drive multicopter 10, the internal combustion engine 7a not only drives the power generation device 8 to generate electricity, but also mechanically transmits energy to the rotor 22 to rotate the rotor 22. On the other hand, in the series hybrid drive multicopter 10, all of the rotors 12 are rotated by the electric power generated by the power generation device 8. For this reason, in the series hybrid drive multicopter 10, if the power generation device 8 is, for example, a fuel cell, the internal combustion engine 7a is not an essential component.

[0041] Next, the configuration and operation of the sensing system according to the embodiment of the present disclosure will be described with reference to FIGS.

[0042] Conventionally, landing judgment for a multicopter has mainly been performed using an imaging device or LiDAR. Landing judgment involves checking whether there are any features or obstacles at the planned landing point and determining whether it is possible to land the multicopter at that point. Various ranging sensors are used to measure the distance between the multicopter and the ground, and LiDAR is one example.

[0043] The sensing system according to the embodiment of the present disclosure is suitable for use in a multicopter, and is capable of detecting obstacles located below the multicopter and determining whether the multicopter has landed. This can contribute to reducing the cost of the sensing system. Furthermore, the system can be made more robust against changes in the brightness of the surrounding environment.

[0044] FIG. 3 is a block diagram showing an example configuration of the sensing system 100. The sensing system 100 according to an embodiment of the present disclosure includes a ranging sensor 81, an altitude sensor 82, an acquisition device 83, and a processing device 84. The ranging sensor 81 and the altitude sensor 82 are included in a sensor group 4b. The devices included in the sensing system 100, such as the control device 4a and the communication device 4c, are communicatively connected to each other via, for example, a CAN bus. The sensing system 100 can be mounted on various types of multicopters as described above. The multicopters are not limited to quad-type multicopters and may be, for example, hexa-type multicopters (hexacopters) having six rotors or octo-type multicopters (octocopters) having eight rotors.

[0045] Examples of the distance measurement sensor 81 include the aforementioned laser range finder or a laser sensor such as a two-dimensional or three-dimensional LiDAR. However, the distance measurement sensor 81 may also be an ultrasonic sensor or a millimeter-wave radar. The distance measurement sensor 81 is used to measure the distance between the multicopter and, for example, an object or the ground located directly below the multicopter. Such distance measurement does not necessarily require LiDAR, which acquires a data set indicating the two-dimensional or three-dimensional distribution of the object or the ground, so a relatively inexpensive laser range finder can be suitably employed. The use of a laser range finder can contribute to reducing the cost of the system.

[0046] The ranging sensor 81 outputs first sensor data indicating the distance between the multicopter and an object or the ground located below the multicopter. More specifically, the ranging sensor 81 outputs first sensor data indicating the distance between the object or the ground located below the multicopter and the ranging sensor 81 mounted on the multicopter. In other words, the first sensor data indicates the height of the multicopter (or the ranging sensor 81) from the object or the ground located below the multicopter.

[0047] "Feature" refers to anything on the ground. Feature primarily refers to any obstacle that prevents a multicopter from landing on the ground. Examples of features include grass, trees, roads, waterways, fields with crops, ditches, rivers, ponds, bridges, forests, mountains, vehicles, rocks, buildings, and railroad tracks.

[0048] Examples of the altitude sensor 82 include a barometric pressure sensor or a GNSS receiver. The altitude sensor 82 outputs second sensor data indicating the altitude of the multicopter. The altitude is the height of a measurement point in the air above mean sea level. The altitude of the multicopter is the height of the multicopter above mean sea level, and more specifically, the height of the altitude sensor 82 mounted on the multicopter above mean sea level. For example, a GNSS receiver may measure the height of the multicopter above mean sea level. In this case, to obtain a more accurate altitude, the measured height must be corrected to the height of the multicopter above the geoid surface (or simply "geoid"), taking into account the effects of gravity. Such correction processing may be performed by the GNSS receiver itself or, for example, by a processing device 84 (described later). Combining a barometric pressure sensor and a GNSS receiver and mutually complementing their data can improve the accuracy of the measured altitude.

[0049] The acquisition device 83 in this embodiment is, for example, the above-mentioned communication device 4c. The acquisition device 83 may further include a storage device that stores the acquired map data.

[0050] The storage device may be, for example, a writable memory (e.g., PROM), a rewritable memory (e.g., flash memory), or a read-only memory (ROM). The storage device may store a program that controls the operation of the processing unit 84. The storage device does not have to be a single recording medium, but may be a collection of multiple recording media. Some of the collection of multiple recording media may be removable memory.

[0051] The acquisition device 83 acquires map data including information on the elevation of each point on the map. For example, when the multicopter is powered on or while the multicopter is flying, the acquisition device 83 may access a cloud server that manages map data via a network to acquire the map data and store it in a storage device. Alternatively, the acquired map data may be stored in advance in a storage device and updated after a predetermined period of time has passed.

[0052] Elevation is the height from the geoid to the ground at each point on the map. The map data may be, for example, GIS data including elevation data conforming to a geographic information system (GIS). An example of a data model for elevation data is a digital elevation model (DEM), which is classified as a raster data format. The DEM represents the height from the geoid to the ground with features such as buildings or trees present on the ground removed.

[0053] An example of the processing device 84 is a processor. The processor is one or more semiconductor integrated circuits, and is also called a central processing unit (CPU) or a microprocessor. The processor sequentially executes computer programs stored in a storage device to realize the processes described below. The term "processor" is broadly interpreted as including a field programmable gate array (FPGA), a graphic processor unit (GPU), an application specific integrated circuit (ASIC), or an application specific standard product (ASSP) equipped with a CPU.

[0054] The processing device 84 is configured to estimate the height of a feature located below the multicopter based on the distance indicated by the first sensor data output from the distance measurement sensor 81, the altitude indicated by the second sensor data output from the altitude sensor 82, and the altitude of the point below the multicopter indicated by the map data. The distance indicated by the first sensor data and / or the altitude indicated by the second sensor data may contain an error. In such a case, for example, the processing device 84 may perform preprocessing to correct the first sensor data and / or the second sensor data before performing processing to estimate the height of the feature.

[0055] The processing device 84 in the embodiment of the present disclosure estimates the height of an object located below the multicopter from the relationship between the total altitude and distance of the point below the multicopter and the altitude. Hereinafter, the altitude, altitude, and distance will be referred to as altitude A, altitude B, and distance C, respectively. As shown in the formula in Equation 1, the processing device 84 estimates the height E of the object located below the multicopter from the value obtained by subtracting the total altitude B and distance C from altitude A. [Equation 1] Height E of object = altitude A - sum of altitude B and distance C

[0056] The processing device 84 determines whether or not a feature exists below the multicopter based on the estimated height E of the feature. Preferably, the processing device 84 determines whether or not a feature exists below the multicopter by comparing the absolute value of the feature height E with a threshold value, as shown in Equation 2. The processing device 84 determines that a feature exists below the multicopter if the estimated height E is equal to or greater than the threshold value. In this case, the processing device 84 may output data indicating the estimated height E. On the other hand, the processing device 84 determines that no feature exists below the multicopter if the estimated height E is less than the threshold value. In this case, the processing device 84 may output data indicating that the estimated height E is zero meters. The output data output by the processing device 84 is stored in, for example, a storage device.

[0057] The threshold value may depend on the measurement accuracy of each of the distance measurement sensor 81 and the altitude sensor 82, and may also differ depending on the type of feature to be detected. The threshold value may be set, for example, in the range of several tens of centimeters to several meters, and preferably within several tens of centimeters. [Equation 2] Absolute value of estimated height E≧threshold value

[0058] FIG. 4 is a schematic diagram showing the multicopter 10 landing on the ground 90a. FIG. 4 also shows the ground 90a and a line 90b corresponding to the geoid or mean sea level. In the example shown in FIG. 4, altitude A is equal to elevation B, and distance C is zero. Therefore, altitude A on the right side of equation (1) is equal to the sum of elevation B and distance C. In this case, the processing device 84 estimates the feature height D to be zero meters and determines that no feature exists below the multicopter. If altitude A contains an error, altitude A will not match elevation B. However, by introducing a threshold as shown in equation (2) and comparing the estimated height E with the threshold, it is possible to reduce the impact of errors on the estimation results.

[0059] Before the multicopter 10 takes off, the processing device 84 may determine whether either the distance measurement sensor 81 or the altitude sensor 82 is malfunctioning, based on the result of estimating the height of the feature. If neither the distance measurement sensor 81 nor the altitude sensor 82 is malfunctioning, as described above, before the multicopter 10 takes off, the altitude A will be equal to the altitude B, and the distance C will be zero. Therefore, the processing device 84 should estimate the height D of the feature to be zero meters. However, if the altitude sensor 82 malfunctions, the altitude indicated by the second sensor data output from the altitude sensor 82 will indicate an abnormal value. As a result, the equation altitude A = altitude B will not hold. Therefore, the absolute value of the estimated height E may be greater than or equal to the threshold. In this case, the processing device 84 can determine that there is a high possibility that the altitude sensor 82 is malfunctioning.

[0060] The sensing system 100 may further include a warning device 90, as illustrated in FIG. 3 . The warning device 90 issues a warning to notify of a failure of the distance measurement sensor 81 or the altitude sensor 82. Examples of the warning device 90 include a buzzer that emits a warning sound to notify of a failure of these sensors, or an optical device such as an LED (Light Emitting Diode) lamp. When the processing device 84 detects a failure of the distance measurement sensor 81 or the altitude sensor 82, it may send a warning command to the warning device 90. The warning device 90 may, for example, emit a warning sound in response to the warning command. For example, the warning sound emitted by the buzzer can prompt the pilot to stop flying the multicopter 10. Alternatively, when the processing device 84 detects a failure of the distance measurement sensor 81 or the altitude sensor 82, it may notify, for example, a control device used by the pilot of the detection of the failure. The control device is a device with a communication function located away from the multicopter. The control device may be, for example, a remote control device used by a pilot to remotely control the multicopter, or a mobile terminal such as a smartphone or tablet computer. The control device may include or be connected to a display device.

[0061] If the estimated height E is equal to or greater than the threshold value even though the operation of the altitude sensor 82 is guaranteed, it is highly likely that the value of the first sensor data output from the distance measurement sensor 81 indicates a value other than zero. In other words, it is highly likely that the distance C indicates an abnormal value. In this case, the processing device 84 can determine that the distance measurement sensor 81 is malfunctioning. In this way, the sensing system 100 according to an embodiment of the present disclosure can be suitably used to detect malfunctions in distance measurement sensors or altitude sensors.

[0062] 5 is a schematic diagram showing the multicopter 10 hovering above a ground surface 90a, which shows the flat ground surface 90a and a line 90b corresponding to the geoid or mean sea level.

[0063] In the example shown in FIG. 5 , no feature is present below the multicopter 10. The distance sensor 81 acquires first sensor data indicating the distance C between the ground 90a and the multicopter 10. The altitude sensor 82 acquires second sensor data indicating the altitude A from the line 90b to the multicopter 10. The processing device 84 reads map data from the storage device and acquires, from the map data, the elevation B of a point below the multicopter 10 indicated by the map data. For example, the processing device 84 acquires, from the map data, the elevation B of a point identified by the longitude and latitude of the multicopter's current location, which is included in the position information acquired by the GNSS receiver. The processing device 84 estimates the height of a feature located below the multicopter 10 based on the elevation A, elevation B, and distance C, and determines whether or not a feature is present below the multicopter 10 based on the estimation result.

[0064] 5, the altitude A is equal to the sum of the elevation B and the distance C. Therefore, the processing device 84 estimates the height E of the feature to be zero meters from the formula 1, and compares the estimated result with a threshold value to determine that no feature exists below the multicopter 10.

[0065] 6 is a diagram showing another example of the multicopter 10 hovering above a ground surface 90a, which is included in the slope of a mountain in a mountainous region, and a line 90b corresponding to the geoid or mean sea level.

[0066] 6 , there is no object below the multicopter 10. The altitude A is equal to the sum of the elevation B and the distance C. Therefore, the processing device 84 estimates the height E of the object to be zero meters from the formula 1, compares the estimated result with a threshold value, and determines that there is no object below the multicopter 10.

[0067] Figure 7 is a schematic diagram showing the multicopter 10 hovering above a tree 92. Figure 7 shows flat ground 90a and a line 90b corresponding to the geoid or mean sea level.

[0068] 7 , a tree 92, which is a feature, is present below the multicopter 10. The distance measurement sensor 81 acquires first sensor data indicating the distance C between the tree 92 and the multicopter 10. The altitude sensor 82 acquires second sensor data indicating the altitude A from the line 90b to the multicopter 10. The processing device 84 reads the map data from the storage device and acquires, from the map data, the altitude B of the point below the multicopter 10 indicated by the map data.

[0069] Because a tree 92 exists below the multicopter 10, the altitude A is greater than the sum of the elevation B and the distance C by the height E of the feature. The processing device 84 estimates the height E of the tree 92 using Equation 1. The estimated height E is greater than or equal to the threshold value, and therefore, the processing device 84 determines, using Equation 2, that a feature exists below the multicopter 10. As described above, the sensor group 4b may include an imaging device. The processing device 84 can identify the type of feature from the image captured by the imaging device by applying, for example, image processing techniques such as segmentation to the image including the feature. However, identifying the type of feature is not essential.

[0070] The sensing system 100 can function as a landing determination system for determining whether or not it is possible to land the multicopter 10. The processing device 84 can make a landing determination for the multicopter 10 based on the result of estimating the height of the feature.

[0071] In an example of landing determination, the processing device 84 estimates the height of a feature located below the multicopter 10 in response to a landing command for the multicopter 10. If the estimated height of the feature is less than a threshold, the processing device 84 determines that no feature exists below the multicopter 10. In this case, the processing device 84 causes the multicopter 10 to perform a landing operation in response to the landing command for the multicopter 10. For example, the processing device 84 may send a landing operation permission notification to the control device 4a. The control device 4a begins controlling the landing operation in response to the permission notification. The landing command for the multicopter 10 is transmitted, for example, from a control device of a pilot on the ground and received by the communication device 4c. On the other hand, if the estimated height of the feature is equal to or greater than a threshold, the processing device 84, for example, ignores the landing command for the multicopter 10 and causes the control device 4a to continue flying or hovering. In this case, for example, the processing device 84 may notify a control device remotely used by the pilot via the communication device 4c that a feature exists at the planned landing point and landing is not permitted. Alternatively, the processing device 84 may send a warning command to the warning device 90. For example, in response to the warning command, the warning device 90 emits a warning sound around the multicopter 10.

[0072] In this way, by having the processing device 84 automatically make the landing determination, it is not necessary for the pilot to determine whether landing is possible, thereby improving the safety of the landing operation of the multicopter 10. When monitoring video from an imaging device as in the past, the video may be distorted or cut off due to communication failures, etc., but by automatically determining whether landing has occurred, there is no impact from such communication failures.

[0073] As described above, a work machine can be connected to the multicopter 10, and the work machine or a payload can be suspended, for example, via a cable, wire, wire rope, or rope. In this specification, cables, wires, wire ropes, and ropes are collectively referred to as "work ropes." The multicopter 10 can fly with the work machine or a payload suspended by the work rope. During such flight, it may be necessary to monitor whether the work machine or the payload comes into contact with an obstacle.

[0074] Fig. 8 is a diagram showing a multicopter 10 with a payload 93 suspended by a work rope 94 hovering above a tree 92. Fig. 8 also shows flat ground 90a and a line 90b corresponding to the geoid or mean sea level. However, the illustrated example is not limiting, and a work machine can be suspended from the multicopter 10 by using the work rope 94.

[0075] When the multicopter 10 is flying with the payload 93 suspended, the processing device 84 can determine whether the payload 93 will contact the ground 90a or the tree 92 based on the distance C. For example, information about the payload 93, including the distance F between the multicopter 10 and the payload 93, is pre-stored in the storage device. The processing device 84 reads the information about the distance F (or the length of the work rope 94) from the storage device and compares the distance C with the distance F to determine whether the payload 93 will contact the ground 90a or the tree 92. Specifically, if the distance C is greater than the distance F, the processing device 84 determines that the payload 93 will not contact the ground 90a or the tree 92, and if the distance C is equal to or less than the distance F, the processing device 84 determines that the payload 93 will contact the ground 90a or the tree 92. In this way, the sensing system 100 can be used to determine whether the work machine or the payload will not contact the ground or a feature.

[0076] The multicopter 10 may include a winding mechanism. The winding mechanism has an actuator that performs an operation to wind or unwind the work rope 94. An example of the actuator is an electric motor. The winding mechanism allows the length of the work rope 94 to be adjusted. The length of the work rope 94 may be variable while the multicopter 10 is flying. Therefore, while the winding mechanism is adjusting the length of the work rope 94, the processing device 84 preferably determines whether the payload 93 will contact the ground 90a or the tree 92 multiple times. For example, while the work rope 94 is being unwound, the processing device 84 preferably determines whether the payload 93 will contact the ground 90a or the tree 92 each time the work rope 94 is unwound, taking into account the length of the unwound work rope 94. As the work rope 94 is unwound, the payload 93 approaches the ground 90a or the tree 92. When the load 93 approaches the ground 90a or the tree 92 to a predetermined distance, the processing device 84 may send a warning command to the warning device 90, for example, to generate a warning sound around the multicopter 10.

[0077] The distance measurement sensor 81 included in the sensing system 100 is not limited to the above-mentioned example, and may be mounted on a work machine or a load. When the work machine or a load is suspended below the multicopter, distance measurement by the distance measurement sensor 81 may be hindered. By mounting the distance measurement sensor 81 on the work machine or the load, such a situation can be avoided.

[0078] As another example, if the ground surface 90a is the ground surface of a field where crops are planted, the processing device 84 may determine the growth state of the crops based on the estimated height of the crops, which are features. Thus, the sensing system 100 is useful not only for landing determination, but also for surveying applications such as generating a crop height map, or for growth management applications such as managing the growth rate of crops using a cloud server. Furthermore, as described above, the type of feature can be identified by combining the sensing system 100 with an imaging device.

[0079] According to an embodiment of the present disclosure, the sensing system can estimate the height of a feature located below the multicopter based on distance, altitude, and elevation instead of using images acquired by an imaging device, and can further determine whether a feature exists below the multicopter from the estimation result. Such a determination result can be used to determine whether the multicopter is about to land. This allows for fully automatic landing determination. For example, even if a dynamic obstacle, such as agricultural machinery, temporarily exists in a specific location, such a location can be excluded from the landing site, allowing the multicopter to land in a safe location.

[0080] The number of times the sensing system 100 determines whether the multicopter can land may be one (single) or multiple (continuous). If the obstacle is a moving object such as a vehicle, even if the initial determination indicates that landing is not possible, the obstacle may leave the area over time. Therefore, particularly in consideration of determining whether a dynamic obstacle can be landed, it is preferable to periodically perform the landing determination multiple times, for example.

[0081] As described above, the control device 4a may include, for example, a flight control device such as a flight controller and a host computer (companion computer). The companion computer may perform various processes required for landing determination, and may issue commands related to landing determination based on the results of those processes to the flight controller. Furthermore, some or all of the functions of the electrical components such as the control device 4a mounted on the multicopter 10 may be implemented by one or more servers (computers) 500 or terminal devices (including portable and fixed types) 600 connected to the communication device 4c of the multicopter 10 via a communication network N, as shown in FIG. 9 . An agricultural machine 700 such as a tractor may be connected to such a communication network N, and communication may be performed between the multicopter 10 and the agricultural machine 700. Some of the data used in the processing of the control device 4a and control signals for the multicopter 10 may be transmitted from the agricultural machine 700 to the multicopter 10 via the communication network N.

[0082] A system providing various functions in the embodiments can also be retrofitted to a multicopter that does not have those functions. Such a system can be manufactured and sold independently of the multicopter. A computer program used in such a system can also be manufactured and sold independently of the multicopter. The computer program can be provided, for example, by being stored in a computer-readable non-transitory storage medium. The computer program can also be provided by downloading via a telecommunications line (e.g., the Internet).

[0083] This specification discloses the solutions described in the following items.

[0084] [Item 1] A sensing system used for an unmanned aerial vehicle, comprising: a ranging sensor that outputs first sensor data indicating the distance between the unmanned aerial vehicle and a feature or ground located below the unmanned aerial vehicle; an altitude sensor that outputs second sensor data indicating the altitude of the unmanned aerial vehicle; an acquisition device that acquires map data including information on the elevation of each point on a map; and a processing device that estimates the height of the feature located below the unmanned aerial vehicle based on the distance indicated by the first sensor data, the altitude indicated by the second sensor data, and the elevation of the point below the unmanned aerial vehicle indicated by the map data.

[0085] [Item 2] The processing device estimates the height of the feature located below the unmanned aerial vehicle from the relationship between the altitude and the sum of the altitude and the distance of the point below the unmanned aerial vehicle.

[0086] [Item 3] The processing device estimates the height of the feature located below the unmanned aerial vehicle from the value obtained by subtracting the sum from the altitude, and determines that the feature is located below the unmanned aerial vehicle if the estimated height is equal to or greater than a threshold value. This is the sensing system described in Item 2.

[0087] [Item 4] The sensing system according to item 1 or 2, wherein the processing device determines whether the unmanned aerial vehicle should land based on the result of estimating the height of the feature.

[0088] [Item 5] The processing device, in response to a landing command for the unmanned aerial vehicle, estimates the height of the feature located below the unmanned aerial vehicle, and if the estimated height is less than a threshold, causes the unmanned aerial vehicle to perform a landing operation, is the sensing system described in Item 4.

[0089] [Item 6] A sensing system according to any one of items 1 to 5, wherein before the unmanned aerial vehicle takes off, the processing device determines whether either the ranging sensor or the altitude sensor is faulty based on the result of estimating the height of the feature.

[0090] [Item 7] The sensing system described in Item 1, wherein when the unmanned aerial vehicle is flying with a work machine or a load suspended from it, the processing device determines whether the work machine or the load will come into contact with the feature or ground based on the distance.

[0091] [Item 8] The sensing system according to Item 7, wherein the distance measurement sensor is mounted on the work machine or the load.

[0092] [Item 9] The unmanned aerial vehicle is equipped with a winding mechanism that adjusts the length of the work cable that suspends the work machine or load, and while adjusting the length of the work rope, the processing device determines multiple times whether the work machine or load will come into contact with the feature or ground, in the sensing system described in Item 7 or 8.

[0093] [Item 10] The sensing system according to Item 1, wherein the ground is the ground of a field in which crops are planted, and the processing device determines a growth state of the crops based on the result of estimating the height of the feature.

[0094] [Item 11] An unmanned aerial vehicle comprising: a plurality of rotors; and a sensing system according to any one of items 1 to 10.

[0095] The unmanned aerial vehicle disclosed herein can be widely used not only for aerial photography, surveying, logistics, and pesticide spraying, but also for ground work related to agricultural work, transporting harvested products and agricultural materials, and the like.

[0096] 2: Rotor (propeller), 3: Rotation drive device, 4: Airframe body, 4a: Control device, 4b: Sensor group, 4c: Communication device, 5: Airframe frame, 6: Ground station, 7a: Internal combustion engine, 7b: Fuel tank, 8: Power generation device, 9: Power buffer, 10: Multicopter, 12: Rotor, 14: Motor, 16: ESC, 70: First controller, 76: Power supply device, 81: Distance measurement sensor, 82: Altitude sensor, 83: Storage device, 84: Processing device, 90: Warning device, 100: Sensing system, 200: Work machine

Claims

1. 1. A sensing system for use with an unmanned aerial vehicle, comprising: a ranging sensor that outputs first sensor data indicating a distance between the unmanned aerial vehicle and a feature or ground located below the unmanned aerial vehicle; an altitude sensor that outputs second sensor data indicating the altitude of the unmanned aerial vehicle; an acquisition device for acquiring map data including information on the elevation of each point on a map; a processing device that estimates the height of the feature located below the unmanned aerial vehicle based on the distance indicated by the first sensor data, the altitude indicated by the second sensor data, and the elevation of the point below the unmanned aerial vehicle indicated by the map data; and A sensing system comprising:

2. The sensing system of claim 1, wherein the processing device estimates the height of the feature located below the unmanned aerial vehicle based on the relationship between the altitude and the sum of the altitude and the distance of the point below the unmanned aerial vehicle.

3. The processing device includes: estimating the height of the feature located below the unmanned aerial vehicle from the value obtained by subtracting the sum from the altitude; The sensing system according to claim 2 , wherein if the estimated height is equal to or greater than a threshold, it is determined that the feature is present below the unmanned aerial vehicle.

4. The sensing system according to claim 1 or 2, wherein the processing device determines whether the unmanned aerial vehicle should land based on the result of estimating the height of the feature.

5. The processing device includes: In response to a landing command for the unmanned aerial vehicle, estimate the height of the feature located below the unmanned aerial vehicle; The sensing system of claim 4 , further comprising: causing the unmanned aerial vehicle to perform a landing operation if the estimated height is less than a threshold.

6. A sensing system as described in any one of claims 1 to 3, wherein before the unmanned aerial vehicle takes off, the processing device determines whether either the ranging sensor or the altitude sensor is faulty depending on the result of estimating the height of the feature.

7. The sensing system of claim 1, wherein when the unmanned aerial vehicle is flying with a work machine or a load suspended from it, the processing device determines based on the distance whether the work machine or the load will come into contact with the feature or ground.

8. The sensing system according to claim 7 , wherein the distance measuring sensor is mounted on the work machine or the load.

9. the unmanned aerial vehicle is provided with a winding mechanism that adjusts the length of a work cable that suspends the work machine or a load, 9. The sensing system according to claim 7, wherein the processing device determines whether the work machine or the load comes into contact with the feature or the ground a plurality of times while adjusting the length of the work rope.

10. The ground is the ground of a field where crops are planted, The sensing system according to claim 1 , wherein the processing device determines the growth state of the crops according to the result of estimating the height of the feature.

11. A plurality of rotors; A sensing system according to any one of claims 1 to 3; An unmanned aerial vehicle comprising: