Moving body, information processing device, information processing method, and program product

Abstract

[Object] Provided are a moving body, an information processing device, an information processing method, and a program that can improve the accuracy of determining whether a drone lands on a landing surface. [Solution] The moving body of the present technology includes a control unit. The control unit calculates a temporal change of the landing surface based on images obtained by imaging the landing surface of the moving body in a time series by an imaging unit of the moving body in a hovering state, and determines whether to land the moving body on the landing surface based on the calculated temporal change.

Description

Technical Field

[0001] The present technology relates to a mobile body, an information processing device, an information processing method, and a program. Background Art

[0002] Recently, for example, an unmanned aerial vehicle equipped with a camera has been proposed for use in aerial photography of landscapes and the like. In such an unmanned aerial vehicle, when performing a landing operation, a technique for determining the landing surface environment from an image of the landing surface is employed (for example, Patent Document 1).

[0003] Citation List

[0004] Patent Document

[0005] Patent Document 1: Japanese Unexamined Patent Application Publication (Translation of PCT Application) No. 2017-530043 Summary of the Invention

[0006] Technical Problem

[0007] However, in the technique described in Patent Document 1, it is necessary to mainly determine whether to land the unmanned aerial vehicle from the image information of the landing surface, and there are cases where the determination is incorrect.

[0008] In this regard, the present technology provides a mobile body, an information processing device, an information processing method, and a program that can improve the accuracy of determining whether to land a mobile body on a landing surface.

[0009] Solution to the Problem

[0010] To solve the above problems, a mobile body according to an embodiment of the present technology includes a control unit.

[0011] The control unit calculates the temporal change of the landing surface based on images obtained by imaging the landing surface of the mobile body in time series by an imaging unit of the mobile body in a hovering state, and determines whether to land the mobile body on the landing surface based on the calculated temporal change.

[0012] The mobile body may further include a detection unit that detects the height of the mobile body from the landing surface, wherein the control unit determines whether to land the mobile body on the landing surface based on the height and the temporal change.

[0013] The control unit may perform image processing on the images to remove objects different from the landing surface, and calculate the temporal change based on the images on which the image processing has been performed.

[0014] When the height of the mobile body is greater than a predetermined threshold, the control unit causes the mobile body to approach the landing surface.

[0015] The threshold is the height at which the landing surface changes over time due to the wind generated by the moving body.

[0016] The control unit can determine the environment of the landing surface based on an image, and determine whether to land the moving body on the landing surface based on the determined environment, time change, and height.

[0017] When the time change exceeds the threshold corresponding to the determined environment, the control unit lands the moving body on the landing surface.

[0018] When the determined environment is water surface and the time change is equal to or less than the threshold corresponding to the water surface, the control unit lands the moving body on the landing surface.

[0019] The moving body can be a flying body.

[0020] To solve the above problems, an information processing apparatus according to an embodiment of the present technology includes a control unit.

[0021] The control unit calculates the time change of the landing surface based on an image obtained by imaging the landing surface of the moving body in time series by an imaging unit of the moving body in a hovering state, determines whether to land the moving body on the landing surface based on the calculated time change, and outputs the determination result to the moving body.

[0022] The information processing apparatus can be a server.

[0023] To solve the above problems, an information processing method according to an embodiment of the present technology includes: calculating the time change of the landing surface based on an image obtained by imaging the landing surface of the moving body in time series by an imaging unit of the moving body in a hovering state, and determining whether to land the moving body on the landing surface based on the calculated time change.

[0024] To solve the above problems, a program according to an embodiment of the present technology causes the moving body or the information processing apparatus to perform the following steps: calculating the time change of the landing surface based on an image obtained by imaging the landing surface of the moving body in time series by an imaging unit of the moving body in a hovering state, and determining whether to land the moving body on the landing surface based on the calculated time change. Description of the Drawings

[0025] Figure 1 Figure 1 is a schematic diagram showing a configuration example of an information processing system according to a first embodiment of the present technology.

[0026] Figure 2 Figure 2 is a block diagram showing a configuration example of the information processing system.

[0027] ​​​​​Figure 3 Figure 3 is a block diagram showing an example of the hardware configuration of an information processing apparatus and an unmanned aerial vehicle of an information processing system.

[0028] Figure 4 Figure 4 is a flowchart showing a typical operation flow of the information processing system.

[0029] Figure 5 Figure 5 is a graph showing an example of a determination index regarding whether to land an unmanned aerial vehicle on a landing surface.

[0030] Figure 6 Figure 6 is a graph showing an example of a determination index regarding whether to land an unmanned aerial vehicle on a landing surface.

[0031] Figure 7 Figure 7 is a block diagram showing an example of the configuration of an unmanned aerial vehicle according to a second embodiment of the present technology. DETAILED DESCRIPTION

[0032] Hereinafter, embodiments of the present technology will be described with reference to the accompanying drawings.

[0033] <First Embodiment>

[0034] [Configuration of Information Processing System]

[0035] Figure 1 is a schematic diagram showing an example of the configuration of an information processing system 1 according to the first embodiment. Figure 2 is a block diagram showing an example of the configuration of the information processing system 1. The information processing system 1 includes an unmanned aerial vehicle 10 and a user interface (UI) 30, as Figure 2 shown.

[0036] The unmanned aerial vehicle 10 and the UI 30 are connected to each other so as to be able to communicate with each other via the network N. The network N may be the Internet, a mobile communication network, a local area network, etc., and may be a network combining multiple types of networks.

[0037] (Unmanned Aerial Vehicle)

[0038] As Figure 2 shown, the unmanned aerial vehicle 10 includes a control unit 11, a storage unit 12, a camera 13, a height sensor 14, a flight controller 15, and a motor 16. The unmanned aerial vehicle 10 is an example of the "mobile body" in the claims.

[0039] ​​​​​​​​​The control unit 11 controls the overall operation or part of the unmanned aerial vehicle 10 according to the program stored in the storage unit 12. Functionally, the control unit 11 includes a landing possibility and approach judgment unit 116, an environment discrimination unit 117, a wind vibration characteristic calculation unit 118, and an action generation unit 119.

[0040] The landing possibility and approach judgment unit 116 determines whether the unmanned aerial vehicle 10 can land on the ground at the current position without damage. Here, damage means that the unmanned aerial vehicle 10 overturns and is damaged from that point due to poor footing at the landing point after landing. The same applies to the following descriptions.

[0041] The environment discrimination unit 117 refers to the environment database 122 to discriminate the ground environment at the current position of the unmanned aerial vehicle 10 from the captured image of the landing surface for the unmanned aerial vehicle 10 captured by the camera 13.

[0042] The wind vibration characteristic calculation unit 118 performs predetermined image processing on the images of the landing surface captured by the camera 13 in time series, and calculates the time change of the landing surface as the wind vibration characteristic value. This time change is, for example, the time change of the shape of the landing surface imaged by the camera 13. Here, the "wind vibration characteristic value" is a numerical value indicating how much the physical shape changes over time when the landing surface is subjected to natural wind (trade wind, westerly wind, polar easterly wind, monsoon, local wind, etc.) or wind from the unmanned aerial vehicle 10. The same applies to the following descriptions.

[0043] The action generation unit 119 controls the UI 30 and the flight controller 15 based on instructions from the landing possibility and approach judgment unit 116. For example, when receiving an instruction to land the unmanned aerial vehicle 10 from the UI 30, the action generation unit 119 outputs this instruction to the flight controller 15.

[0044] As Figure 2 shown, the storage unit 22 functionally includes a landing possibility database 121 and an environment database 122.

[0045] The landing possibility database 121 stores determination criteria for whether to land the unmanned aerial vehicle 10 on the ground or whether to approach the ground with the unmanned aerial vehicle 10 ( Figure 5 and Figure 6 ). Such determination criteria correspond to the wind vibration characteristic value and altitude of the unmanned aerial vehicle 10. The landing possibility and approach judgment unit 116 determines the operation to be taken by the unmanned aerial vehicle 10 based on whether these values exceed a predetermined threshold.

[0046] The environmental database 222 stores data (hereinafter referred to as reference data) in which images of the landing surface previously captured by the camera 13 are associated with information about the environment of the landing surface (e.g., sandy, grassy, rocky, water surface, etc.) that appears in the images.

[0047] The camera 13 is a camera that captures images in the landing surface direction (vertically downward direction) relative to the unmanned aerial vehicle 10. The camera 13 advantageously has a resolution that can discriminate objects on the landing surface. The camera 13 outputs images that are continuous in time series rather than a single image to the environment discrimination unit 117 and the wind vibration characteristic calculation unit 118 during operation. The camera 13 is an example of the "imaging unit" in the claims.

[0048] The altitude sensor 14 is a sensor that detects the altitude of the unmanned aerial vehicle 10 from the landing surface. The altitude sensor 14 can be a distance sensor that uses infrared or ultrasonic waves to detect the altitude from the landing surface, or it can be an atmospheric pressure sensor. Alternatively, the unmanned aerial vehicle 10 can be a sensor that combines a distance sensor and an atmospheric pressure sensor.

[0049] The flight controller 15 is a board for controlling the attitude of the unmanned aerial vehicle 10. The flight controller 15 includes a microcontroller and an inertial measurement unit (IMU). The flight controller 15 performs arithmetic processing in the microcontroller based on data from the control unit 11 and controls the unmanned aerial vehicle 10 to have an appropriate attitude based on commands obtained from the UI 30.

[0050] The motor 16 controls the rotational speed of the propellers of the unmanned aerial vehicle 10. The motor 16 can be, for example, a brushed motor or a brushless motor, and any type of motor can be used.

[0051] The unmanned aerial vehicle 10 can be of the fixed-wing aircraft type or the rotary-wing aircraft type. When the unmanned aerial vehicle 10 is of the rotary-wing aircraft type, the unmanned aerial vehicle 10 can be, for example, a triaxial aircraft, a quadcopter, a hexacopter, or an octocopter, and its type is not limited.

[0052] (UI)

[0053] The UI 30 is an interface for exchanging information with the unmanned aerial vehicle 10. The UI 30 is usually a graphical user interface (GUI) such as a smartphone, but is not limited thereto, and can be, for example, a web user interface (WUI), a character user interface (CUI), a tactile interface, a touch interface, a batch interface, a perceptual user interface (PUI), a reflective interface, a tangible user interface (TUI), a text user interface, a voice user interface, or a zoom interface.

[0054] (Hardware Configuration)

[0055] Figure 3 is a block diagram showing an example of the hardware configuration of the unmanned aerial vehicle 10 and the information processing device 20. The unmanned aerial vehicle 10 and the information processing device 20 to be described later can be implemented by the information processing device 100.

[0056] The information processing device 100 includes a central processing unit (CPU) 101, a read-only memory (ROM) 102, and a random access memory (RAM) 103. The control units 11 and 21 can be the CPU 101.

[0057] In addition, the information processing device 100 may include a host bus 104, a bridge 105, an external bus 106, an interface 107, an input device 108, an output device 109, a storage device 110, a drive 111, a connection port 112, and a communication device 113.

[0058] Furthermore, the information processing device 100 may include an imaging device 114 and a sensor 115. The information processing device 100 may include a processing circuit such as a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a graphics processing unit (GPU) as an alternative to or in addition to the CPU 101.

[0059] The CPU 101 serves as an arithmetic processing unit and a control unit, and controls the overall operation or a part thereof of the information processing device 100 according to various programs recorded in the ROM 102, the RAM 103, the storage device 110, or recorded on the removable recording medium 40. The storage units 12 and 22 can be the ROM 102, the RAM 103, the storage device 110, or the removable recording medium 40.

[0060] The ROM 102 stores programs, calculation parameters, etc. to be used by the CPU 101. The RAM 103 temporarily stores programs to be used in the execution of the CPU 101, parameters appropriately changed during the execution of the program, etc.

[0061] The CPU 101, the ROM 102, and the RAM 103 are interconnected via a host bus 104 including an internal bus such as a CPU bus. In addition, the host bus 104 is connected to an external bus 106 such as a peripheral component interconnect / interface (PCI) bus via a bridge 105.

[0062] The input device 108 is a device for user operation such as a mouse, keyboard, touch panel, button, switch, or joystick. The input device 108 can be, for example, a remote control device using infrared rays or other radio waves, or can be an external connection device 50 such as a mobile phone corresponding to the operation of the information processing apparatus 100.

[0063] The input device 108 includes an input control circuit for generating an input signal based on the information input by the user and outputting the generated input signal to the CPU 101. By operating the input device 108, the user inputs various types of data or indicates a processing operation to the information processing apparatus 100.

[0064] The output device 109 is configured by a device capable of notifying the acquired information to the user by using senses such as vision, hearing, and touch. For example, the output device 109 can be a display device such as a liquid crystal display (LCD) or an organic electroluminescence (EL) display, a sound output device such as a speaker or headphones, or a vibrator.

[0065] The output device 109 outputs the result obtained by the processing of the information processing apparatus 100 as a video such as text or an image, a sound such as voice or audio, or a vibration.

[0066] The storage device 110 is an example of a data storage device configured as a storage unit of the information processing apparatus 100. For example, the storage device 110 is configured by a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or a magneto-optical storage device. The storage device 110 stores, for example, a program to be executed by the CPU 101, various types of data, and various types of data acquired from the outside.

[0067] The drive 111 is a reader / writer for a removable recording medium 40 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and is built in the information processing apparatus 100 or externally attached to the information processing apparatus 900.

[0068] The drive 111 reads the information recorded on the removable recording medium 40 mounted thereon and outputs the read information to the RAM 103. In addition, the drive 111 writes a record to the removable recording medium 40 mounted thereon.

[0069] The connection port 112 is a port for connecting a device to the information processing apparatus 100. The connection port 112 can be, for example, a universal serial bus (USB) port, an IEEE1394 port, or a small computer system interface (SCSI) port.

[0070] In addition, the connection port 112 can be an RS-232C port, an optical audio terminal, a High-Definition Multimedia Interface (HDMI) (registered trademark) port, etc. The externally connected device 50 is connected to the connection port 112. Therefore, various types of data can be exchanged between the information processing apparatus 100 and the externally connected device 50.

[0071] The communication device 113 is, for example, a communication interface including communication devices for connecting to a network N or the like. The communication device 113 can be, for example, a communication card for a local area network (LAN), Bluetooth (registered trademark), Wi-Fi, or wireless USB (WUSB).

[0072] In addition, the communication device 113 can be a router for optical communication, a router for asymmetric digital subscriber line (ADSL), or a modem for various types of communication. The communication device 113 transmits signals and the like to the Internet or other communication devices and receives them from the Internet or other communication devices by using a predetermined protocol such as TCP / IP.

[0073] In addition, the network N connected to the communication device 113 is a network connected in a wired or wireless manner, and can include, for example, the Internet, a home LAN, infrared communication, radio wave communication, and satellite communication.

[0074] For example, the imaging device 114 is a device that uses an image pickup device such as a complementary metal oxide semiconductor (CMOS) or a charge coupled device (CCD), and various components such as a lens for controlling the formation of a subject image on the image pickup device to capture an image of the real space and then generate a captured image. The imaging device 114 can capture a still image or can capture a moving image. The camera 13 corresponds to the imaging device 114.

[0075] For example, the sensor 115 is various sensors such as an acceleration sensor, an angular velocity sensor, a geomagnetic sensor, an illuminance sensor, a temperature sensor, an atmospheric pressure sensor, and a sound sensor (microphone). The sensor 115 acquires information about the state of the information processing apparatus 100 itself, such as the housing posture of the information processing apparatus 100, and information about the surrounding environment of the information processing apparatus 100, such as the brightness and noise around the information processing apparatus 100.

[0076] In addition, the sensor 115 can further include a Global Positioning System (GPS) receiver that receives GPS signals to measure the latitude, longitude, and altitude of the device. The altitude sensor 14 corresponds to the sensor 115.

[0077] The configuration example of the information processing system 1 has been described above. Each of the above-described constituent components can be configured using general components, or can be configured by components dedicated to the functions of each component. Such a configuration can be changed as appropriate according to the technical level at the time of implementation.

[0078] [Operation of Unmanned Aerial Vehicle]

[0079] Figure 4 is a flowchart showing a typical operation process of the information processing system 1. Hereinafter, the operation of the unmanned aerial vehicle 10 will be described as appropriate with reference to Figure 4 to describe the operation of the unmanned aerial vehicle 10.

[0080] (Step S101: Landing Command)

[0081] First, the user operating the unmanned aerial vehicle 10 outputs, via the UI 30, an instruction to land the unmanned aerial vehicle 10 in a hovering state on the landing surface. In response to the instruction from the UI 30, the motion generation unit 119 outputs an instruction to the landing possibility and approach determination unit 116 for determining whether to land the unmanned aerial vehicle 10 on the landing surface and further for determining whether to approach the landing surface with the unmanned aerial vehicle 10.

[0082] (Step S102: Discriminate Environment)

[0083] In response to the instruction from the motion generation unit 119, the landing possibility and approach determination unit 116 outputs an instruction to the environment discrimination unit 117 to discriminate the environment of the landing surface imaged by the camera 13.

[0084] The environment discrimination unit 212 discriminates the environment of the landing surface from the images continuously captured by the camera 13 at a predetermined frame rate by referring to the reference data stored in the environment database 122, and outputs the discrimination result to the landing possibility and approach determination unit 116.

[0085] (Step S103: Calculate Aeroelastic Characteristics Value)

[0086] The aeroelastic characteristics calculation unit 118 performs image processing on the images continuously captured by the camera 13 to remove objects in the images that are different from the landing surface (for example, moving bodies such as people, insects, or cars). Specifically, the aeroelastic characteristics calculation unit 118 identifies and removes moving bodies in the images in which the landing surface is imaged by techniques such as object detection or semantic segmentation using machine learning, for example.

[0087] Next, the wind vibration characteristic calculation unit 118 calculates the wind vibration characteristic value of the landing surface in the image that has undergone image processing. At this time, for example, the wind vibration characteristic calculation unit 118 detects the feature points of the landing surface from the images of the landing surface continuously captured by the camera 13, and calculates the temporal change of the feature points between frames as the wind vibration characteristic value. Such temporal change is, for example, the moving distance between the frames of the detected feature points.

[0088] The wind vibration characteristic calculation unit 118 detects feature points for each of the plurality of images in which the landing surface is imaged at a predetermined frame rate. The feature points are, for example, points at the boundaries between different regions where at least one of the values indicating luminance, color, or distance is equal to or greater than a predetermined value, and correspond to edges (points where the luminance suddenly changes), corners (black points of line segments or sudden transitions of edges), etc.

[0089] The wind vibration characteristic calculation unit 118 detects feature points from the images of the landing surface captured by image processing according to a predetermined algorithm such as Scale-Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), Rotation-Invariant Fast Feature (RIFF), Binary Robust Independent Elementary Features (BREIF), Binary Robust Invariant Scalable Keypoints (BRISK), Oriented FAST and Rotated BRIEF (ORB), or Compact and Real-Time Descriptor (CARD).

[0090] The wind vibration characteristic calculation unit 118 outputs the calculated wind vibration characteristic value and the sensor data acquired from the altitude sensor 14 to the landing possibility and approach determination unit 116.

[0091] (Step S104: Is the altitude above the threshold?)

[0092] Figure 5 and Figure 6 are both graphs showing examples of determination indexes regarding whether to land the unmanned aerial vehicle 10 on the landing surface. The landing possibility and approach determination unit 116 reads the determination index corresponding to the environment of the landing surface imaged by the camera 13 based on the discrimination result of the environment discrimination unit 117 ( Figure 5 and Figure 6 ), and determines whether the wind vibration characteristic value acquired from the wind vibration characteristic calculation unit 118 and the sensor data (the altitude of the unmanned aerial vehicle 10 from the landing surface) acquired from the altitude sensor 14 exceed a predetermined threshold with reference to the determination index.

[0093] Here, if the altitude of the unmanned aerial vehicle 10 exceeds the thresholds L1 and L3 corresponding to the environment of the landing surface determined in the previous step 102 (Yes in step S104), it is difficult for the camera 13 to satisfactorily image the landing surface. In addition, the wind pressure of the propellers of the unmanned aerial vehicle 10 cannot be sufficiently transmitted to the landing surface. Therefore, the landing possibility and approach determination unit 116 outputs an instruction to the motion generation unit 119 to cause the unmanned aerial vehicle 10 to approach the landing surface to a predetermined altitude (step S105). In response to the instruction from the landing possibility and approach determination unit 116, the motion generation unit 214 controls the rotational speed of the motor 16 via the flight controller 15. Thus, the unmanned aerial vehicle 10 approaches the landing surface to the preset altitude and hovers while maintaining an altitude equal to or less than the thresholds L1 and L3. Next, the control unit 11 executes steps S101 to S104 again while the unmanned aerial vehicle 10 is hovering at an altitude equal to or less than the thresholds L1 and L3.

[0094] On the other hand, if the altitude of the unmanned aerial vehicle 10 is equal to or less than the thresholds L1 and L3 (No in step S104), the control unit 11 executes step S106 to be described later.

[0095] (Step S106: Is landing possible?)

[0096] The landing possibility and approach determination unit 116 determines whether to land the unmanned aerial vehicle 10 on the landing surface based on the environment of the landing surface determined by the environment determination unit 117 and the wind vibration characteristic value calculated by the wind vibration characteristic calculation unit 118. Hereinafter, some determination examples corresponding to the respective environments of the landing surface will be described.

[0097] - Determination example 1: If the environment of the landing surface is rock or sand

[0098] If the environment of the landing surface determined in the previous step S102 is a rock ground and the wind vibration characteristic value calculated in the previous step S103 is equal to or less than the threshold L2 corresponding to the rock ground (No in step S106), the landing possibility and approach determination unit 116 determines that the environment of the landing surface is an environment in which even if the landing surface receives sufficient wind pressure from the unmanned aerial vehicle 10, the shape of the landing surface does not change much.

[0099] Therefore, if the unmanned aerial vehicle 10 lands on the landing surface in such an environment, the unmanned aerial vehicle 10 may fall and be damaged because the landing surface does not fully absorb the landing impact. Therefore, the landing possibility and approach determination unit 116 outputs an instruction not to land the unmanned aerial vehicle 10 on the landing surface to the motion generation unit 119.

[0100] In response to an instruction from the landing possibility and proximity determination unit 116, the action generation unit 10 notifies the UI 30 of the fact that the unmanned aerial vehicle 10 cannot land on the landing surface, and controls the rotational speed of the motor 16 via the flight controller 15. In this way, the unmanned aerial vehicle 10 is maintained in a hover at a height equal to or less than the threshold L1 (step S108).

[0101] On the other hand, if the environment of the landing surface determined in the previous step S102 is a rocky ground and the wind vibration characteristic value calculated in the previous step S103 exceeds the threshold L2 corresponding to the rocky ground ("Yes" in step S106), the landing possibility and proximity determination unit 116 determines that the environment of the landing surface is one in which the landing impact is absorbed by the landing surface when the unmanned aerial vehicle 10 lands on the landing surface. Therefore, the landing possibility and proximity determination unit 116 outputs an instruction to cause the unmanned aerial vehicle 10 to land on the landing surface to the action generation unit 119 (step S107).

[0102] In response to an instruction from the landing possibility and proximity determination unit 116, the action generation unit 119 notifies the UI 30 that the unmanned aerial vehicle 10 has landed on the landing surface, and controls the rotational speed of the motor 16 via the flight controller 15. As a result, the unmanned aerial vehicle 10 lands on the landing surface.

[0103] - Determination example 2: If the environment of the landing surface is grassland or a place where metal is stacked on the landing surface

[0104] If the environment of the landing surface determined in the previous step S102 is one in which metal such as wire rods is stacked on the landing surface, and the wind vibration characteristic value calculated in the previous step S103 is equal to or less than the threshold L2 corresponding to such an environment ("No" in step S106), the landing possibility and proximity determination unit 116 determines that the environment of the landing surface is one in which the metal stacking does not change much even if the landing surface receives sufficient wind pressure from the unmanned aerial vehicle 10.

[0105] Therefore, if the unmanned aerial vehicle 10 lands on the landing surface in such an environment, the unmanned aerial vehicle 10 may fall and be damaged. Therefore, the landing possibility and proximity determination unit 116 outputs an instruction not to cause the unmanned aerial vehicle 10 to land on the landing surface to the action generation unit 119.

[0106] In response to an instruction from the landing possibility and approach judgment unit 116, the action generation unit 10 notifies the UI 30 of the fact that the unmanned aerial vehicle 10 cannot land on the landing surface, and controls the rotational speed of the motor 16 via the flight controller 15. In this way, the unmanned aerial vehicle 10 is maintained in a hover at a height equal to or less than the threshold value L1 (step S108).

[0107] On the other hand, if the environment of the landing surface determined in the previous step S102 is grassland and the wind vibration characteristic value calculated in the previous step S103 exceeds the threshold value L2 corresponding to grassland ("Yes" in step S106), the landing possibility and approach judgment unit 116 determines that the environment of the landing surface is an environment in which the landing impact is absorbed by the landing surface when the unmanned aerial vehicle 10 lands on the landing surface. Therefore, the landing possibility and approach judgment unit 116 outputs an instruction to cause the unmanned aerial vehicle 10 to land on the landing surface to the action generation unit 119 (step S107).

[0108] In response to an instruction from the landing possibility and approach judgment unit 116, the action generation unit 214 notifies the UI 30 that the unmanned aerial vehicle 10 has landed on the landing surface, and controls the rotational speed of the motor 16 via the flight controller 15. As a result, the unmanned aerial vehicle 10 lands on the landing surface.

[0109] - Judgment Example 3: If the environment of the landing surface is a place where paper is stacked on the landing surface

[0110] If the environment of the landing surface determined in the previous step S102 is an environment in which a desktop tray or a paper document box is stacked, and the wind vibration characteristic value calculated in the previous step S103 is equal to or less than the threshold value L2 corresponding to such an environment ("No" in step S106), the landing possibility and approach judgment unit 116 determines that the environment of the landing surface is an environment in which the paper stack does not change much even if the landing surface receives sufficient wind pressure from the unmanned aerial vehicle 10.

[0111] Therefore, if the unmanned aerial vehicle 10 lands on the landing surface in such an environment, the unmanned aerial vehicle 10 may fall and be damaged. Therefore, the landing possibility and approach judgment unit 116 outputs an instruction not to cause the unmanned aerial vehicle 10 to land on the landing surface to the action generation unit 119.

[0112] In response to an instruction from the landing possibility and approach judgment unit 116, the action generation unit 10 notifies the UI 30 of the fact that the unmanned aerial vehicle 10 cannot land on the landing surface, and controls the rotational speed of the motor 16 via the flight controller 15. In this way, the unmanned aerial vehicle 10 is maintained in a hover at a height equal to or less than the threshold value L1 (step S108).

[0113] On the other hand, if the environment of the landing surface determined in the previous step S102 is an environment where papers such as leaflets or their deformations are stacked, and the wind vibration characteristic value calculated in the previous step S103 exceeds the threshold L2 corresponding to such an environment ("Yes" in step S106), then the landing possibility and approach determination unit 116 determines that the environment of the landing surface is an environment where the landing impact is absorbed by the landing surface when the unmanned aerial vehicle 10 lands on the landing surface. Therefore, the landing possibility and approach determination unit 116 outputs an instruction to land the unmanned aerial vehicle 10 on the landing surface to the action generation unit 119 (step S107).

[0114] In response to the instruction from the landing possibility and approach determination unit 116, the action generation unit 119 notifies the UI 30 that the unmanned aerial vehicle 10 has landed on the landing surface and controls the rotational speed of the motor 16 via the flight controller 15. As a result, the unmanned aerial vehicle 10 lands on the landing surface.

[0115] - Judgment Example 4: If the environment of the landing surface is water or a floor with a water pattern

[0116] If the environment of the landing surface determined in the previous step S102 is water and the wind vibration characteristic value calculated in the previous step S103 exceeds the threshold L4 corresponding to water ("No" in step S106), then the landing possibility and approach determination unit 116 outputs an instruction not to land the unmanned aerial vehicle 10 on the landing surface to the action generation unit 119 because if the unmanned aerial vehicle 10 lands on the landing surface in such an environment, the unmanned aerial vehicle 10 may be submerged and damaged.

[0117] In response to the instruction from the landing possibility and approach determination unit 116, the action generation unit 10 notifies the UI 30 of the fact that the unmanned aerial vehicle 10 cannot land on the landing surface and controls the rotational speed of the motor 16 via the flight controller 15. In this way, the unmanned aerial vehicle 10 is maintained in a hover at a height equal to or less than the threshold L3 (step S108).

[0118] On the other hand, if the environment of the landing surface determined in the previous step S102 is water, and the wind vibration characteristic value calculated in the previous step S103 is equal to or less than the threshold L4 corresponding to water ("Yes" in step S106), then the landing possibility and approach determination unit 116 determines that the environment of the landing surface is a floor with a water pattern.

[0119] Therefore, since the UAV 10 is not submerged even if it lands on the landing surface of such an environment, the landing possibility and proximity determination unit 116 outputs an instruction to land the UAV 10 on the landing surface to the action generation unit 119 (step S107).

[0120] In response to the instruction from the landing possibility and proximity determination unit 116, the action generation unit 119 notifies the UI 30 that the UAV 10 has landed on the landing surface and controls the rotational speed of the motor 16 via the flight controller 15. As a result, the UAV 10 lands on the landing surface.

[0121] [Actions and effects]

[0122] Until now, in cases such as aerial photography of landscapes, the opportunities to use UAVs have increased. It can be easily anticipated that the development environment of UAVs will develop in the future, so there will be UAVs that can fly autonomously without an operator or UAVs that can be remotely controlled.

[0123] In such UAVs, the operator is not nearby or not present, so the UAV itself needs to determine whether it can land on the landing surface to prevent battery depletion and the UAV from crashing in cases such as when the battery power is low and an emergency landing must be performed. Therefore, for autonomously flying UAVs or remotely controllable UAVs, it is essential to adopt a technology that accurately determines whether it is possible to land on the landing surface to prevent damage. Even for current UAVs, such a technology is desirable.

[0124] Therefore, in view of the above situation, a technology that determines whether a UAV can land on the landing surface using the image information of the landing surface has been used in recent years. However, in this technology, it is necessary to determine whether the UAV has landed only based on the image information of the landing surface, and this determination may be made incorrectly.

[0125] On the other hand, the UAV 10 according to the embodiment calculates the wind vibration characteristic value of the landing surface using the images obtained by imaging the landing surface in time series by the camera 13, and determines whether to land on the landing surface based on the calculated wind vibration characteristic value.

[0126] Therefore, it is determined whether the unmanned aerial vehicle 10 is to land on the landing surface by considering not only the image of the landing surface to be imaged but also the temporal change in the physical shape of the landing surface (e.g., the softness of the landing surface). Therefore, compared with the determination based only on the image information of the landing surface, the accuracy of determining whether it is possible to land in order to prevent damage to the unmanned aerial vehicle is improved.

[0127] In addition, if the altitude exceeds a threshold value corresponding to the environment of the landing surface, the unmanned aerial vehicle 10 executes control to approach the landing surface. Therefore, even if the environment of the landing surface is windless, the wind pressure from the unmanned aerial vehicle 10 is transmitted to the landing surface, so that the wind vibration characteristic value of the landing surface can be calculated. Therefore, since the unmanned aerial vehicle 10 can calculate the wind vibration characteristic value of the landing surface in any environment of the landing surface, the unmanned aerial vehicle 10 can accurately determine whether it is possible to land in order to prevent damage to it regardless of the environment of the landing surface.

[0128] <Second Embodiment>

[0129] Figure 7 FIG. is a block diagram showing a configuration example of an information processing system 2 according to the second embodiment. Hereinafter, components similar to those of the first embodiment will be denoted by similar reference numerals, and the description thereof will be omitted or simplified.

[0130] The information processing system 2 according to the second embodiment is different from the information processing system 2 of the first embodiment in that the information processing device 20 determines whether to land the unmanned aerial vehicle 10 on the landing surface.

[0131] [Configuration of Information Processing System]

[0132] As Figure 7 shown, the information processing system 2 includes an unmanned aerial vehicle 10, an information processing device 20, and a UI 30. The information processing device 20 is generally a server device, but is not limited thereto, and may be any other computer such as a PC.

[0133] The information processing device 20 includes a control unit 21 and a storage unit 22. The control unit 21 controls the overall operation or a part thereof of the information processing device 20 according to a program stored in the storage unit 22.

[0134] The control unit 21 functionally includes a landing possibility and approach determination unit 116, an environment determination unit 117, a wind vibration characteristic calculation unit 118, and an action generation unit 119.

[0135] The storage unit 22 functionally includes a landing possibility database 121 and an environment database 122.

[0136] [Operation of the Information Processing Device]

[0137] The information processing device 20 according to the second embodiment performs operations according to the flowchart shown in Figure 4 . Note that, except that the information processing device 20 executes steps S101 to S108, the information processing system 2 operates in the same manner as in the first embodiment, and thus the detailed description thereof will be omitted.

[0138] [Actions and Effects]

[0139] In the information processing system 2 of the second embodiment, the information processing device 20 determines whether to land the unmanned aerial vehicle 10 on the landing surface. In other words, in order to avoid damage to the unmanned aerial vehicle 10, the information processing device 20 executes a part of the arithmetic processing performed by the unmanned aerial vehicle 10. Therefore, the arithmetic load of the unmanned aerial vehicle 10 can be greatly reduced. In addition, since it is not necessary to increase the arithmetic processing ability of the unmanned aerial vehicle 10, the design cost of the unmanned aerial vehicle 10 is suppressed.

[0140] [Modification Example]

[0141] Although the embodiments of the present technology have been described above, the present technology is not limited thereto, and of course, various changes can be made thereto.

[0142] For example, in the above embodiment, the feature points of the landing surface are detected from the image obtained by imaging the landing surface, and the temporal change of the detected feature points is calculated as the wind vibration characteristic value of the landing surface, but the present technology is not limited thereto. For example, the temporal change of the pixel value of the image obtained by imaging the landing surface can be calculated as the wind vibration characteristic value of the landing surface.

[0143] [Supplement]

[0144] In the above embodiment, the description is made on the assumption that the unmanned aerial vehicle 10 is an aircraft, but the present technology is not limited thereto. The present technology can be applied to other moving bodies in addition to aircraft, and its application is not particularly limited. Note that, in addition to unmanned aerial vehicles, aircraft can include airplanes, unmanned drones, unmanned helicopters, etc.

[0145] In addition, the effects described herein are merely illustrative or exemplary and not restrictive. In other words, the present technology can provide other effects clear to those skilled in the art from the description herein as a supplement or alternative to the above effects.

[0146] Although the preferred embodiments of the present technology have been described in detail above with reference to the accompanying drawings, the present technology is not limited to such examples. It is obvious that those skilled in the art of the present technology can conceive of various alternatives or modifications within the scope of the technical idea described in the claims. It should be understood that such alternatives or modifications also naturally fall within the technical scope of the present technology.

[0147] Note that the present technology may have the following configurations.

[0148] (1) A mobile body, comprising:

[0149] A control unit that

[0150] calculates a temporal change of a landing surface based on images obtained by imaging the landing surface of the mobile body in time series by an imaging unit of the mobile body in a hovering state, and

[0151] determines whether to land the mobile body on the landing surface based on the calculated temporal change.

[0152] (2) The mobile body according to (1), further comprising:

[0153] A detection unit that detects a height of the mobile body from the landing surface, wherein

[0154] the control unit determines whether to land the mobile body on the landing surface based on the height and the temporal change.

[0155] (3) The mobile body according to (1) or (2), wherein the control unit

[0156] performs image processing on the images to remove objects different from the landing surface, and

[0157] calculates the temporal change based on the images on which the image processing has been performed.

[0158] (4) The mobile body according to (2), wherein

[0159] when the height of the mobile body is greater than a predetermined threshold, the control unit causes the mobile body to approach the landing surface.

[0160] (5) The mobile body according to (4), wherein

[0161] the threshold is a height at which the landing surface changes over time due to the wind generated by the mobile body.

[0162] (6) The mobile body according to (4) or (5), wherein the control unit

[0163] discriminates the environment of the landing surface based on the images, and

[0164] Determine whether to land the mobile object on the landing surface based on the determined environment, time change, and altitude.

[0165] (7) The mobile object according to (6), wherein

[0166] When the time change exceeds the threshold corresponding to the determined environment, the control unit lands the mobile object on the landing surface.

[0167] (8) The mobile object according to (6), wherein

[0168] When the determined environment is water surface and the time change is equal to or less than the threshold corresponding to the water surface, the control unit lands the mobile object on the landing surface.

[0169] (9) The mobile object according to any one of (1) to (8), wherein the mobile object is a flying object.

[0170] (10) An information processing device, comprising:

[0171] A control unit, which

[0172] Calculates the time change of the landing surface based on the images obtained by imaging the landing surface of the mobile object in time series by the imaging unit of the mobile object in a hovering state,

[0173] Determines whether to land the mobile object on the landing surface based on the calculated time change, and

[0174] Outputs the determination result to the mobile object.

[0175] (11) The information processing device according to (10), wherein the information processing device is a server.

[0176] (12) An information processing method, comprising:

[0177] Calculates the time change of the landing surface based on the images obtained by imaging the landing surface of the mobile object in time series by the imaging unit of the mobile object in a hovering state, and

[0178] Determines whether to land the mobile object on the landing surface based on the calculated time change.

[0179] (13) A program for causing a mobile object or an information processing device to execute the following steps:

[0180] Calculates the time change of the landing surface based on the images obtained by imaging the landing surface of the mobile object in time series by the imaging unit of the mobile object in a hovering state, and

[0181] Determines whether to land the mobile object on the landing surface based on the calculated time change.

[0182] List of reference signs

[0183] Information processing systems 1, 2

[0184] Unmanned aerial vehicle 10

[0185] Information processing devices 20, 100

[0186] UI 30

Claims

1. A mobile body, comprising: A control unit, the control unit Calculates the temporal change of the landing surface based on images obtained by imaging the landing surface of the mobile body in a time series by an imaging unit of the mobile body in a hovering state, Determines the environment of the landing surface based on the images, and Lands the mobile body on the landing surface based on the calculated temporal change exceeding a threshold corresponding to the determined environment, wherein different thresholds correspond to respective different determined environments.

2. The mobile body according to claim 1, further comprising: A detection unit, the detection unit detects the height of the mobile body from the landing surface, wherein, The control unit determines whether to land the mobile body on the landing surface based on the height and the temporal change.

3. The mobile body according to claim 1, wherein The control unit Performs image processing on the images to remove objects different from the landing surface, and Calculates the temporal change based on the images on which the image processing has been performed.

4. The mobile body according to claim 2, wherein, When the height of the mobile body is greater than a predetermined threshold, the control unit causes the mobile body to approach the landing surface.

5. The mobile body according to claim 4, wherein, The predetermined threshold is the height at which the landing surface changes over time due to the wind generated by the mobile body.

6. The mobile body according to claim 4, wherein, The control unit determines whether to land the mobile body on the landing surface based on the determined environment, the temporal change, and the height.

7. The mobile body according to claim 6, wherein, When the determined environment is a water surface and the temporal change is equal to or less than a threshold corresponding to the water surface, the control unit lands the mobile body on the landing surface.

8. The mobile body according to claim 1, wherein The mobile body is a flying body.

9. An information processing device, comprising: A control unit, the control unit Calculates the temporal change of the landing surface based on images obtained by imaging the landing surface of the mobile body in a time series by an imaging unit of the mobile body in a hovering state, Determines the environment of the landing surface based on the images, Lands the mobile body on the landing surface based on the calculated temporal change exceeding a threshold corresponding to the determined environment, wherein different thresholds correspond to respective different determined environments, and Outputs the determination result to the mobile body.

10. The information processing device according to claim 9, wherein, The information processing device is a server.

11. An information processing method, comprising: Calculating the temporal change of the landing surface based on images obtained by imaging the landing surface of the mobile body in a time series by an imaging unit of the mobile body in a hovering state, Determining the environment of the landing surface based on the images, and Landing the mobile body on the landing surface based on the calculated temporal change exceeding a threshold corresponding to the determined environment, wherein different thresholds correspond to respective different determined environments.

12. A program product that causes a mobile body or an information processing device to perform the following steps Calculate the temporal change of the landing surface based on images obtained by imaging the landing surface of the moving body in a time series with an imaging unit of the moving body in a hovering state. Discriminate the environment of the landing surface based on the images, and Land the moving body on the landing surface based on the calculated time change exceeding a threshold value corresponding to the determined environment, where Different thresholds correspond to the respective different discriminated environments.

Images