Information processing apparatus, landing suitability determination method, and storage medium

Abstract

To provide an information processing apparatus, a landing suitability determination method, and a storage medium that can select an appropriate landing site taking into account the influence of a downblast hitting a ground object and rebounding, the present invention detects a ground object existing in the vicinity of a landing candidate site of a UAV 1, estimates the degree of influence on landing of a downblast hitting the ground object and rebounding when the UAV 1 lands, that is, the degree of influence in the landing candidate site, and determines whether the landing candidate site is suitable for landing based on the estimated degree of influence.

Description

Technical Field

[0001] This invention relates to the technical field of methods for reducing the impact of ground effects during the landing of unmanned aerial vehicles. Background Technology

[0002] When unmanned aerial vehicles (UAVs) or other drones land, the impact of the wind (downwash) generated by the UAV's rotating propellers hitting the ground at the landing site and bouncing back can sometimes make landing control difficult. This impact is known as the ground effect. Patent Document 1 discloses a technique to reduce the impact of the ground effect during landing by determining whether the area of ​​the intended landing area is larger than the projected area of ​​the aircraft and whether the intended landing area is a horizontal or near-horizontal surface, thereby determining a suitable landing area.

[0003] Background Technology Documents

[0004] Patent documents

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2019-64280 Summary of the Invention

[0006] [The problem the invention aims to solve]

[0007] However, sometimes the downdraft not only hits the ground at the landing site but also bounces off objects on the ground around the landing site. Therefore, even if the landing site is level, the impact of the downdraft hitting objects on the ground around the landing site can make it difficult to control the landing of the unmanned aerial vehicle.

[0008] Therefore, the present invention was made in view of the aforementioned problems, and one of the objectives is to provide an information processing device, a landing suitability determination method, and a program capable of selecting an appropriate landing site that takes into account the impact of downdrafts hitting objects on the ground and causing them to bounce.

[0009] [Technical means to solve the problem]

[0010] To address the aforementioned problem, the invention described in technical solution 1 is characterized by comprising: a detection mechanism for detecting ground objects existing around a potential landing site for an unmanned aerial vehicle (UAV); an estimation mechanism for estimating the impact of downdrafts generated during UAV landing hitting and rebounding from the ground objects on the landing, i.e., the impact within the potential landing site; and a determination mechanism for determining whether the potential landing site is suitable for landing based on the impact estimated by the estimation mechanism. Thus, an appropriate landing site that takes into account the impact of downdrafts hitting and rebounding from ground objects can be selected.

[0011] The invention described in technical solution 2 is characterized in that, in the information processing apparatus described in technical solution 1, the estimation mechanism estimates the impact degree based on data representing the relationship between the attributes of ground objects and the ease or difficulty of airflow through the ground objects, and the attributes of the ground objects detected by the detection mechanism. Therefore, the impact degree on landing candidate sites can be estimated more efficiently and effectively.

[0012] The invention described in technical solution 3 is characterized in that, in the information processing apparatus described in technical solution 2, the estimation mechanism estimates the impact degree based on the ease or difficulty of airflow to a specific ground object in each location where the surrounding area of ​​the landing candidate site is hypothetically subdivided into specific first intervals. This reduces noise generated when assessing the ease or difficulty of airflow to a specific location, thereby improving the accuracy of estimating the impact degree on the landing candidate site.

[0013] The invention described in technical solution 4 is characterized in that, in the information processing apparatus described in technical solutions 2 or 3, the estimation mechanism, in addition to the attributes of the ground object detected by the detection mechanism, also estimates the influence degree based on at least one of the 3D dimensions and 3D shape of the ground object detected by the detection mechanism. This improves the accuracy of estimating the landing influence degree on the landing candidate site.

[0014] The invention described in technical solution 5 is characterized in that, in the information processing device described in any of technical solutions 1 to 4, the landing candidate site is a location that hypothetically subdivides the landing candidate area into specific second intervals. Therefore, within the landing candidate area, landing candidate sites with minimal impact from downwinds hitting objects on the ground can also be selected as landing sites.

[0015] The invention described in technical solution 6 is characterized in that, in the information processing apparatus described in technical solution 5, the estimation mechanism estimates the impact degree according to each of the plurality of landing candidate sites, and the determination mechanism determines whether each of the landing candidate sites is suitable for landing based on the impact degree of each of the landing candidate sites. Thus, a more suitable landing candidate site can be selected as the landing site from the plurality of landing candidate sites.

[0016] The invention described in technical solution 7 is characterized in that, in the information processing device described in technical solutions 5 or 6, a specific mechanism is further provided to select a flat area of ​​at least no obstacles and a certain size or larger from a preset landing target area as the landing candidate area. Thus, by pre-reducing the landing target area, futile exploration can be avoided, and the computational load before selecting a landing site can be reduced.

[0017] The invention described in technical solution 8 is characterized in that, in the information processing apparatus described in technical solutions 5 or 6, a specific mechanism is further provided to select a predetermined landing target area, a specific flat area that is at least free of obstacles and of a specific size or larger, and an area whose surface properties are suitable for landing, as the potential landing area. Thus, by pre-reducing the landing target area, futile exploration can be avoided, and the computational load before selecting a landing site can be reduced.

[0018] The invention described in technical solution 9 is characterized in that, in the information processing apparatus described in any of technical solutions 1 to 8, the detection mechanism detects ground objects present around the landing candidate site based on sensing information obtained by the unmanned aerial vehicle (UAV) sensing the perimeter of the landing candidate site. Therefore, because sensing information obtained by the UAV sensing the area intended for landing can be used, the accuracy of estimating the impact of landing on the landing candidate site can be improved.

[0019] The invention described in technical solution 10 is characterized by being a landing suitability determination method executed by one or more computers, and includes the following steps: detecting ground objects existing around the landing candidate site of the unmanned aerial vehicle; estimating the impact of the downwind generated when the unmanned aerial vehicle lands on the ground objects and their rebound on the landing, that is, the impact of the landing candidate site; and determining whether the landing candidate site is suitable for landing based on the estimated impact.

[0020] The invention described in technical solution 11 is characterized by enabling a computer to function as the following mechanisms: a detection mechanism for detecting ground objects around a potential landing site for an unmanned aerial vehicle (UAV); an estimation mechanism for estimating the impact of the downdraft generated when the UAV lands on the ground objects and their rebound, which is the impact of the potential landing site; and a determination mechanism for determining whether the potential landing site is suitable for landing based on the impact estimated by the estimation mechanism.

[0021] According to the present invention, it is possible to select an appropriate landing site that takes into account the impact of downdrafts hitting objects on the ground and causing them to bounce. Attached Figure Description

[0022] Figure 1 This is a diagram illustrating a summary structure of the flight management system S.

[0023] Figure 2 This is a diagram illustrating a general structural example of UAV1.

[0024] Figure 3 This is a diagram illustrating a general configuration example of management server 2.

[0025] Figure 4 This is a diagram showing an example of the function blocks of the control unit 23.

[0026] Figure 5 This is a conceptual diagram representing an example of the landing target area AR0 and the landing candidate areas AR1 and AR2.

[0027] Figure 6 This is a concept diagram representing an example of the properties of ground objects existing around the landing candidate site (1).

[0028] Figure 7 This is an example of a graph that uses a chart format to represent the relationship between the properties of objects on the ground and the ease of air circulation around those objects.

[0029] Figure 8 This is a conceptual diagram representing an example of the ease of air circulation for ground objects existing around a landing candidate site (1).

[0030] Figure 9 This is a flowchart illustrating an example of the processing performed between UAV1 and management server 2 within the flight management system S.

[0031] Figure 10 This is a flowchart illustrating an example of the processing performed between UAV1 and management server 2 within the flight management system S. Detailed Implementation

[0032] Hereinafter, one embodiment of the present invention will be described with reference to the accompanying drawings. Furthermore, the following embodiment applies the present invention to a flight management system for enabling unmanned aerial vehicles (UAVs) to fly for a specific purpose. Examples of such specific purposes include, for instance, handling (delivery), measurement, photography, inspection, and monitoring.

[0033] [1. Composition and Operation Summary of Flight Management System S]

[0034] First, refer to Figure 1 The structure and operation overview of the flight management system S in this embodiment will be explained. Figure 1 This is a diagram illustrating a general structural example of a flight management system S. For example... Figure 1 As shown, the flight management system S comprises an unmanned aerial vehicle (hereinafter referred to as "UAV" or "unmanned aerial vehicle") 1 and a management server 2 (an example of an information processing device), both of which are connected to a communication network NW. Here, the communication network NW consists of, for example, the Internet, a mobile communication network, and its wireless base stations. UAV1 is an example of an unmanned aerial vehicle, also known as a drone or multi-rotor drone. UAV1 can fly from the ground to the landing area according to remote operator control, or fly autonomously in the air. Management server 2 is a server that manages the landing area and UAV1.

[0035] The flight management system S of this embodiment can select (determine) a landing site for the UAV1 where the impact of the downdraft generated by the UAV1's propeller rotation on ground objects in the vicinity (in other words, where landing control is easier) is minimal (the impact is less severe). Thus, even in locations without pre-installed takeoff and landing facilities (takeoff and landing docks), the UAV1 can land stably. Here, the landing site is selected from, for example, residential land, parking lots, parks, logistics warehouse land, building rooftops, near damaged buildings, or adjacent areas such as cliffs or embankments. Ground objects can be natural or man-made objects, especially objects that exist in contact with the ground, and can be planar or three-dimensional. Examples of attributes (in other words, types) of ground objects include building walls, fences (e.g., masonry walls, metal mesh fences), hedges (e.g., tree hedges, bamboo fences), trees, items (e.g., goods stacked on pallets), mobile objects such as cars or bicycles, rubble, roads, etc. Ground objects can also be referred to as obstacles. In addition, in this specification, "ground" may also include the roof surface of a building.

[0036] [1-1. Composition and Function of UAV1]

[0037] Next, refer to Figure 2 The structure and function of UAV1 will be explained. Figure 2 This is a diagram illustrating a general structural example of UAV1. (For example...) Figure 2 As shown, the UAV1 includes a drive unit 11, a positioning unit 12, a communication unit 13, a sensor unit 14, a storage unit 15, and a control unit 16 (an example of a computer). Furthermore, the UAV1 includes a battery (not shown) that supplies power to all parts of the UAV1, and a horizontally rotating rotor (propeller). Additionally, the UAV1 may include a holding mechanism for holding the items being transported. This holding mechanism may include a storage section for storing the items. In this case, for example, an opening and closing door may be provided on the lower surface of the storage section. Furthermore, the holding mechanism may also include a wire and a reel (winch) for feeding or winding the wire.

[0038] The drive unit 11 includes a motor and a rotating shaft. The drive unit 11 drives the motor and rotating shaft, which in turn rotate multiple rotors according to control signals output from the control unit 16. The positioning unit 12 includes a radio wave receiver and an altitude sensor. The positioning unit 12 receives radio waves, for example, transmitted from GNSS (Global Navigation Satellite System) satellites, and detects the current horizontal position (latitude and longitude) of the UAV1 based on these radio waves. Furthermore, the current horizontal position of the UAV1 can be corrected based on images captured by the camera of the sensor unit 14. Alternatively, the current horizontal position of the UAV1 can also be corrected using the RTK (Real-Time Kinematic) method, which utilizes radio waves received from a base station (a base station capable of communicating with the UAV1) at a specific location. The position information indicating the current position detected by the positioning unit 12 is output to the control unit 16. Additionally, the positioning unit 12 can also detect the current vertical position (altitude) of the UAV1 using an altitude sensor such as a barometric pressure sensor. In this case, the location information includes altitude information indicating the altitude of UAV1.

[0039] The communication unit 13 has wireless communication capabilities and is responsible for controlling communication via the communication network NW. The sensor unit 14 has various sensors for the flight control of the UAV1. These sensors include, for example, optical sensors, weather sensors, 3-axis angular velocity sensors, 3-axis accelerometers, and geomagnetic sensors. The optical sensors are configured with cameras (e.g., RGB cameras, depth cameras) and continuously sense the actual space within the sensing range (e.g., the range falling within the camera's field of view). Here, sensing means, for example, measuring, photographing, or perceiving arbitrary quantities (e.g., physical quantities).

[0040] Additionally, the optical sensor may include a LiDAR (Light Detection and Ranging, or Laser Imaging Detection and Ranging) sensor that measures the shape of objects on the ground or the distance between objects and the ground. Furthermore, the optical sensor may include a temperature sensor that non-contactly senses the temperature (infrared) of the ground containing objects. Additionally, the weather sensor is used to sense weather conditions such as wind speed (wind intensity) and wind direction in the surrounding environment of the UAV1. The sensing information obtained by the sensor unit 14 can be output to the control unit 16. The sensing information includes at least one of the following images: an RGB image, a depth image, a distance image, and a temperature image sensed by the optical sensor. Furthermore, the sensing information may also include weather information (e.g., wind speed and wind direction) sensed by the weather sensor.

[0041] The storage unit 15 is composed of non-volatile memory and the like, storing various programs and data. Additionally, the storage unit 15 stores the UAV1's identification ID (identification information). The control unit 16 includes a CPU (Central Processing Unit), ROM (Read Only Memory), and RAM (Random Access Memory), and executes various controls according to the programs stored in the ROM (or storage unit 15). For example, the control unit 16 performs flight control to make the UAV1 fly towards the landing target area. In this flight control, the rotor speed, UAV1's position, attitude, and direction of travel are controlled using position information obtained from the positioning unit 12, sensing information obtained from the sensor unit 14, and landing target area information. Thus, the UAV1 can autonomously move to the airspace above the landing target area.

[0042] The landing area information includes, for example, the center location (latitude and longitude) of the landing area and the width of the landing area. Here, the width of the landing area is represented, for example, by an area with a radius of tens of meters based on the center location, or an area tens of meters long × tens of meters wide. The landing area information can be set at the departure point (flight start point) of UAV1, or it can be set by sending it from the management server 2. Furthermore, during the flight of UAV1, the location information of UAV1 and the airframe ID of UAV1 are sequentially sent to the management server 2 via the communication unit 13.

[0043] When UAV1 reaches the airspace above the landing target area (e.g., at an altitude of 30m), the control unit 16 causes the sensor unit 14 to sense (e.g., as a first-stage long-range sensing) the area encompassing the landing target area from the airspace above, and obtains sensing information obtained through this sensing (hereinafter referred to as "first sensing information"). This sensing can also be performed continuously in a time sequence. The first sensing information obtained by the sensor unit 14 sensing the area encompassing the landing target area, along with the UAV1's position information and the UAV1's body ID, is transmitted to the management server 2 via the communication unit 13.

[0044] Furthermore, when the control unit 16 receives landing candidate area information via the communication unit 13, indicating a landing candidate area (i.e., a landing candidate area contained within the landing target area) specified by the management server 2 based on the first sensing information from the landing target area, flight control to the landing candidate area is performed. Here, the landing candidate area information includes, for example, the center position (latitude and longitude) of the landing candidate area and the size of the landing candidate area. The size of the landing candidate area can also be represented in the same way as the landing target area. Afterwards, when the UAV1 receives the landing candidate area information, decreases its altitude, and reaches the airspace above the landing candidate area (e.g., at an altitude of 10m), the control unit 16 causes the sensor unit 14 to sense (e.g., close-range sensing as a second stage) the range containing the landing candidate area from the airspace above, and obtains sensing information obtained by the sensing (hereinafter referred to as "second sensing information"). The sensing can also be performed continuously in a time sequence. The second sensing information, obtained by the sensor unit 14 sensing the range including the landing candidate area, together with the UAV1's position information and the UAV1's airframe ID, is sent to the management server 2 via the communication unit 13.

[0045] Subsequently, when the control unit 16 receives landing site information via the communication unit 13, indicating the landing site finally selected by the management server 2 based on the second sensing information, landing control is performed to proceed to the landing site. The landing site information includes, for example, the location (latitude and longitude) of the landing site and the two-dimensional dimensions of the landing site (e.g., ym xm). During landing control, the rotor speed, the position, attitude, and direction of travel of the UAV1 are controlled using the landing site information, the position information obtained from the positioning unit 12, and the sensing information obtained from the sensor unit 14.

[0046] Here, the control unit 16 can perform landing control according to the landing method corresponding to the configuration of obstacles (e.g., ground objects that may obstruct landing) around the landing site. For example, if there are obstacles around the landing site (e.g., all four sides), the control unit 16 causes the UAV1 to descend vertically from directly above the landing site. On the other hand, if the obstacles are located in any direction around the landing site, the control unit 16 causes the UAV1 to descend at an angle towards the landing site from a direction where the obstacles are not present. In this case, to avoid prolonged exposure to wind, the control unit 16 may also accelerate the descent of the UAV1. The landing method corresponding to the configuration of obstacles can be determined by the control unit 16 or by the management server 2. If the landing method is determined by the management server 2, landing method information indicating the landing method corresponding to the configuration of obstacles is sent from the management server 2 to the UAV1 along with landing site information. In this case, the control unit 16 performs landing control based on the received landing method information using the landing method corresponding to the configuration of obstacles around the landing site.

[0047] Furthermore, landing control of the UAV1 includes not only contact between the UAV1 and the ground at the landing site, but also stopping (hovering) the UAV1 in the air at a vertical distance of tens of centimeters to about 2 meters from the ground. The latter assumes the UAV1 is used for transporting goods. In this case, while hovering, the UAV1 releases the goods by detaching them from its holding mechanism, or by opening the storage door of the holding mechanism. Afterward, the UAV1 returns without contacting the ground. Alternatively, in this case, the goods can also be released when the goods or their storage compartment descends vertically and contacts the ground by feeding a wire from the UAV1's reel.

[0048] [1-2. Composition and Functions of Management Server 2]

[0049] Next, refer to Figure 3 The composition and functions of management server 2 will be explained. Figure 3 This is a diagram illustrating a general configuration example of management server 2. (Example) Figure 3As shown, the management server 2 includes a communication unit 21, a storage unit 22, and a control unit 23 (an example of a computer). The communication unit 21 is responsible for controlling communication via the communication network NW. The communication unit 21 receives sensing information, location information, and aircraft ID sent from the UAV1. The management server 2 can identify the current location of the UAV1 using the UAV1's location information. The storage unit 22 is composed of, for example, a hard disk drive, and stores various programs and data. In addition, a flight management database 221 is built in the storage unit 22. The flight management database 221 is a database used to manage flight-related information of the UAV1. The flight management database 221 establishes correspondences and stores (logs) information such as landing target area information, landing site information, and aircraft information. Here, the aircraft information includes the aircraft ID and aircraft dimensions of the UAV1 flying towards the landing target area. The aircraft dimensions are, for example, the two-dimensional dimensions of the UAV1 (vertical ym × horizontal xm). In addition, the flight management database 221 can also store landing method information.

[0050] The control unit 23 includes a CPU, ROM, and RAM. Figure 4 This is a diagram illustrating a functional block example of the control unit 23. The control unit 23 operates according to, for example, a program (program code group) stored in ROM or storage unit 22, such as... Figure 4 As shown, the following mechanisms function as: a sensing information acquisition unit 231, a landing candidate area specifying unit 232 (an example of a specific mechanism), a landing candidate site specifying unit 233, a ground object specifying unit 234 (an example of a detection mechanism), an impact estimation unit 235 (an example of an estimation mechanism), a landing suitability determination unit 236 (an example of a determination mechanism), a landing site selection unit 237, and an information provision unit 238. Additionally, the program can be stored in the storage unit 22 from a computer-readable storage medium (CD, DVD, USB memory, etc.) containing the program (computer program).

[0051] The sensing information acquisition unit 231 acquires first sensing information from the UAV1 via the communication unit 21, obtained by the UAV1 sensing a range including the landing target area. Additionally, the sensing information acquisition unit 231 acquires second sensing information from the UAV1 via the communication unit 21, obtained by the UAV1 sensing a range including the landing candidate area. Here, the range including the landing candidate area includes the perimeter of the landing candidate site, which will be described later.

[0052] The landing candidate area designation unit 232, based on the first sensing information acquired by the sensing information acquisition unit 231, designates a flat area (a planar area) of at least no obstacles and a certain size or larger from a preset landing target area (e.g., 50m x 50m). By pre-reducing the landing target area in this way, futile exploration can be avoided, and the amount of computation until a landing site is selected can be reduced. The specific size is set to be at least larger than the size of the UAV1. The flat area is, for example, an area with a slope (inclination relative to the horizontal plane) of less than a threshold. The flat area can also be designated from the 3D shape of the landing target area. The 3D shape of the landing target area can be designated, for example, by performing SLAM (Simultaneous Localization and Mapping) processing on the first sensing information. Furthermore, the positions of various points in the landing candidate area can be designated based on, for example, the position information of the UAV1 that transmitted the first sensing information and the distance of the UAV1 to each point.

[0053] Furthermore, the landing candidate area designation unit 232 can designate areas from a preset landing target area that are unobstructed, flat, and of a certain size or larger, and whose surface properties are suitable for landing, as landing candidate areas. Here, "surface" refers to the surface of the landing target area as seen from above, and is distinguished from the ground surface. Examples of surface properties include concrete, water, trees, soil, grass, and roads. For example, it can be determined that concrete, soil, and grass are suitable for landing, while water, trees, and roads are not suitable for landing. In addition, the surface properties can also be inferred from a pre-learned semantic segmentation model α. Semantic segmentation is a method of classifying each pixel in an image based on surrounding pixel information. The semantic segmentation model α is, for example, a learned model that takes an RGB image containing first sensing information as input and outputs the attribute value of each pixel in the RGB image. The attribute value represents the properties of the surface and varies depending on each attribute of the surface.

[0054] Figure 5 This is a conceptual diagram representing an example of the landing target area AR0 and the landing candidate areas AR1 and AR2. Furthermore, Figure 5 In the example, the 3D shape of the landing object region AR0 is omitted. Figure 5In the example, the landing target area AR0 is hypothetically subdivided into intervals BLx (a specific example of a second interval) corresponding to the 2D dimensions required for landing with UAV1 (e.g., 5m x 5m). The surface attributes of each interval BLx are represented according to different patterns (concrete, grass, water, trees, and roads in this example). Here, the shape of the interval BLx can be rectangular, circular, or elliptical. When using the semantic segmentation model α, the most frequently occurring attribute among the surface attributes corresponding to each pixel in a given interval BLx can be set as the surface attribute of that interval BLx. Thus, by representing the surface attributes according to each interval BLx, noise generated when generating specific surface attributes can be reduced. Furthermore, in Figure 5 In the example, from the landing target area AR0, two landing candidate areas AR1 and AR2 with a narrower range than the landing target area AR0 are specified. In the case of specifying multiple landing candidate areas AR1 and AR2, the landing candidate area AR1 with a wider range can be prioritized for use in subsequent processing based on connectivity.

[0055] Furthermore, the landing candidate area designation unit 232 can also designate areas from the landing target area that are flat and free of obstacles of a certain size or larger, and whose wind speed contained in the first sensing information is below a threshold, based on the first sensing information obtained by the sensing information acquisition unit 231. Thus, areas with the flatness required for landing and where the wind is not strong within the landing target area can be designated as landing candidate areas.

[0056] The landing candidate site designation unit 233 designates one or more landing candidate sites for UAV1 from the landing candidate area designated by the landing candidate area designation unit 232. Here, the two-dimensional dimensions of the UAV1's landing candidate site are the two-dimensional dimensions required for UAV1 landing (above the UAV1's fuselage dimensions). For example, such as... Figure 5 As shown, each interval BLx obtained by hypothetically subdividing the landing candidate area AR1 can also be used as a landing candidate site for UAV1 (1) to (15). Thus, even in the landing candidate area AR1, any one of the landing candidate sites ((1) to (15)) with relatively small impact from the downwind hitting the ground object can be selected as the landing site.

[0057] The ground object identification unit 234 detects ground objects existing around the landing candidate site based on the second sensing information acquired by the sensing information acquisition unit 231, and identifies the attributes of the ground objects (which may be the material of the ground objects). At this time, at least one of the three-dimensional dimensions and the three-dimensional shape of the ground object can be identified. While it is desirable that the perimeter of the landing candidate site encompasses the entire area around the landing candidate site (e.g., all four directions), it can also be any one of the surrounding areas (sometimes only ground objects existing in any one direction are detected). For example, the ground object identification unit 234 can identify the attributes of ground objects existing around the landing candidate site by using image analysis of data representing the relationship between the appearance features and attributes of the ground objects. Furthermore, the three-dimensional dimensions and three-dimensional shape of the ground objects can be identified, for example, by performing SLAM processing on the second sensing information. In addition, when multiple landing candidate sites are specified by the landing candidate site specification unit 233, ground objects around each landing candidate site can be detected, and at least one of the attributes of the ground objects, their 3D size, and 3D shape can be specified.

[0058] Furthermore, the attributes of ground objects can be determined (estimated) from a pre-learned semantic segmentation model β. In this case, the ground object determination unit 234 uses the semantic segmentation model β to detect ground objects existing around the landing candidate sites and determine the attributes of the ground objects. For example, ground objects are detected by inputting an RGB image contained in the second sensing information into the semantic segmentation model β, and the attribute value of each pixel in the RGB image is output from the semantic segmentation model β. The attribute value is a value representing the attribute of the ground object, which is different for each attribute of the ground object. Multiple adjacent pixels with the same attribute value are grouped together to form one ground object. By using the semantic segmentation model β, the attributes of ground objects existing around multiple landing candidate sites can be determined at once. In addition, for the semantic segmentation model β, not only RGB images are input, but also depth images or temperature images are input, thereby improving the estimation accuracy of ground objects. In addition, for the semantic segmentation model β, in addition to images such as the RGB image contained in the second sensing information, images such as the RGB image contained in the first sensing information can also be input.

[0059] Figure 6 This is a conceptual diagram representing an example of the properties of ground objects existing around the landing candidate site (1). Figure 6In the example, the hypothetical subdivision of the area surrounding the landing candidate site (1) is divided into intervals Bly (an example of a specific first interval). In each interval Bly where ground objects are detected, the attributes of the ground objects are represented. However, in the interval Bly where no ground objects are detected, the attributes of the ground objects are not represented. Here, the shape of the interval Bly can be rectangular, circular, or elliptical. Furthermore, the size of the interval Bly can be smaller than that shown in the figure; the smaller the size of the interval Bly, the higher the precision of the attribute specificity. On the other hand, as... Figure 6 As shown, if each interval of a certain size (BLy) is used to determine the properties of specific ground objects, then noise generated when those properties are being generated can be reduced. Furthermore, in Figure 6 In the example, although the dimensions of interval Bly (e.g., 1m x 1m) are narrower than the dimensions of interval BLx (e.g., 5m x 5m), the dimensions of interval Bly and interval BLx can be the same. Additionally, in Figure 6 In the example, although the perimeter of the landing waiting site (1) is defined as a range equivalent to two intervals Bly (e.g., 2m) outward from the boundary BO of the landing waiting site (1), the range is not particularly limited and may be a narrower range (e.g., one interval Bly) or a wider range (e.g., three intervals Bly).

[0060] The impact estimation unit 235 estimates the impact of the downwind generated during UAV1's landing, which bounces off the ground object, on the landing. This impact is the impact of the landing candidate site (i.e., the impact on landing at the candidate site), and is, for example, a value representing the degree of the impact. When multiple landing candidate sites are specified, the impact is estimated for each candidate site. Furthermore, the impact on landing at a candidate site can also be referred to as the landing suitability of that candidate site. For example, the greater the impact on landing at a candidate site, the more difficult it is to control landing at that candidate site, thus reducing the landing suitability.

[0061] As a better example, the impact estimation unit 235 can estimate the impact on landing candidate sites based on data representing the relationship between the properties of ground objects and the ease of airflow of ground objects, and the properties of ground objects detected by the ground object specific unit 234. This allows for a more efficient and better estimation of the impact on landing candidate sites. Here, the ease of airflow can also be represented numerically, with higher values ​​for easier airflow. Figure 7 This is an example of a graph that uses a chart format to represent the relationship between the properties of objects on the ground and the ease of air circulation around those objects. Figure 7In the example, for walls (building walls), the ease of airflow is set to "0" points; for trees, it's set to "5" points; and for roads, it's set to the maximum value of "10" points. Furthermore, the ease of airflow for each ground object (or a single ground object) located around a specific landing candidate site is calculated, with a larger sum of these ease of airflow resulting in a smaller impact. Additionally, in... Figure 7 In the example, although the properties of ground objects are categorized as walls (building walls), metal fences, hedges, trees, items, moving objects, rubble, and roads, the properties described here can be further subdivided, and the ease of airflow can be set according to each attribute of the categorization. For example, items, moving objects, and rubble can also be categorized by size as large, medium, and small.

[0062] Figure 8 This is a conceptual diagram illustrating the ease of airflow around ground objects in the vicinity of a landing candidate site (1). Figure 8 In the example, with Figure 6 Similarly, the area surrounding the landing candidate site (1) is hypothetically subdivided into intervals Bly, where each Bly interval represents the ease of airflow to ground objects. Furthermore, the Bly intervals where no ground objects are detected are scored, for example, according to user or system administrator settings. Figure 8 This example represents a user setting the ease of air circulation to the maximum value, which is "10". In Figure 8 In the example, based on the airflow ease of a specific ground object in each interval Bly, the impact on landing candidate site (1) is estimated. For example, the impact on landing candidate site (1) is calculated as the reciprocal of the sum of airflow ease. The smaller the size of interval Bly, the more accurate the specific airflow ease can be. On the other hand, as Figure 8 As shown, if the airflow ease is determined according to a certain size for each interval Bly, the noise generated when determining the airflow ease can be reduced, thereby improving the accuracy of estimating the impact on the landing candidate site (1). Furthermore, the airflow ease for each interval Bly can be determined by considering the wind speed and direction contained in the second sensing information. For example, the airflow ease determined based on the properties of ground objects can also be corrected according to the wind speed and direction of the interval Bly where the ground objects are located. For example, it can be corrected in such a way that the stronger the wind blowing towards the landing candidate site (1), the lower the airflow ease.

[0063] Furthermore, even for the same ground object, the ease of airflow can vary depending on the object's 3D dimensions and 3D shape. Therefore, the influence estimation unit 235 can estimate the impact on landing candidate sites based not only on the ground object's attributes detected by the ground object identification unit 234, but also on at least one of the ground object's 3D dimensions and 3D shape detected by the ground object identification unit 234. This improves the accuracy of estimating the impact on landing candidate sites. For example, the ease of airflow determined based on the ground object's attributes is corrected according to the ground object's 3D dimensions, and the influence is calculated such that the larger the sum of the corrected ease of airflow, the smaller the influence. For example, corrections can be made such that the higher the ground object, the lower the ease of airflow. Additionally, the ease of airflow determined based on the ground object's attributes is corrected according to the ground object's 3D shape, and the influence is calculated such that the larger the sum of the corrected ease of airflow, the smaller the influence. For example, corrections can be made based on the principle that the more curved (curved) an object on the ground is, the easier it is for air to circulate.

[0064] Furthermore, the impact estimation unit 235 can estimate the impact on the landing candidate site based on data representing the relationship between the attributes of ground objects and the difficulty of airflow through the ground objects, and the attributes of the ground objects detected by the ground object specific unit 234. Here, the difficulty of airflow can also be represented numerically, with higher values ​​for more difficult airflow. For example, in the data representing the relationship between the attributes of ground objects and the difficulty of airflow through the ground objects, the difficulty of airflow through walls is set to "10" points, the difficulty of airflow through trees is set to "5" points, and the difficulty of airflow through roads is set to "0" points. Moreover, the impact is calculated by considering the airflow difficulty of all ground objects (or one ground object) specifically existing around the landing candidate site, with the larger the sum of the specific airflow difficulties, the greater the impact. In this case, Figure 8 In the example, the impact on landing candidate site (1) is estimated based on the airflow difficulty of a specific ground object in each interval Bly. For example, the sum of airflow difficulties is used as the impact in landing candidate site (1). Furthermore, the airflow difficulty of each interval Bly can be specified by taking into account the wind speed and wind direction contained in the second sensing information. For example, the airflow difficulty specified based on the attributes of the ground object is corrected according to the wind speed and wind direction of the interval Bly in which the ground object exists. For example, it is corrected in such a way that the stronger the wind blowing towards landing candidate site (1), the higher the airflow difficulty.

[0065] Furthermore, similar to the ease of airflow, the impact estimation unit 235 can estimate the impact on the landing candidate site based on at least one of the three-dimensional dimensions and three-dimensional shape of the ground object detected by the ground object specification unit 234, in addition to the attributes of the ground object detected by the ground object specification unit 234. For example, the airflow difficulty specified based on the attributes of the ground object is corrected according to the three-dimensional dimensions of the ground object, and the greater the sum of the corrected airflow difficulties, the greater the impact. For example, the correction is performed in a way that the higher the height of the ground object, the greater the airflow difficulty. Additionally, the airflow difficulty specified based on the attributes of the ground object is corrected according to the three-dimensional shape of the ground object, and the greater the sum of the corrected airflow difficulties, the greater the impact. For example, the correction is performed in a way that the more curved the ground object, the lower the ease of airflow.

[0066] Furthermore, the data representing the relationship between the attributes of a ground object and the ease (or difficulty) of airflow over the ground object can also be data representing the relationship between the attributes of the ground object and the influence corresponding to the ease (or difficulty) of airflow over the ground object. In this case, the relationship between the ease (or difficulty) of airflow over the ground object and the influence is predetermined. Moreover, from the data, the influence corresponding to the attributes of the ground object detected by the ground object specific unit 234 is estimated as the landing influence on the landing candidate site.

[0067] The landing suitability determination unit 236 determines whether a candidate landing site is suitable for landing based on the impact level estimated by the impact level estimation unit 235. For example, if the impact level is below a threshold (in other words, the landing suitability is relatively high), the candidate landing site is determined to be suitable for landing. On the other hand, if the impact level is above the threshold (in other words, the landing suitability is relatively low), the candidate landing site is determined to be unsuitable for landing. Furthermore, when the impact level estimation unit 235 estimates the impact level of multiple candidate landing sites, the landing suitability determination unit 236 determines whether each candidate landing site is suitable for landing based on the impact level of each candidate landing site.

[0068] The landing site selection unit 237 selects a candidate landing site deemed suitable for landing by the landing suitability determination unit 236 as the landing site for the UAV1. Furthermore, if multiple candidate landing sites are deemed suitable, the candidate landing site with the lowest estimated impact (in other words, the highest landing suitability) from among these candidate sites can be selected as the landing site. This allows for the selection of a more suitable candidate landing site from among multiple candidate landing sites. Alternatively, the candidate landing site selected by the user of the UAV1 (e.g., an operator remotely controlling the UAV1) from among the multiple candidate landing sites deemed suitable for landing can also be selected as the landing site.

[0069] Information providing unit 238 provides (sends) landing target area information to UAV1 via communication unit 21. Additionally, information providing unit 238 provides UAV1 with landing candidate area information, indicating landing candidate areas specified by landing candidate area specifying unit 232, via communication unit 21. Furthermore, information providing unit 238 provides UAV1 with landing site information, indicating landing sites selected by landing site selection unit 237, via communication unit 21.

[0070] [2. Actions of Flight Management System S]

[0071] Next, refer to Figure 9 and Figure 10 The actions of the flight management system S will be explained. Figure 9 and Figure 10 This is a flowchart illustrating an example of the processing performed between UAV1 and management server 2 within the flight management system S. Figure 9 In step S1, the management server 2 sends the landing object area information, which includes the sensing command for the landing object area, to the UAV1 via the communication network NW.

[0072] Next, after UAV1 obtains (receives) the landing target area information from management server 2, it begins flying from its departure point towards the landing target area (step S2). Next, after UAV1 reaches the airspace above the landing target area (e.g., at an altitude of 30m) (step S3), it activates sensor unit 14 to begin sensing the area including the landing target area and obtains the first sensing information obtained from the sensing (step S4). This sensing can be performed continuously while UAV1 is moving or while hovering. Next, UAV1 transmits the first sensing information obtained in step S4 and UAV1's airframe ID to management server 2 via communication network NW (step S5).

[0073] Next, after the management server 2 obtains the first sensing information and the aircraft ID from the UAV1 through the sensing information acquisition unit 231, it determines a landing candidate area from the landing target area based on the first sensing information by the landing candidate area determination unit 232 (step S6). Next, the management server 2 sends the landing candidate area information containing the sensing command for the landing candidate area determined in step S7 to the UAV1 via the communication network NW (step S7).

[0074] Next, after obtaining the landing candidate area information from the management server 2, UAV1 moves towards the airspace above the landing candidate area while decreasing its altitude (step S8). Next, after UAV1 reaches the airspace above the landing candidate area (e.g., at an altitude of 10m) (step S9), the sensor unit 14 is activated to begin sensing the area including the landing candidate area, and the second sensing information obtained from the sensing is acquired (step S10). This sensing can be performed continuously by UAV1 while moving or while hovering. Next, UAV1 sends the second sensing information obtained in step S10 and UAV1's aircraft ID to the management server 2 via the communication network NW (step S11).

[0075] Next, after obtaining the second sensing information and the aircraft ID from the UAV1 through the sensing information acquisition unit 231, the management server 2 specifies a landing candidate site with the required 2D dimensions for landing of the UAV1 through the landing candidate site designation unit 233 (step S12). Alternatively, the required 2D dimensions for landing of the UAV1 can be set based on the corresponding aircraft dimensions established in the flight management database 221 with the aircraft ID of the UAV1. Next, the management server 2 selects one of the landing candidate sites specified in step S12 (step S13).

[0076] Next, based on the acquired second sensing information, the management server 2 detects ground objects around the landing candidate site selected in step S13 using the ground object identification unit 234 (step S14). Next, the management server 2 identifies the attributes of the ground objects detected in step S14 using the ground object identification unit 234 (step S15). At this time, at least one of the 3D dimensions and 3D shape of the ground objects detected in step S14 can also be identified.

[0077] Next, the management server 2 estimates the landing impact on the candidate landing site selected in step S13 using the impact estimation unit 235 (step S16). For example, the impact estimation unit 235 refers to data representing the relationship between the attributes of a ground object and the ease (or difficulty) of airflow over the ground object, establishes an airflow ease (or airflow difficulty) corresponding to the attributes of the ground object specified in step S15, and estimates the landing impact on the candidate landing site selected in step S13 according to a specific calculation formula based on the specified airflow ease (or airflow difficulty). The estimated impact is then mapped to the candidate landing site and stored.

[0078] Furthermore, if multiple ground objects are detected around the selected landing candidate site in step S13, the impact on the landing candidate site is estimated by summing the ease (or difficulty) of airflow for each ground object. In addition to the attributes of the ground objects specified in step S15, the impact estimation unit 235 can also estimate the impact of the landing candidate site on the landing based on at least one of the 3D dimensions and 3D shapes of the ground objects specified in step S15.

[0079] Next, based on the impact estimated in step S16, management server 2 determines, through landing suitability determination unit 236, whether the landing candidate site selected in step S13 is suitable for landing (step S17). If the landing candidate site is determined to be suitable for landing (step S17: YES), the information of the landing candidate site (e.g., location and 2D dimensions) is entered into the candidate list (step S18), and the process proceeds to step S19. On the other hand, if the landing candidate site is determined to be unsuitable for landing (step S17: NO), the process proceeds to step S19.

[0080] Next, in Figure 10 In step S19, the management server 2 determines whether there are any unselected landing candidate locations among the landing candidate locations specified in step S12. If it is determined that there are unselected landing candidate locations (step S19: Yes), the process returns to step S13, selects an unselected landing candidate location, and proceeds to step S14 as described above. On the other hand, if it is determined that there are no unselected landing candidate locations (step S19: No), the process proceeds to step S20.

[0081] In step S20, the management server 2 determines whether a landing waitlist is logged in the waitlist. If it is determined that no landing waitlist is logged in the waitlist (step S20: No), the process ends. In this case, other landing waitlist areas may be specified, and the same process as described above may be performed. On the other hand, if it is determined that a landing waitlist is logged in the waitlist (step S20: Yes), the process proceeds to step S21.

[0082] In step S21, the management server 2 selects a landing site for UAV1 from the waiting list of waiting sites registered by the landing site selection unit 237. For example, if one landing site is registered in the waiting list, that landing site is set as the landing site for UAV1. On the other hand, if multiple landing sites are registered in the waiting list, the landing site with the least impact as estimated in step S16 is selected as the landing site for UAV1 from among the multiple landing sites.

[0083] Furthermore, the management server 2 can also send 3D map data, representing the locations of the multiple landing candidate sites registered in the candidate list and the surrounding ground objects, to the terminal of the user using the UAV1 (e.g., an operator remotely controlling the UAV1). In this case, the 3D map representing the locations of the multiple landing candidate sites registered in the candidate list and the surrounding ground objects is displayed on the user's terminal. If the user specifies a desired landing candidate site from the multiple landing candidate sites shown on the 3D map, information representing the specified landing candidate site is sent from the user's terminal to the management server 2. The landing candidate site specified by the user is then selected as the landing site for the UAV1 by the landing site selection unit 237.

[0084] Next, management server 2 sends the landing site information, representing the landing site selected in step S21, to UAV1 via the communication network NW (step S22). Furthermore, in step S21, management server 2 may also designate ground objects detected in step S14 as obstacles and determine a landing method corresponding to the configuration of the obstacles. In this case, management server 2 sends the landing site information along with landing method information representing the determined landing method to UAV1 via the communication network NW.

[0085] Next, after obtaining the landing site information from the management server 2, UAV1 performs landing control on the landing site shown in the landing site information (step S23). Furthermore, after obtaining the landing site information and landing method information from the management server 2, UAV1 performs landing control according to the landing method corresponding to the configuration of obstacles surrounding the landing site shown in the landing site information. Afterwards, UAV1 returns to its starting point, for example.

[0086] As explained above, according to the first embodiment, since the management server 2 is configured to detect ground objects around the landing candidate site of the UAV1, it estimates the impact of the downdraft generated when the UAV1 lands on the rebound of the ground objects, which is the impact on the landing, i.e., the impact in the landing candidate site. Based on the estimated impact, it determines whether the landing candidate site is suitable for landing, so it is possible to select an appropriate landing site that takes into account the impact of the downdraft rebounding from the ground objects (i.e., a landing site with less downdraft impact). In other words, even if the landing candidate site is surrounded by ground objects and the distance to the ground objects is relatively close, if the downdraft impact is small, then the site can be selected as the landing site. For example, although in the past it was impossible to land if the distance to the ground objects was too close, according to this embodiment, if the ground objects are trees or metal mesh fences with good ventilation, then it can be determined that it is a suitable landing site, even if the two-dimensional dimensions of the landing candidate site are narrow. Furthermore, according to the described embodiment, even when there are ground objects in the landing area, an appropriate landing plan can be selected in addition to a safer landing site.

[0087] Furthermore, the described embodiment is one embodiment of the present invention, and the present invention is not limited to the described embodiment. Various modifications, such as changes to the configuration, can be made from the described embodiment without departing from the spirit of the present invention, and such modifications are also included within the technical scope of the present invention. In the described embodiment, although the management server 2 is configured to detect ground objects existing around the landing candidate site of the UAV1, estimate the impact of the downdraft generated when the UAV1 lands on the ground objects and their rebound, and perform processing based on the estimated impact to determine whether the landing candidate site is suitable for landing, the processing can also be performed by the UAV1. In the case, the control unit 16 of the UAV1 functions as the landing candidate area specifying unit 232, the landing candidate site specifying unit 233, the ground object specifying unit 234, the impact estimation unit 235, the landing suitability determination unit 236, and the landing site selection unit 237. Figure 9 Steps S6, S12 to S18, and... are shown. Figure 10The processing steps S19 to S21 shown are performed by the control unit 16. Alternatively, in this case, the semantic segmentation model α and semantic segmentation model β may be pre-stored in the storage unit 15.

[0088] Furthermore, in the described embodiment, by sensing the area including the target landing area and the area including the candidate landing area during flight of a UAV1 configured for a predetermined landing, the accuracy of estimating the impact of landing on candidate landing sites can be improved because sensing information obtained just before landing of the UAV1 can be used. However, the sensing can also be performed in advance by an aircraft other than the UAV1 intended for landing (e.g., before the UAV1 begins flight). In this case, the first sensing information is mapped to the target landing area information and stored in a database, and the second sensing information is mapped to the candidate landing area information and stored in a database. The first sensing information and the second sensing information are retrieved from the database by the management server 2 or the UAV1. Additionally, although this embodiment is described using a UAV as an example of an unmanned aerial vehicle, the present invention can also be applied to other types of flying robots, such as UAVs.

[0089] [Explanation of Symbols]

[0090] 1 UAV

[0091] 2. Management Server

[0092] 11 Drive Unit

[0093] 12 Positioning Unit

[0094] 13 Ministry of Communications

[0095] 14. Sensor Department

[0096] 15. Storage Department

[0097] 16 Control Department

[0098] 21 Ministry of Communications

[0099] 22 Storage Department

[0100] 23 Control Department

[0101] 231 Sensing Information Acquisition Unit

[0102] 232 Landing Candidate Area Specific Unit

[0103] 233 Landing Reserve Site Specific Department

[0104] 234 Specific parts of objects on the ground

[0105] 235. Estimated Influence

[0106] 236 Landing Suitability Assessment Department

[0107] 237 Landing Site Selection Department

[0108] 238 Information Provision Department

[0109] S Flight Management System.

Claims

1. An information processing device, characterized in that... have: Testing agencies inspect ground objects around the landing sites for unmanned aerial vehicles (UAVs); The presumption mechanism presumes the impact of the wind blown down by the unmanned aerial vehicle during landing on the rebound of the ground objects on the landing, which is the impact of the alternative landing site. and The determining body, based on the degree of impact estimated by the prescribing body, determines whether the alternative landing site is suitable for landing; The estimation mechanism estimates the degree of influence based on data representing the relationship between the properties of ground objects and the ease or difficulty of airflow over the ground objects, and the properties of the ground objects detected by the detection mechanism.

2. The information processing apparatus according to claim 1, wherein the estimation mechanism estimates the influence degree based on the ease or difficulty of air circulation of a specific ground object in each location where the surrounding area of ​​the landing candidate site is hypothetically subdivided into specific first intervals.

3. The information processing apparatus according to claim 1 or 2, wherein the estimation mechanism, in addition to the attributes of the ground object detected by the detection mechanism, estimates the influence degree based on at least one of the three-dimensional dimensions and three-dimensional shape of the ground object detected by the detection mechanism.

4. The information processing apparatus according to claim 1 or 2, wherein the landing waiting area is a location that hypothetically subdivides the landing waiting area into specific second intervals.

5. The information processing apparatus according to claim 4, wherein the estimation mechanism estimates the impact degree according to each of the plurality of said landing candidate sites. The determining body determines whether each of the landing candidate sites is suitable for landing based on the degree of influence of each of the landing candidate sites.

6. The information processing apparatus according to claim 4 further comprises a specific mechanism for selecting a flat area of ​​at least no obstacles and a specific size or larger from a preset landing target area as the landing candidate area.

7. The information processing apparatus according to claim 4 further comprises a specific mechanism for selecting a predetermined landing target area, a specific flat area that is at least free of obstacles and of a specific size or larger, and an area whose surface properties are suitable for landing as the landing candidate area.

8. The information processing apparatus according to claim 1 or 2, wherein the detection mechanism detects ground objects present in the vicinity of the landing candidate site based on sensing information obtained by the unmanned aerial vehicle sensing the vicinity of the landing candidate site.

9. A method for determining landing suitability, characterized in that... Performed by one or more computers, and includes the following steps: Detect ground objects around the landing candidate site for unmanned aerial vehicles; The impact of the wind blown down by the unmanned aerial vehicle during landing on the rebound of the ground objects is estimated to have an impact on the landing, which is the impact in the landing candidate site. and Based on the estimated impact level, it is determined whether the proposed landing site is suitable for landing; The estimation step estimates the influence based on data representing the relationship between the properties of ground objects and the ease or difficulty of airflow over the ground objects, and the properties of the ground objects detected by the detection step.

10. A computer-readable storage medium, characterized in that... It stores a computer program that, when executed by a computer, performs the following steps: Detect ground objects around the landing candidate site for unmanned aerial vehicles; The impact of the wind blown down by the unmanned aerial vehicle during landing on the rebound of the ground objects is estimated to have an impact on the landing, which is the impact in the landing candidate site. and Based on the estimated impact level, it is determined whether the proposed landing site is suitable for landing; The estimation step estimates the influence based on data representing the relationship between the properties of ground objects and the ease or difficulty of airflow over the ground objects, and the properties of the ground objects detected by the detection step.

Images