Information processing method, information processing device, and computer program

Abstract

To improve the energy efficiency of a flying body capable of flying in a three-dimensional space. An information processing method according to the present disclosure generates, on the basis of map data indicating a flight environment of a flying body capable of flying in a three-dimensional space, a plurality of candidate routes that are candidates for a route in which the flying body flies, and at least one of the plurality of candidate routes includes a route in which the flying body flies in a direction including a component in a direction opposite to the direction of gravity.

Description

Information processing method, information processing device, and computer program

[0001] The present disclosure relates to an information processing method, an information processing device, and a computer program.

[0002] The use of drones has been increasing in recent years. There is demand for drones to inspect industrial infrastructure and for aerial photography. Accordingly, drones equipped with autonomous flight functions, such as autonomously avoiding obstacles to carry out transportation and inspection tasks and autonomously returning when the battery is low, are emerging. Since the battery life of drones is the limiting factor in their operating time, there is a demand for maximizing energy efficiency.

[0003] There are methods for finding the shortest route for drones using route planning technology to maximize energy efficiency. However, the shortest route found by conventional route planning technology does not fully consider factors other than route length, and does not necessarily match the optimal energy efficiency that takes into account the characteristics of the drone.

[0004] International Publication No. 2023 / 112242

[0005] The present disclosure provides an information processing method, an information processing device, and a computer program that improve the energy efficiency of an aircraft capable of flying in three-dimensional space.

[0006] The information processing method disclosed herein generates multiple candidate routes that are candidates for routes that the aircraft will fly based on map data that shows the flight environment of an aircraft capable of flying in three-dimensional space, and at least one of the multiple candidate routes includes a route in which the aircraft will fly in a direction that includes a component in the opposite direction to the direction of gravity.

[0007] 1 is a diagram showing a communication system according to an embodiment of the present disclosure. FIG. 1 is a block diagram of a drone and an operating device. FIG. 2 is a diagram showing an example of power consumption characteristics. FIG. 3 is a diagram showing an example of a plurality of paths for avoiding an obstacle. FIG. 4 is a diagram showing an example of energy consumption. FIG. 5 is a diagram showing an example of a display including an obstacle. FIG. 6 is a diagram showing another example of a display including an obstacle. FIG. 7 is a flowchart of an operation example of Example 2. FIG. 8 is an explanatory diagram of a specific example of Example 2. FIG. 9 is a flowchart of an operation example of Example 3. FIG. 10 is an explanatory diagram of a specific example of Example 3. FIG. 11 is a flowchart of an operation example of Example 4. FIG. 12 is an explanatory diagram of a specific example of Example 4. FIG. 13 is a flowchart of an operation example of Example 5. FIG. 14 is an explanatory diagram of a specific example of Example 5. FIG. 15 is an explanatory diagram of a specific example of Example 5.

[0008] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In one or more embodiments shown in the present disclosure, elements included in each embodiment can be combined with each other, and the combined result also forms part of the embodiment shown in the present disclosure.

[0009] FIG. 1 illustrates a communication system according to an embodiment of the present disclosure. The communication system of FIG. 1 includes a drone 10, which is an aircraft capable of flying in three-dimensional space, and an operation device 20 that allows a user 30, who is the operator of the drone 10, to give various instructions to the drone 10. The drone 10 is capable of autonomous flight in any three-dimensional environment, such as a mountain forest, a factory, a news reporting location, a stadium, or a neighborhood. The drone 10 is equipped with a camera 11 that can capture images of the environment during flight. The aircraft capable of flying in three-dimensional space is not limited to a drone, and may be a manned aircraft such as a helicopter.

[0010] The operation device 20 can be operated by the user 30. The operation device 20 is a control device that gives various instructions to the drone 10 or performs various settings on the drone 10. The operation device 20 is an operation tablet, a remote control system, a simulation terminal, or the like. The operation device 20 includes a display unit 22 that displays images to the user 30, and a stick (instruction unit) 21 through which the user 30 inputs various instructions. The display unit 22 displays interface images (e.g., various menu screens) for the user 30 to give various instructions, map data showing the real-time flight environment while the drone 10 is flying, location information showing the current position of the drone 10 during flight, video data (images) captured by the drone 10, and the like.

[0011] The drone 10 flies autonomously in three-dimensional space, but may be manually controlled by a user 30. In this case, the flight of the drone 10 may be controlled by the operation device 20. Furthermore, the direction or zoom mechanism of the photographing camera 107 of the drone 10 may be controlled by the operation device 20.

[0012] It is assumed that the drone 10 is directly connected to and communicates with the control device 20 wirelessly, but a base station may be interposed between the drone 10 and the control device 20. In this case, communication between the control device 20 and the drone 10 is performed via the base station. Some of the functions of the control device 20 may be provided in the base station.

[0013] 2 is a block diagram of the drone 10 and the operation device 20. [Drone 10] The drone 10 includes an environment acquisition unit 101, a communication unit 102, a power consumption characteristics storage unit 103, a flight control unit 104, an action planning unit 105, a drive system 106, and a photographing camera 107. All or at least some of the functions of blocks 101 to 107 may be realized by causing a computer including a processor such as a CPU (Central Processing Unit) to execute a program, or may be realized by circuits such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array), or may be realized by a combination of these.

[0014] The communication unit 102 wirelessly communicates with the communication unit 206 of the control device 20. When the drone 10 transmits and receives information to and from the control device 20 via a base station, the communication unit 102 wirelessly communicates with the base station.

[0015] The environment acquisition unit 101 acquires information about the surrounding environment of the space in which the drone 10 flies. For example, it acquires map data (which may be three-dimensional topographical information (including contour lines)) from a storage unit (not shown) within the drone 10 or an external server. It also acquires detection information from various sensors equipped in the drone 10 and understands the surrounding environment from the detection information. The various sensors include a photographing camera 107, a GPS (Global Positioning Satellite) sensor, and an IMU (Inertial Measurement Unit). The environment acquisition unit 101 analyzes the surrounding environment and acquires more detailed environmental information. For example, this information includes the drone 10's own position, information about surrounding three-dimensional objects (occupancy information), and information about obstacle objects (obstacle information). The three-dimensional object information (occupancy information) includes semantic information (information about the type of area), such as forest, building, or river. Analyzing the information about the surrounding environment also includes analyzing camera images using semantic segmentation.

[0016] The aircraft information storage unit 108 stores information on the specifications of the drone 10. For example, this information includes the ID, weight, and shape of the drone's own aircraft, the ID, capacity, and remaining amount of the installed battery, the ID and type of various installed sensors, and the ID, type, and location of accessories such as propellers.

[0017] The power consumption characteristics storage unit 103 stores information about the power consumption characteristics of the aircraft. The power consumption characteristics may be highly accurate characteristics obtained by examining the aircraft's own characteristics, or may be based on general aircraft characteristics. Examples are shown below. There are power consumption characteristics based on aircraft movement, which represent the relationship between power consumption according to the aircraft's speed, acceleration, angular velocity, angular acceleration, attitude velocity, and other movements based on the aircraft's position and attitude when flying in three-dimensional space. Here, the position and attitude movements may be based on the results of actual aircraft movement or target values ​​such as a target route or trajectory generated in action planning or control. There are power consumption characteristics based on ascending flight, which represent the loss due to the movement component in the direction of gravity when flying in three-dimensional space. There are power consumption characteristics based on flight (including hovering), which represent the power required for flight. These characteristics relate to the minimum power consumed when flying, hovering, or moving. There are power consumption characteristics based on physical characteristics such as the weight, size, and shape of the aircraft and accessories. For example, there is the relationship between weight, size, and shape and the aircraft's energy consumption. A heavy drone consumes more power than a lighter one, and a large drone with the same weight consumes more power than a small one due to the greater air resistance. ・Power consumption characteristics are based on the physical characteristics of the object being transported by the drone, such as the weight, size, and shape. ・Power consumption characteristics are based on the aircraft's inherent characteristics, such as wear and tear and deterioration over time. As the drone ages, the performance of the drive system 106 declines, resulting in increased power consumption.

[0018] The various power consumption characteristics described above may be represented by a generalized simplified model, or may be represented by an optimized model that reflects the detailed power consumption characteristics of the target aircraft that will actually fly.

[0019] FIG. 3 shows an example of power consumption characteristics.

[0020] Figure 3(A) shows the relationship between speed and energy consumption, showing that there are speeds at which energy efficiency is high depending on the aircraft.

[0021] Figure 3(B) shows the relationship between energy consumption and altitude. The energy consumption due to movement in the direction opposite to gravity during flight in three-dimensional space is shown for each weight of the drone (including the weight of the drone itself, accessories, etc.). Movement in the direction opposite to gravity requires a large amount of energy consumption, and increasing altitude with a heavy weight results in even greater energy consumption. The characteristics related to energy consumption shown in Figure 3(B) include the relationship between the distance traveled (ascended) in the direction opposite to the direction of gravity and the energy consumption. Alternatively, the characteristics related to energy consumption shown in Figure 3(B) include the relationship between the distance traveled (ascended) in the direction opposite to the direction of gravity, the energy consumption, and the weight of the drone. Note that the relationship between energy consumption and altitude may also be defined for movement in the direction of gravity.

[0022] Figure 3(C) shows the relationship between the change in velocity and energy consumption. The velocity change is based on the velocity change Δv = v1 - v0, where v0 and v1 are the velocity vectors in three-dimensional space at times t0 and t1. For example, the energy consumption required to move from t0 to t1 can be expressed based on the impulse received by the center of gravity between t0 and t1. The velocity change includes changes in direction and acceleration / deceleration. Furthermore, in addition to simple velocity change, the direction of gravity and plane components perpendicular to gravity can be treated separately. For example, this can be expressed as the total change in the angle of change in direction per unit time, calculated from the rate of change of direction and the rate of change of velocity in a plane perpendicular to gravity. This shows that the more changes in direction, the greater the energy consumption. Similarly, the characteristic of greater energy consumption during acceleration and deceleration is shown. Furthermore, this can be based not only on the change in velocity (acceleration), but also on changes in angular velocity and attitude velocity. This includes the characteristic that less attitude change reduces energy consumption, not just velocity change.

[0023] FIG. 4 shows an example of a route P01 that avoids a building (obstacle) B from above when moving from point A to point B, and a route P02 that avoids the building (obstacle) B from the side (changing course).

[0024] Figure 5 shows an example of the energy consumption when the drone 10 flies along route P1 (Figure 5(A)) and when the drone flies along route P02 (Figure 5(B)). When the drone 10 ascends and when it changes course, it consumes more energy than usual. When ascending, the energy consumption is greater than when it changes course.

[0025] The drone 10 consumes energy required to stay in the air while flying (the example in the figure assumes the energy consumption required when flying straight at a certain speed), and additional energy consumption is required when ascending or changing course.

[0026] Route P01 has a shorter flight distance to point B than route P02. With route P01, the drone 10 arrives at point B earlier, so the energy consumption required to stay in the air is smaller, but the energy consumption required to climb is greater. With route P02, the drone 10 arrives at point B later than with route P01, so the energy consumption required to stay in the air is greater, but the additional energy consumption caused by changing course is smaller. As a result, in this example, route P02, which avoids the obstacle from the side, consumes less total energy than route P01, which avoids the obstacle from above. In other words, there is a trade-off between arriving at the destination earlier and energy consumption.

[0027] The behavior planning unit 105 acquires information about the movement task of the drone 10 from the control device 20, and based on the movement task, performs a path plan or trajectory plan taking into account power consumption characteristics to generate route or trajectory candidates and determine the route or trajectory to be used from the candidates. A route is a sequence of positions (three-dimensional positions) passed through from a departure point to a destination. A trajectory is a route to which information capable of identifying the time at which each position included in the route should be passed may be added. The information capable of identifying the time may be the time itself, the speed at which the drone 10 passes the position, or other types of information. In other words, a trajectory includes a path and additional information, so generating a trajectory encompasses generating a route. The following explanation assumes the case of generating a route. In the case of a trajectory, the term "route" in the following explanation may be replaced with "trajectory" as appropriate. For example, "route characteristics" may be replaced with "trajectory characteristics."

[0028] The behavior planning unit 105 includes a characteristic acquisition unit 111 , a path / trajectory planning unit 112 , and an energy estimation unit 113 .

[0029] The characteristics acquisition unit 111 acquires information on the power consumption characteristics of the device from the power consumption characteristics storage unit 103 .

[0030] The route / trajectory planning unit 112 generates route candidates (route candidates) for accomplishing the movement task based on the power consumption characteristics of the aircraft and the energy consumption required to fly the route. The energy consumption is calculated using the energy estimation unit 113. The section (part or all of the route) to be calculated is specified in the energy estimation unit 113 to obtain information on the energy consumption. One or more route candidates are generated. Information on the one or more generated candidates may be sent to the operation device 20 and presented to the user 30. A candidate selected by the user 30 from the presented candidates may be adopted.

[0031] The user 30 may specify one or more types of target conditions (modes) for generating route candidates in advance. When there is one target condition and only one candidate satisfying this target condition is generated, the generated candidate may be unconditionally adopted as the route to be used. When there are multiple target conditions and each candidate is generated, the candidates may be evaluated and compared to determine the route to be used. The user may be prompted to determine the route to be used. When there is one target condition and multiple candidates are generated, the candidates may be evaluated and compared to determine the route to be used. The user may be prompted to determine the route to be used. The path / trajectory planner 112 may be based on the processing of conventional path planning techniques such as A* and RRT*. The path / trajectory planner 112 may be based on the processing of conventional trajectory planning techniques such as MPC. The base method for path planning or trajectory planning is not limited.

[0032] Examples of goal conditions include selecting the candidate with the lowest or lowest possible energy consumption, or selecting the candidate that will arrive at the destination the fastest or as quickly as possible. Conditions for how to avoid obstacles when they exist include "side avoidance priority" and "overhead avoidance priority." "Side avoidance priority" means that if there is an obstacle in the drone's direction of travel (forward), the drone prioritizes turning left or right to avoid the obstacle, while "overhead avoidance priority" means that if there is an obstacle in the drone's direction of travel (forward), the drone prioritizes climbing higher than the obstacle to avoid it. Side avoidance typically requires less power than overhead avoidance. Side avoidance, overhead avoidance, etc. can be defined as costs (values) and included as parameters in the function (evaluation function) that evaluates the route.

[0033] The evaluation function may also include other parameters such as energy consumption, arrival time, flight distance, and flight environment (mountainous area, residential area, over a river, etc.). A route may be determined so as to optimize or sub-optimize the output value (score) of the evaluation function. For example, if a low score is desired, one or more routes with low scores may be selected as candidates. By using side dodging, overhead dodging, etc. as parameters with costs (values), it is possible to make it easier to avoid sideways or overhead. The evaluation function may be generated in advance by machine learning. The evaluation function is not limited to a specific type of function. The evaluation function may be a regression function, a neural network, or the like.

[0034] The energy estimation unit 113 estimates the energy consumption required to fly the section specified by the route / trajectory planning unit 112. The specified section may be any section, or may be a portion of the entire section from the departure point to the destination. Information on the estimated energy consumption is used by the route / trajectory planning unit 112 to determine one or more flight paths. When estimating the energy consumption, for example, the energy consumption may be calculated based on an assumption that the drone 10 flies the route at a constant speed and the power consumption characteristics of the aircraft. When estimating the energy consumption of a trajectory, the energy consumption may be calculated based on the power consumption characteristics of the aircraft, assuming that the drone 10 actually passes each position at each time. This will be described in more detail below.

[0035] The energy estimation unit 113 has a model that reflects the power consumption characteristics of the aircraft and calculates the energy required for the drone 10 to fly a specified section. As an example, the model receives route information for the section and the drone's flight conditions as input, and outputs the energy consumed by the drone 10 when flying that section. The model may be an approximation model that combines physical estimation models, a table based on actual measurements, or an estimation model using machine learning. The drone's flight conditions may be the speed and attitude at each position. For example, when flying a drone along a certain route, the drone's attitude and speed at each position may be calculated, and the attitude and speed at each position may be used as flight conditions. The attitude and speed at each position may be assumed values, or when calculating a trajectory, the speed at each position included in the trajectory may be used to calculate the attitude required at each position.

[0036] The path / trajectory planning unit 112 may perform path planning using the consumed energy estimated by the energy estimation unit 113 as a cost during the process of path planning, such as A* or RRT*. In the path planning, a score of the path may be calculated. Furthermore, the path / trajectory planning unit 112 may perform trajectory planning using the consumed energy estimated by the energy estimation unit 113 as a cost during the process of trajectory planning, such as MPC. In the trajectory planning, a score of the trajectory may be calculated.

[0037] (Example of Model of Energy Consumption in a Section) The model held by the energy estimation unit 113 is represented by M. The following equation (1) is used to estimate the energy consumption E required when flying from a certain attitude (represented as ps) at a certain position of the aircraft to a certain attitude (represented as pe) at another position at a speed vi. E = f(ps, pe, vi, M) (1)

[0038] The following equation (2) is an estimation formula for the consumed energy E required for flying from the initial state to the final state when a sequence of target positions and attitudes is input. p includes the position (x, y, z) and attitude (rx, ry, rz) of the aircraft as six parameters. T includes a sequence of p0 to pT as a sequence of p values. The formula for consumed energy E includes T and M as arguments. As in the case of equation (1), M represents the model held by the energy estimation unit 113. p=[x, y, z, rx, ry, rz] T= [p0, p1, ..., pT] E= f(T, M) (2)

[0039] (Improving Accuracy of Energy Estimation Unit 113) The energy estimation unit 113 may perform more accurate energy estimation by updating the power consumption characteristics of the aircraft during or before the flight of the drone 10.

[0040] Data related to the power consumption characteristics of the aircraft from past flight history may be stored in the aircraft information storage unit 108 or another storage unit, and the energy estimation unit 113 may use this data to update the model that estimates energy consumption. For example, if the model is an approximation model that combines physical estimation models, the latest data is used to correct the internal coefficient parameters. Alternatively, if the model is a table based on actual measurements, the table is updated to increase the weight of the latest data. If the model is an estimation model using machine learning, the latest data is used to perform re-learning. In this way, by updating the model, losses due to individual differences and aging, and the effects of the size and shape of the item being transported, etc. are corrected, allowing for more accurate energy estimation.

[0041] The flight control unit 104 generates control command values ​​necessary to fly the drone 10 according to the route determined by the action planning unit 105 (e.g., a route selected by the user 30 from multiple route candidates). For example, the flight control unit 104 calculates the attitude, speed, etc. of the drone 10 and generates the control command values. The flight control unit 104 controls the flight of the drone 10 by outputting the generated control command values ​​to the drive system 106. This allows the drone 10 to fly autonomously to its destination.

[0042] [Operation Device 20] The operation device 20 includes a candidate presentation unit 201, a selection unit 202, a route specification presentation unit 203, a setting unit 204, a control unit 205, a communication unit 206, a display unit 207, and an instruction unit 208. All or at least some of the functions of blocks 201 to 208 may be realized by causing a computer including a processor such as a CPU (Central Processing Unit) to execute a program, or may be realized by a circuit such as an ASIC or FPGA, or may be realized by a combination of these.

[0043] The communication unit 206 communicates wirelessly with the communication unit 102 of the drone 10. When the control device 20 transmits and receives information to and from the drone 10 via a base station, the communication unit 206 communicates wirelessly with the base station.

[0044] The instruction unit 208 is an interface for the user 30 to input various instructions or data, and corresponds to the instruction unit 21 in FIG.

[0045] The display unit 207 displays information or data on a screen for the user 30. The display unit 207 corresponds to the display unit 22 in FIG. 1. The display unit 207 is a display capable of displaying data or information, and is, for example, a liquid crystal display panel or an organic EL panel. The display unit 207 may be a touch panel. In this case, the instruction unit 208 is included in the display unit 207.

[0046] The setting unit 204 sets information about the departure point and destination, and one or more target conditions (modes) for generating route candidates based on input from the user 30. Examples of target conditions include a target condition for minimizing or optimizing energy consumption, a target condition for prioritizing avoiding obstacles by going around them from the left or right rather than ascending, which consumes a lot of power (power consumption for descent may be considered in addition to ascending), a target condition for considering safety (for example, a target condition for prioritizing flying over areas unlikely to be traversed by people (high-priority areas), such as over rivers or mountainous areas, over areas likely to be traversed by people), a target condition for arriving at the destination in a short time, a target condition for shortening the flight time or flight distance to the destination, and a target condition for minimizing the load on the aircraft or its components. The setting unit 204 sends information about the departure point, destination, and target conditions to the drone 10 via the communication unit 102. This information corresponds to the movement task. A configuration is also possible in which the user 30 does not input the target conditions. In this case, the drone 10 may determine the target conditions as default conditions. When there is cargo to be transported by the drone 10, the setting unit 204 may set information about the cargo (carried item) based on input from the user 30 and transmit the information to the drone 10. The information about the transported item may include, for example, at least one of the size, shape, and weight of the transported item.

[0047] The candidate presentation unit 201 acquires information on one or more route candidates generated by the drone 10 based on the one or more target conditions, and displays the route candidates on the display unit 207. In this way, the route candidates are presented to the user 30. For example, the route candidates are displayed superimposed on map data.

[0048] The route identification and presentation unit 203 acquires, from the drone 10, information on the route characteristics of the one or more route candidates generated by the drone 10, associates the information with the route candidates, and displays the information on the display unit 207. In this way, the route characteristics of each candidate are presented to the user 30. The information on the route candidates and the information on the route characteristics may be acquired from the drone 10 simultaneously, or may be acquired at different times.

[0049] Examples of route characteristics are listed below. Information on energy consumption (Wh) or power consumption (W). Examples include total energy consumption, energy consumption per section, percentage of energy consumption relative to battery capacity, and energy consumption (power consumption) per unit time. Information on the safety of the drone or protected object. Safety includes safety for the drone itself, such as from collisions, and safety for the protected object, such as from contact with the protected object or from the drone colliding with the protected object when it falls. Protected objects include humans and non-human objects. A route with many obstacles may increase the risk of the drone 10 being damaged by contact with the obstacle (reducing the safety of the drone). Furthermore, adopting a route that does not pass over people increases human safety. Information on the load on the drone. For example, the load on the drone (risk of damage) may be low because there are few or no obstacles to avoid, or the load on the drone may be low because there is no need to travel to a high altitude. Alternatively, the load may be quantified. Load information may be values ​​for each component (motor, propeller, battery, etc.). - There is the time to arrive at the destination (estimated time). - There are aircraft characteristics such as the remaining battery level at departure, the remaining battery level at arrival, and aircraft size. - There is the shortest distance to the destination (the distance if the aircraft were to fly ideally along the route). - There is the shortest distance to an obstacle. - There is a mark or string of characters that suggests low energy consumption. An example of a mark is a mark indicating that the aircraft is eco-friendly, and an example of a string of characters is "route with minimal energy consumption."

[0050] Specific examples of information regarding the safety of the drone or object include: information regarding objects that the drone may collide with, and the location or object that the drone may fall into if it falls. For example, information on trees, tree branches, wires, steel towers, buildings, cliffs, etc.; information regarding objects to be protected (such as people) if the drone comes into contact with them or falls onto them; information on the presence or passage of people (roads, houses, etc.); information on places such as rivers, lakes, oceans, and forests where people may be present for swimming or boating; information on areas on a map; information visualizing areas such as rivers, forests, and places where people walk; information regarding operational status related to safety; information on whether the drone 10 is within or outside the user's line of sight (information on whether the user 30 can see the drone 10), communication conditions (information on whether the radio wave environment is good or bad, or information on electromagnetic wave strength, etc.), temperature, wind, etc.; and information on each section of the route.

[0051] The selection unit 202 selects a route candidate based on an input from the user 30, with reference to route characteristics displayed in association with one or more route candidates, and inputs information indicating the selected route candidate. The selection unit 202 sends selection result information indicating the selected route candidate to the drone 10.

[0052] The behavior planning unit 105 of the drone 10 identifies the route candidate selected by the user 30 based on the selection result information and determines it as the route to be used for actual flight (target route). The behavior planning unit 105 or the route / trajectory planning unit 112 generates flight control command values ​​based on this target route and sends them to the flight control unit 104.

[0053] Examples of generating safe route candidates are shown below. - Generating route candidates that avoid passing over areas where people or other objects to be protected exist as much as possible (for example, generating a route that prioritizes passing over a river or forest). - Generating route candidates that are unlikely to encounter objects to be protected (such as people) even if the drone falls (for example, generating a route candidate that passes over the sea). - Generating route candidates that make it easy to recover the drone 10 in the event of a fall (for example, generating a route candidate that passes over a cliff, or generating a route candidate that passes over a forest within a certain distance from the road). - Generating route candidates that avoid trees as much as possible. - Generating route candidates that avoid high-voltage power line locations as much as possible. - Generating route candidates that maintain a distance from obstacles that is greater than or equal to a certain margin. - Generating route candidates that do not fall out of the user's line of sight or that minimize the time that the drone 10 is out of the user's line of sight. In this case, the drone 10 acquires location information of the control device 20 (location information of the user 30) from the control device 20. - Generating route candidates that ensure communication between the control device 20 and the drone 10 is as uninterrupted as possible. For example, generating route candidates that maintain a certain level of received power strength or higher as much as possible. - Route candidates are generated so as to avoid as many strong magnetic field locations (locations where observation noise tends to be large) as possible. - Route candidates are generated so as to avoid as many locations as possible where it is difficult to receive GPS signals. - No-entry areas are set, and route candidates are generated so as not to pass through the no-entry areas. The above-mentioned locations where it is difficult to receive GPS signals, etc. may be set as no-entry areas.

[0054] An example of generating route candidates with low load on the aircraft is shown below.

[0055] A route candidate is generated for flight that optimizes the load on the drone 10. The load on the drone can be either the load on the drone itself or the load on components (motor, propeller, battery, etc.) mounted on the drone. The load is calculated assuming that the drone 10 moves along the candidate route, and may be calculated based on general load characteristics or more precisely based on the drone's own load characteristics. The route candidate may be determined by comprehensively assessing the usage status and lifespan of each component.

[0056] Specifically, route candidates may be generated that minimize the rate of change in motor or propeller load (i.e., suppress the frequency or total amount of acceleration / deceleration and course changes). Generally, avoiding sudden acceleration / deceleration or changes in direction is gentler on most parts. The smaller the integral value of the angle of the direction of travel during movement along a route, the smaller the load, and this integral value may be used as the load.

[0057] Route candidates may be generated so as to minimize the maximum load on the motor or propeller (for example, to avoid centrifugal force caused by turning at high speed). For this reason, an upper limit may be added as a constraint to the maximum speed of the drone 10. Furthermore, when generating route candidates that include return, route candidates are generated so that the drone can return with the remaining battery charge above a threshold. For example, candidates that allow the drone to return with an SOC (State Of Charge) of 20% or 30% or higher are generated.

[0058] Specific examples of the operation of this embodiment will be described below as Examples 1 to 5.

[0059] [Example 1] In a travel task from a departure point to a destination, a route is generated based on a target condition, with priority given to sideways maneuvers. The route can be re-planned by presenting generated route candidates to the user 30 before departure as well as during flight, prompting the user 30 to select one.

[0060] (Before Flight) The user 30 can select and set a "top avoidance priority" or "side avoidance priority" mode as the target condition on the operation device 20. The user 30 empirically believes that when the drone 10 avoids an obstacle, avoiding it to the side consumes less battery power than avoiding it from above, and so sets the target condition to "side avoidance priority" mode. Information on the mode (target condition) set by the user 30 is transmitted to the drone 10. The behavior planning unit 105 of the drone 10 generates a route to the destination that avoids obstacles to the side as much as possible. The flight control unit 104 generates control command values ​​to fly along the generated route and provides the control command values ​​to the drive system 106. This allows the drone 10 to fly autonomously. The generated route candidates may be presented to the user 30. The drone 10 transmits its position information, image data from the image capturing camera 107, and the like to the operation device 20, and the user 30 can check this information in real time on the display unit of the operation device. The map data may be pre-stored in the operation device 20 or may be transmitted to the operation device 20 by the drone 10.

[0061] (After Flight) Shortly after the flight, an obstacle is observed in front of the drone 10, and this obstacle is displayed on the display unit 207 of the operation device 20.

[0062] FIG. 6 shows an example of a display on the display unit 207, including an obstacle (building) B1 observed ahead of the drone 10. The departure point S, destination G, and the drone 10's current position C are schematically shown. A route P11 from departure to the present is also shown. A route P12 for flight in side-avoidance priority mode is displayed as a planned route after the current position C. At this time, a route P13 for flight in top-avoidance priority mode (top-avoidance priority route) is also displayed. Information on routes P12 and P13 is transmitted from the action planning unit 105 to the operation device 20 and displayed on the display unit 207. The characters "top-avoidance priority route" and "side-avoidance priority route" in the figure are text strings displayed on the display unit 207 and correspond to examples of route characteristics. While routes P12 and P13 are displayed, the user 30 selects one of them and inputs the selection result via the instruction unit 208. If the user 30 does not select either route or selects route P12, the drone 10 continues flying along route P12, which is the planned route. If route P13 is selected, the flight control unit 104 controls the drone 10 to avoid building B1 via route P13.

[0063] The side-avoidance priority mode may be further divided into a right-avoidance priority mode and a left-avoidance priority mode. Alternatively, other modes, such as a mode that minimizes energy consumption, may be used.

[0064] FIG. 7 shows another example of the display on the display unit 207, including building B2 observed ahead. Elements identical to those in FIG. 6 are designated by the same reference numerals, and detailed descriptions are omitted. Before starting flight, the user 30 selects right-side avoidance priority mode, and route P21 (right-side avoidance priority route) for flight in right-side avoidance priority mode is displayed as the planned route. At this time, route P22 (top-side avoidance priority route) for flight in top-side avoidance priority mode, route P23 (left-side avoidance priority route) for flight in left-side avoidance priority mode, and route P24 (power-saving priority route) for flight in power-saving priority mode are also displayed. While routes P21 to P24 are displayed, the user 30 can select one of them and input the selection result via the instruction unit 208. If the user 30 does not select any of them or if route P21 is selected, the drone 10 continues flying along route P21, the planned route. If any of routes P22 to P24 is selected, the flight control unit 104 controls the flight of the drone 10 so as to avoid building B2 along the selected route.

[0065] As described above, according to this embodiment, the drone 10 can fly while avoiding obstacles and reflecting the user's wishes in real time.

[0066] [Example 2] In this example, flight begins according to a target route determined as a route from a departure point to a destination, and thereafter, if an obstacle is detected based on observation information from the drone 10, the route is dynamically changed based on the power consumption characteristics of the drone 10 and the user's target conditions. This allows efficient flight to continue while autonomously avoiding the obstacle.

[0067] FIG. 8 is a flowchart of an example of operation of this embodiment. The behavior planning unit 105 of the drone 10 reads the power consumption characteristics of the drone 10 from the power consumption characteristics storage unit 103 (S101). The behavior planning unit 105 determines a target route based on information about the departure point and destination (S102). A condition that minimizes energy consumption is used as the target condition. The target route may also be specified by the user 30, and the behavior planning unit 105 may acquire information about the target route from the operation device 20. The behavior planning unit 105 acquires map data from the environment acquisition unit 101 (S103). The flight control unit 104 generates control command values ​​for flight along the target route and sends the control command values ​​to the drive system 106, thereby starting flight of the drone 10 (S104).

[0068] The behavior planning unit 105 acquires detection data from various sensors via the environment acquisition unit 101, acquires or updates information about obstacles, and acquires or updates the position (self-position) of the drone 10 (S105). The sensor detection data includes image capture data from the image capture camera 107, and may also include detection data from the distance measurement sensor or radar if the drone 10 is equipped with a distance measurement sensor such as Lidar or radar.

[0069] The behavior planning unit 105 generates one or more candidate routes that avoid the obstacles based on information about the detected obstacles and the vehicle's own position (S106). The behavior planning unit 105 estimates the energy consumption of each candidate route (S107), calculates a score for each candidate route based on the estimated results, and evaluates or compares each candidate route based on the score (S108). The behavior planning unit 105 selects the candidate route with the lowest energy consumption (S109) and updates the target route using the selected candidate route (S110). Here, the candidate route includes not only a route that reaches the destination but also partial routes along the way to the destination. For example, comparing the costs and scores of searched nodes generated as intermediate results during the route search process can also be considered a comparison of candidate routes.

[0070] The flight control unit 104 controls the drive system 106 to fly along the updated target route (S111). This allows the drone 10 to avoid obstacles. The behavior planning unit 105 determines whether the drone 10 has arrived at the destination (S112). If the drone 10 has arrived at the destination, the process ends. If the drone 10 has not arrived at the destination, the process returns to step S105. As an example, the loop from steps S105 to S112 (replanning of behavior) is repeated every few milliseconds to a few seconds.

[0071] Below, several specific examples of flying the drone 10 according to this embodiment will be shown using Figures 9 and 10.

[0072] FIG. 9(A) shows an example in which there are no obstacles between the departure point S and the destination G. A dotted line connecting the departure point S and the destination G in a straight line is virtually shown as the task route SL. The information shown in FIG. 9(A) may be displayed as an image on the display unit 207 of the operation device 20. Here, the target route is a route that moves along a predetermined altitude (e.g., 5 m) on the task route. Since there are no obstacles on the task route, there are also no obstacles on the target route, and the drone 10 flies along route P31 along the target route and lands at or near destination G.

[0073] Figure 9(B) shows an example in which a building B3 exists on the task route SL set between the starting point S and the destination G. The information shown in Figure 9(B) may be displayed as an image on the display unit 207 of the operation device 20. In this case, too, the target route is a route that moves along a predetermined altitude (e.g., 5 m) on the task route SL. Building B1, which is 15 m wide on both sides and 10 m high, exists at the center of the task route SL.

[0074] Drone 10 was unable to observe building B3 immediately after departure, but then the obstacle information was updated and building B3 was observed ahead.

[0075] Two candidate routes P32 and P33 that avoid the obstacles are generated by the behavior planning unit 105. Candidate route P32 is a route that avoids building B3 from the side, and candidate route P33 is a route (shortest route) that avoids building B3 from above.

[0076] The energy consumption of each candidate route is estimated, and a score for each candidate route is calculated based on the estimation results. The candidate routes are compared by score, and the target route is updated based on the optimal candidate route. In this example, since route P32, which avoids building B3 from the side, consumes less energy than route P33, which avoids building B3 from above, candidate route P32, which avoids building B3 from the side, is selected, and the target route is updated based on candidate route P32.

[0077] Figure 10(A) shows an example in which a building B4 exists on a task route SL set between a starting point S and a destination G. The information shown in Figure 10(A) may be displayed as an image on the display unit 207 of the operation device 20. Here too, the target route is a route that moves along a predetermined altitude (for example, 5 m) on the task route SL. A building B1, 20 m wide and 10 m high, exists on both sides of the task route SL at the center.

[0078] Immediately after departure, the drone 10 was unable to observe building B2, but subsequently, the obstacle information was updated, and building B4 was observed ahead, and processing similar to that shown in FIG. 9B was performed. As a result of this processing, multiple candidate routes that avoid the obstacle are generated. In this example, of these candidate routes, candidate route P34 that avoids the obstacle from above is selected rather than a candidate route (not shown) that avoids the obstacle from the side. Because building B2 is widely spread out horizontally, avoiding it from the side consumes more energy than avoiding it from above. As a result, candidate route P34, which is closest to the shortest route, is selected, and the target route is updated to candidate route P34.

[0079] FIG. 10(B) shows the same starting point S and destination G as FIG. 10(A), and the same task route SL and building B2 as FIG. 10(A), but differs in that the drone 10 is carrying a load 11. The information shown in FIG. 10(B) may be displayed as an image on the display unit 207 of the operation device 20. The fact that the drone 10 is carrying the load 11 may be read as characteristic information from the power consumption characteristic storage unit 103. In FIG. 10(A) described above, a candidate route that avoids building B2 from above was selected from the multiple candidate routes generated. However, in FIG. 10(B), candidate route P35 that avoids building B2 from the side is selected because the energy consumption in the direction opposite to gravity required to lift the loaded load is large.

[0080] As shown in Figures 9 and 10, according to this embodiment, it is possible to select an energy-efficient route depending on differences in the environment, such as the position and size of obstacles, and the state of the aircraft, such as whether or not it is carrying cargo.

[0081] [Example 3] In this example, the aircraft flies from the departure point to the destination according to a target route, dynamically changing the route selection mode (target conditions used) while autonomously avoiding obstacles, thereby achieving flight that meets the user's wishes as closely as possible. The target route may be any route generated from the departure point and the destination, such as a route that passes through points specified by the user 30, a route that smoothly connects each point from the departure point to the destination by appropriately interpolating the specified points, a route that travels linearly in the air at a predetermined altitude from the departure point to the destination, or a route defined by other methods. When initially determining the target route, a standard mode may be set in advance, and the action planning unit 105 may calculate the target route in the standard mode. The standard mode may be any mode, such as a battery-priority mode, a speed-priority mode, a shortest mode, or a low-load (aircraft-friendly) mode.

[0082] 11 is a flowchart of an example of operation of the present embodiment 3. The route selection mode is dynamically changed while autonomously avoiding obstacles.

[0083] The behavior planning unit 105 of the drone 10 reads a predetermined target route (S201). The behavior planning unit 105 reads information on a preferred mode, which is a mode (target condition) to be used preferentially or as a default (S202). The behavior planning unit 105 also reads information on candidate modes (candidate target conditions) that can be used as candidates when selecting a route during flight (S203).

[0084] The priority mode and candidate mode may be set in advance by the user 30 on the operation device 20 and the set information may be transmitted to the drone 10, or initial values ​​may be set in advance in the drone 10. The initial values ​​may be registered in advance by the user 30. Examples of the priority mode and candidate mode include a battery priority mode, a speed priority mode, a shortest mode, and a low load (kind to the aircraft) mode.

[0085] The flight control unit 104 generates a control command value for flying along the target route and sends the control command value to the drive system 106, thereby starting the flight of the drone 10 (S204).

[0086] The behavior planning unit 105 acquires sensor detection data via the environment acquisition unit 101, acquires or updates information about obstacles, and acquires or updates the position of the drone 10 (self-position) (S205). The sensor detection data includes image capture data from the imaging camera 107, and may also include detection data from the range finding sensor or radar if the drone 10 is equipped with a range finding sensor such as Lidar. Note that, due to differences between the map and the actual topography or situation, obstacles (buildings, etc.) may be present in locations along the target route where no obstacles were expected to exist.

[0087] The behavior planning unit 105 generates candidate routes that avoid the obstacles in priority mode based on information about the detected obstacles and the drone's own position (S206), and also generates candidate routes in candidate mode (S207). The behavior planning unit 105 transmits information about the candidate routes generated in priority mode and candidate mode to the operation device 20. The behavior planning unit 105 may also transmit information about the route characteristics of each candidate route to the operation device 20. In the operation device 20, the candidate presentation unit 201 displays each candidate route on the display unit 207 so that it is overlaid on a map. Furthermore, the route identification presentation unit 203 may associate the route characteristics of each candidate route with each candidate route and display them on the display unit 207. The user 30 selects one candidate route from the candidate routes displayed on the display unit 207. The selection of a candidate route can also be referred to as the selection of a mode or route characteristics corresponding to the candidate route. The operation device 20 transmits information identifying the selected route candidate (e.g., information about the mode corresponding to the selected route candidate) to the drone 10. If there is no input from the user 30 within a certain period of time, a candidate route in the priority mode is selected.

[0088] The behavior planning unit 105 of the drone 10 determines whether there has been a change in mode from the information received from the operation device 20, and if there has been a change in mode, updates the target route to a candidate route for the changed mode (S210). The flight control unit 104 controls the drive system 106 to fly along the updated target route (S211). If there has been no change in mode, the target route is updated to a candidate route for the current mode, and the flight control unit 104 controls the drive system 106 to fly along the target route (S211).

[0089] The behavior planning unit 105 determines whether the destination has been reached (S212), and if the destination has been reached, the process ends, but if the destination has not been reached, the process returns to step S205. As an example, the loop from steps S205 to S212 (re-planning of behavior) is repeated every few milliseconds to a few seconds.

[0090] In the above operation, a mode is selected (a candidate route is selected) when an obstacle is detected, but a mode may be selected in response to the occurrence of an event other than the detection of an obstacle. For example, a mode may be selected in response to the arrival of a fixed time. A mode may also be selected in response to receiving request information from the user 30 not to change the mode.

[0091] According to this embodiment, the user 30 can thus realize the flight as desired by selecting the route candidate or route characteristics for the desired flight while flying the drone 10 while viewing the route candidate corresponding to each mode displayed on the display unit 207.

[0092] A specific example of flying the drone 10 according to this embodiment will be described below with reference to FIG. 12.

[0093] 12 shows an example in which movement is started along a target route P11 set between a departure point S and a destination G, and then, at a position C, two buildings B11 and B12 are detected near the target route P41. The information shown in FIG. 12 may be displayed as an image on the display unit 207 of the controller device 20. Note that the user 30 has previously set, on the controller device 20, the battery priority mode as the priority mode, and the speed priority mode, the shortest mode, and the low load mode (a mode that is gentle on the aircraft) as the candidate modes.

[0094] The drone 10 starts flying from a departure point S, flies along a target route P11, and detects buildings B11 and B12 ahead at position C. To avoid the two buildings B11 and B12 visible ahead, the drone 10 generates candidate routes in battery-priority mode, speed-priority mode, shortest mode, and low-load mode. Information about each candidate route is transmitted to the controller 20 and displayed on the display unit 207 of the controller 20. At this time, a character string identifying the route is displayed as information about the route characteristics associated with each candidate route. For example, the character strings "speed-priority," "shortest distance," "battery-priority (low energy consumption)," and "low-load (gentle on the aircraft)" are displayed. This allows the user 30 to easily determine the characteristics of each candidate route.

[0095] The user 30 can achieve the flight they desire by selecting the route they want to take while flying while viewing each candidate route. In this case, the user 30 has ample battery power remaining and wants the drone 10 to arrive at the destination quickly, so they touch the screen to select candidate route P44 in speed priority mode. By selecting candidate route P44 in speed priority mode, the drone will avoid flying over buildings B11 and B12, which will increase energy consumption, but will allow the drone to arrive at destination G more quickly.

[0096] The display of the above-described candidate routes and the like may be turned on / off using a candidate display button (instruction unit 208) of the operation device 20. When turned off, the drone 10 may continue flying in priority mode without confirming with the user 30. In addition, the display of candidate routes and the like may be performed when an event other than the detection of an obstacle occurs, such as when it is detected that the map and the actual terrain differ.

[0097] [Example 4] In this example, a route (target route) from a departure point to a destination is generated in advance by simulation, and the generated target route is flown by the drone 10. In Examples 1 to 3, the route to be taken when an obstacle is detected is determined in real time while reflecting instructions from the user 30 in real time, but in Example 4, a target route is determined in advance taking into account the presence of obstacles, etc.

[0098] Fig. 13 shows a flowchart of an operation example of Example 4. Fig. 13(A) shows an operation flow for generating a route (target route) by simulation before the departure of the drone 10. Fig. 13(B) shows an operation flow for actually flying the drone 10 according to the generated target route.

[0099] 13A, the action planning unit 105 reads information on the power consumption characteristics to be used and information on the task route (S301). The task route specifies a start point and an end point using three-dimensional coordinates on a map. A more detailed specification may be made by inputting one or more via points. In this case, the task route is information that specifies in advance the points to be passed through or the desired points to be passed through.

[0100] The behavior planning unit 105 reads map data (3D map) of the target flight area (S302) and also reads information on one or more target conditions to be used (S303). For each target condition, the behavior planning unit 105 generates candidate routes that avoid obstacles based on the points specified in the task route and taking into account the location of the obstacles (S304). The behavior planning unit 105 calculates the energy consumption required to fly along each candidate route and calculates an evaluation value (score). The score calculation formula (function) includes the energy consumption as a variable and may also include other variables, each of which is weighted by a pre-learned coefficient (parameter). Examples of other variables include the time required for travel (required time) and the amount of deviation from the task route. The amount of deviation may be, for example, a penalty value calculated by counting the number of points included in the task route if the robot does not pass within a certain distance of that point, or the amount of deviation may be calculated using other methods. Furthermore, virtually defined areas may also be taken into account. For example, a plan may be created that takes into account the flight area, altitude, no-entry areas, and other factors set before the flight. The evaluation value is calculated so that, for example, the smaller the energy consumption, the higher the evaluation. Depending on the form of the function, the higher the evaluation, the higher the score value, or the higher the evaluation, the lower the score value. The action planning unit 105 selects one candidate route based on the score of each candidate route, and determines the selected candidate route as the target route (S305). Information about the target route may be transmitted to the operation device 20 and displayed on the display unit 207.

[0101] 13(B), the flight control unit 104 reads information about the target route (S311) and generates control command values ​​to fly along the target route. The drive system 106 controls the rotor based on the control command values ​​to rotate the propellers, causing the drone 10 to take off and fly toward the destination (S312, S313). The flight control unit 104 ends the process when the drone 10 arrives at the destination (S314). Note that the flight control unit 104 may autonomously perform obstacle avoidance operations if it detects from observation information (photography data, etc.) that the map data and the actual terrain differ.

[0102] A specific example of flying the drone 10 according to this embodiment will be described below with reference to FIG. 14 .

[0103] 14 shows an example of determining in advance, by simulation, a route (target route) for delivering a package by drone 10 in an area with mountains, rivers, buildings (houses, buildings), etc. A three-dimensional map of the target flight area is shown schematically, and it is assumed that the flight route of drone 10 is to be determined from a departure point S to a destination G. The target flight area includes various natural features such as a river 31, mountains 32, forests 34, and lakes 35, as well as an area including multiple buildings 33 such as houses and buildings.

[0104] The behavior planning unit 105 generated routes A and B as candidate routes based on a task route that specifies a starting point S and a destination G. Note that in the figure, the line connecting the starting point S and the destination G is shown as a schematic representation of the task route. Route B was generated based on the target condition of shortest flight distance, and route A was generated based on the target condition of prioritizing power saving (low energy consumption). Route B passes through mountain 32, which requires the drone to increase its altitude to avoid obstacles such as mountain walls and trees along the way. However, route A has flat terrain and no obstacles that require it to increase its altitude along the way. Scores were calculated based on estimated energy consumption values ​​for routes A and B, and route A was selected as the target route based on a comparison of the scores. When actually flying the drone 10, route A is loaded into the drone, and the flight control unit 104 generates control command values ​​for the drivetrain 106 to fly the drone 10. In this example, the target condition of shortest flight distance and the target condition of power saving are used. However, various sets of target conditions, such as the target condition of prioritizing side-dodging and the target condition of power saving, may also be used.

[0105] As described above, an energy-efficient target route can be generated from map information (topographical information) and task routes, enabling the drone 10 to fly with high battery efficiency. Since a target route that avoids obstacles is determined before departure, there is also the effect of reducing the amount of processing required by the drone 10 during flight.

[0106] [Example 5] In this example, similar to Example 4, a target route from a departure point to a destination is generated in advance by simulation, and the drone 10 is caused to fly along the generated target route. In Example 4 described above, the target route was determined autonomously by the drone 10, but in Example 5, instructions from the user 30 are reflected in the determination of the target route. This allows the drone 10 to fly along a route that reflects the wishes of the user 30.

[0107] Fig. 15 shows a flowchart of an operation example of Example 5. Fig. 15(A) shows an operation flow for generating a route (target route) by simulation before the departure of the drone 10. Fig. 15(B) shows an operation flow performed when the drone 10 is actually flown according to the generated target route.

[0108] In FIG. 15A, the action planning unit 105 reads information on the power consumption characteristics to be used and information on the task route (S401).

[0109] The behavior planning unit 105 reads map data (4D map) of the target flight area (S402) and also reads information on one or more target conditions to be used (S403). For each target condition, the behavior planning unit 105 generates candidate routes that avoid obstacles based on the points specified in the task route (S404). The behavior planning unit 105 calculates route characteristics, such as the energy consumption, battery consumption (%), and flight time required for flying along each candidate route. Information on each candidate route and information on the route characteristics are transmitted to the control device 20 and displayed on the display unit 207 (S405). The display unit 207 of the control device 20 displays the candidate routes and route characteristics on a map. When the user 30 selects a candidate route using the instruction unit 208, information on the selected candidate route is transmitted to the drone 10. The behavior planning unit 105 of the drone 10 receives information on the candidate route selected by the user 30 (S406) and determines the candidate route indicated by the received information as the target route (S407).

[0110] 15(B), the flight control unit 104 reads information about the target route (S411) and generates control command values ​​to fly along the target route. The drive system 106 controls the rotor based on the control command values ​​to rotate the propellers, causing the drone 10 to take off and fly toward the destination (S412, S413). The flight control unit 104 ends the process when the drone 10 arrives at the destination (S414). Note that the flight control unit 104 may autonomously perform obstacle avoidance operations if it detects from observation information (photography data, etc.) that the map data and the actual terrain differ.

[0111] Specific examples of flying the drone 10 according to this embodiment will be described below with reference to FIGS. 16 to 18.

[0112] 16 shows an example of determining in advance, by simulation, a route (target route) for delivering a package by drone 10 in an area with mountains, rivers, buildings (houses, buildings), etc. A three-dimensional map of the target flight area is shown schematically, and it is assumed that the flight route of drone 10 is to be determined from a departure point S to a destination G. The target flight area includes various natural features such as a river 31, mountains 32, forests 34, and lakes 35, as well as an area including multiple buildings 33 such as houses and buildings.

[0113] The user 30 specifies a task route by specifying a starting point and a destination using three-dimensional coordinates on a map using the operation device 20. If there is a desired point to pass through, the user may specify that point as part of the task route. The operation device 20 transmits information about the task route to the drone 10. At this time, information about target conditions previously registered by the user 30 may also be transmitted to the drone 10.

[0114] The behavior planning unit 105 of the drone 10 generates candidate routes according to each target condition. Here, the candidate routes generated include an express route P51 related to arrival time, a power-saving route P52 related to energy consumption, and a safe route P53 related to safety, and the route characteristics of each candidate route are also calculated. Note that different types of route characteristics may be calculated for each candidate route, or the same type of information may be calculated for all candidate routes. Information on these candidate routes and route characteristics is transmitted to the operation device 20 and displayed on the display unit 207. The information shown in FIG. 16 is displayed on the display unit 207 as an image.

[0115] The safe route P53 is generated with consideration given to avoiding collisions with people when the vehicle falls and reducing the risk of damaging objects due to collisions. In this example, the safe route P53 is generated to pass through areas where people are unlikely to be present, such as the foot of a mountain 32, above a river 31, and through a forest. The safe route may be determined by calculating a safety level based on parameters according to the type of area to be traversed and the distance traveled, and determining a route with the highest safety level or above a threshold. Multiple safe routes may be generated and displayed, or a different candidate safe route may be presented in response to a request from the user 30. Multiple routes may be calculated for each of the power-saving route and the express route, and the multiple routes may be displayed in the same manner, or a different candidate may be displayed in response to a request from the user 30.

[0116] In the example of Figure 16, the route characteristics of each candidate route are displayed as the required time and the estimated remaining battery level (%) at arrival. The character strings "safe route," "power-saving route," and "express route" are also examples of route characteristics. By viewing this information, the user 30 can easily understand the intention behind the generation of the candidate route, and can also easily understand which of the candidate routes will enable the flight that meets the user's wishes. The route characteristics shown in the figure are just an example, and other types of information, such as power consumption, battery consumption (%), and aircraft load, may also be displayed.

[0117] 17 shows another example of the display of route characteristics of each candidate route. Battery consumption (%) is displayed as the route characteristic of the power-saving route P52. Character strings indicating that the express route P51 is the fastest route and that it avoids obstacles from above are displayed as the route characteristic of the express route P51. Character strings indicating that the safe route P53 passes over a river are displayed as the route characteristic.

[0118] 18 shows another example of displaying the route characteristics of each candidate route. Energy consumption (Wh) is displayed as the route characteristic of the energy-saving route P52. The route characteristics of the express route P51 and the safe route P53 are the same as those in the example of FIG. 17.

[0119] The user 30 selected route P52, which consumes less battery power, because the user plans to have the drone 10 displayed on the display unit 207 continue to perform other tasks after it arrives at destination G.

[0120] When actually flying the drone 10, the route P52 is loaded into the drone 10, and the flight control unit 104 generates control command values ​​for the drive system 106 to fly the drone 10.

[0121] As described above, by displaying several candidate routes to the user 30 based on the map information (topographical information) and the task route and allowing the user 30 to select one candidate route, the flight route desired by the user 30 can be realized.

[0122] In this example, route candidates are generated for each target condition, but weights may be assigned to multiple target conditions to generate candidate routes that reflect the multiple target conditions with their respective weights. For example, energy consumption priority and safety priority may be combined as target conditions, and candidate routes may be generated with weights assigned to each of them. A simple method is to divide the distance to the destination into sections where energy consumption priority is applied and sections where safety priority is applied according to the respective weights, and apply the respective characteristics to each section to calculate a route that combines both target conditions.

[0123] As described above, according to this embodiment, a route can be generated that minimizes energy consumption while taking into account gravity in three-dimensional space (power consumption increases during ascent) so that a drone can accomplish a movement task. For example, an energy-efficient route can be generated when obstacles can be avoided from above and from the sides.

[0124] Furthermore, according to this embodiment, by displaying multiple candidate routes to the drone user 30 and providing a means for selecting a candidate route, it is possible to realize flight that conforms to the user's 30 intentions.

[0125] Furthermore, according to this embodiment, by displaying candidate drone routes and route characteristics for each candidate route in the target flight environment (where various terrains and obstacles may exist), the user 30 can select a candidate route after understanding the characteristics of each candidate route.

[0126] Furthermore, the effects of the present disclosure described in this specification are merely examples, and other effects may also be present.

[0127] The present invention is not limited to the above-described embodiments, and the components can be modified and embodied in practice without departing from the spirit of the invention. Furthermore, various inventions can be created by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be omitted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined.

[0128] The present disclosure can also be configured as follows. [Item 1] An information processing method comprising: generating a plurality of candidate routes as candidates for a route to be flown by an aircraft capable of flying in a three-dimensional space, based on map data showing a flight environment of the aircraft, and at least one of the plurality of candidate routes includes a route in which the aircraft flies in a direction including a component in the opposite direction to the direction of gravity. [Item 2] The information processing method described in Item 1, wherein the route in which the aircraft flies in a direction including a component in the opposite direction to the direction of gravity is a route in which the aircraft rises above an obstacle in order to avoid the obstacle in its direction of travel. [Item 3] The information processing method described in Item 1 or 2, wherein a target route to be flown by the aircraft is selected from the plurality of candidate routes based on energy consumption required for the aircraft to fly along the plurality of candidate routes. [Item 4] The information processing method of any one of items 1 to 3, comprising: displaying information indicating a plurality of the candidate routes on a user's operation device; acquiring selection result information indicating a candidate route selected by the user from the plurality of candidate routes; and determining a target route to be flown by the aircraft from the plurality of candidate routes based on the acquired selection result information. [Item 5] The information processing method of item 4, comprising generating a control command value to control the flight of the aircraft according to the target route; and controlling the flight of the aircraft by supplying the control command value to a drive system. [Item 6] The information processing method of any one of items 1 to 5, comprising generating at least one of the plurality of candidate routes based on power consumption characteristics of the aircraft, the power consumption characteristics including a characteristic related to energy consumption required for flying in a direction including a component in the opposite direction to the direction of gravity. [Item 7] The information processing method of item 6, wherein the characteristic related to energy consumption includes a relationship between the distance of the component in the opposite direction to the direction of gravity and the energy consumption. [Item 8] The information processing method according to item 6, wherein the characteristics related to the consumed energy include a relationship between a distance of a component in a direction opposite to the direction of gravity, the consumed energy, and the weight of the aircraft. [Item 9] The information processing method according to any one of items 1 to 8, wherein a plurality of the candidate routes are generated based on a plurality of target conditions related to the flight of the aircraft.[Item 10] The information processing method according to Item 9, wherein the plurality of goal conditions include a goal condition for avoiding an obstacle present in the flying vehicle's direction of travel by flying in a direction including a component opposite to the direction of gravity, and a goal condition for avoiding the obstacle by making a detour laterally. [Item 11] The information processing method according to Item 9, wherein the plurality of goal conditions include at least two of a goal condition for arriving at the destination in a short time, a goal condition for arriving at the destination in a short distance, a target condition for preferentially passing through areas with a higher priority among a plurality of types of areas present in the flight environment, a target condition for reducing energy consumption until arriving at the destination, and a target condition for reducing load on a fuselage body or components of the flying vehicle. [Item 12] The information processing method according to any one of Items 9 to 11, wherein the plurality of goal conditions are received from a user's operation device, and the received plurality of target conditions are used. [Item 13] The information processing method according to Item 4 or 5, wherein information indicating route characteristics of the plurality of candidate routes is further associated with the plurality of candidate routes and displayed on the operation device of the user. [Item 14] The information processing method according to Item 13, wherein the information indicating the route characteristics of the plurality of candidate routes includes at least one of information on a type of area through which the candidate route passes, information on energy consumption required when flying along the candidate route, information on an arrival time when flying along the candidate route, information on the flight time or flight distance required when flying along the candidate route, and information on a direction in which the candidate route avoids an obstacle present in the flight direction of the aircraft. [Item 15] The information processing method according to Item 4 or 5, wherein the aircraft detects an obstacle present in the flight direction of the aircraft based on data detected by a sensor provided on the aircraft while it is flying, and the plurality of candidate routes are routes that avoid the obstacle, and include a route that avoids the obstacle by ascending in a direction that includes a component in the opposite direction to the direction of gravity, and a route that avoids the obstacle by detouring laterally.[Item 16] The information processing method of any one of Items 1 to 15, further comprising: generating a plurality of candidate routes based on a departure point and a destination of the aircraft before departure; detecting obstacles present in a flight environment of the aircraft based on the map data; and at least one of the plurality of candidate routes including a route that avoids the obstacle present in the flight direction of the aircraft by flying in a direction that includes a component in the opposite direction to the direction of gravity. [Item 17] The information processing method of any one of Items 1 to 16, wherein the aircraft is a drone. [Item 18] An information processing device comprising: a behavior planning unit that generates a plurality of candidate routes that are candidates for a route along which the aircraft will fly, based on map data that shows a flight environment of the aircraft capable of flying in three-dimensional space; and at least one of the plurality of candidate routes including a route that avoids an obstacle present in the flight environment by flying in a direction that includes a component in the opposite direction to the direction of gravity when the aircraft avoids the obstacle. [Item 19] A computer program that causes a computer to execute a step of generating multiple candidate routes that are candidates for routes that an aircraft capable of flying in a three-dimensional space will fly, based on map data that shows the flight environment of the aircraft, and at least one of the multiple candidate routes includes a route that the aircraft will use to avoid an obstacle in the flight environment by flying in a direction that includes a component with the direction of gravity.

[0129] REFERENCE SIGNS LIST 10 Drone 11 Baggage 11 Photographing camera 20 Operation device 21 Instruction unit (stick) 22 Display unit 30 User 101 Environment acquisition unit 102 Communication unit 103 Power consumption characteristic storage unit 104 Flight control unit 105 Action planning unit 106 Drive system 107 Photographing camera 108 Aircraft information storage unit 111 Characteristics acquisition unit 112 Trajectory planning unit 113 Energy estimation unit 201 Candidate presentation unit 202 Selection unit 203 Route identification presentation unit 204 Setting unit 205 Control unit 206 Communication unit 207 Display unit 208 Instruction unit

Claims

1. An information processing method comprising: generating a plurality of candidate routes that are candidates for routes that the aircraft will fly based on map data showing the flight environment of an aircraft capable of flying in three-dimensional space; and at least one of the plurality of candidate routes includes a route in which the aircraft will fly in a direction that includes a component in the opposite direction to the direction of gravity.

2. The information processing method described in claim 1, wherein the route flying in a direction including a component in the opposite direction to the direction of gravity is a route that avoids an obstacle by rising above the obstacle in the flying object's direction of travel.

3. The information processing method according to claim 1, further comprising the step of selecting a target route for the aircraft to fly from among the plurality of candidate routes based on the energy consumption required for the aircraft to fly along the plurality of candidate routes.

4. An information processing method as described in claim 1, comprising: displaying information indicating a plurality of the candidate routes on a user's operation device; acquiring selection result information indicating a candidate route selected by the user from the plurality of the candidate routes; and determining a target route for the aircraft to fly from among the plurality of the candidate routes based on the acquired selection result information.

5. The information processing method according to claim 4, further comprising generating a control command value for controlling the flight of the aircraft in accordance with the target route, and controlling the flight of the aircraft by supplying the control command value to a drive system.

6. The information processing method described in claim 1, wherein at least one of the multiple candidate routes is generated based on power consumption characteristics of the aircraft, the power consumption characteristics of the aircraft including characteristics related to the energy consumption required to fly in a direction that includes a component opposite to the direction of gravity.

7. The information processing method according to claim 6, wherein the characteristic relating to the consumed energy includes a relationship between the distance of the component in the direction opposite to the direction of gravity and the consumed energy.

8. The information processing method according to claim 6, wherein the characteristics relating to the consumed energy include a relationship between the distance of the component in the direction opposite to the direction of gravity, the consumed energy, and the weight of the flying object.

9. The information processing method according to claim 1, further comprising generating a plurality of the candidate routes based on a plurality of target conditions related to the flight of the aircraft.

10. The information processing method according to claim 9, wherein the plurality of target conditions include a target condition for avoiding an obstacle present in the direction of travel of the aircraft by flying in a direction that includes a component in the opposite direction to the direction of gravity, and a target condition for avoiding the obstacle by detouring laterally.

11. The information processing method of claim 9, wherein the multiple target conditions include at least two of a target condition for arriving at the destination in a short time, a target condition for arriving at the destination in a short distance, a target condition for preferentially passing through areas with higher priority among multiple types of areas present in the flight environment, a target condition for reducing energy consumption until arriving at the destination, and a target condition for reducing load on the main body or parts of the aircraft.

12. The information processing method according to claim 9, further comprising the steps of: receiving the plurality of goal conditions from a user's operation device; and using the received plurality of goal conditions.

13. The information processing method according to claim 4, further comprising the step of: displaying information indicating route characteristics of the plurality of candidate routes on the operation device of the user in association with the plurality of candidate routes.

14. The information processing method described in claim 13, wherein the information indicating the route characteristics of the plurality of candidate routes includes at least one of the following: information regarding the type of area through which the candidate routes pass, information regarding the energy consumption required when flying along the candidate routes, information regarding the arrival time when flying along the candidate routes, information regarding the flight time or flight distance required when flying along the candidate routes, and information regarding the direction in which the candidate routes avoid obstacles present in the direction of travel of the aircraft.

15. The information processing method of claim 4, further comprising: detecting an obstacle present in the flying vehicle's direction of travel based on data detected by a sensor equipped on the flying vehicle while the flying vehicle is in flight; and the plurality of candidate routes are routes that avoid the obstacles, including a route that avoids the obstacle by ascending in a direction that includes a component in the opposite direction to the direction of gravity, and a route that avoids the obstacle by detouring laterally.

16. The information processing method of claim 1, further comprising: generating a plurality of candidate routes based on the departure point and destination of the aircraft before departure of the aircraft; detecting obstacles present in the flight environment of the aircraft based on the map data; and at least one of the plurality of candidate routes includes a route that avoids the obstacles present in the flight direction of the aircraft by flying in a direction that includes a component in the opposite direction to the direction of gravity.

17. The information processing method according to claim 1, wherein the aircraft is a drone.

18. An information processing device comprising: an action planning unit that generates a plurality of candidate routes that are candidates for routes that the aircraft will fly based on map data that shows a flight environment of the aircraft capable of flying in three-dimensional space, at least one of the plurality of candidate routes including a route that allows the aircraft to avoid an obstacle present in the flight environment by flying in a direction that includes a component in the opposite direction to the direction of gravity.

19. A computer program that causes a computer to execute a step of generating multiple candidate routes that are candidates for routes that the aircraft will fly based on map data that shows the flight environment of an aircraft capable of flying in three-dimensional space, wherein at least one of the multiple candidate routes includes a route that allows the aircraft to avoid an obstacle present in the flight environment by flying in a direction that includes a component with the direction of gravity.

Images