Aircraft control device, aircraft control method, and program

Abstract

To enable operation of a UAV at a low risk in a place where a human exist and a surrounding air area.SOLUTION: A flying body control device 100 includes a detection unit 101 that detects a target from video data output by an imaging device 201 for imaging a lower part of a flying body 200, a fall range estimation unit 102 that estimates a fall range of the flying body 200, and a control unit 103 that controls the flying body 200 so that a position of the detected target is not overlapped with the fall range.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an aircraft control device and an aircraft control method for controlling the flight route of an aircraft, particularly an unmanned aerial vehicle (UAV), and a method for implementing the same. Nopu Regarding the program. [Background technology]

[0002] In recent years, UAVs have been attracting great expectations as a means of effectively utilizing low altitude airspace. This is because small UAVs have become available at low cost due to the miniaturization and high output of electric motors and batteries. In particular, from the standpoint of maneuverability and stability, many small UAVs are multicopter types equipped with multiple rotors.

[0003] Furthermore, due to the advancement of computer performance and miniaturization, UAVs are equipped with computers, unlike conventional remote-controlled model helicopters, etc. For this reason, methods for controlling the flight route of a UAV include manual operation by the pilot, as well as autonomous flight along a coordinate route set in advance using a GPS (Global Positioning System).

[0004] Furthermore, when flying a UAV autonomously, it is necessary to avoid collisions between the UAV and obstacles such as steel towers and power lines. For this reason, Patent Document 1 proposes a technology in which the coordinates of obstacles are input to the UAV in advance along with the flight route, so that the UAV automatically avoids the obstacle when it gets too close to it.

[0005] Furthermore, UAVs are expected to be used in the field of crime prevention. For example, Patent Document 2 discloses a technology in which, when a surveillance camera installed in a building detects an intruder, a UAV is dispatched to the location and the camera mounted on the UAV detects and tracks the intruder. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2003-127994 A (Fig. 1) [Patent Document 2] JP 2014-149620 A (Figure 1) Summary of the Invention [Problem to be solved by the invention]

[0007] However, the use of UAVs, especially multicopter-type UAVs, poses a high risk of sudden falls, which can be caused by piloting errors, running out of battery or fuel, changes in environmental conditions such as strong winds, or communication failures.

[0008] If a UAV were to fall on a pedestrian due to such a sudden drop, it could be a life-or-death situation. Furthermore, it is difficult to predict such a sudden drop. For this reason, the only measures to prevent a UAV from suddenly falling at present are to limit UAV flights to operations in spaces where there are no people, or to operations in emergency situations where the risk of falling is acceptable, in order to avoid the risk of falling.

[0009] One example of the objective of the present invention is to provide an aircraft control device, an aircraft control method, and a program that solve the above problem and enable UAVs to be operated with low risk even in areas where people are present and in the airspace nearby. [Means for solving the problem]

[0010] In order to achieve the above object, an aircraft control device according to one aspect of the present invention comprises: A detection unit that detects an object from video data output by an imaging device that captures an image below the flying object; A fall range estimation unit that estimates a fall range of the flying object; A control unit that controls the flying object so that the detected position of the target does not overlap with the falling range; The present invention is characterized by comprising:

[0011] In order to achieve the above object, a method for controlling an aircraft according to one aspect of the present invention includes: (a) detecting an object from image data output by an imaging device that captures an image below the aircraft; (b) estimating a fall range of the flying object; (c) controlling the flying object so that the detected position of the target does not overlap with the falling range; The present invention is characterized by having the following.

[0012] Furthermore, in order to achieve the above object, a computer-readable recording medium according to one aspect of the present invention comprises: On the computer, (a) detecting an object from image data output by an imaging device that captures an image below the aircraft; (b) estimating a fall range of the flying object; (c) controlling the flying object so that the detected position of the target does not overlap with the falling range; The present invention is characterized in that it records a program including instructions for executing the above. Effect of the Invention

[0013] As described above, according to the present invention, UAVs can be operated with low risk even in areas where people are present and in the airspace nearby. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a block diagram showing the configuration of an aircraft control device according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing an example of a flying object to be controlled and its falling range in the first embodiment of the present invention. [Diagram 3]FIG. 3 is a diagram showing an example of the relationship between the flying object and the wind direction. [Figure 4] FIG. 4 is a diagram showing an example of a drop range according to wind speed and altitude. [Diagram 5] FIG. 5 is a diagram showing an example of the positional relationship between a person on the ground and a drop range. [Figure 6] FIG. 6 is a diagram showing a case where the altitude of the flying object 200 is lowered from the state shown in FIG. [Figure 7] FIG. 7 is a flow diagram showing the operation of the aircraft control device in the first embodiment of the present invention. [Figure 8] FIG. 8 is a block diagram showing the configuration of an aircraft control device in accordance with the second embodiment of the present invention. [Figure 9] FIG. 9 is a flow diagram showing the operation of the aircraft control device in the second embodiment of the present invention. [Figure 10] FIG. 10 is a block diagram showing the configuration of an aircraft control device according to the third embodiment of the present invention. [Figure 11] FIG. 11 is a flow diagram showing the operation of the aircraft control device in the third embodiment of the present invention. [Figure 12] FIG. 12 is a block diagram showing another example of the configuration of an aircraft control device in the third embodiment of the present invention. [Figure 13] FIG. 13 is a block diagram showing an example of a computer that realizes the flying object control device according to the first to third embodiments of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] (Embodiment 1) Hereinafter, an aircraft control device, an aircraft control method, and a program according to a first embodiment of the present invention will be described with reference to FIGS.

[0016] [Device configuration] First, the configuration of the flying object control device in the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the flying object control device in the first embodiment of the present invention.

[0017] The aircraft control device 100 shown in Fig. 1 is a device for controlling an aircraft 200. Here, the aircraft 200 includes any aircraft capable of flight, such as an airplane, a helicopter, an airship, etc. Also, in Fig. 1, an unmanned aircraft (UAV) is shown as an example of the aircraft 200, but the aircraft 200 may be manned or unmanned.

[0018] 1, the flying object control device 100 includes a detection unit 101, a fall range estimation unit 102, and a control unit 103. Among these, the detection unit 101 detects an object from video data output by an imaging device 201 that captures an image of the area below the flying object 200. The object may be anything that needs to be prevented from colliding with the flying object 200 due to the falling of the flying object 200, such as a person, a baby stroller, a bicycle, an automobile, a train, a ship, an airplane, an endangered wild animal, or a building.

[0019] Furthermore, the fall range estimation unit 102 estimates a fall range of the flying object 200. Specifically, the fall range estimation unit 102 estimates a fall range in the case where the flying object 200 falls due to a stop or deterioration of the function of the flying object 200. The control unit 103 controls the flying object 200 so that the position of the detected target does not overlap with the fall range.

[0020] In this way, the flying object control device 100 detects an object with which it is desired to avoid collision with the flying object 200 from the image captured by the imaging device 201 mounted on the flying object 200. Furthermore, if there is a possibility of collision with the object when the flying object 200 falls, the flying object control device 100 controls the flying object 200 so that it does not collide with the object. Therefore, the flying object control device 100 makes it possible to operate the flying object 200 with low risk even in places where people are present and in the airspace nearby.

[0021] Next, the configuration of the flying object control device in the present embodiment 1 will be described more specifically with reference to Fig. 2 in addition to Fig. 1. Fig. 2 is a diagram showing an example of a flying object to be controlled and its falling range in the present embodiment 1 of the present invention.

[0022] As shown in FIG. 2, in the first embodiment, a multicopter type UAV having four propellers is used as the flying object 200. The flying object 200 is equipped with an imaging device 201 for photographing an object present below the flying object 200. In FIG. 2, 206 indicates an object for collision avoidance. In FIG. 2, the object 206 indicates a person. Furthermore, as shown in FIG. 1, the flying object 200 includes a motor 203 for driving each propeller, and a motor driving unit 202 for supplying power to each motor in response to an instruction from the control unit 103.

[0023] Specific examples of the imaging device 201 include a visible light camera, a far-infrared camera, and a ToF (Time of Flight) camera. The imaging device 201 may also be configured by combining two or more of these.

[0024] 2, the shooting direction of the imaging device 201 is set, for example, vertically below the flying object 200. However, the shooting direction is not limited to vertically below, and may be set below the flying object 200 so that a falling range described below can be photographed. For example, the shooting direction may be a direction inclined downward from the traveling direction of the flying object 200.

[0025] Also, for example, as disclosed in Patent Document 2, if the flying object 200 is equipped with a camera for tracking a person, the imaging device 201 may serve as the tracking camera or may be a camera separate from the tracking camera.

[0026] As described above, the detection unit 101 detects a person or the like who is a target for collision avoidance from the video data output by the imaging device 201. As a method for detecting a target by the detection unit 101, various known detection methods can be used.

[0027] For example, when the imaging device 201 is a visible light camera, a person can be detected by using a recognition dictionary that is constructed in advance using a machine learning technique and in which feature amounts corresponding to recognition targets are registered. Specifically, first, various rectangular areas are cut out from an image identified by video data, and then feature amounts are calculated from each of the cut-out rectangular areas. Thereafter, the calculated feature amounts are compared with the recognition dictionary, thereby identifying an area showing a person, and the identified area is detected as a person.

[0028] In addition, examples of learning algorithms for constructing a recognition dictionary include SVM (Support Vector Machine), cascade classifiers, etc. In addition, examples of features that can be used include HoG (Histograms of Oriented Gradients) features, Haar-Like features, etc.

[0029] Also, for example, if the imaging device 201 is a far-infrared camera, a person can be detected by extracting pixels whose pixel values ​​correspond to a temperature close to body temperature from an infrared image specified in the video data, and joining these adjacent pixels to extract a two-dimensional area. Furthermore, even if the imaging device 201 is a far-infrared camera, a person can be detected by using a recognition dictionary, as in the case of a visible light camera. Specifically, in this case, a location where the temperature is changing rapidly is extracted from an infrared image specified in the video data by image processing. Then, a feature amount is calculated from the extracted location, and the calculated feature amount is then compared with the recognition dictionary, thereby detecting a person.

[0030] In addition, when the imaging device 201 is a ToF camera, a person can be detected by using a recognition dictionary, as in the case of a visible light camera. Specifically, in this case, a portion where the distance between adjacent pixels is discontinuous is extracted from a distance image specified by video data. Then, a feature amount is calculated from the extracted portion, and the calculated feature amount is then compared with the recognition dictionary, thereby detecting a person.

[0031] Furthermore, when the detection unit 101 detects an object, it identifies the position of the object on the ground surface. Specifically, it is assumed that a three-dimensional coordinate system specific to the flying object 200, internal parameters of the image capture device 201, external parameters of the image capture device 201 relative to the three-dimensional coordinate system specific to the flying object 200, and the height of the flying object 200 above the ground are known. In this case, the detection unit 101 identifies the coordinates of the object in the video, and performs geometric calculations using the identified coordinates and the above-mentioned known information, thereby being able to identify the position of the object in the video on the ground surface.

[0032] The three-dimensional coordinate system specific to the flying object 200 is a coordinate system set inside the flying object 200 in order to identify the position of each part of the flying object 200, and may be set in advance. The internal parameters of the imaging device 201, such as the focal length, may be measured in advance.

[0033] The external parameters of the imaging device 201 with respect to the three-dimensional coordinate system specific to the flying object 200 are, for example, the angle of the optical axis of the lens and the position of the lens with respect to the flying object 200. When the position and orientation of the imaging device 200 are fixed with respect to the flying object 200, the external parameters only need to be measured in advance. On the other hand, when the position and orientation of the imaging device 200 are movable with respect to the flying object 200, the external parameters are calculated by integrating the measured values ​​in advance and position information from a sensor that detects the position of the imaging device 200 with respect to the flying object 200, and executing computational geometry.

[0034] For example, if the imaging device 201 is a ToF camera, the height above ground of the flying object 200 may be measured by calculating the distance from the flying object 200 to the ground based on distance information output by the ToF camera. Also, if the flying object 200 is equipped with a device for measuring altitude, such as a radio altimeter, the height above ground of the flying object 200 may be calculated based on information from this device.

[0035] In addition, in the detection unit 101, a coordinate system is set to represent the position of the target on the ground surface. Here, this coordinate system is referred to as a "ground plane coordinate system." In the first embodiment, any coordinate system may be set as the ground plane coordinate system, and is not particularly limited.

[0036] Specifically, the ground plane coordinate system may be a coordinate system having an origin at the point where a perpendicular line extending vertically from the target flying object 200 toward the ground surface intersects with the ground surface, an axis parallel to the front direction (travel direction) of the flying object 200 is the x-axis, and an axis perpendicular to the x-axis is the y-axis. This coordinate system is set with the target flying object 200 at the center, and the origin moves from moment to moment as the flying object 200 moves. However, in this embodiment 1, if the relative position of the object with respect to the flying object 200 can be determined, the flying object 200 can be controlled, so this ground plane coordinate system can be said to be a coordinate system that meets the objectives of the present invention.

[0037] Furthermore, the ground plane coordinate system may be a coordinate system set in the image from the imaging device 201, i.e., an image coordinate system. Further, the ground plane coordinate system may be a global coordinate system such as the Earth Centered Earth Fixed (ECEF) Cartesian coordinate system used in GPS and the like. When the global coordinate system is used, the position of the target in the global coordinate system is calculated from the position of the target aircraft 200 in the global coordinate system and the relative position of the target with respect to the target aircraft 200. As a result, the position of the target on the ground surface is expressed by the global coordinate system.

[0038] In the first embodiment, the fall range estimation unit 102 estimates a fall range on the ground that is expected when the flying object 200 to be controlled starts falling at the current time. If the flying object 200 is a multicopter type, it is difficult for the flying object 200 to glide when falling. For this reason, as shown in FIG. 2, for example, in the absence of wind, the flying object 200 is considered to fall within a circle 205 centered on a point 204 located directly below the flying object 200. Therefore, in this case, the fall range estimation unit 102 estimates the circle 205 as the fall range. In addition, since the radius of the circle 205 changes depending on the altitude of the flying object 200, the fall range estimation unit 102 sets the radius of the circle 205 based on the relationship between the preset altitude and the radius of the circle 205.

[0039] More practically, the drop range is significantly affected by the wind. For this reason, in estimating the drop range, it is preferable to use the wind direction and wind speed as parameters in addition to the altitude of the flying object 200. For this reason, in the first embodiment, it is preferable that the drop range estimation unit 102 stores in advance, as a database, drop ranges for combinations of at least the altitude from the ground and the wind speed, and estimates the drop range by referring to the database. This point will be described with reference to Figs. 3 and 4.

[0040] Fig. 3 is a diagram showing an example of the relationship between the flying object and the wind direction. Fig. 4 is a diagram showing an example of the drop range according to the wind speed and altitude. As shown in Fig. 3, it is assumed that the flying object 200 is affected by the wind. In this case, the drop range estimation unit 102 sets the direction (indicated by arrow 301) obtained by projecting a virtual arrow indicating the wind direction onto the ground along the vertical direction as the main axis direction, and sets the direction perpendicular to this (indicated by arrow 302) as the sub-axis direction.

[0041] 4, drop ranges 401-403 are set for various combinations of the wind speed and the current altitude of the flying object 200 as parameters. The set drop ranges 401-403 are associated with the corresponding combinations and stored as a database.

[0042] In addition, instead of using a database, the fall range estimation unit 102 can also perform a physical simulation of what would happen if the flying object 200 were to fall, based on the wind direction, wind speed, and altitude of the flying object 200, to calculate the fall range in real time.

[0043] 3 and 4, the wind direction and wind speed can be estimated by using the attitude control information of the flying object 200 at the time when the autonomous control was functioning well before the start of the fall. However, in this case, the wind direction and wind speed are estimated on a three-dimensional coordinate system specific to the flying object 200, so the fall range estimation unit 102 sets the fall range in the ground plane coordinate system using computational geometry, similar to the case of target detection by the detection unit 101.

[0044] 3 and 4, the direction 301 obtained by projecting a virtual arrow indicating the wind direction vertically onto the ground is set as the main axis direction, but in the absence of wind, the main axis direction is undefined. However, in the absence of wind, the falling range is concentric with the current position of the flying object 200 as the center, so there is no problem even if the main axis direction is set arbitrarily.

[0045] In the present embodiment 1, the control unit 103 determines whether the target position and the drop range overlap on the ground plane coordinate system, since the target position and the drop range are expressed in the ground plane coordinate system. If the determination result shows that the target and the drop range overlap, the control unit 103 controls the position of the flying object 200 so that the target detected by the detection unit 101 is not located within the drop range estimated by the drop range estimation unit 102. Specifically, the control unit 103 adjusts the output of each motor 203 via the motor drive unit 202 to make the flying object 200 perform an evasive maneuver so that the target position and the drop range do not overlap.

[0046] There are two main avoidance movements of the flying object 200 to avoid overlapping of the target position with the falling range. One is an avoidance movement in which the flying object 200 moves horizontally so that the detected target does not enter the falling range. The other is an avoidance movement in which the flying object 200 descends to lower its altitude so that the detected target does not enter the falling range.

[0047] The advantage of the former is that the relative position of the drop range with respect to the position of the flying object 200 does not change basically, so it is easy to control since it is only necessary to move the drop range horizontally so that the detected target does not enter the drop range. The direction and distance of the horizontal movement of the drop range correspond to the direction and distance to which the flying object 200 should be moved.

[0048] The latter has the advantage that the area of ​​the falling range can be narrowed down and made smaller, and that the impact can be reduced if the flying object does fall. The reason that the area of ​​the falling range can be made smaller is that the latter evasive maneuver shortens the time it takes for the flying object 200 to fall to the ground, and the amount of horizontal movement of the flying object 200 during the fall is reliably reduced.

[0049] Here, the advantages of the avoidance maneuver to lower the altitude of the flying object will be specifically described with reference to Fig. 5 and Fig. 6. Fig. 5 is a diagram showing an example of the positional relationship between a person on the ground and the drop range. Fig. 6 is a diagram showing a case where the altitude of the flying object 200 is lowered from the state shown in Fig. 5.

[0050] As shown in Fig. 5, for example, when multiple people 502 are present on the ground and flying object 200 is located at the current altitude, it is assumed that no matter how flying object 200 moves horizontally, people 502 will fall within falling range 501. In such a state, if flying object 200 is made to perform an evasive maneuver to lower its altitude, falling range 601 will be narrower than falling range 501 as shown in Fig. 6, so that falling range 601 and the positions of people 602 can be prevented from overlapping.

[0051] [Device operation] Next, the operation of the aircraft control device 100 in the first embodiment of the present invention will be described with reference to FIG. 7. FIG. 7 is a flow chart showing the operation of the aircraft control device in the first embodiment of the present invention. In the following description, FIGS. 1 to 6 will be referred to as appropriate. In addition, in this embodiment, an aircraft control method is implemented by operating the aircraft control device 100. Therefore, the description of the aircraft control method in this embodiment will be replaced by the following description of the operation of the aircraft control device 100.

[0052] 7, first, the detection unit 101 acquires video data output by the imaging device 201 (step A1). Next, the detection unit 101 detects an object from the video data acquired in step A1 (step A2). Also, in step A2, the detection unit 101 applies various parameters to the coordinates of the object in the image to perform geometric calculations and identify the position of the detected object on the ground plane.

[0053] Next, the drop range estimation unit 102 estimates the drop range when the flying object 200 falls (step A3). At this time, the drop range estimation unit 102 estimates the drop range taking into account the wind direction and wind speed in the airspace in which the flying object 200 is flying.

[0054] Thereafter, the control unit 103 determines whether the object detected in step A2 overlaps with the drop range estimated in step A3, and if so, controls the flying object 200 so that the object does not enter the drop range (step A4). Specifically, the control unit 103 causes the flying object 200 to perform avoidance movements such as horizontal movement and / or lowering in altitude, thereby eliminating the overlap between the object and the drop range.

[0055] In addition, in step A4, for example, if the control unit 103 controls the flying body 200 to move horizontally but is unable to eliminate the overlap between the position of the detected target and the fall range, the control unit 103 may control the flying body 200 to lower the altitude of the flying body 200, so that the position of the detected target does not overlap with the fall range.

[0056] [Effects of the first embodiment] As described above, in the present embodiment 1, it is possible to reduce the risk of falling during operation of the flying object 200. According to the present embodiment 1, it is possible to operate the flying object 200 with low risk even in a place where people are present and in the airspace nearby.

[0057] [program] The program in the present embodiment 1 may be a program that causes a computer to execute steps A1 to A4 shown in Fig. 7. By installing and executing this program in a computer, the aircraft control device 100 and the aircraft control method in the present embodiment 1 can be realized. In this case, a CPU (Central Processing Unit) of the computer functions as a detection unit 101, a fall range estimation unit 102, and a control unit 103, and performs processing.

[0058] The program in this embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as any one of the detection unit 101, the fall range estimation unit 102, and the control unit 103.

[0059] (Embodiment 2) Next, an aircraft control device, an aircraft control method, and a program according to a second embodiment of the present invention will be described with reference to FIGS.

[0060] [Device configuration] First, the configuration of the flying object control device in the second embodiment will be described with reference to Fig. 8. Fig. 8 is a block diagram showing the configuration of the flying object control device in the second embodiment of the present invention.

[0061] As shown in Fig. 8, the flying object control device 120 in the second embodiment further includes a movement range estimation unit 104 in addition to the detection unit 101, the fall range estimation unit 102, and the control unit 103 also shown in Fig. 1. In this respect, the flying object control device 120 in the second embodiment differs from the flying object control device 100 in the first embodiment shown in Fig. 1. The following description will focus on the differences from the first embodiment.

[0062] Assuming that the flying object 200 has started to fall, the movement range estimation unit 104 estimates the range within which the object already detected by the detection unit 101 may move on the ground from the start of the fall to the time it reaches the ground (hereinafter referred to as the "expected movement range").

[0063] For example, assume that the flying object 200 starts falling from the current time and takes 5 seconds to reach the ground, and the target person's running speed is a maximum of 7 m / sec. In this case, the movement range estimation unit 104 estimates a circular range with a radius of 35 m centered on the target person's location as the expected movement range. The expected movement range is also ultimately expressed in the ground plane coordinate system.

[0064] Also, in the second embodiment, unlike the first embodiment, when the movement range estimation unit 104 estimates the predicted movement range of the target, the control unit 103 determines whether or not the predicted movement range estimated by the movement range estimation unit 104 overlaps with the drop range. If the determination result indicates that the two overlap, the control unit 103 controls the flying object 200 so that the two do not overlap. As described above, the predicted movement range is also expressed in the ground plane coordinate system like the drop range, so that the control unit 103 can easily determine whether or not there is an overlap.

[0065] [Device operation] Next, the operation of the flying object control device 120 in the second embodiment of the present invention will be described with reference to Fig. 9. Fig. 9 is a flow diagram showing the operation of the flying object control device in the second embodiment of the present invention. In the following description, Fig. 8 will be referred to as appropriate. In addition, in this second embodiment, an flying object control method is implemented by operating the flying object control device 120. Therefore, the description of the flying object control method in this second embodiment will be replaced with the following description of the operation of the flying object control device 120.

[0066] As shown in Fig. 9, first, the detection unit 101 acquires video data output by the imaging device 201 (step B1). Next, the detection unit 101 detects an object from the video data acquired in step B1 (step B2). In addition, in step B2, the detection unit 101 identifies the position of the detected object on the ground plane. Note that steps B1 and B2 are the same as steps A1 and A2 shown in Fig. 7, respectively.

[0067] Next, the movement range estimation unit 104 estimates the expected movement range of the object detected in step B2 (step B3). Specifically, the movement range estimation unit 104 calculates the movement speed of the object on the video data, and further calculates the time it will take for the flying object 200 to reach the ground if it starts falling at the current time from the altitude and speed of the flying object 200. Then, the movement range estimation unit 104 estimates the expected movement range using the calculated movement speed and time of the object.

[0068] Next, the drop range estimation unit 102 estimates the drop range in the case where the flying object 200 falls (step B4). After that, the control unit 103 determines whether the expected movement range estimated in step B3 and the drop range estimated in step B4 overlap, and if they overlap, controls the flying object 200 so that the expected movement range and the drop range do not overlap (step B5). Specifically, the control unit 103 causes the flying object 200 to perform avoidance movements of either or both of horizontal movement and lowering of altitude, thereby eliminating the overlap between the expected movement range and the drop range. Note that step B4 is the same step as step A3 shown in FIG. 7.

[0069] In addition, in step B5, for example, if the control unit 103 controls the flying body 200 to move horizontally but is unable to eliminate the overlap between the position of the detected target and the fall range, the control unit 103 may control the flying body 200 to lower the altitude of the flying body 200 so that the predicted movement range and the fall range do not overlap.

[0070] [Effects of the second embodiment] As described above, according to the second embodiment, it is possible to more reliably avoid collisions between the flying object 200 and objects during a fall, thereby making it possible to further reduce the risk of the flying object 200 falling when operating the flying object 200.

[0071] In the above example, the flying object 200 is controlled so that the expected movement range and the falling range do not overlap, but the second embodiment is not limited to this example. For example, the movement range estimation unit 104 may calculate a numerical value representing the possibility that the flying object will collide with an object when it falls, such as the probability of the object being present when the flying object falls, for each set area, instead of the above-mentioned expected movement range. In this case, the control unit 103 controls the flying object 200 to move to an area where the value is the minimum. According to this aspect, it is possible to reduce the risk that the flying object 200 will fall in a place where more people are present, and optimal control of the flying object 200 is possible.

[0072] [program] The program in the second embodiment may be a program that causes a computer to execute steps B1 to B5 shown in Fig. 9. By installing and executing this program in a computer, it is possible to realize the aircraft control device 120 and the aircraft control method in the second embodiment. In this case, a CPU (Central Processing Unit) of the computer functions as a detection unit 101, a drop range estimation unit 102, a control unit 103, and a movement range estimation unit 104, and performs processing.

[0073] The program in this embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as any one of the detection unit 101, the fall range estimation unit 102, the control unit 103, and the movement range estimation unit 104.

[0074] (Embodiment 3) Next, an aircraft control device, an aircraft control method, and a program according to a third embodiment of the present invention will be described with reference to FIGS.

[0075] [Device configuration] First, the configuration of the flying object control device in the third embodiment will be described with reference to Fig. 10. Fig. 10 is a block diagram showing the configuration of the flying object control device in the third embodiment of the present invention.

[0076] As shown in Fig. 10, the flying object control device 130 in the present embodiment 3 further includes a fall risk estimation unit 105 in addition to the detection unit 101, the fall range estimation unit 102, and the control unit 103 also shown in Fig. 1. In this respect, the flying object control device 130 in the present embodiment 3 differs from the flying object control device 100 in the embodiment 1 shown in Fig. 1. The following description will focus on the differences from the embodiment 1.

[0077] 10, the fall risk estimation unit 105 estimates the risk of the flying object 200 falling based on the current state of the flying object 200. Specifically, the fall risk estimation unit 105 monitors, for example, wind speed, air turbulence, remaining battery level, reception state of radio waves for remote control, temperature of the motor 203, temperature of the computer constituting the flying object control device 130, etc. from moment to moment, and acquires the values ​​of each item (referred to as "monitoring information").

[0078] Since each of the monitored items increases the risk of falling if its value exceeds a certain level, a threshold is set in advance for each item. Therefore, the fall risk estimation unit 105 judges whether the value of each monitored item is equal to or greater than a predetermined threshold indicating the risk of falling.

[0079] If the result of the determination is that the threshold value is exceeded for one or more items, the fall risk estimation unit 105 determines that the risk of falling is increasing, and outputs a message to that effect to the control unit 103. On the other hand, if the result of the determination is that the threshold value is not exceeded for any item, the fall risk estimation unit 105 determines that the risk of falling at the current time is low, and outputs a message to that effect to the control unit 103.

[0080] When the fall risk estimation unit 105 determines that there is a high risk of falling, the control unit 103 controls the flying object 200 so that the position of the object detected by the detection unit 101 does not overlap with the fall range estimated by the fall range estimation unit 102.

[0081] Furthermore, when the fall risk estimation unit 105 judges that there is little risk of falling, the control unit 103 can allow the position of the target detected by the detection unit 101 to overlap with the fall range estimated by the fall range estimation unit 102. In this case, the control unit 103 does not cause the flying object 200 to perform evasive maneuvers.

[0082] [Device operation] Next, the operation of the flying object control device 130 in the third embodiment of the present invention will be described with reference to FIG. 11. FIG. 11 is a flow diagram showing the operation of the flying object control device in the third embodiment of the present invention. In the following description, FIG. 10 will be referred to as appropriate. In addition, in this third embodiment, the flying object control method is implemented by operating the flying object control device 130. Therefore, the description of the flying object control method in this third embodiment will be replaced with the following description of the operation of the flying object control device 130.

[0083] As shown in FIG. 11, first, the detection unit 101 acquires video data output by the imaging device 201 (step C1). Next, the detection unit 101 detects an object from the video data acquired in step C1 (step C2). Also, in step C2, the detection unit 101 identifies the position of the detected object on the ground plane. Next, the fall range estimation unit 102 estimates the fall range when the flying object 200 falls (step C3). Note that steps C1 to C3 are respectively similar to steps A1 to A3 shown in FIG. 7.

[0084] Next, the fall risk estimation unit 105 estimates the risk of the flying object 200 falling based on the current situation of the flying object 200 (step C4). Specifically, in step C4, the fall risk estimation unit 105 judges whether the value of each item constantly monitored is equal to or greater than a predetermined threshold. If the result of the judgment shows that one or more items exceed the threshold, the fall risk estimation unit 105 judges that the risk of falling is increasing, and if not, judges that the risk of falling is low.

[0085] Thereafter, the control unit 103 checks whether the risk is determined to be high in step C4, and if the risk is determined to be high, controls the flying object 200 so that the object does not enter the falling range (step C5).

[0086] [Effects of the third embodiment] According to the third embodiment described above, when the risk of falling is low, the flying object 200 can fly above people, so that the flying object 200 can perform work above densely populated areas, pass through such airspace, etc. As a result, the efficiency of work performed by the flying object 200 can be improved.

[0087] 12, the flying object control device 130 in the third embodiment can also include the movement range estimation unit 104 shown in FIG. 8 in the second embodiment. In this case, when it is determined in step C4 that there is high risk, the control unit 103 controls the flying object 200 so that the falling range does not overlap with either the target position or the predicted movement range.

[0088] [program] The program in the third embodiment may be a program that causes a computer to execute steps C1 to C5 shown in Fig. 11. By installing and executing this program in a computer, the aircraft control device 130 and the aircraft control method in the third embodiment can be realized. In this case, a CPU (Central Processing Unit) of the computer functions as a detection unit 101, a fall range estimation unit 102, a control unit 103, and a fall risk estimation unit 105, and performs processing.

[0089] The program in this embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as any one of the detection unit 101, the fall range estimation unit 102, the control unit 103, and the fall risk estimation unit 105.

[0090] (physical configuration) Here, a computer that realizes an aircraft control device by executing the programs in the first to third embodiments will be described with reference to Fig. 13. Fig. 13 is a block diagram showing an example of a computer that realizes an aircraft control device in the first to third embodiments of the present invention.

[0091] The computer shown in Fig. 13 is an embedded computer mounted on an aircraft. However, in the first to third embodiments, the computer may be a computer that performs data communication with the aircraft from the ground. In this case, the aircraft control device is constructed in a location separate from the aircraft.

[0092] 13, a computer 110 includes a CPU 111, a main memory 112, a storage device 113, and a communication interface 114. These components are connected to each other via a bus 115 so as to be able to communicate data with each other.

[0093] The CPU 111 loads the programs (codes) according to this embodiment stored in the storage device 113 onto the main memory 112, and executes these in a predetermined order to perform various calculations.

[0094] A specific example of the storage device 113 is a semiconductor storage device such as a flash memory. The main memory 112 is typically a volatile storage device such as a dynamic random access memory (DRAM). The communication interface 114 mediates data transmission between the CPU 111 and other computers.

[0095] The program in this embodiment is provided in a state of being stored in a computer-readable recording medium 116. Specific examples of the recording medium 116 include general-purpose semiconductor storage devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), magnetic storage media such as a flexible disk, and optical storage media such as a CD-ROM (Compact Disk Read Only Memory). The program in this embodiment may be provided via the communication interface 114.

[0096] The flying object control device in this embodiment can be realized not by a computer with a program installed, but by using hardware corresponding to each part. Furthermore, the flying object control device may be realized in part by a program and the remaining part by hardware.

[0097] A part or all of the above-described embodiment can be expressed by (Additional Notes 1) to (Additional Notes 18) described below, but is not limited to the following descriptions.

[0098] (Appendix 1) A detection unit that detects an object from video data output by an imaging device that captures an image below the flying object; A fall range estimation unit that estimates a fall range of the flying object; A control unit that controls the flying object so that the detected position of the target does not overlap with the falling range; An aircraft control device comprising:

[0099] (Appendix 2) The drop range estimation unit estimates the drop range based on an altitude of the flying object from the ground and wind information that specifies a wind direction and a wind speed. 2. An aircraft control device as described in appendix 1.

[0100] (Appendix 3) the fall range estimation unit holds in advance a database of fall ranges for combinations of at least heights from the ground and wind speeds, and estimates the fall range by referring to the database; 3. An aircraft control device as described in appendix 2.

[0101] (Appendix 4) A movement range estimation unit is configured to estimate, as an expected movement range, a range in which the detected object may move on the ground from when the flying object starts to fall until when the flying object reaches the ground, The control unit controls the flying object so that the estimated predicted movement range and the falling range do not overlap. 2. An aircraft control device as described in appendix 1.

[0102] (Appendix 5) When the overlap between the detected position of the target and the falling range cannot be eliminated even if the control unit controls the flying object to move in a horizontal direction, the control unit controls the flying object to lower an altitude of the flying object so that the detected position of the target does not overlap with the falling range. 2. An aircraft control device as described in appendix 1.

[0103] (Appendix 6) A falling risk estimation unit is further provided for estimating a risk of the flying object falling based on a state of the flying object, When the estimated risk exceeds a predetermined level, the control unit controls the flying object so that the falling range does not overlap with either the detected position of the target or the predicted movement range. 5. An aircraft control device as described in appendix 4.

[0104] (Appendix 7) (a) detecting an object from image data output by an imaging device that captures an image below the aircraft; (b) estimating a fall range of the flying object; (c) controlling the flying object so that the detected position of the target does not overlap with the falling range; A method for controlling an aircraft, comprising:

[0105] (Appendix 8) In the step (b), the falling range is estimated based on an altitude of the aircraft from the ground and wind information that specifies a wind direction and a wind speed. 8. An aircraft control method as described in Supplementary Note 7.

[0106] (Appendix 9) In the step (b), a fall range for at least a combination of an altitude from the ground and a wind speed is stored in advance as a database, and the fall range is estimated by referring to the database. 9. An aircraft control method as described in appendix 8.

[0107] (Appendix 10) (d) estimating, as an expected movement range, a range in which the detected object may move on the ground from when the flying object starts to fall until when the flying object reaches the ground; In the step (c), the flying object is controlled so that the estimated predicted movement range and the falling range do not overlap. 8. An aircraft control method as described in Supplementary Note 7.

[0108] (Appendix 11) In the step (c), if the overlap between the detected position of the target and the falling range cannot be eliminated even if the aircraft is controlled to move in a horizontal direction, the aircraft is controlled to lower its altitude so that the detected position of the target does not overlap with the falling range. 8. An aircraft control method as described in Supplementary Note 7.

[0109] (Appendix 12) (e) estimating a risk of the aircraft falling based on the status of the aircraft; In the step (c), when the estimated risk exceeds a predetermined level, the flying object is controlled so that the falling range does not overlap with either the detected position of the object or the predicted movement range. 11. An aircraft control method as described in claim 10.

[0110] (Appendix 13) On the computer, (a) detecting an object from image data output by an imaging device that captures an image below the aircraft; (b) estimating a fall range of the flying object; (c) controlling the flying object so that the detected position of the target does not overlap with the falling range; A program that executes the following.

[0111] (Appendix 14) In the step (b), the falling range is estimated based on an altitude of the aircraft from the ground and wind information that specifies a wind direction and a wind speed. 14. The program according to claim 13.

[0112] (Appendix 15) In the step (b), a fall range for at least a combination of an altitude from the ground and a wind speed is stored in advance as a database, and the fall range is estimated by referring to the database. 15. The program according to claim 14.

[0113] (Appendix 16) The computer includes: (d) estimating, as an expected movement range, a range in which the detected object may move on the ground from when the flying object starts to fall until when the flying object reaches the ground; In the step (c), the flying object is controlled so that the estimated predicted movement range and the falling range do not overlap. 14. The program according to claim 13.

[0114] (Appendix 17) In the step (c), if the overlap between the detected position of the target and the falling range cannot be eliminated even if the aircraft is controlled to move in a horizontal direction, the aircraft is controlled to lower its altitude so that the detected position of the target does not overlap with the falling range. 14. The program according to claim 13.

[0115] (Appendix 18) The computer includes: (e) estimating a risk of the aircraft falling based on the status of the aircraft; In the step (c), when the estimated risk exceeds a predetermined level, the flying object is controlled so that the falling range does not overlap with either the detected position of the object or the predicted movement range. 17. The program according to claim 16.

[0116] Although the present invention has been described above with reference to the embodiment, the present invention is not limited to the above embodiment. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0117] This application claims priority based on Japanese Patent Application No. 2015-165850, filed August 25, 2015, the disclosure of which is incorporated herein in its entirety. [Industrial Applicability]

[0118] As described above, according to the present invention, an aircraft such as a UAV can be operated with low risk even in places where people are present and in the airspace nearby. The present invention is useful for various aircraft. [Explanation of symbols]

[0119] 100 Aircraft control device 101 Detection unit 102 Fall range estimation unit 103 Control section 104 Moving range estimation unit 105 Fall Risk Estimation Department 110 Computer 111 CPU 112 Main memory 113 Storage device 114 Communication Interface 115 Bus 116 Recording media 200 Flying Objects 201 Imaging device 202 Motor drive unit 203 Motor 204 Point directly below the aircraft 205 Falling range circle 206 subjects (people) 301 The direction obtained by projecting an imaginary arrow showing the wind direction onto the ground (main axis direction) 302 Direction perpendicular to the main axis (sub-axis direction) 401, 402, 403 Fall ranges set for each combination of wind speed and altitude 501 Fall range overlapping with human presence 502 people 601 Fall range without overlap with human presence 602 people

Claims

1. A fall range estimation unit that estimates a fall range of the flying object based on an altitude of the flying object from the ground and wind information that specifies a wind direction and a wind speed; Based on the state of the flying object, a risk of the flying object falling is estimated, and if it is determined that there is a risk of the flying object falling, A control unit that controls the flying object based on a position of an object detected from video data output by an imaging device that captures an image below the flying object and the falling range; Equipped with the fall range estimation unit estimates the fall range by referring to a database that stores in advance fall ranges for combinations of at least heights from the ground and wind speeds; The control unit controls the flying object so that the detected position of the target does not overlap with the estimated falling range. An aircraft control device comprising:

2. 2. The flying object control device according to claim 1, A movement range estimation unit is configured to estimate, as an expected movement range, a range in which the detected object may move on the ground from when the flying object starts to fall until when the flying object reaches the ground, The control unit controls the flying object so that the predicted movement range and the falling range do not overlap. Aircraft control device.

3. 2. The flying object control device according to claim 1, When the overlap between the detected position of the target and the falling range cannot be eliminated even if the control unit controls the flying object to move in a horizontal direction, the control unit controls the flying object to lower an altitude of the flying object so that the detected position of the target does not overlap with the falling range. Aircraft control device.

4. 3. The flying object control device according to claim 2, A falling risk estimation unit is further provided for estimating a risk of the flying object falling based on a state of the flying object, When the estimated risk exceeds a predetermined level, the control unit controls the flying object so that the falling range does not overlap with either the detected position of the target or the predicted movement range. Aircraft control device.

5. The computer (a) estimating a fall range of the flying object based on an altitude of the flying object from the ground and wind information that specifies a wind direction and a wind speed; (b) estimating a risk of the flying object falling based on the state of the flying object, and when it is determined that there is a risk of the flying object falling, controlling the flying object based on the position of an object detected from video data output by an imaging device that captures an image below the flying object and the falling range; Run In the step (a), estimating the range of fall by referring to a database that stores in advance a range of fall for a combination of at least an altitude from the ground and a wind speed; In the step (b), the flying object is controlled so that the detected position of the target does not overlap with the falling range. A method for controlling an aircraft.

6. On the computer, (a) estimating a fall range of the flying object based on an altitude of the flying object from the ground and wind information that specifies a wind direction and a wind speed; (b) estimating a risk of the flying object falling based on the state of the flying object, and when it is determined that there is a risk of the flying object falling, controlling the flying object based on the position of an object detected from video data output by an imaging device that captures an image below the flying object and the falling range; Run the command, In the step (a), estimating the range of fall by referring to a database that stores in advance a range of fall for a combination of at least an altitude from the ground and a wind speed; In the step (b), the flying object is controlled so that the detected position of the target does not overlap with the falling range. program.

Images