Mobile Systems

Abstract

[Problem] To ensure safety during aerial photography. [Solution] A mobile body system 1 comprising a mobile body 100 that flies in a subject area F and a flight mode switching unit 330 for switching flight modes M102 and M105 in which flyable areas for the mobile body and mobile-body geofences G100 and G200 obtained by defining a region encompassing flyable areas are associated with each other, wherein the flight mode switching unit transitions to a second flight mode M105 via third flight modes M104 and M106 when switching from the first flight mode M102 to the second flight mode, and a third geofence of the third flight mode differs from a first geofence G100 of the first flight mode and a second geofence G200 of the second flight mode.

Description

Mobile system, aerial photography system, aerial photography method, and aerial photography program

[0001] The present invention relates to a mobile system, an aerial photography system, an aerial photography method, and an aerial photography program.

[0002] Patent Literature 1 discloses a camera viewpoint display system that detects the aircraft position and nose direction of an aircraft, detects the pan angle and tilt angle of a camera device mounted on the aircraft, calculates the camera viewpoint from these pieces of data, and displays the viewpoint on a map on a monitor screen. In this system, an operator controls the aircraft's position and attitude, the camera's shooting direction, etc., while grasping the aircraft's position and heading from a ground station.

[0003] JP 2006-281830 A

[0004] To effectively capture sports and other events, it is necessary to capture images that are suited to the details of the subject, such as the game situation. More specifically, it is preferable to capture images by switching between different capture modes, which define the flight area and capture direction of the aircraft, depending on the content. In addition, safety can be ensured by defining a geofence for the aircraft for each capture mode.

[0005] The system described in Patent Document 1 is complicated because it requires simultaneous control of the camera and the aircraft. Furthermore, the operation of switching between shooting modes in the system described in Patent Document 1 is cumbersome and not easy. In particular, when the geofence needs to be switched in response to a change in the flight area of ​​the aircraft, if the switching is not performed at the appropriate time, control interference may occur, and the aircraft may not move as intended.

[0006] The present invention has been made in consideration of the above-mentioned problems, and has an object to provide a mobile body system that can ensure safety during aerial flight.

[0007] In order to achieve the above-mentioned object, a mobile body system according to one aspect of the present invention comprises a mobile body that flies in a target area, and a flight mode switching unit that switches flight modes that correspond to a flyable area of ​​the mobile body and a geofence of the mobile body that defines an area that encompasses the flyable area, and when switching from a first flight mode to a second flight mode, the flight mode switching unit transitions to the second flight mode via a third flight mode, and the third geofence in the third flight mode is different from the first geofence in the first flight mode and the second geofence in the second flight mode.

[0008] The third geofence may define an area covering an integrated area formed by combining a first area defined by the first geofence and a second area defined by the second geofence.

[0009] The third geofence may define an area covering an integrated area that combines a first area defined by the first geofence, a second area defined by the second geofence, and a gap between the first area and the second area.

[0010] One of the first flight mode and the second flight mode may be an edge flight mode in which the aircraft flies along the outer edge of the target area, and the other of the first flight mode and the second flight mode may be an intra-target area flight mode in which the aircraft flies above the target area.

[0011] The device may further include a flight path generation unit that generates a flight path of the moving body in the third flight mode, and that changes the flight path when at least a portion of the flight path is generated outside the third geofence.

[0012] When it is detected that the moving body has deviated outside of any of the set first geofence, second geofence, or third geofence, the moving body may be caused to perform at least one of the following actions: landing, hovering, or moving toward the inside of the first area, the second area, or the integrated area.

[0013] In addition, in order to achieve the above-mentioned object, an aerial photography system according to one aspect of the present invention comprises a moving body flying in a target area, an event detection unit that detects an event based on an image acquired by a camera photographing the target area or input from an external system, and a mode switching determination unit that determines whether or not to switch flight modes that define at least a flyable area of ​​the moving body, the flight modes including at least an outer edge flight mode in which the moving body flies above the outer edge along part or all of the outer edge of a court configured within the target area, and an intra-court flight mode in which the moving body flies above the interior of the court, and the mode switching determination unit determines whether or not to switch flight modes depending on the event detected by the event detection unit.

[0014] The mode switching determination unit may switch the flight mode in accordance with an event related to a match being held in the target area, which event is detected from an image captured by the camera.

[0015] In the in-court flight mode, the mobile body may further be provided with an in-court flight control unit that generates a flight path within the court for automatically following a specific player or ball in the target area, or a flight path for moving the mobile body to a target position specified by the user.

[0016] The device may further include a display control unit that displays information about the moving object on an operation screen, and the display control unit may be configured to display the specific player or the ball that is being followed on the operation screen when the moving object is flying using the automatic following.

[0017] The in-court flight control unit may generate the flight path in the in-court flight mode by connecting a plurality of preset shooting positions.

[0018] The device may further include a display control unit that displays information about the moving object on an operation screen, and the display control unit may be configured to display information about the destination shooting position on the operation screen while flying between the shooting positions in the in-court flight mode.

[0019] The target area is composed of the court and an area outside the court, which are partitioned by the connection of a pair of goal lines facing each other and a pair of touch lines facing each other, and further includes an outer edge flight control unit that controls the flight of the moving body in the outer edge flight mode, and the outer edge flight control unit may be configured to move the moving body along the touch lines in the outer edge flight mode and follow the moving body to a specific player or the ball to take automatic photographs.

[0020] The outer edge flight control unit may be configured to move the moving body to the outside court area outside the touch line when the ball goes beyond the touch line on which the moving body is flying and take a picture directly below the moving body or toward the court.

[0021] The flight modes may further include a fixed position flight mode in which the moving body flies at a fixed position, and may further include a fixed position flight control unit that controls the operation of the moving body in the fixed position flight mode, and the fixed position flight control unit may hover at a predetermined position in the fixed position flight mode, and control the direction of the nose or the direction of the camera to follow a specific player or the ball and automatically take photographs.

[0022] When the in-court flight control unit detects an obstacle on the flight path or near the moving body in the in-court flight mode, it may perform at least one of the following actions: changing the connection of the shooting position to regenerate a flight path that bypasses the obstacle, deciding to fly at a higher altitude than the flight path, starting to move the moving body on the flight path after hovering for a predetermined time, switching to manual control after hovering for a predetermined time, and displaying a message prompting the user to re-input the target position after hovering for a predetermined time.

[0023] The outer edge flight control unit may generate a flight path for the moving body in the outer edge flight mode, and when an obstacle is detected on the flight path or near the moving body, may perform at least one of the following actions: regenerate a flight path that bypasses the obstacle to the inside of the court; decide to fly at a higher altitude than the flight path; start moving the moving body on the flight path after hovering for a predetermined time; switch to manual control after hovering for a predetermined time; and display a message prompting the user to re-input the target position after hovering for a predetermined time.

[0024] In order to achieve the above-mentioned object, an aerial photography method according to another aspect of the present invention comprises an event detection step of detecting an event based on an image acquired by a camera photographing a target area or input from an external system, and a mode switching determination step of determining whether or not to switch flight modes, which determine at least a flyable area of ​​a mobile object flying within the target area, and the flight modes include at least an outer edge flight mode in which the mobile object flies above the outer edge of a court configured within the target area along part or all of the outer edge of the court, and an intra-court flight mode in which the mobile object flies above the interior of the court, and the mode switching determination step determines whether or not to switch flight modes depending on the event detected by the event detection step.

[0025] To achieve the above object, another aspect of the present invention provides an aerial photography program that causes a computer to execute an event detection command that detects an event based on an image acquired by a camera photographing a target area or an input from an external system, and a mode switching determination command that determines whether to switch flight modes that define a flyable area for a mobile object flying within the target area, the flight modes including at least an outer edge flight mode in which the mobile object flies along and over part or all of the outer edge of a court within the target area, and an inside court flight mode in which the mobile object flies over the court, and the mode switching determination command determines whether to switch flight modes in response to the event detected by the event detection command. The computer program can be provided by being stored on various data-readable recording media or by being downloadable via a network such as the Internet.

[0026] According to the present invention, it is possible to ensure safety during aerial flight.

[0027] 1 is a diagram illustrating the overall configuration of a mobile system according to an embodiment of the present invention; FIG. 2 is a simplified external perspective view of a drone according to the embodiment; and FIG. 3 is a functional configuration diagram of the drone according to the embodiment. (a) is a simplified external front view of a control device according to the embodiment, and (b) is a schematic diagram illustrating the direction in which the drone moves or turns in response to input from the control device.

[0024] FIG. 3 is a functional configuration diagram of a server according to the embodiment.

[0025] FIG. 4 is a schematic diagram illustrating an example of a drone's pre-set shooting position in a field to be photographed where the drone flies.

[0026] FIG. 5 is a schematic diagram illustrating an example of a geofence setting in the field to be photographed, showing (a) a first example, (b) a second example, (c) a third example, and (d) a fourth example.

[0027] FIG. 5 is a schematic state transition diagram illustrating the transition of flight modes of the drone.

[0028] FIG. 6 is a schematic state transition diagram illustrating the transition of drone states according to the drone's aircraft state.

[0029] FIG. 7 is a schematic state transition diagram illustrating the transition of drone states according to the drone's aircraft behavior state.

[0029] FIG. 8 is a schematic state transition diagram illustrating the transition of game states in a stadium as an example of a field to be photographed.

[0029] FIG. 9 is a schematic state transition diagram illustrating the transition of offensive and defensive states in the stadium. 16A and 16B are diagrams showing an example of a correspondence between the state of a game in the stadium and the flight mode of the drone. FIG. 16B is a diagram showing a schematic diagram of a photographing position and a flight path to which the photographing position can be transferred. FIG. 16C is a flowchart of control performed during flight of the drone. FIG. 16D is a flowchart of control of flight restrictions in the drone (details of S1002 in FIG. 16). FIG. 16E is a flowchart of control of switching flight modes in the drone (details of S1010 in FIG. 16). FIG. 16F is a diagram showing a first example of a screen displayed on a terminal of the aerial photography system. FIG. 16F is a diagram showing a second example of a screen displayed on a terminal of the aerial photography system.

[0028] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted. Furthermore, the embodiments described below are merely examples, and other known elements or alternative means may be adopted depending on the application, purpose, scale, etc.

[0029] <A. One embodiment> [A-1. Configuration] (A-1-1. Overall configuration) Fig. 1 is a diagram showing the overall configuration of an aerial photography system 1 (hereinafter also referred to as "system 1") according to one embodiment of the present invention. System 1 uses a drone 100 (an example of a moving object) to photograph aerial images of competitions taking place at a stadium F (Fig. 7), events taking place at an event venue, etc. Stadium F is an example of a target area.

[0030] In the following explanation, the system 1 that photographs soccer as needed will be used as an example, but the system 1 can also be applied to sports and events other than soccer.

[0031] As shown in Figure 1, in addition to the drone 100, the system 1 mainly includes a controller 200 that allows the pilot to operate the drone 100, a server 300 that manages the flight and photography of the drone 100, an external input device 600, and an external system 700.

[0032] The drone 100 and the controller 200 are connected to each other via wireless communication (which may include communication via a base station 800). The controller 200 and the server 300 are connected to each other via a communication network 400 such as an internet line. The drone 100 acquires satellite signals from an artificial satellite 500 to determine its own location, etc.

[0033] The external input device 600 is a device separate from the controller 200 that can send and receive information to and from the system 1, and is configured, for example, as a mobile device such as a smartphone or tablet. The external input device 600 can be operated, for example, by a manager, coach, bench player, referee, or court facility personnel of a game being played at the stadium F. The external input device 600 has a function for receiving, for example, an emergency command to stop filming, and the drone 100 performs emergency evacuation based on the command. The external input device 600 may also receive an input to switch the flight mode of the drone 100. Furthermore, the external input device 600 may be equipped with a display device that may display information similar to that displayed on the display unit 201 of the controller 200. In particular, the external input device 600 may acquire event information occurring during the game. The event information is referenced when a user of the external input device 600 makes an input to switch the flight mode of the drone 100.

[0034] The external system 700 may be any system configured separately from the system 1. For example, systems such as a court facility system, a match management system, and a referee support system may be applied as systems deployed for a competition held at the stadium F, as well as systems such as a weather observation system or an earthquake observation system deployed independently of the competition. Multiple external systems 700 may be connected to the system 1. The system 1 may receive an emergency command to stop filming or a command to switch the flight mode of the drone 100 from various external systems 700. Furthermore, the various external systems 700 may acquire event information occurring during the competition.

[0035] A court facilities system as an example of the external system 700 may, for example, obtain the brightness of the captured image from the system 1 and control the illuminance adjustment or blinking of the lighting in the stadium F. The court facilities system may also receive a request for lighting illuminance from the system 1 and control the illuminance adjustment or blinking.

[0036] The configuration of system 1 is not limited to that shown in FIG. 1 , and the drone 100, controller 200, server 300, and base station 800 may be connected to each other so that they can communicate with each other via a communication network 400 such as an internet line. In this case, the drone 100 may communicate wirelessly directly with the communication network 400 using a communication method such as LTE without going through the controller 200. Therefore, the drone 100, controller 200, and base station 800 do not need to communicate wirelessly directly; they only need to be connected to the communication network 400 in remote locations. Therefore, this system configuration is suitable for cases where the drone 100 and controller 200 are located in remote locations (for example, when a pilot remotely controls the drone).

[0037] In addition, in system 1, the drone 100, the controller 200, the base station 800, and the server 300 are each connected to each other so that they can communicate with each other via a communication network 400 such as an Internet line, and the drone 100 and the base station 800 may be communicatively connected to the communication network 400 via satellite communication via an artificial satellite 500.

[0038] Furthermore, the system 1 may be configured redundantly, with multiple servers 300 connected to one drone 100 via multiple communication networks 400. In this case, even if an abnormality occurs in the server 300 or the communication network 400, the operation of the system 1, and therefore image capture by the drone 100, can be continued by the other redundant servers 300 and communication networks 400, thereby improving the reliability of the system 1. Note that, in the above two configurations, the drone 100 and the controller 200 can be controlled even when they are remotely located, and therefore the configuration is suitable for remote control, but is not limited to this and can also be applied to visual flight, in which the pilot manually controls the drone 100 while watching it.

[0039] The device described in the above embodiment may be realized as a single device, or may be realized by a plurality of devices (e.g., drone 100, controller 200, cloud server 300) that are partly or entirely connected via communication network 400. For example, each functional unit and memory unit of server 300 may be implemented in a different server 300, drone 100, and controller 200 that are connected to each other via communication network 400.

[0040] (A-1-2. Drone 100) (A-1-2-1. Overview of the drone 100) Fig. 2 is a simplified perspective view of the exterior of the drone 100 of this embodiment. Fig. 3 is a functional configuration diagram of the drone 100 of this embodiment. As described above, the drone 100 takes aerial photographs of competitions taking place at the stadium F (Fig. 7), events taking place at event venues, and the like.

[0041] In this specification, the term "drone" refers to any flying object that has the ability to autonomously control its attitude, regardless of the power source (electricity, prime mover, etc.), the control method (wireless or wired, fully autonomous flight type or partially manual flight type, etc.), and whether it is manned or unmanned. Drones are also sometimes called unmanned aerial vehicles (UAVs), flying objects, multicopters, RPAS (remote piloted aircraft systems), or UAS (unmanned aircraft systems), etc.

[0042] As shown in FIG. 2 , the exterior of the drone 100 is primarily composed of a housing 101 and multiple propellers 122. The housing 101 is, for example, a substantially rectangular parallelepiped, but may have any shape. Laterally extending rod-shaped connecting parts 102 are connected to the left and right sides of the housing 101. A propeller 122 and a motor 121 that rotates each propeller 122 are connected to the other end of each connecting part 102. The motor 121 is, for example, an electric motor. Note that, in this embodiment, four connecting parts 102, four propellers 122, and four motors 121 are provided, but the number is not limited thereto. The propeller 122 may be a single propeller or multiple propellers arranged coaxially. The number and shape of the blades of each propeller are not particularly limited.

[0043] Additionally, a propeller guard (not shown) may be provided on the outside of the propeller 122 to prevent the propeller from interfering with obstacles.

[0044] The housing 101 holds, for example, a photographing camera 141 by a camera holder 142 below the housing 101. An obstacle detection camera 131 is also provided on the front surface of the housing 101. In this embodiment, the obstacle detection camera 131 is a so-called dual camera made up of two cameras that form a pair. The obstacle detection camera 131 is provided so as to capture an image in front of the drone 100. Note that the obstacle detection cameras 131 may be provided not only on the front surface but also on all surfaces of the housing 101, for example, on six surfaces in the case of a housing 101 that is a substantially rectangular parallelepiped.

[0045] The drone 100 is equipped with an alarm device 250 that alerts people around the drone 100 to the presence of the drone 100. The alarm device 250 has, for example, a warning light 251 and a speaker 252. The warning light 251 is provided for each propeller 122 or motor 121, and is disposed, for example, on each side of multiple motors 121. The warning light 251 may be disposed along the cylindrical side surface of the motor 121 so that it can be seen from all directions, including the front. The speaker 252 outputs an alarm sound and is provided in the housing 101 of the drone 100. The speaker 252 is provided, for example, on the underside of the housing 101, and transmits the alarm sound downward from the drone 100.

[0046] (A-1-2-2. Functional blocks of drone 100) As shown in Figure 3, the drone 100 is equipped with an arithmetic unit such as a CPU (Central Processing Unit) for executing information processing, and storage devices such as a RAM (Random Access Memory) and a ROM (Read Only Memory), and thereby has the following functional blocks: a measurement unit 110, a flight function unit 120, an obstacle detection unit 130, an imaging unit 140, and a communication unit 150.

[0047] The measurement unit 110 is a functional unit that measures information related to the drone 100 or its surroundings. The measurement unit 110 includes, for example, a position measurement unit 111, a direction measurement unit 112, an altitude measurement unit 113, and a speed measurement unit 114. In addition to these, the measurement unit 110 may also include various sensors that acquire information such as temperature, air pressure, wind speed, and acceleration.

[0048] The position measurement unit 111 receives signals from the artificial satellites 500 and measures the position (absolute position) of the aircraft based on the signals. The position measurement unit 111 measures its current position using, for example, a Global Navigation Satellite System (GNSS) or a Global Positioning System (GPS), although this is not particularly limited. As a method for measuring its position, for example, a Real Time Kinematic - Global Navigation Satellite System (RTK-GNSS) can also be used. The position information includes at least two-dimensional coordinate information (e.g., latitude and longitude) in a planar view, and preferably includes three-dimensional coordinate information including altitude information.

[0049] Furthermore, the base station 800, which provides information on the reference point of a fixed station used for relative positioning such as RTK, is wirelessly connected to the drone 100 and the controller 200, thereby enabling measurement of the position of the drone 100 with higher accuracy. When RTK measurement is performed using a virtual reference point method with a VRS (Virtual Reference Station), the base station 800 can be omitted, or the accuracy of the position coordinate estimation of the base station 800 or the drone 100 can be further improved.

[0050] The orientation measurement unit 112 measures the orientation (nose direction, heading direction) of the aircraft. The orientation measurement unit 112 is composed of a geomagnetic sensor that measures the nose orientation (heading direction) of the aircraft of the drone 100 by measuring geomagnetism, a compass, etc.

[0051] The altitude measurement unit 113 measures the altitude above ground (hereinafter also referred to as "flight altitude") as the distance from the ground below the drone 100 (vertically downward).

[0052] The speed measurement unit 114 detects the flight speed of the drone 100. The speed measurement unit 114 may measure the speed using a known sensor such as a gyro sensor.

[0053] (A-1-2-3. Flight function unit 120) The flight function unit 120 is a mechanism and functional unit that causes the drone 100 to fly, and generates thrust in the airframe to lift the drone 100 and move it in a desired direction. As shown in Figures 2 and 3, the flight function unit 120 has multiple motors 121, multiple propellers 122, and a flight control unit 123.

[0054] The flight control unit 123 independently controls the multiple motors 121 to rotate each propeller 122, causing the drone 100 to perform various operations such as taking off, moving forward, turning, and landing, and controls the attitude angle control and flight operations of the drone 100 from takeoff to flight and landing.

[0055] The flight control unit 123 includes a processing unit, also referred to as a flight controller. The processing unit may include one or more processors, such as a programmable processor (e.g., a central processing unit (CPU), an MPU, or a DSP). The processing unit has access to a memory (storage unit). The memory stores logic, code, and / or program instructions that the processing unit can execute to perform one or more steps. The memory may include a separable medium, such as an SD card or RAM, or an external storage device. Various data acquired by the measurement unit 110 or video or still image data captured by the imaging camera 141 may be directly transmitted to and stored in the memory. Note that each data may also be recorded in an external memory.

[0056] The processing unit includes a control module configured to control the airframe state of the drone 100. For example, the control module controls the flight function unit 120 (thrust generating unit) of the drone 100 to adjust the spatial configuration, attitude angle, angular velocity, angular acceleration, angular jerk rate, and / or acceleration of the drone 100, which has six degrees of freedom (translational motion x, y, and z, and rotational motion θx, θy, and θz).

[0057] The flight control unit 123 can control the flight of the drone 100 based on control signals from the controller 200 or based on a preset autonomous flight program. The flight control unit 123 can also control the flight of the drone 100 by controlling the motor 121 based on various information such as the field to be photographed, flight permitted / prohibited areas, information on the corresponding flight geofences, map information including two-dimensional or three-dimensional map data, current position information of the drone 100, attitude information (heading information), speed information, and acceleration information, or any combination of these.

[0058] Examples of Shooting Target Field and Shooting Position In this specification, "shooting target field" or "target area" refers to a two-dimensional location (for example, stadium F) that is the subject of shooting.

[0059] 7 is a schematic diagram showing an example of a stadium F, which is an example of a field to be photographed by a drone, viewed from above. The stadium F is composed of a court F100, which is generally rectangular and defined by a linear outer edge, and an outer court area F200, which is a predetermined area that covers the outer edge of the court F100. The outer edge of the court F100 is formed by mutually opposing goal lines F110a, F110b and mutually opposing touch lines F111a, F111b, which are connected at approximately right angles. The connection points between the goal lines F110a, F110b and the touch lines F111a, F111b are corners F112a, F113a, F112b, and F113b.

[0060] Goals F120a and F120b are provided approximately in the center of the pair of goal lines F110a and F110b, respectively. Penalty areas F130a and F130b are defined in predetermined areas within the court F100 adjacent to the goals F120a and F120b, and penalty lines F140a and F140b are drawn on the outer edges of the penalty areas.

[0061] A halfway line F150 is drawn in the center of the court F100, connecting the midpoints of the pair of touchlines F111a and F111b to divide the court F100 into approximately equal parts. The halfway line F150 is approximately parallel to the goal lines F110a and F110b.

[0062] Note that the goal lines F110a, F110b, touchlines F111a, F111b, penalty lines F140a, F140b, and halfway line F150 are required by the rules for players to play the game, and therefore, all of these lines are generally drawn in a visible manner, but the technical scope of the present invention is not limited to this. Also, while this explanation uses a soccer stadium as an example, the games photographed by the system of the present invention are not limited to soccer and include any other game, such as tennis. Furthermore, the subject of photography is not limited to sports, and the system can also be applied to other events (concerts, ceremonies, etc.).

[0063] A plurality of predetermined shooting positions L101 to L105 and L206 to L215 are defined in the stadium F. The shooting positions L101 to L105 and L206 to L215 may be two-dimensional coordinates on a plane, or may be three-dimensional coordinate information that also defines the height at each position. The flight height of the drone 100 may be manually controlled based on input from the controller 200.

[0064] Photographing positions L101 to L105 are defined, for example, on the touchline F111b, at approximately equal intervals along the touchline F111b. For example, photographing position L101 is a point located in a range including the intersection of the halfway line F150 and the touchline F111b and a point slightly outside the court F100. Photographing positions L103 and L105 are points near the corners F112a or F112b on either side of the touchline F111b. Photographing positions L102 and L104 are points between photographing positions L103 and L105 and photographing position L101. Note that the above-mentioned positions are merely examples and are not limited to these, and any appropriate positions may be used.

[0065] Photographing positions L206 to L215 are points defined within the court F100. For example, photographing positions L206 and L211 are points near the center of the penalty lines F140a and F140b on a line parallel to the goal lines F110a and F110b, so-called goal-front photographing positions. Photographing positions L207 and L212 are photographing positions closer to the touchline F111a or F111b and closer to the halfway line F150 than photographing positions L206 and L215. More specifically, photographing positions L207 and L212 are points on an imaginary line connecting photographing position L101 and goals F120a and F120b, for example, approximately in the center of the imaginary line. Photographing positions L209 and L215 are points symmetrical to photographing positions L207 and L212. Photographing position L208 is a point between photographing position L207 and the halfway line F150, photographing position L210 is a point between photographing position L209 and the halfway line F150, photographing position L213 is a point between photographing position L212 and the halfway line F150, and photographing position L214 is a point between photographing position L215 and the halfway line F150.

[0066] In the off-court area F200, an evacuation point H200 is set to which the drone 100 is to be evacuated if an abnormality or malfunction of the drone 100 or the system 1 is detected. The abnormality referred to here refers to an abnormality related to the stability of the drone 100's aerial movement. The abnormality may include, for example, a case where the computational load associated with the operation control (behavior control, shooting control, etc.) of the drone 100 exceeds a load threshold. Alternatively, the abnormality may include a transient abnormality related to the environment, such as a case where a measured value of the drone 100's behavior control value (e.g., speed) exceeds an allowable value due to the influence of strong winds or the like.

[0067] The evacuation point H200 is set at a location different from the shooting positions L101 to L105 and L206 to L215, and in this embodiment, it is set outside the touchline F111a and along the touchline F111a. There may be multiple evacuation points H200; in this embodiment, there are three evacuation points. The evacuation point H220 is set near an extension of the halfway line F150. The evacuation points H210 and H230 are set closer to the goals F120a and F120b than the shooting positions L206 and L211. The evacuation points H210 and H230 are set, for example, at the ends of the area defined by the geofence G200 described below. At the evacuation point H200, for example, the drone 100 is replaced or the battery installed in the drone 100 is replaced.

[0068] In the stadium F, multiple geofences G100 and G200 are set, encompassing at least the respective shooting positions L101-L105 and L206-L215. A geofence indicates a virtual boundary line that divides an area. In particular, the geofence in this embodiment indicates a fence that separates a permitted flight area where the drone 100 is permitted to fly or move from a prohibited flight area. A geofence is a boundary line that divides an area that extends three-dimensionally, including both planar and vertical dimensions. When a moving object such as the drone 100 is detected as having come into contact with a geofence or departing the geofence, flight or movement is restricted to prevent the drone from flying outside the permitted flight area. That is, for example, the flight control unit 340, 350, or 360 described below lands the drone 100. The flight control unit 340, 350, or 360 may also cause the drone 100 to hover. In addition, the flight control unit 340, 350, or 360 may move the drone 100 toward the inside of the geofence G100, the geofence G200, or the third geofence. Detailed control of the geofence will be described later.

[0069] The boundary line of the geofence in the height direction may include an upper limit and a lower limit. In this embodiment, the geofences G100 and G200 that are applied to flight permission and prohibition are switched in accordance with the control of the system 1 while the drone 100 is flying. Although the number of geofences G100 and G200 depicted in the figure is two, the number is arbitrary, and specifically may be three or more.

[0070] The geofence G100 is an area that encompasses the shooting positions L101 to L105 and defines an area that encompasses the area on and near the touchline F111b. In other words, the geofence G100 is defined near the outer edge of the court F100, with a portion extending into the outer court area F200. The geofence G200 is a geofence that is primarily applied in the outer edge flight mode M102, which will be described later. The geofence G200 is an area that encompasses the shooting positions L206 to L215 and is set at least inside the court F100. This geofence G200 is a geofence that is primarily applied in the inner court flight mode M105, which will be described later.

[0071] The areas defined by the multiple geofences G100, G200 at least partially contact or overlap with each other. The areas defined by the multiple geofences G100, G200 also overlap in the height direction. The heights of the multiple geofences G100, G200 may be different from each other. Specifically, the lower limit of the altitude of the geofence G200 set inside the stadium F is set higher than the lower limit of the altitude of the geofence G100 set on the outer edge of the stadium F. Because there is a high probability that subjects to be photographed, such as players, are present inside the stadium F, setting the lower limit of the altitude of the geofence high and flying the drone 100 at a sufficient height can prevent interference with the movements of the subjects to be photographed and collisions with the subjects to be photographed or the ball.

[0072] Figure 8 is a diagram illustrating an embodiment of a geofence defined near the outer edge of the court F100. Figure 8(a) is a diagram illustrating the geofences G100 and G200 described in Figure 7. Figure 8(b) is another example of a geofence defined near the outer edge of the court F100. In this example, instead of the geofence G100, a geofence G100a is placed that defines an area that covers the entire perimeter of the court F100.

[0073] 8(c) is a diagram showing an example of a geofence G100b that defines an area covering two adjacent sides of the outer edge of the court F100, instead of the geofence G100. The geofence G100b defines an L-shaped area connecting an area along the goal line F110a and an area along the touchline F111a. FIG. 8(d) is a diagram showing an example of a geofence G100c that defines an area along the goal line F110b, instead of the geofence G100.

[0074] The geofence may be set in advance or may be set by the user. For example, if there is a fixed obstacle near the court F100, the user can set the geofence so as not to interfere with the obstacle. In addition to the geofence, the user can also set an area in which the drone 100 can fly when the geofence is set, and store the area in association with the geofence.

[0075] (A-1-2-4. Obstacle detection unit 130) Returning to the explanation of Figure 3, the obstacle detection unit 130 is a functional unit that detects obstacles around the drone 100. Obstacles may include, for example, people, players, objects, animals such as birds, fixed equipment, and the ball. The obstacle detection unit 130 measures the position, velocity vector, etc. of obstacles located below the drone 100, etc., based on the acquired image.

[0076] The obstacle detection unit 130 includes, for example, an obstacle detection camera 131, a ToF (Time of Flight) sensor 132, and a laser sensor 133. The ToF sensor 132 measures the time it takes for a laser pulse emitted from the sensor to return to a light-receiving element within the sensor, and converts this time into distance to measure the distance to an object. The laser sensor 133 uses, for example, a LiDAR (Light Detection and Ranging) method to irradiate an object with light such as near-infrared light, visible light, or ultraviolet light, and captures the reflected light with an optical sensor to measure the distance.

[0077] In FIG. 2, in this embodiment, the obstacle detection camera 131 is shown to be positioned facing forward, but the type, position and number of the camera 131, the ToF sensor 132 and the laser sensor 133 are arbitrary, and the ToF sensor 132 or the laser sensor 133 may be positioned instead of the camera 131, or the ToF sensor 132 or the laser sensor 133 may be provided on all six surfaces of the housing 101, i.e., the front, back, top, bottom and both side surfaces.

[0078] (A-1-2-5. Filming Unit 140) The filming unit 140 is a functional unit that captures video of competitions at the stadium F (FIG. 7) or events at the event venue, and includes a filming camera 141, a camera holding unit 142, and a filming control unit 143. As shown in FIG. 2, the filming camera 141 (imaging device) is disposed at the bottom of the main body of the drone 100, and outputs image data relating to peripheral images captured around the drone 100. The filming camera 141 is a video camera (color camera) that captures video. The video may include audio data acquired by a microphone (not shown). Additionally or alternatively, the filming camera 141 may also capture still images.

[0079] The orientation of the photographic camera 141 (the attitude of the photographic camera 141 relative to the housing 101 of the drone 100) can be adjusted by a camera actuator (not shown) incorporated in the camera holding unit 142. The photographic camera 141 may have an automatic control function for parameters such as exposure, contrast, or ISO. The camera holding unit 142 may have a so-called gimbal control mechanism that suppresses transmission of shaking or vibration of the aircraft to the photographic camera 141. The photographing control unit 143 controls the photographic camera 141 and the camera holding unit 142 to adjust the orientation of the photographic camera 141, the photographing magnification (zoom amount), the camera's photographing conditions, etc. Image data acquired by the photographic camera 141 can be transmitted to a memory unit of the drone 100 itself, the controller 200, the server 300, etc.

[0080] (A-1-2-6. Communication Unit 150) The communication unit 150 is capable of radio wave communication via the communication network 400 and includes, for example, a radio wave communication module. The communication unit 150 is capable of communication with the controller 200 and the like via the communication network 400 (including the wireless base station 800).

[0081] (A-1-3. Controller 200) (A-1-3-1. Overview of Controller 200) FIG. 4 is a simplified front view of the exterior of the controller 200 of this embodiment. FIG. 5 is a functional configuration diagram of the controller 200 of this embodiment. The controller 200 is a mobile information terminal that controls the drone 100 through operation by the operator and displays information received from the drone 100 (e.g., position, altitude, remaining battery level, camera footage, etc.). Note that in this embodiment, the flight status (altitude, attitude, etc.) of the drone 100 may be remotely controlled by the controller 200, or may be autonomously controlled by the drone 100. For example, when the operator transmits a flight command to the drone 100 via the controller 200, the drone 100 performs autonomous flight. Furthermore, manual operation may be possible during basic operations such as takeoff and return, and in emergencies.

[0082] As shown in Fig. 4, controller 200 includes, as its hardware configuration, a display unit 201 and an input unit 202. Display unit 201 and input unit 202 are connected to each other so that they can communicate with each other via wire or wirelessly. Display unit 201 may be configured as a touch panel or LCD monitor that is integrally incorporated into controller 200, or may be configured as a display device such as an LCD monitor, tablet terminal, or smartphone that is connected to controller 200 via wire or wirelessly. Display unit 201 as a hardware configuration may be configured as a touch panel display by integrally incorporating an element that accepts input such as touch.

[0083] The input unit 202 is a mechanism through which the pilot inputs operational commands such as flight direction and takeoff / landing when piloting the drone 100. As shown in FIG. 4A , the input unit 202 includes a left slider 326L, a right slider 326R, a left input stick 327L, a right input stick 327R, a power button 328, and a return button 329. The left slider 326L and the right slider 326R are controls that accept, for example, 0 / 1 input or one-dimensional, stepless or gradual information input. The pilot performs input by sliding the sliders with their left and right index fingers while holding the controller 200 in their hands. The left input stick 327L and the right input stick 327R are controls that accept multi-dimensional, stepless or gradual information input, such as joysticks. The left input stick 327L and the right input stick 327R may also accept 0 / 1 input by pressing them. The power button 328 and the feedback button 329 are operators that are pressed down, and are configured by mechanical switches or the like.

[0084] The left input stick 327L and the right input stick 327R accept input operations that instruct three-dimensional flight operations of the drone 100, including takeoff, landing, ascent, descent, right turn, left turn, forward movement, backward movement, left movement, and right movement. Fig. 4(b) is a schematic diagram showing the movement direction or rotation direction of the drone 100 corresponding to each input of the left input stick 327L and the right input stick 327R shown in Fig. 4(a). Note that this correspondence is an example.

[0085] As shown in FIG. 5, the controller 200 includes a computing device such as a CPU for executing information processing, and storage devices such as RAM and ROM, which constitute the software configuration, primarily functional blocks of a display control unit 210, an input control unit 220, and a communication unit 240.

[0086] (A-1-3-2. Display control unit 210) The display control unit 210 displays to the pilot status information of the drone 100 and the like obtained from the drone 100 or the server 300. The display control unit 210 can display images relating to various information such as the field to be photographed, permitted / prohibited flight areas, flight geofence, map information, current position information of the drone 100, attitude information (directional information), speed information, acceleration information, and remaining battery power. The "current position information" referred to here is sufficient to include information on the horizontal position of the current position of the drone 100 (i.e., latitude and longitude), and does not have to include altitude information (absolute altitude or relative altitude).

[0087] The display control unit 210 has a mode display unit 211 and a shooting state display unit 212 .

[0088] The mode display unit 211 is a functional unit that displays at least the state, i.e., the mode, to which the drone 100 belongs on the display unit 201. The mode to which the drone 100 belongs is, for example, the flight mode shown in Fig. 9, but instead of or in addition to this, the display unit 201 may display the aircraft state shown in Fig. 10, the aircraft action state shown in Fig. 11, the game state shown in Fig. 12, or the offensive and defensive states shown in Fig. 13.

[0089] As shown in Figure 19, the screen G1 displayed on the display unit 201 displays, for example, a display field G21 for the flight mode to which the drone 100 belongs, as well as a status display field G22 showing the aircraft status, aircraft behavior status, game status, and offensive and defensive status.

[0090] The shooting status display unit 212 shown in Fig. 5 is a functional unit that displays, on the display unit 201, images captured by the shooting camera 141 mounted on the drone 100. As shown in Fig. 19, the screen G1 displayed on the display unit 201 displays, for example, an image field G40 in which images being captured by the drone 100 are displayed. Details of the screen G1 and each mode will be described later.

[0091] 5 accepts various inputs from a user such as a pilot. The input control unit 220 of this embodiment mainly has the following functional units: an aircraft position operation unit 221, an aircraft attitude operation unit 222, a camera attitude operation unit 223, a camera zoom operation unit 224, a flight mode switching unit 225, a target position acceptance unit 226, a power supply input unit 227, and a return input unit 228.

[0092] The aircraft position operation unit 221 includes an up / down movement input unit 221a and a left / right movement input unit 221b. The aircraft attitude operation unit 222 includes a forward / backward movement input unit 222a and a yaw rotation input unit 222b.

[0093] The up / down movement input unit 221a is an input unit used by the pilot to move the drone 100 up and down, and acquires input to the right input stick 327R. That is, when the right input stick 327R is moved upward (toward the back when held in the hand), the drone 100 rises, and when the right input stick 327R is moved downward (toward the front when held in the hand), the drone 100 descends. The left / right movement input unit 221b is an input unit used by the pilot to move the drone 100 left and right, and acquires input to the right input stick 327R. That is, when the right input stick 327R is moved to the right, the drone 100 moves right, and when the right input stick 327R is moved to the left, the drone 100 moves left.

[0094] The forward / backward movement input unit 222a is an input unit used by the pilot to move the drone 100 forward or backward, and acquires input to the left input stick 327L. That is, when the left input stick 327L is moved upward (toward the rear when held in the hand), the drone 100 moves forward, and when the left input stick 327L is moved downward (toward the front when held in the hand), the drone 100 moves backward. The yaw rotation input unit 222b is an input unit used by the pilot to yaw rotate the drone 100, and acquires input to the left input stick 327L. That is, when the left input stick 327L is moved to the right, the drone 100 turns right, and when the left input stick 327L is moved to the left, the drone 100 turns left.

[0095] The camera attitude operation unit 223 is an input unit for operating the camera holding unit 142 via the photography control unit 143 and for operating the orientation of the photography camera 141 relative to the housing 101 of the drone 100. The camera attitude operation unit 223 acquires input to the right slider 326R. The camera attitude operation unit 223 accepts operation of either or both of the pitch angle and yaw angle of the photography camera 141 relative to the housing 101.

[0096] The camera zoom operation section 224 is an input section for operating the imaging magnification of the imaging camera 141, and acquires input to the left slider 326L.

[0097] The flight mode switching unit 225 is an input unit for switching flight modes. Flight modes selectable by the flight mode switching unit 225 include at least, for example, an outer edge flight mode M102 (see FIG. 9 ), an inside-court flight mode M105 (see FIG. 9 ), and a fixed position flight mode M103 or M107 (see FIG. 9 ). The flight mode switching unit 225 accepts the switching of flight modes via, for example, a touch panel display integrated with the display unit 201.

[0098] The target position receiving unit 226 is a functional unit that receives input of a target shooting position to which the drone 100 should head. The target position receiving unit 226 receives input of a point on the stadium F. For example, in a state in which at least a portion of an image or schematic diagram of the stadium F is displayed on the display unit 201, the target position receiving unit 226 may receive input of the target shooting position via a touch panel display that is configured integrally with the display unit 201. In cases in which a point that can be selected as the target shooting position, i.e., a shooting position, is specified in advance, the target position receiving unit 226 may receive input of the selection of the target shooting position.

[0099] Flight Modes Here, we will explain the types of flight modes that can be set in drone 100 and examples of their state transitions. As shown in Fig. 9, the flight modes of drone 100 mainly include advance preparation mode M100, off-court takeoff and landing mode M101, outer edge flight mode M102, off-court fixed position flight mode M103, on-court entry mode M104, on-court flight mode M105, off-court exit mode M106, on-court fixed position flight mode M107, and on-court takeoff and landing mode M108.

[0100] The advance preparation mode M100 is a mode in which advance settings such as geofence setting are performed. The advance preparation mode M100 transitions to an off-court takeoff and landing mode M101. In this off-court takeoff and landing mode M101, the drone 100 takes off from point L101g (see FIG. 15 ). Note that in the off-court takeoff and landing mode M101, the drone 100 may take off from a point outside the court F100 that is different from point L101g.

[0101] The off-court takeoff and landing mode M101 is the mode to which the drone 100 belongs when control starts or ends. The drone 100 transitions from the off-court takeoff and landing mode M101 to the perimeter flight mode M102.

[0102] The outer edge flight mode M102 is a mode in which the drone flies above the outer edge along part or all of the outer edge of the court F100 to photograph the playing field F, and more specifically, is a mode in which the drone flies at one of the photographing positions L101 to L105 (FIG. 7). In the main embodiment in this description, the outer edge flight mode M102 is a mode in which the drone flies above the touch line F111b. However, the "outer edge" in which the drone flies in the outer edge flight mode M102 is a concept that includes not only directly above the touch line F111b but also slightly outside the court F100.

[0103] In outer flight mode M102, the drone 100 receives user instructions via the target position receiving unit 226 of the controller 200 and flies at one of the specified shooting positions L101 to L105. The shooting angle may be manually controlled in response to the user's instructions, or may be fixed at a predetermined angle. In outer flight mode M102, the drone 100 may change its shooting position while keeping the shooting angle fixed, thereby following and shooting a specific player using so-called dolly shooting.

[0104] The outer edge flight mode M102 can transition to an outside court takeoff and landing mode M101, an outside court fixed position flight mode M103, or an inside court approach mode M104.

[0105] The off-court fixed position flight mode M103 is a mode in which the drone 100 flies in a fixed position outside the area of ​​the court F100. The off-court fixed position flight mode M103 can transition to the outer edge flight mode M102. The on-court entry mode M104 is a mode in which a series of processes required for the drone 100 to enter the area of ​​the court F100 are performed. The drone 100 transitions to the on-court flight mode M105 via the on-court entry mode M104.

[0106] The in-court flight mode M105 is a mode in which the drone flies above the court F100 to photograph the playing field F, specifically, in which the drone flies at one of the photographing positions L206 to L215 (FIG. 7). In the in-court flight mode M105, as in the outer edge flight mode M102, the drone receives a user command to select a photographing position via the target position receiving unit 226 of the controller 200, and flies at one of the specified photographing positions L206 to L215. The photographing direction (photographing angle) may be manually controlled in response to a user instruction, or may be fixed at a predetermined angle.

[0107] The in-court flight mode M105 can transition to an out-of-court exit mode M106, an in-court fixed position flight mode M107, or an in-court takeoff and landing mode M108.

[0108] The court exit mode M106 is a mode in which the drone 100 performs a series of processes required for the drone 100 to exit the area of ​​the court F100. The drone 100 transitions to the outer perimeter flight mode M102 via the court exit mode M106. Note that the court exit mode M106 and the court entry mode M104 can transition to each other.

[0109] The on-court fixed position flight mode M107 is a mode in which the drone flies in a fixed position within the area of ​​the court F100. The on-court fixed position flight mode M107 can transition to the on-court flight mode M105. The on-court takeoff and landing mode M108 is a mode in which the drone takes off and lands within the area of ​​the court F100, and is a mode to which the drone transitions mainly when a command to land on the spot is issued by manual intervention. A drone that takes off in the on-court takeoff and landing mode M108 transitions to the on-court flight mode M105.

[0110] Returning to Figure 5, the power input unit 227 is a functional unit that receives a command to turn the power of the controller 200 on or off via a power button 328.

[0111] The return input unit 228 is a functional unit that accepts a command via the return button 329 to return the drone 100 located in the stadium F (Figure 7) to the target landing point L101g (see Figure 15).

[0112] Furthermore, instead of or in addition to the above configuration, the input control unit 220 may be configured to receive touch input to the display unit 201 and transmit a control command to the drone 100 in response to the input. More specifically, for example, when the user performs a selection operation on appropriate information such as a map or a schematic diagram displayed on the display unit 201, a route to the selected point may be automatically generated, causing the drone 100 to fly autonomously.

[0113] (A-1-3-4. Communication Unit 240) The communication unit 240 is a functional unit that transmits and receives signals between the controller 200 and appropriate components included in the system 1. The controller 200 has a communication function that performs wireless communication with the drone 100 using Wi-Fi, 2.4 GHz, and frequency bands of 5.6 to 5.8 GHz. The controller 200 also has a wireless communication function that enables communication with the server 300 via the communication network 400 using a communication standard such as LTE (Long Term Evolution). The communication unit 240 transmits, for example, various input signals from a user such as a pilot to the drone 100, the server 300, or the like. The communication unit 240 also receives signals from the drone 100, the server 300, or the like.

[0114] (A-1-4. Server 300) (A-1-4-1. Overview of Server 300) Fig. 6 is a functional configuration diagram of the server 300 of this embodiment. The server 300 manages or controls the flight and photography of the drone 100. The server 300 has an input / output unit (not shown) for inputting or outputting various types of information (image output, audio output).

[0115] The server 300 may be a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing.

[0116] The server 300 is equipped with an arithmetic unit such as a CPU for executing information processing, and storage devices such as RAM and ROM, and as a software configuration, it mainly comprises functional blocks such as a pre-setting unit 310, an event detection unit 320, a flight mode switching unit 330, an outer edge flight control unit 340, an in-court flight control unit 350, a fixed position flight control unit 360, a communication unit 370, and a memory unit 380.

[0117] (A-1-4-2. Pre-setting unit 310) The pre-setting unit 310 is a functional unit that performs settings necessary for the flight of the drone 100 before the drone 100 flies over the field to be photographed. The pre-setting unit 310 mainly has a geofence setting unit 311.

[0118] The geofence setting unit 311 is a functional unit that sets a geofence for the drone 100. The geofence includes information on the planar direction and the height direction.

[0119] The geofence setting unit 311 sets a geofence according to the flight mode. That is, for example, the geofence setting unit 311 activates the geofence G100 in the outer perimeter flight mode M102 (see FIG. 9 ). The geofence setting unit 311 also activates the geofence G200 in the inner court flight mode M105 (see FIG. 9 ). The geofence setting unit 311 also sets a geofence different from the geofence G100 or the geofence G200, a so-called third geofence, in intermediate modes that intervene in the transition between the outer perimeter flight mode M102 and the inner court flight mode M105, i.e., the outer court exit mode M106 and the inner court entry mode M104. The outer perimeter flight mode M102 and the inner court flight mode M105 are examples of the first or second flight mode in the claims. The outer court exit mode M106 and the inner court entry mode M104 are examples of the third flight mode in the claims.

[0120] When the geofence in the first flight mode and the geofence in the second flight mode, which is the flight mode at the destination, at least partially overlap, the third geofence is a geofence that covers an integrated area that combines the first area defined by the geofence in the first flight mode and the second area defined by the geofence in the second flight mode. In this embodiment, the geofence G100 in the outer edge flight mode M102 and the geofence G200 in the inner court flight mode M105 overlap. Therefore, the third geofence is a geofence that defines the area that combines the geofence G100 and the geofence G200.

[0121] Furthermore, if the geofence for the first flight mode and the geofence for the second flight mode do not overlap, the third geofence may be a geofence covering an area that combines a first area defined by the first geofence corresponding to the first flight mode, a second area defined by the second geofence corresponding to the second flight mode, and a gap between the first and second areas. This configuration ensures safety by preventing deviation from the geofence during mode transitions, even when transitioning between flight modes in which the geofences do not overlap. The geofence setting unit 311 stores information about the set geofence in the storage unit 380.

[0122] (A-1-4-3. Event detection unit 320) The event detection unit 320 is a functional unit that detects the state of the subject to be photographed or the drone 100. The event detection unit 320 detects an event based on the camera image from the photographing camera 141 or an input from the external system 700. The detection criteria for each event are stored in, for example, the storage unit 380, and the event detection unit 320 detects an event by referring to the storage unit 380. The event detection unit 320 may also detect an event by analysis using a neural network. The detection process by the event detection unit 320 can be performed using any known appropriate image analysis technology.

[0123] The event detection unit 320 particularly detects an event that triggers a change in the flight mode or shooting conditions of the drone 100. The event detection unit 320 mainly includes an aircraft state acquisition unit 321, an aircraft behavior state acquisition unit 322, a game state acquisition unit 323, and an offensive / defensive state acquisition unit 324.

[0124] The aircraft state acquisition unit 321 is a functional unit that acquires the aircraft state of the drone 100. Fig. 10 is a diagram showing the state transitions of the aircraft state of the drone 100. The aircraft states are broadly divided into, for example, a normal operation flight mode M200, a detection and judgment mode M210, and an action mode M220. When the drone 100 starts flying, the drone 100 transitions to the normal operation flight mode M200.

[0125] When any detection or reception occurs in the normal operation flight mode M200, the aircraft state transitions to a detection and judgment mode M210. The detection and judgment mode M210 includes an abnormality detection mode M211, a failure detection mode M212, a manual intervention mode M213, and a low battery mode M214.

[0126] Specifically, if an abnormality is detected in the normal operation flight mode M200, the mode transitions to the abnormality detection mode M211. This abnormality is a transient, reversible disturbance such as a decrease in radio wave intensity or strong wind. If the abnormality is resolved in the abnormality detection mode M211, the mode transitions to the normal operation flight mode M200.

[0127] In the normal operation flight mode M200, if a failure of the aircraft or system is detected, the drone 100 transitions to the failure detection mode M212. If a manual control command is received, the drone 100 transitions to the manual intervention mode M213. If it is detected that the remaining battery charge is lower than a predetermined value, the drone 100 transitions to the low battery mode M214. Furthermore, if a manual control command is received in the abnormality detection mode M211, the failure detection mode M212, or the low battery mode M214, the drone 100 transitions to the manual intervention mode M213. The drone 100 transitions to the action mode M220 corresponding to the detection and determination mode M210.

[0128] The action mode M220 is a state in which the drone 100 performs a series of operations preset for each state. The action mode M220 includes a landing at an evacuation point mode M221, an emergency stop mode M222, a landing in place mode M223, a return mode M224, and a fixed position flight mode M225.

[0129] The landing at evacuation point mode M221 is set to an operation of flying the drone 100 to the evacuation point H200 and landing it. The landing at evacuation point mode M221 is entered when the abnormality is not resolved in the abnormality detection mode M211.

[0130] The emergency stop mode M222 is set to stop the propellers 122 immediately. In the emergency stop mode M222, the drone 100 falls freely. The emergency stop mode M222 can be selected in the manual intervention mode M213 when the propellers 122 are about to come into contact with a person or object.

[0131] The on-the-spot landing mode M223 is set to perform a soft landing on the spot, and the return mode M224 is set to perform a return to the takeoff and landing point.

[0132] The fixed position flight mode M225 is a state in which the drone 100 flies at a fixed position, and can transition to the normal operation flight mode M200 based on a user operation. The user operation is input, for example, by selecting a button displayed on the display unit 201. Furthermore, in the fixed position flight mode M225, if an event that can cause a transition from the normal operation flight mode M200 to the detection and judgment mode M210, i.e., an abnormality, a malfunction, manual intervention, or a low battery, is detected, the drone 100 transitions from the fixed position flight mode M225 to the detection and judgment mode M210 via the normal operation flight mode M200. In the fixed position flight mode M225, the drone 100 can transition to the return mode M224 based on a user operation.

[0133] The drone 100 in the abnormality detection mode M211 and the failure detection mode M212 transitions to the evacuation point landing mode M221. The drone 100 in the manual intervention mode M213 transitions to one of the evacuation point landing mode M221, emergency stop mode M222, in-place landing mode M223, return mode M224, and fixed position flight mode M225, depending on the input command. The drone 100 in the low battery mode M214 transitions to the return mode M224.

[0134] The drone 100 in the normal operation flight mode M200 can also transition to the return mode M224 based on a user operation. The user operation is input by selecting a button displayed on the display unit 201, for example.

[0135] The aircraft behavior state acquisition unit 322 is a functional unit that acquires the aircraft behavior state of the drone 100. Each mode of the aircraft behavior state is a submode of the aircraft state that is performed to realize the transition of the aircraft state. Figure 11 is a diagram showing the state transition of the aircraft behavior state. The aircraft behavior state is broadly divided into, for example, a takeoff mode M300, an escape mode M310, a normal mode M320, and a landing mode M340.

[0136] Takeoff mode M300 is a mode in which the drone 100 takes off. The state transition of the aircraft's behavioral state starts from takeoff mode M300. When the drone 100 takes off, the aircraft's behavioral state transitions from takeoff mode M300 to evacuation mode M310 or normal mode M320. Evacuation mode M310 mainly includes an evacuation point arrival stationary mode M311 and an evacuation moving mode M312. Furthermore, normal mode M320 includes a point arrival stationary mode M321 and a moving mode M322. Evacuation mode M310 and normal mode M320 can transition to each other via temporary suspension mode M330. Note that this is just one example.

[0137] When transitioning from takeoff mode M300 to evacuation mode M310, the drone 100 in takeoff mode M300 transitions to evacuation point arrival stationary mode M311. The evacuation point arrival stationary mode M311 is a mode in which the drone moves to the evacuation point H200 and remains stationary there, i.e., hovers. When moving from the evacuation point H200 to another location for the purpose of evacuation, the drone 100 in the evacuation point arrival stationary mode M311 transitions to evacuation moving mode M312. When the drone 100 reaches a predetermined destination or there is no movement command due to manual intervention, the aircraft behavior state transitions from the evacuation point arrival stationary mode M311 or the evacuation moving mode M312 to the temporary suspension mode M330.

[0138] Furthermore, when transitioning from takeoff mode M300 to normal mode M320, the drone 100 in takeoff mode M300 transitions to point arrival stationary mode M321. Point arrival stationary mode M321 is a mode in which the drone moves to a predetermined destination and remains stationary, i.e., hovers. When moving to another location in normal use, the drone 100 in point arrival stationary mode M321 transitions to moving mode M322. When the drone 100 reaches the predetermined destination or no movement command is issued by manual intervention, the aircraft behavior state transitions from point arrival stationary mode M321 or moving mode M322 to pause mode M330.

[0139] The drone 100 in the evacuation point arrival stationary mode M311, the point arrival stationary mode M321, the moving mode M322, and the temporary suspension mode M330 can transition to the landing mode M340. The aircraft's behavior state ends processing in the landing mode M340.

[0140] The game status acquisition unit 323 shown in FIG. 6 is a functional unit that acquires the game status of a competition held at stadium F. The game status acquisition unit 323 detects the game status by processing captured images. The game status acquisition unit 323 may also acquire the game status based on decision-related information input by a referee to the external input device 600 or a referee support system, which is an example of the external system 700. Furthermore, the game status acquisition unit 323 may acquire the game status based on information input from the external input device 600 held by a team member, for example, a manager or coach.

[0141] FIG. 12 is a diagram showing an example of a state transition of a game state. This figure shows an example of a soccer game state. The game state includes a pre-game state M400, a normal play state M410, and an end-of-game state M460. The state transition starts from the pre-game state M400, and transitions from the pre-game state M400 to the normal play state M410. The normal play state M410 is a state in which the game is progressing. When the game ends, transition occurs from the normal play state M410 to the end-of-game state M460. It should be noted that a transition from the normal play state M410 to the end-of-game state M460 may occur not only at the end of the game, but also during a break during the game, such as halftime.

[0142] The game state also includes a play suspended without a foul play state M420 and a play suspended with a foul play state M440. When play is suspended without a foul play in the normal play state M410, the state transitions to the play suspended without a foul play state M420. The play suspended without a foul play state M420 is entered, for example, when the ball crosses the goal line F110a, F110b or the touchline F111a, F111b and goes out of the court. The play suspended without a foul play state M420 transitions to a throw-in state M421, a goal kick state M422, or a corner kick state M423 in accordance with events that occur according to the rules of the game, such as the type of line the ball crossed or the affiliation of the player who sent the ball out of the court. The throw-in state M421, the goal kick state M422, or the corner kick state M423 transitions to the normal play state M410 when the event in each state ends.

[0143] In the normal play state M410, if a foul occurs, or more precisely, if the referee recognizes the occurrence of a foul, the state transitions to the foul state M431. Furthermore, if an offside occurs or is recognized by the referee, the state transitions to the offside state M432. Then, from the foul state M431 or the offside state M432, the state transitions to the foul play suspended state M440. In the foul play suspended state M440, the state transitions to the free kick state M441 or the penalty kick state M442 depending on the location of the foul and the event that occurred. In the free kick state M441, a so-called indirect free kick may be taken instead of a free kick. The free kick state M441 may also be subdivided into a free kick state for the attacking team and a free kick state for the defending team. In the free kick state M441 and the penalty kick state M442, once the events in each state are completed, the game resumes and the game state transitions to the normal play state M410.

[0144] When the match ends, the normal play state M410 transitions to an after-match state M460, and the state transition of the match state ends.

[0145] Furthermore, the normal play state M410 can transition to a penalty shootout state M443. Although not shown in the drawings, the penalty shootout state M443 may transition to a state M460 after the match ends, thereby ending the state transition.

[0146] 12 , some game states may trigger flight mode switching, while other game states may not. For example, flight mode switching may be performed based on transitions to the shaded states in the figure, i.e., the pre-game state M400, goal kick state M422, corner kick state M423, free kick state M441, penalty kick state M442, player substitution state M450, and end-of-game state M460. Note that when transitioning from another game state to the normal play state M410, the flight mode may be switched to one corresponding to the offensive or defensive state.

[0147] The offensive and defensive status acquisition unit 324 is a functional unit that acquires the offensive and defensive status of the teams in the match held at the stadium F. The offensive and defensive status acquisition unit 324 detects the offensive and defensive status by processing the captured images. The offensive and defensive status acquisition unit 324 may also acquire the offensive and defensive status based on decision-related information input by the umpire to the external input device 600 or a referee support system, which is an example of the external system 700. Furthermore, the offensive and defensive status acquisition unit 324 may acquire the offensive and defensive status based on information input from the external input device 600 held by a team member, for example, a manager or coach.

[0148] 13 is a diagram showing an example of a state transition between offensive and defensive states. The figure shows an example of an offensive and defensive state in soccer. When a match starts, the offensive and defensive state transitions to an offensive state M510 or a defensive state M520. Furthermore, the offensive state M510 and the defensive state M520 transition to each other via an offensive / defensive change state M530 or an offensive / defensive uncertainty state M540.

[0149] The offensive state M510 is a state in which one pre-designated team (hereinafter also referred to as "Team A") is on the offensive. For example, the offensive state is a state in which Team A is in possession of the ball and advancing toward the other team (hereinafter also referred to as "Team B"), but is not limited to this and may be a predetermined state determined by any determination criterion stored in advance in the storage unit 380.

[0150] The offensive state M510 includes a team A offensive (own side) state M511, a team A offensive (opponent's side) state M512, and a team A quick attack state M513. The team A offensive (own side) state M511 and the team A offensive (opponent's side) state M512, and the team A offensive (own side) state M511 and the team A quick attack state M513 can transition to each other. Also, the team A quick attack state M513 can transition to the team A offensive (opponent's side) state M512.

[0151] The defensive state M520 includes a team A defensive (opponent's territory) state M521, a team A defensive (owner's territory) state M522, and a team B fast attack state M523. The team A defensive (owner's territory) state M521 and the team A defensive (owner's territory) state M522, and the team A defensive (opponent's territory) state M521 and the team B fast attack state M523 can transition to each other. In addition, the team B fast attack state M523 can transition to the team A defensive (owner's territory) state M522.

[0152] The offense / defense change state M530 and the offense / defense uncertain state M540 can be transitioned to from any of the following states: Team A offensive (home side) state M511, Team A offensive (opponent's side) state M512, Team A quick attack state M513, Team A defensive (opponent's side) state M521, Team A defensive (home side) state M522, and Team B quick attack state M523.

[0153] Instead of or in addition to the above-described acquisition units 321 to 324, the event detection unit 320 may determine an event based on input information from the external system 700. For example, the event detection unit 320 may determine a disturbance such as a strong wind as an event based on input information from a weather information system, which is an example of the external system 700. The event detection unit 320 may also determine an event based on input information from a court facility system, which is another example of the external system 700, or facility information input by a person involved with the court facility.

[0154] (A-1-4-4. Flight mode switching unit 330) The flight mode switching unit 330 is a functional unit that switches flight modes in accordance with the detection results by the event detection unit 320. The flight mode switching unit 330 mainly has a mode switching determination unit 331, a flight-permitted area switching unit 332, a geofence switching unit 333, and a flight path generation unit 334.

[0155] The mode switching determination unit 331 is a functional unit that determines whether or not a flight mode needs to be switched. The mode switching determination unit 331 determines whether or not a flight mode needs to be switched in accordance with an event detected by the event detection unit 320. The mode switching determination unit 331 references a table T1 (see FIG. 14 ) that associates the state detected by the event detection unit 320 with the flight mode in that state, and determines that a flight mode needs to be switched if the flight modes are different.

[0156] As shown in FIG. 14 , the event-flight mode table T1 is a table in which events detected as game states are associated with flight modes to be selected for those events. More specifically, for example, in the goal kick state M422 and the defending free kick state M441, the outer perimeter flight mode M102 is associated to avoid the risk of the ball hitting the drone 100. Also, in the penalty kick state M442 and the attacking free kick state M441, the inside court flight mode M105 is associated to capture the goal scene from close up. In the pre-game state M400, the penalty shootout state M443, and the player substitution state M450, the movement of the target to be focused on is small or nonexistent, so the fixed position flight mode M103 or M107 is associated. Whether the off-court fixed position flight mode M103 or the on-court fixed position flight mode M107 is to be used may be determined depending on the flight mode at the time of transition to the pre-match state M400, the penalty shootout state M443, or the player substitution state M450. In the post-match state M460, the on-court flight mode M105 is associated in order to photograph the facial expressions of the players up close.

[0157] With the above-described configuration, it is possible to appropriately photograph a competition or event taking place in the target area F using a flight mode suitable for the event.

[0158] The flight permission area switching unit 332 is a functional unit that switches the flight permission area in response to switching of the flight mode.

[0159] The geofence switching unit 333 is a functional unit that switches geofences in response to flight mode switching. For example, when the flight mode is the edge flight mode M102, the geofence switching unit 333 sets a geofence G100. Furthermore, when the flight mode is the inside-court flight mode M105, the geofence switching unit 333 sets a geofence G200. Furthermore, in the intermediate modes that intervene in the transition between the edge flight mode M102 and the inside-court flight mode M105, i.e., the inside-court entry mode M104 and the outside-court exit mode M106, a third geofence different from the geofences G100 and G200 in the edge flight mode M102 and the inside-court flight mode M105 is set.

[0160] The flight path generation unit 334 is a functional unit that generates a flight path of the drone 100 when moving with a flight mode switch. The flight path generation unit 334 determines, for example, a shooting position at which the drone transitions to the in-court entry mode M104 or the out-of-court exit mode M106. The flight path generation unit 334 also determines a shooting position at which the drone transitions from the in-court entry mode M104 to the in-court flight mode M105, or a shooting position at which the drone transitions from the out-of-court exit mode M106 to the outer edge flight mode M102. The flight path generation unit 334 also generates a specific flight path for the in-court entry mode M104 or the out-of-court exit mode M106. This flight path is generally generated within the area of ​​the third geofence.

[0161] (A-1-4-5. Outer edge flight control unit 340) The outer edge flight control unit 340 is a functional unit that controls the flight of the drone 100 in outer edge flight mode. The outer edge flight control unit 340 has a target position acquisition unit 341 and a flight path generation unit 342.

[0162] The target position acquisition unit 341 is a functional unit that acquires a target position to which the drone 100 should head. The target position acquisition unit 341 acquires a target position that is located within the range of the flight area in the outer flight mode M102. The target position acquisition unit 341 may acquire a target position that is input by the user and that is received via the target position receiving unit 226 of the controller 200, for example.

[0163] The flight path generation unit 342 is a functional unit that generates a flight path along which the drone 100 moves in the outer flight mode M102. In other words, the flight path generation unit 342 generates a flight path within the flight area available in the outer flight mode M102. When the drone 100 belongs to the flight area available in the outer flight mode M102 and the acquired target position also belongs to the flight area available in the outer flight mode M102, the flight path generation unit 342 generates a flight path from the current location to the target position. Furthermore, when the drone 100 moves across the outer flight mode M102 and the on-court flight mode M105, the flight path generation unit 342 may generate a flight path within the flight range in the outer flight mode M102. The flight path generation unit 342 moves the moving object along, for example, the touch line F111b.

[0164] When the ball crosses the touchline F111b where the drone 100 is flying and enters the off-court area F200, the flight path generation unit 342 moves the drone 100 to the off-court area F200, outside the touchline F111b. The flight path generation unit 342 may also cause the drone 100 to capture images directly below the drone 100 or from that point toward the court F100. This configuration enables tracking and capturing images of the ball even when the ball rolls into the off-court area F200. The geofence G100 for the outer flight mode M102 may be pre-set to extend beyond the touchline F111b to the outside of the court F100. This configuration allows the drone 100 to reliably remain within the geofence G100, even when the drone 100 flies slightly outside the touchline F111b to track the ball, as described above.

[0165] If the flight path generation unit 342 detects an obstacle on the flight path or near the drone 100, it regenerates a flight path that bypasses the obstacle toward the inside of the court F100. Furthermore, if the flight path generation unit 342 detects an obstacle, it may hover for a predetermined period of time and then move along the originally generated flight path. This is because, while the safety of the drone 100 is not ensured in the off-court area F200 of the stadium F, the safety of the drone 100 within the court F100 is highly likely to be ensured. Obstacles may be detected, for example, by the obstacle detection unit 130 of the drone 100 or by information from an external system 700, etc. Furthermore, if an obstacle is detected, the outer edge flight control unit 340 may switch to manual control after having the flight path generation unit 342 hover for a predetermined period of time. Furthermore, when an obstacle is detected, the outer edge flight control unit 340 may cause the flight path generation unit 342 to hover for a predetermined period of time, and then display a message via the display control unit 210 prompting the user to re-input the target position.

[0166] (A-1-4-6. Intra-court flight control unit 350) The intra-court flight control unit 350 is a functional unit that controls the flight of the drone 100 in intra-court flight mode M105. The intra-court flight control unit 350 has a target position acquisition unit 351 and a flight path generation unit 352. The target position acquisition unit 351 is a functional unit that acquires a target position located within the range of the flight area in intra-court flight mode M105.

[0167] The flight path generation unit 352 generates a flight path for the drone 100 to travel in the intra-court flight mode M105. That is, the flight path generation unit 352 generates a flight path within the flyable area in the intra-court flight mode M105. More specifically, the flight path generation unit 352 generates a flight path in the intra-court flight mode M105 by connecting multiple preset shooting positions. Like the flight path generation unit 342 of the outer edge flight control unit 340, the flight path generation unit 352 may generate a flight path within the flight range in the intra-court flight mode M105 when the current location and the target location belong to flyable areas of different flight modes.

[0168] If the flight path generation unit 352 detects an obstacle on the flight path or near the drone 100, it regenerates a flight path that bypasses the obstacle. The obstacle is detected, for example, by the obstacle detection unit 130. The flight path generation unit 352 may regenerate the flight path by changing the connection between multiple preset shooting positions, or may change the flight path to a higher altitude while maintaining the flight path on a plane. Furthermore, if the flight path generation unit 352 detects an obstacle, it may hover for a predetermined period of time and then resume movement along the originally generated flight path. Furthermore, if the intra-court flight control unit 350 detects an obstacle, it may cause the flight path generation unit 352 to hover for a predetermined period of time and then switch to manual control. Furthermore, if the intra-court flight control unit 350 detects an obstacle, it may cause the flight path generation unit 352 to hover for a predetermined period of time and then display a message via the display control unit 210 prompting the user to re-input the target position. The obstacle may be, for example, a bird, fixed equipment, or a player. The obstacles also include balls.

[0169] In this embodiment, flight control in the outer flight mode M102 is performed by the outer flight control unit 340, and flight control in the inner court flight mode M105 is performed by the inner court flight control unit 350. The specified shooting positions for each of the outer flight mode M102 and inner court flight mode M105 are presented as options, and a flight path to the selected target shooting position is generated. However, the technical scope of the present invention is not limited to this. The pilot 200 may control the shooting position and orientation of the drone 100 to fly freely within any location within the geofences G100 and G200 set for each flight mode.

[0170] Furthermore, the functional configuration of the flight path generation units 334, 342, and 352 is an example, and for example, a single flight path generation unit may generate the flight path without subdivision.

[0171] 15 is a diagram schematically illustrating the routes that the drone 100 can take, defined by the image capture positions L101 to L105, L206 to L215, and the evacuation point H200. Point L101g on the ground at image capture position L101 is the takeoff and landing point for the drone 100. The position transition of the drone 100 begins with the step of taking off from point L101g and arriving at image capture position L101. The drone 100 also descends at image capture position L101 and lands at point L101g, completing image capture.

[0172] At each of the image capture positions L101 to L105 and L206 to L215, the drone 100 may only be able to transition to adjacent image capture positions. For example, the point to which the drone 100 at image capture position L105 can transition while maintaining the outer edge flight mode is image capture position L104. Furthermore, the points to which the drone 100 at image capture position L105 can transition after switching to the inside court flight mode M105 are image capture positions L106 and L107. The flight path generation unit 334, 342, or 352 (see FIG. 6) generates a flight path for the drone 100 by referencing the transitionable path.

[0173] When the target position receiving unit 226 receives the selection of a shooting position, the drone 100 transitions to the selected shooting position via the available shooting positions. For example, when the drone 100 is at shooting position L105 and shooting position L215 is selected, the flight path generating unit 334, 342, or 352 (see FIG. 6 ) generates a flight path that transitions through shooting positions L105, L207, L208, L213, L212, and L215 in this order, and the drone 100 flies along this flight path.

[0174] Note that the drone 100 may transition to an adjacent image capture position when the transition of the image capture position involves a flight mode switch, and may transition directly to a non-adjacent image capture position when the transition does not involve a flight mode switch. For example, when the drone moves from image capture position L105 to image capture position L215, it may transition from image capture position L105 to image capture position L207 with a mode switch, and then move linearly within the area within the geofence G200 from image capture position L107 to image capture position L215.

[0175] The outer edge flight control unit 340 and the inner court flight control unit 350 autonomously fly the drone 100 in each flight area according to the flight mode. For example, the outer edge flight control unit 340 and the inner court flight control unit 350 may perform dolly photography within each flight area, i.e., cause the drone 100 to automatically track and photograph a specific object such as a ball or a designated player. The outer edge flight control unit 340 and the inner court flight control unit 350 may also automatically control the direction of the nose of the drone 100 or the direction of the photographing camera 141, i.e., the photographing direction. The outer edge flight control unit 340 and the inner court flight control unit 350 may also automatically control the flight height of the drone 100. The autonomous flight mode may differ depending on the flight mode. For example, dolly photography may be performed when controlled by the outer edge flight control unit 340, while automatic tracking photography with only the photographing direction fixed and the photographing position may be performed when controlled by the inner court flight control unit 350, or automatic tracking photography with both the position and the photographing direction may be performed. In addition, the outer edge flight control unit 340 and the in-court flight control unit 350 may generate a flight path within the court (within the stadium F) to move the drone 100 to a target position specified by the user in each flight area.

[0176] (A-1-4-7. Fixed position flight control unit 360) The fixed position flight control unit 360 is a functional unit that controls flight of the drone 100 in the off-court fixed position flight mode M103 and the on-court fixed position flight mode M107. In the fixed position flight mode, the fixed position flight control unit 360 hovers at a predetermined position and controls the direction of the nose or the direction of the shooting camera 141 to follow a specific player or the ball and perform automatic shooting. Note that the above-mentioned "control of direction" is a concept that includes control not only in the left-right direction (the so-called "pan direction") but also in the up-down direction (the so-called "tilt direction").

[0177] The fixed position flight control unit 360 includes a mode switching permission determination unit 361. The mode switching permission determination unit 361 is a functional unit that determines whether or not flight mode switching is permitted in the fixed position flight mode M103 or M107.

[0178] (A-1-4-8. Communication unit 370) The communication unit 370 has a modem or the like (not shown) and is capable of communicating with the drone 100, the controller 200, etc. via the communication network 400. The communication unit 370 may, for example, monitor the state of the drone 100 and its surroundings and notify the controller 200.

[0179] (A-1-4-9. Storage unit 380) The storage unit 380 is a functional unit, such as a database, that stores information related to flight control of the drone 100. The storage unit 380 stores, for example, the coordinates of multiple shooting positions L101 to L105, L210 to L215 in the stadium F. These coordinates may be two-dimensional coordinates on a plane or three-dimensional coordinates that include information in the height direction.

[0180] The storage unit 380 also stores an event-flight mode table T1 shown in FIG. 14. Furthermore, the storage unit 380 may store strong wind conditions detected as events in association with fixed position flight modes M103 or M107. This is because in the case of strong winds, the drone 100 may not be able to fly in the intended direction, so it is safer not to move the drone 100. The event-flight mode table T1 is recorded in a rewritable manner. Furthermore, multiple event-flight mode tables T1 may be stored.

[0181] Flowcharts Figure 16 is a flowchart showing the overall flow of aerial photography control in this embodiment. Figure 17 is a subroutine of the flight restriction process S1002 in Figure 16. Figure 18 is a subroutine of the flight mode switching process S1010 in Figure 16.

[0182] The control shown in the flowchart of Fig. 16 is executed in a periodic loop. As shown in Fig. 16, if it is detected that the drone 100 is approaching the vicinity of the geofences G100 and G200 while flying (YES in step S1001), the process proceeds to flight restriction processing in step S1002. The subroutine of the flight restriction processing S1002 will be described in Fig. 17.

[0183] If it is not detected in step S1001 that the drone is approaching the vicinity of the geofences G100 and G200 (NO in step S1001), it is detected whether or not there is an obstacle in the path or near the drone 100 (step S1003). If an obstacle is detected in step S1003 (YES in step S1003), the drone 100 is caused to hover or a detour route is generated, and the flight route of the drone 100 is changed to the detour route (step S1004).

[0184] If no obstacle is detected in step S1003 (NO in step S1003), it is detected whether or not an action determination has been made on the aircraft state (step S1005). If an action determination on the aircraft state has been made in step S1005 (YES in step S1005), the process proceeds to step S1006, where an event type determination process is executed (step S1006).

[0185] If no action is detected in step S1005 (NO in step S1005), it is determined whether or not there is an input from the controller 200 by the user (step S1007). If an input from the controller 200 is detected in step S1007 (YES in step S1007), a command based on the input is executed (step S1008).

[0186] If no input from the controller 200 is detected in step S1007 (NO in step S1007), it is determined whether or not an event has occurred (step S1009). If an event has been detected (YES in step S1009), the process proceeds to flight mode switching processing in step S1010 (step S1010). If no event has been detected in step S1009 (NO in step S1009), the process returns to step S1001 and repeats steps S1001 to S1009.

[0187] As shown in Figure 16, the overall processing of aerial photography control is performed in the following order: geofence restriction, obstacle detection, control based on aircraft status, control based on user input, and control based on in-game events such as the game status or offensive / defensive status. In other words, each control is executed before control based on in-game events. This order is determined by the priority of performing safe control processing. This configuration can more reliably ensure the safety of the drone 100 flight.

[0188] 17 , first, the flight control unit 123 of the drone 100 issues an operation command to restrict the drone 100 from advancing outside the geofence (step S1101). In step S1101, a restriction is set on the flight target position so that the drone 100 does not advance outside the geofence even when the drone 100 is manually operated.

[0189] Next, if the drone 100 does not advance outside the geofence (NO in step S1102), the processing ends.

[0190] If the drone 100 advances outside the geofence (YES in step S1102), the process proceeds to step S1103. Possible causes of this situation include, for example, the wind being too strong and causing the drone to be swept away, or a malfunction of the drone 100 preventing it from flying in the intended direction. In step S1103, the flight control unit 123 issues an operation command to return to the geofence. More specifically, in step S1103, a flight target position command to return to the geofence, i.e., an operation command to set a predetermined point inside the geofence as the flight target position, is issued to the drone 100.

[0191] Next, the information measured by the measurement unit 110 of the drone 100, such as information on position, orientation, altitude, or speed, is referenced to determine whether the drone 100 is approaching the inside of the geofence (step S1104). In this case, step S1104 is executed a predetermined time after step S1103. Note that in step S1104, it is sufficient if the drone 100 is closer to the inside of the geofence than at the time of step S1103, and it is not necessary to determine whether the drone 100 is located inside the geofence.

[0192] In step S1104, if it is determined that the drone 100 is not approaching the geofence (NO in step S1104), it is determined that the above operation command is ineffective, and the flight control unit 123 forces the drone 100 to land (step S1105).

[0193] If the drone 100 is approaching the geofence in step S1104 (YES in step S1104), the process proceeds to step S1106. In step S1106, information measured by the measurement unit 110 of the drone 100, such as the position and altitude, is referenced to determine whether the drone 100 is located within the geofence. If the drone 100 is located within the geofence (YES in S1106), the process ends. If the drone 100 is not located within the geofence (NO in S1106), the process returns to step 1104, and the drone 100 continues operating based on the operation command to return to the geofence until it returns to the geofence.

[0194] 18 is a diagram showing an example of a switching process flow for switching flight modes. Fig. 18 shows a process flow involving switching from outer edge flight mode M102 to inner court flight mode M105, and in particular, a process flow including switching flight modes and generating a flight path when moving from photographing position L101 to photographing position L215. Note that the following description will be given assuming that automatic switching of flight modes has occurred based on event detection, but the flight mode switching process itself is similar when the flight mode switch is input through manual intervention.

[0195] 18 , first, the flight mode switching unit 330 switches the flight mode from the outer edge flight mode M102 to the inner court entry mode M104, and the geofence switching unit 333 activates the geofence corresponding to the inner court entry mode M104 (step S1201). The geofence corresponding to the inner court entry mode M104 is an integrated geofence defined by an area combining the geofence G100 of the outer edge flight mode M102 and the geofence G200 of the inner court flight mode M105.

[0196] Next, the flight path generation unit 334 generates a flight path from the image capture position L101 to the image capture position L215 (step S1202). The flight path may be a flight path that transitions between adjacent image capture positions along the path shown in FIG. 15, or may be a flight path that linearly connects the image capture position L101 to the image capture position L215. Note that the flight path may be generated by the flight path generation unit 342 or 352 instead of or in addition to the flight path generation unit 334.

[0197] Next, it is determined whether the generated flight path is generated within the integrated geofence (step S1203). If at least a portion of the flight path is not generated within the integrated geofence (NO in step S1203), the flight path generation unit 334 corrects the flight path (step S1204), and the process returns to step S1203. This configuration prevents the drone 100 from going outside the geofence while moving, ensuring high safety. If the flight path is generated within the integrated geofence (YES in step S1203), the process proceeds to step S1205.

[0198] Next, a command to move to the shooting position L215 is sent (step S1205), and the drone 100 starts moving.

[0199] During movement, the system 1 determines whether or not the controller 200 has issued a command to exit the court F100 (step S1206). If no exit command has been received (NO in step S1206), step S1206 is continued until the drone 100 has completed its movement into the court F100 (NO in step S1207). Once the drone 100 has completed its movement into the court F100 (YES in step S1207), the flight mode switching unit 330 changes the flight mode to the in-court flight mode M105, and the geofence switching unit 333 switches the geofence to the geofence G200 corresponding to the in-court flight mode M105 (step S1208). The drone 100 continues flying, and when it reaches the destination, the photographing position L215 (YES in step S1220), the process ends.

[0200] In step S1206, if a command to exit court F100 is received from the controller 200 (YES in step S1206), the process proceeds to step S1209. Next, the flight path generation unit 334 generates a flight path to the input destination outside court F100 (step S1209). Next, the flight control unit 123 starts moving the drone 100 along that flight path. The drone 100 continues moving until it has completed moving to the destination outside court F100 (NO in step S1210), and when it has completed moving to that destination (YES in step S1210), the process proceeds to step S1211.

[0201] In step S1211, the flight mode switching unit 330 changes the flight mode to the outer edge flight mode M102, and the geofence switching unit 333 switches the geofence to the geofence G100 corresponding to the outer edge flight mode M102 (step S1211). Next, the drone 100 continues moving within the geofence G100, and when the movement to the destination is completed, the processing ends (step S1220).

[0202] Display Examples of the Display Unit 201 FIGS. 19 and 20 are examples of screens G1 and G2 displayed on the display unit 201 of the controller 200 by the display control unit 210. FIG.

[0203] The screen G1 shown in FIG. 19 displays a field map G10, which shows a bird's-eye view of the stadium F and the shooting positions L101-L215; an icon G11 indicating the position information of the drone 100; a shooting range G12 captured by the shooting camera 141; a display field G21 showing the flight mode to which the drone 100 belongs; a status display field G22 showing the status detected by the event detection unit, such as the aircraft status, aircraft behavior status, game status, and offensive / defensive status; a mode switch button G31 for switching between flight modes; a control switch button G32 for switching between automatic and manual control; and a video field G40 displaying images captured by the drone 100. In the example of FIG. 19, the drone is flying in edge flight mode at shooting position L101. The mode switch button G31 displays available modes, and in FIG. 19, it indicates that the drone can be switched to inside-court flight mode. The control switch button G32 also displays whether the drone can be switched to automatic or manual control. In FIG. 19, it is shown that switching to manual intervention is possible.

[0204] The position and shooting direction of the drone 100 may be controlled manually, or automatic tracking control of the ball or a specific player may be performed. When automatic tracking control is performed, information about the ball or specific player to be tracked may be displayed on the screen G1.

[0205] FIG. 20 shows an example of a screen G2 displayed by the display control unit 210 when the drone 100 is moving. FIG. 20 shows that the drone 100 is flying from the image capture position L208 to the image capture position L206 in the on-court flight mode M105. Screen G2 also displays information about the image capture position L206, which is the destination, i.e., the target position. More specifically, the drone 100 is flying between the image capture positions L206 and L208, and an arrow extending from the drone 100 toward the destination image capture position L206 is displayed. Furthermore, the target image capture position L206 is displayed in a different manner from the other image capture positions, e.g., highlighted. More specifically, the image capture position L206 is displayed in bold or a large font. Furthermore, the status display field G22 indicates that the aircraft behavior status is "moving."

[0206] The nose direction of the drone 100 does not necessarily have to be the direction of travel of the drone 100, but may be directed in any direction. Furthermore, the nose direction of the drone 100 does not have to be constant while moving, and for example, the drone 100 may move while photographing a player or the ball by yaw rotation.

[0207] [A-2. Effect of this embodiment] According to this embodiment, safety during aerial photography can be ensured. More specifically, when switching flight modes in which different geofences are set, by setting a third geofence that is different from the geofences before and after the switch, it is possible to ensure that the aircraft flies within the geofence even during the switch, prevent interference between the geofence and the flight path, and ensure safe switching between flight modes.

[0208] The present invention is not limited to the above-described embodiment, and it goes without saying that various configurations can be adopted based on the contents of the present specification.

[0209] The series of processes described in connection with the above embodiment may be implemented using software, hardware, or a combination of software and hardware. A computer program for implementing each function of the server 300 according to this embodiment may be created and installed on a PC or the like. A computer-readable recording medium storing such a computer program may also be provided. Examples of the recording medium include a magnetic disk, an optical disk, a magneto-optical disk, and a flash memory. The computer program may also be distributed, for example, via the communication network 400 without using a recording medium.

[0210] The flowcharts used in the above embodiments do not necessarily have to be executed in the order shown in the drawings. Some processing steps may be executed in parallel. In addition, additional processing steps may be employed, and some processing steps may be omitted.

[0211] 1 Aerial photography system 100 Drone (mobile body) 141 Photography camera 200 Controller 220 Input control unit 300 Server 320 Event detection unit 330 Flight mode switching unit 331 Mode switching determination unit 380 Memory unit F Stadium F100 Court F200 Area outside the court M102 Outer edge flight mode M105 Inner court flight mode

Claims

1. A moving object flying over a target area; a flight mode switching unit that switches flight modes in which a flyable area of the moving body and a geofence of the moving body that defines an area that includes the flyable area are associated with each other; Equipped with When switching from a first flight mode to a second flight mode, the flight mode switching unit transitions to the second flight mode via a third flight mode, The third geofence in the third flight mode is different from the first geofence in the first flight mode and the second geofence in the second flight mode, and the third geofence defines an area covering an integrated area that combines a first area defined by the first geofence and a second area defined by the second geofence. Mobile systems.

2. A mobile object flying over a target area; a flight mode switching unit that switches flight modes in which a flyable area of the moving body and a geofence of the moving body that defines an area that includes the flyable area are associated with each other; Equipped with When switching from a first flight mode to a second flight mode, the flight mode switching unit transitions to the second flight mode via a third flight mode, The third geofence in the third flight mode is different from the first geofence in the first flight mode and the second geofence in the second flight mode, and the third geofence defines an area covering an integrated area that combines a first area defined by the first geofence, a second area defined by the second geofence, and a gap between the first area and the second area. Mobile systems.

3. One of the first flight mode and the second flight mode is an edge flight mode in which the aircraft flies on the edge of the target area, and the other of the first flight mode and the second flight mode is an intra-target area flight mode in which the aircraft flies above the target area.

3. A mobile system according to claim 1 or 2.

4. a flight path generation unit that generates a flight path of the moving object in the third flight mode; The flight path generation unit changes the flight path when at least a portion of the flight path is generated outside the third geofence.

3. A mobile system according to claim 1 or 2.

5. When it is detected that the moving body has deviated from the outside of any one of the first geofence, the second geofence, or the third geofence that has been set, the moving body is caused to perform at least one of the following actions: landing, hovering, or moving toward the inside of the first area, the second area, or the integrated area.

3. A mobile system according to claim 1 or 2.