Aerial photographing system, aerial photographing method and aerial photographing program

Abstract

To secure safety in aerial photographing.SOLUTION: An aerial photographing system includes: a movable body which flies in a target area; an event detection unit which detects an event on the basis of an image acquired by a camera which images the target area or an input from an external system; and a mode switching determination unit which determines whether it is necessary to switch a flight mode for which at least a flight-available area of the movable body is defined. The flying mode includes at least an outer edge flight mode in which the mobile body flies above outer edges along part or all of the edges of a court formed within the target area, and an intra-court flying mode in which the mobile body flies above and within the court. The mode switching determination unit determines whether it is necessary to switch the flight mode according to the event detected by the event detection unit.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to an aerial photography system, an aerial photography method, and an aerial photography program. [Background technology]

[0002] Patent Document 1 discloses a camera viewpoint display system that detects the aircraft position and nose direction, as well as the pan and tilt angles of a camera device mounted on the aircraft, calculates the camera viewpoint from these pieces of data, and displays the viewpoint on a map on a monitor screen. In this system, an operator controls the aircraft's position and attitude, as well as the camera's shooting direction, while monitoring the aircraft's position and heading at a ground station. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2006-281830 A Summary of the Invention [Problem to be solved by the invention]

[0004] To effectively capture sports and other events, it is necessary to capture images that are suited to the details of the subject, such as the game situation. More specifically, it is preferable to capture images by switching between different capture modes, which define the flight area and capture direction of the aircraft, depending on the content. In addition, safety can be ensured by defining a geofence for the aircraft for each capture mode.

[0005] The system described in Patent Document 1 is complicated because it requires simultaneous control of the camera and the aircraft. Furthermore, the operation of switching between shooting modes in the system described in Patent Document 1 is cumbersome and not easy. In particular, when it is necessary to switch geofences in response to changes in the aircraft's flight area, failure to switch at the appropriate time can result in control interference, potentially preventing the aircraft from moving as intended.

[0006] The present invention has been made in consideration of the above-mentioned problems, and has an object to provide a mobile body system that can ensure safety during aerial flight. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, a mobile body system according to one aspect of the present invention comprises a mobile body that flies in a target area, and a flight mode switching unit that switches flight modes that correspond to a flyable area of ​​the mobile body and a geofence of the mobile body that defines an area that encompasses the flyable area, and when switching from a first flight mode to a second flight mode, the flight mode switching unit transitions to the second flight mode via a third flight mode, and the third geofence in the third flight mode is different from the first geofence in the first flight mode and the second geofence in the second flight mode.

[0008] The third geofence may define an area covering an integrated area that combines a first area defined by the first geofence and a second area defined by the second geofence.

[0009] The third geofence may define an area covering an integrated area that combines a first area defined by the first geofence, a second area defined by the second geofence, and a gap between the first area and the second area.

[0010] One of the first flight mode and the second flight mode may be an edge flight mode in which the aircraft flies along the outer edge of the target area, and the other of the first flight mode and the second flight mode may be an intra-target area flight mode in which the aircraft flies above the target area.

[0011] The device may further include a flight path generation unit that generates a flight path of the moving body in the third flight mode, and that changes the flight path when at least a portion of the flight path is generated outside the third geofence.

[0012] When it is detected that the moving body has deviated outside of any of the first geofence, the second geofence, or the third geofence that has been set, the moving body may be caused to perform at least one of the following actions: landing, hovering, or moving toward the inside of the first area, the second area, or the integrated area.

[0013] In addition, in order to achieve the above-mentioned object, an aerial photography system according to one aspect of the present invention comprises a moving body flying in a target area, an event detection unit that detects an event based on an image acquired by a camera photographing the target area or input from an external system, and a mode switching determination unit that determines whether or not to switch flight modes that define at least a flyable area of ​​the moving body, the flight modes including at least an outer edge flight mode in which the moving body flies above the outer edge along part or all of the outer edge of a court configured within the target area, and an intra-court flight mode in which the moving body flies above the interior of the court, and the mode switching determination unit determines whether or not to switch flight modes depending on the event detected by the event detection unit.

[0014] The mode switching determination unit may switch the flight mode in accordance with an event related to a match being held in the target area, which event is detected from an image captured by the camera.

[0015] In the in-court flight mode, the mobile body may further be provided with an in-court flight control unit that generates a flight path within the court for automatically following a specific player or ball in the target area, or a flight path for moving the mobile body to a target position specified by the user.

[0016] The device may further include a display control unit that displays information about the moving object on an operation screen, and the display control unit may be configured to display the specific player or the ball that is being followed on the operation screen when the moving object is flying using the automatic following.

[0017] The in-court flight control unit may generate the flight path in the in-court flight mode by connecting a plurality of preset shooting positions.

[0018] The device may further include a display control unit that displays information about the moving object on an operation screen, and the display control unit may be configured to display information about the destination shooting position on the operation screen while flying between the shooting positions in the in-court flight mode.

[0019] The target area is composed of the court and an area outside the court, which are partitioned by the connection of a pair of goal lines facing each other and a pair of touch lines facing each other, and further includes an outer edge flight control unit that controls the flight of the moving body in the outer edge flight mode, and the outer edge flight control unit may be configured to move the moving body along the touch lines in the outer edge flight mode and follow the moving body to a specific player or the ball to take automatic photographs.

[0020] The outer edge flight control unit may be configured to move the moving body to the outside court area outside the touch line when the ball goes beyond the touch line on which the moving body is flying and take a picture directly below the moving body or towards the court.

[0021] The flight modes may further include a fixed position flight mode in which the moving body flies at a fixed position, and may further include a fixed position flight control unit that controls the operation of the moving body in the fixed position flight mode, and the fixed position flight control unit may hover at a predetermined position in the fixed position flight mode, and control the direction of the nose or the direction of the camera to follow a specific player or the ball and automatically take photographs.

[0022] When the in-court flight control unit detects an obstacle on the flight path or near the moving body in the in-court flight mode, it may perform at least one of the following actions: changing the connection of the shooting position to regenerate a flight path that bypasses the obstacle, deciding to fly at a higher altitude than the flight path, starting to move the moving body on the flight path after hovering for a predetermined time, switching to manual control after hovering for a predetermined time, and displaying a message prompting the user to re-input the target position after hovering for a predetermined time.

[0023] The outer edge flight control unit may generate a flight path for the moving body in the outer edge flight mode, and when an obstacle is detected on the flight path or near the moving body, may perform at least one of the following actions: regenerate a flight path that bypasses the obstacle to the inside of the court; decide to fly at a higher altitude than the flight path; start moving the moving body on the flight path after hovering for a predetermined time; switch to manual control after hovering for a predetermined time; and display a message prompting the user to re-input the target position after hovering for a predetermined time.

[0024] In order to achieve the above-mentioned object, an aerial photography method according to another aspect of the present invention comprises an event detection step of detecting an event based on an image acquired by a camera photographing a target area or input from an external system, and a mode switching determination step of determining whether or not to switch flight modes, which determine at least a flyable area of ​​a mobile object flying within the target area, and the flight modes include at least an outer edge flight mode in which the mobile object flies above the outer edge of a court configured within the target area along part or all of the outer edge of the court, and an intra-court flight mode in which the mobile object flies above the interior of the court, and the mode switching determination step determines whether or not to switch flight modes depending on the event detected by the event detection step.

[0025] In order to achieve the above-mentioned object, an aerial photography program according to another aspect of the present invention causes a computer to execute an event detection command that detects an event based on an image acquired by a camera photographing a target area or input from an external system, and a mode switching determination command that determines whether or not to switch flight modes that define at least a flyable area for a moving object flying within the target area, the flight modes including at least an outer edge flight mode in which the object flies above the outer edge of a court configured within the target area along part or all of the outer edge of the court, and an intra-court flight mode in which the object flies above the court, and the mode switching determination command determines whether or not to switch flight modes depending on the event detected by the event detection command. The computer program can be provided by being stored on various data-readable recording media, or can be provided so as to be downloadable via a network such as the Internet. [Effects of the Invention]

[0026] According to the present invention, it is possible to ensure safety during aerial flight. [Brief explanation of the drawings]

[0027] [Figure 1] 1 is an overall configuration diagram of a mobile body system according to an embodiment of the present invention; [Figure 2] FIG. 1 is a simplified perspective view showing the appearance of a drone according to the embodiment. [Figure 3] FIG. 2 is a functional configuration diagram of the drone according to the embodiment. [Figure 4] (a) A simplified front view of the exterior of the control device of the embodiment, (b) a schematic diagram showing the direction in which the drone moves or turns in response to input from the control device. [Figure 5] FIG. 2 is a functional configuration diagram of the control device according to the embodiment. [Figure 6] FIG. 2 is a functional configuration diagram of a server according to the embodiment. [Figure 7]1 is a schematic diagram showing an example of a drone's shooting position that is set in advance in a field to be photographed in which the drone flies. FIG. [Figure 8] 10A to 10D are schematic diagrams showing examples of geofence settings in the field to be photographed, showing (a) a first example, (b) a second example, (c) a third example, and (d) a fourth example. [Figure 9] FIG. 2 is a schematic state transition diagram showing the transition of flight modes of the drone. [Figure 10] FIG. 1 is a schematic state transition diagram showing the state transition of the drone according to the drone's aircraft state. [Figure 11] 1 is a schematic state transition diagram showing the state transition of the drone according to the aircraft behavior state of the drone. [Figure 12] FIG. 10 is a schematic state transition diagram showing state transitions of a game state in a stadium as an example of a field to be photographed. [Figure 13] FIG. 2 is a schematic state transition diagram showing the state transition of the offensive and defensive states in the stadium. [Figure 14] 10 is a table showing an example of a correspondence relationship between the game state in the stadium and the flight mode of the drone. [Figure 15] 10 is a schematic diagram showing a photographing position and a flight path that can be transitioned from the photographing position. FIG. [Figure 16] 10 is a flowchart of control performed during flight of the drone. [Figure 17] 17 is a flowchart (details of S1002 in FIG. 16) relating to the control of flight restrictions in the drone. [Figure 18] 17 is a flowchart (details of S1010 in FIG. 16) relating to flight mode switching control in the drone. [Figure 19] FIG. 3 is a diagram showing a first example of a screen displayed on a terminal of the aerial photography system. [Figure 20] FIG. 10 is a diagram showing a second example of a screen displayed on a terminal of the aerial photography system. DETAILED DESCRIPTION OF THE INVENTION

[0028] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted. Further, the embodiments shown below are merely examples, and other known elements and alternative means can be adopted according to the use, purpose, scale, etc.

[0029] <A. One Embodiment> [A-1. Configuration] (A-1-1. Overall Configuration) FIG. 1 is an overall configuration diagram of an aerial photography system 1 (hereinafter also referred to as "system 1") according to an embodiment of the present invention. The system 1 is for aerial photography of competitions held in a stadium F (FIG. 7), events held in an event venue, etc. by a drone 100 (an example of a moving body). The stadium F is an example of a target area.

[0030] In the following description, the system 1 for photographing soccer will be described as an example as needed, but the present system 1 is also applicable to competitions and events other than soccer.

[0031] As shown in FIG. 1, in addition to the drone 100, the system 1 mainly includes a controller 200 for an operator to operate the drone 100, a server 300 for managing the flight and photography of the drone 100, an external input device 600, and an external system 700.

[0032] The drone 100 and the controller 200 are connected to each other via wireless communication (which may include that via a base station 800). The controller 200 and the server 300 are connected to each other via a communication network 400 such as an Internet line. The drone 100 acquires a satellite signal from a satellite 500 for specifying its own position and the like.

[0033] The external input device 600 is a device separate from the controller 200 that can send and receive information to and from the system 1, and is configured, for example, as a mobile terminal such as a smartphone or tablet. The external input device 600 can be operated, for example, by a manager, coach, bench player, referee, or court facility personnel of a game being played at the stadium F. The external input device 600 has a function for receiving, for example, an emergency command to stop filming, and the drone 100 performs emergency evacuation based on the command. The external input device 600 may also receive an input to switch the flight mode of the drone 100. Furthermore, the external input device 600 may be equipped with a display device that may display information similar to that displayed on the display unit 201 of the controller 200. In particular, the external input device 600 may acquire event information occurring during the game. The event information is referenced when a user of the external input device 600 makes an input to switch the flight mode of the drone 100.

[0034] The external system 700 may be any system configured separately from the system 1. For example, systems such as a court facility system, a match management system, and a referee support system may be applied as systems deployed for a competition held at the stadium F, and systems such as a weather observation system or an earthquake observation system deployed independently of the competition may also be applied. Multiple external systems 700 may be connected to the system 1. The system 1 may receive an emergency command to stop filming or a command to switch the flight mode of the drone 100 from various external systems 700. Furthermore, the various external systems 700 may acquire event information occurring during the competition.

[0035] A court equipment system as an example of external system 700 may, for example, obtain the brightness of a captured image from system 1 and control the brightness adjustment or blinking of the lighting in the stadium F. The court equipment system may also receive a request for lighting brightness from system 1 and control the brightness adjustment or blinking.

[0036] The configuration of system 1 is not limited to that shown in Fig. 1 , and the drone 100, controller 200, server 300, and base station 800 may be connected to each other so that they can communicate with each other via a communication network 400 such as an internet line. In this case, the drone 100 may communicate wirelessly directly with the communication network 400 using a communication method such as LTE without going through the controller 200. Therefore, the drone 100, controller 200, and base station 800 do not need to communicate wirelessly directly, and only need to be able to connect to the communication network 400 in remote locations. Therefore, this system configuration is suitable for cases where the drone 100 and controller 200 are located in remote locations (for example, when a pilot remotely controls the drone).

[0037] In addition, in system 1, the drone 100, the controller 200, the base station 800, and the server 300 are each connected to each other so that they can communicate with each other via a communication network 400 such as an Internet line, and the drone 100 and the base station 800 may be communicatively connected to the communication network 400 via satellite communication via an artificial satellite 500.

[0038] Furthermore, the system 1 may have multiple servers 300 connected to one drone 100 via multiple communication networks 400, i.e., the system may be redundant. In this case, even if an abnormality occurs in the server 300 or the communication network 400, the operation of the system 1, and therefore image capture by the drone 100, can be continued by the other redundant servers 300 and communication networks 400, thereby improving the reliability of the system 1. Note that in the above two configurations, the drone 100 and the controller 200 can be controlled even when they are remotely located, and therefore the configuration is suitable for remote control, but is not limited to this and can also be applied to visual flight in which the pilot manually controls the drone 100 while watching it.

[0039] The device described in the above embodiment may be realized as a single device, or may be realized by a plurality of devices (e.g., drone 100, controller 200, cloud server 300) that are partly or entirely connected via a communication network 400. For example, each functional unit and memory unit of server 300 may be implemented in a different server 300, drone 100, or controller 200 that are connected to each other via the communication network 400.

[0040] (A-1-2. Drone 100) (A-1-2-1. Overview of Drone 100) Fig. 2 is a simplified perspective view of the exterior of the drone 100 of this embodiment. Fig. 3 is a functional configuration diagram of the drone 100 of this embodiment. As described above, the drone 100 takes aerial photographs of competitions taking place at the stadium F (Fig. 7), events taking place at event venues, etc.

[0041] In this specification, the term "drone" refers to any flying object that has the ability to autonomously control its attitude, regardless of the power source (electricity, prime mover, etc.), the control method (wireless or wired, fully autonomous flight type or partially manual flight type, etc.), and whether it is manned or unmanned. Drones are also sometimes referred to as unmanned aerial vehicles (UAVs), flying objects, multicopters, RPASs (remote piloted aircraft systems), or UASs (unmanned aircraft systems), etc.

[0042] As shown in FIG. 2, the exterior of the drone 100 is mainly composed of a housing 101 and multiple propellers 122. The housing 101 is, for example, a substantially rectangular parallelepiped, but may have any shape. Rod-shaped connecting parts 102 extending laterally are connected to the left and right side surfaces of the housing 101. The other ends of the connecting parts 102 are respectively connected to propellers 122 and motors 121 that rotate the propellers 122. The motors 121 are, for example, electric motors. Note that in this embodiment, there are four connecting parts 102, four propellers 122, and four motors 121, but the number is not limited to this. The propeller 122 may be composed of a single propeller or multiple propellers arranged coaxially. The number and shape of the blades of each propeller are not particularly limited.

[0043] Additionally, a propeller guard (not shown) may be provided on the outside of the propeller 122 to prevent the propeller from interfering with obstacles.

[0044] In the housing 101, for example, a photographing camera 141 is held by a camera holder 142 below the housing 101. In addition, an obstacle detection camera 131 is disposed on the front surface of the housing 101. In this embodiment, the obstacle detection camera 131 is a so-called dual camera configured with two cameras that form a pair. The obstacle detection camera 131 is disposed so as to capture an image in front of the drone 100. Note that the obstacle detection cameras 131 may be disposed not only on the front surface but also on all surfaces of the housing 101, for example, on six surfaces in the case of a housing 101 that is a substantially rectangular parallelepiped.

[0045] The drone 100 is equipped with an alarm device 250 that alerts people around the drone 100 to the presence of the drone 100. The alarm device 250 has, for example, a warning light 251 and a speaker 252. The warning light 251 is provided for each propeller 122 or motor 121, and is disposed, for example, on each side surface of the multiple motors 121. The warning light 251 may be disposed along the cylindrical side surface of the motor 121 so that it can be seen from all directions, including the front. The speaker 252 outputs an alarm sound and is provided in the housing 101 of the drone 100. The speaker 252 is provided, for example, on the underside of the housing 101, and transmits the alarm sound downward from the drone 100.

[0046] (A-1-2-2. Drone 100 functional blocks) As shown in Figure 3, the drone 100 is equipped with an arithmetic unit such as a CPU (Central Processing Unit) for performing information processing, and storage devices such as a RAM (Random Access Memory) and a ROM (Read Only Memory), and thereby has the following main functional blocks: a measurement unit 110, a flight function unit 120, an obstacle detection unit 130, an imaging unit 140, and a communication unit 150.

[0047] The measurement unit 110 is a functional unit that measures information related to the drone 100 or its surroundings. The measurement unit 110 includes, for example, a position measurement unit 111, a direction measurement unit 112, an altitude measurement unit 113, and a speed measurement unit 114. In addition to these, the measurement unit 110 may also include various sensors that acquire information such as temperature, air pressure, wind speed, and acceleration.

[0048] The position measurement unit 111 receives signals from the artificial satellites 500 and measures the position (absolute position) of the aircraft based on the signals. The position measurement unit 111 measures its current position using, for example, a Global Navigation Satellite System (GNSS) or a Global Positioning System (GPS), although this is not particularly limited. As a method for measuring its position, for example, a Real Time Kinematic - Global Navigation Satellite System (RTK-GNSS) can also be used. The position information includes at least two-dimensional coordinate information (e.g., latitude and longitude) in a planar view, and preferably includes three-dimensional coordinate information including altitude information.

[0049] Furthermore, the base station 800, which provides information on the reference point of a fixed station used for relative positioning such as RTK, is connected to the drone 100 and the controller 200 so that the drone 100 can communicate wirelessly with the base station 800, thereby enabling the drone 100 to measure its position with higher accuracy. When RTK measurement is performed using a virtual reference point method using a VRS (Virtual Reference Station), the base station 800 can be omitted, or the accuracy of the position coordinate estimation of the base station 800 or the drone 100 can be further improved.

[0050] The orientation measurement unit 112 measures the orientation of the drone (nose direction, heading direction). The orientation measurement unit 112 is composed of a geomagnetic sensor that measures the nose orientation (heading direction) of the drone 100 by measuring geomagnetism, a compass, etc.

[0051] The altitude measurement unit 113 measures the altitude above the ground (hereinafter also referred to as "flight altitude") as the distance below (vertically downward from) the ground surface of the drone 100.

[0052] The speed measurement unit 114 detects the flight speed of the drone 100. The speed measurement unit 114 may measure the speed using a known sensor such as a gyro sensor.

[0053] (A-1-2-3. Flight function unit 120) The flight function unit 120 is a mechanism and function unit that causes the drone 100 to fly, and generates thrust in the airframe to lift the drone 100 and move it in a desired direction. As shown in Figures 2 and 3, the flight function unit 120 has multiple motors 121, multiple propellers 122, and a flight control unit 123.

[0054] The flight control unit 123 independently controls the multiple motors 121 to rotate each propeller 122, causing the drone 100 to perform various operations such as taking off, moving forward, turning, and landing, and controls the attitude angle control and flight operations of the drone 100 from takeoff to flight and landing.

[0055] The flight control unit 123 includes a processing unit, also referred to as a flight controller. The processing unit may include one or more processors, such as a programmable processor (e.g., a central processing unit (CPU), an MPU, or a DSP). The processing unit has access to a memory (storage unit). The memory stores logic, code, and / or program instructions that the processing unit can execute to perform one or more steps. The memory may include, for example, a separable medium such as an SD card or RAM, or an external storage device. Various data acquired by the measurement unit 110 or video or still image data captured by the imaging camera 141 may be directly transmitted to and stored in the memory. Note that each data may also be recorded in an external memory.

[0056] The processing unit includes a control module configured to control the airframe state of the drone 100. For example, the control module controls the flight function unit 120 (thrust generating unit) of the drone 100 to adjust the spatial configuration, attitude angle, angular velocity, angular acceleration, angular jerk rate, and / or acceleration of the drone 100, which has six degrees of freedom (translational motion x, y, and z, and rotational motion θx, θy, and θz).

[0057] The flight control unit 123 can control the flight of the drone 100 based on a control signal from the controller 200 or based on a preset autonomous flight program. The flight control unit 123 can also control the flight of the drone 100 by controlling the motor 121 based on various information such as the field to be photographed, flight permitted / prohibited areas, information on the corresponding flight geofences, map information including two-dimensional or three-dimensional map data, current position information of the drone 100, attitude information (heading information), speed information, and acceleration information, and any combination of these.

[0058] ●Examples of shooting fields and shooting positions In this specification, the term "field of interest" or "area of ​​interest" refers to a two-dimensional location (for example, stadium F) that is the subject of photography.

[0059] FIG. 7 is a schematic diagram showing an example of a stadium F, which is an example of a field to be photographed by a drone, as viewed from above. The stadium F is composed of a court F100, which is roughly rectangular and defined by a linear outer edge, and an outer court area F200, which is a predetermined area that covers the outer edge of the court F100. The outer edge of the court F100 is formed by mutually opposing goal lines F110a, F110b and mutually opposing touch lines F111a, F111b, which are connected at roughly right angles. The connection points between the goal lines F110a, F110b and the touch lines F111a, F111b are corners F112a, F113a, F112b, and F113b.

[0060] Goals F120a and F120b are provided approximately in the center of the pair of goal lines F110a and F110b, respectively. Penalty areas F130a and F130b are defined in predetermined areas within the court F100 adjacent to the goals F120a and F120b, and penalty lines F140a and F140b are drawn on the outer edges of the penalty areas.

[0061] A halfway line F150 is drawn in the center of the court F100, connecting the midpoints of a pair of touchlines F111a and F111b to divide the court F100 into approximately equal parts. The halfway line F150 is approximately parallel to the goal lines F110a and F110b.

[0062] The goal lines F110a, F110b, touchlines F111a, F111b, penalty lines F140a, F140b, and halfway line F150 are required by the rules for players to play, and therefore are generally drawn in a manner that makes them visible, but the technical scope of the present invention is not limited to this. Also, while this explanation uses a soccer stadium as an example, the sports that can be photographed using the system of the present invention are not limited to soccer and include any other sports, such as tennis. Furthermore, the subject of photography is not limited to sports, and the system can also be applied to other events (concerts, ceremonies, etc.).

[0063] A plurality of predetermined shooting positions L101-L105, L206-L215 are defined in the stadium F. The shooting positions L101-L105, L206-L215 may be two-dimensional coordinates on a plane, or may be three-dimensional coordinate information that also defines the height at each position. The flight height of the drone 100 may be manually controlled based on input from the controller 200.

[0064] Photographing positions L101 to L105 are defined, for example, on the touchline F111b, at approximately equal intervals along the touchline F111b. For example, photographing position L101 is a point located in a range including the intersection of the halfway line F150 and the touchline F111b and a point slightly outside the court F100. Photographing positions L103 and L105 are points near the corners F112a or F112b on both sides of the touchline F111b. Photographing positions L102 and L104 are points between photographing positions L103 and L105 and photographing position L101. Note that the above-mentioned positions are merely examples and are not limited to these, and any appropriate positions may be used.

[0065] Photographing positions L206 to L215 are points defined within the court F100. For example, photographing positions L206 and L211 are points near the center of the penalty lines F140a and F140b that are parallel to the goal lines F110a and F110b, and are so-called goal-front photographing positions. Photographing positions L207 and L212 are photographing positions closer to the touchline F111a or F111b and closer to the halfway line F150 than photographing positions L206 and L215. More specifically, photographing positions L207 and L212 are points on an imaginary line connecting photographing position L101 and goals F120a and F120b, and are, for example, points approximately in the center of the imaginary line. Photographing positions L209 and L215 are points symmetrical to photographing positions L207 and L212. Photographing position L208 is a point between photographing position L207 and the halfway line F150, photographing position L210 is a point between photographing position L209 and the halfway line F150, photographing position L213 is a point between photographing position L212 and the halfway line F150, and photographing position L214 is a point between photographing position L215 and the halfway line F150.

[0066] In the off-court area F200, an evacuation point H200 is set to which the drone 100 is to be evacuated if an abnormality or malfunction of the drone 100 or the system 1 is detected. The abnormality referred to here is an abnormality related to the stability of the drone 100's aerial movement. The abnormality may include, for example, a case where the computational load associated with the operation control (behavior control, shooting control, etc.) of the drone 100 exceeds a load threshold. Alternatively, the abnormality may include a transient abnormality related to the environment, such as a case where a measured value of the drone 100's behavior control value (e.g., speed) exceeds an allowable value due to the influence of strong winds or the like.

[0067] The evacuation point H200 is set at a location different from the shooting positions L101 to L105 and L206 to L215, and in this embodiment, it is set outside of and along the touchline F111a. There may be multiple evacuation points H200, and in this embodiment, there are three. The evacuation point H220 is set near an extension of the halfway line F150. The evacuation points H210 and H230 are set closer to the goals F120a and F120b than the shooting positions L206 and L211. The evacuation points H210 and H230 are set, for example, at the ends of an area partitioned by a geofence G200, which will be described later. At the evacuation point H200, for example, the drone 100 is replaced or the battery installed in the drone 100 is replaced.

[0068] In the stadium F, multiple geofences G100 and G200 are set, encompassing at least the respective shooting positions L101-L105 and L206-L215. A geofence indicates a virtual boundary line that divides an area. In particular, the geofence in this embodiment indicates a fence that separates a fly-permitted area where the drone 100 is permitted to fly or move from a no-fly area. A geofence is a boundary line that divides an area that extends three-dimensionally, including both planar and vertical dimensions. When a moving object such as the drone 100 is detected as having come into contact with a geofence or deviating outside the geofence, the flight or movement of the drone is restricted to prevent the drone from flying outside the fly-permitted area. That is, for example, the flight control unit 340, 350, or 360, described below, lands the drone 100. Alternatively, the flight control unit 340, 350, or 360 may cause the drone 100 to hover. Additionally, the flight control unit 340, 350, or 360 may move the drone 100 toward the inside of the geofence G100, the geofence G200, or the third geofence. Detailed control of the geofence will be described later.

[0069] The boundary line of the geofence in the height direction may include an upper limit and a lower limit. In this embodiment, the geofences G100 and G200 that are applied to flight permission and prohibition are switched according to the control of the system 1 while the drone 100 is flying. Although the number of geofences G100 and G200 depicted in the figure is two, the number can be any number, and specifically, may be three or more.

[0070] The geofence G100 is an area that encompasses the shooting positions L101 to L105, and defines an area that encompasses the area on and near the touchline F111b. In other words, the geofence G100 is defined near the outer edge of the court F100, and a portion of it extends into the outer court area F200. The geofence G200 is a geofence that is primarily applied in the outer edge flight mode M102, which will be described later. The geofence G200 is an area that encompasses the shooting positions L206 to L215, and is set at least inside the court F100. This geofence G200 is a geofence that is primarily applied in the inner court flight mode M105, which will be described later.

[0071] The areas defined by the multiple geofences G100, G200 at least partially contact or overlap each other. The areas defined by the multiple geofences G100, G200 also overlap in the vertical direction. The heights of the multiple geofences G100, G200 may differ from each other. Specifically, the lower limit of the altitude of the geofence G200 set inside the stadium F is set higher than the lower limit of the altitude of the geofence G100 set on the outer edge of the stadium F. Because there is a high probability that subjects to be photographed, such as players, are present inside the stadium F, setting the lower limit of the altitude of the geofence high and flying the drone 100 at a sufficient height can prevent interference with the movements of the subjects to be photographed and collisions with the subjects to be photographed or the ball.

[0072] FIG. 8 illustrates an embodiment of a geofence defined near the outer edge of the court F100. FIG. 8(a) is a diagram showing the geofences G100 and G200 described in FIG. 8(b) is another example of a geofence defined near the outer edge of the court F100. In this example, instead of the geofence G100, a geofence G100a is placed that defines an area that covers the entire perimeter of the court F100.

[0073] 8(c) is a diagram showing an example of a geofence G100b, which is an alternative to the geofence G100 and defines an area covering two adjacent sides of the outer edge of the court F100. The geofence G100b defines an L-shaped area connecting an area along the goal line F110a and an area along the touchline F111a. FIG. 8(d) is a diagram showing an example of a geofence G100c that defines an area along the goal line F110b, instead of the geofence G100.

[0074] The geofence may be set in advance or may be set by the user. For example, if there is a fixed obstacle near the court F100, the user can set the geofence so as not to interfere with the obstacle. In addition to the geofence, the user can also set an area in which the drone 100 can fly when the geofence is set, and store the area in association with the geofence.

[0075] (A-1-2-4. Obstacle detection unit 130) Returning to the explanation of Figure 3, the obstacle detection unit 130 is a functional unit that detects obstacles around the drone 100. Obstacles may include, for example, people, players, objects, animals such as birds, fixed equipment, and the ball. The obstacle detection unit 130 measures the position, velocity vector, etc. of obstacles located below the drone 100, etc., based on the acquired image.

[0076] The obstacle detection unit 130 includes, for example, an obstacle detection camera 131, a ToF (Time of Flight) sensor 132, and a laser sensor 133. The ToF sensor 132 measures the time it takes for a laser pulse emitted from the sensor to return to a light receiving element within the sensor, and converts this time into distance to measure the distance to an object. The laser sensor 133 uses, for example, a LiDAR (Light Detection and Ranging) method to irradiate an object with light such as near-infrared light, visible light, or ultraviolet light, and captures the reflected light with an optical sensor to measure the distance.

[0077] In FIG. 2, in this embodiment, the obstacle detection camera 131 is shown to be positioned facing forward, but the type, position and number of the camera 131, ToF sensor 132 and laser sensor 133 are arbitrary, and the ToF sensor 132 or laser sensor 133 may be positioned instead of the camera 131, or the ToF sensor 132 or laser sensor 133 may be provided on all six surfaces of the housing 101, i.e., the front, back, top, bottom and both side surfaces.

[0078] (A-1-2-5. Filming unit 140) The photographing unit 140 is a functional unit that photographs images of competitions at the stadium F (FIG. 7) or events at the event venue, and includes a photographing camera 141, a camera holding unit 142, and a photographing control unit 143. As shown in FIG. 2, the photographing camera 141 (imaging device) is disposed at the bottom of the main body of the drone 100, and outputs image data relating to peripheral images photographed around the drone 100. The photographing camera 141 is a video camera (color camera) that photographs moving images. The moving images may include audio data acquired by a microphone (not shown). Additionally or alternatively, the photographing camera 141 may also be configured to photograph still images.

[0079] The orientation of the photographic camera 141 (the attitude of the photographic camera 141 relative to the housing 101 of the drone 100) can be adjusted by a camera actuator (not shown) incorporated in the camera holding unit 142. The photographic camera 141 may have an automatic control function for parameters such as exposure, contrast, or ISO. The camera holding unit 142 may have a so-called gimbal control mechanism that suppresses transmission of shaking or vibration of the aircraft to the photographic camera 141. The photographing control unit 143 controls the photographic camera 141 and the camera holding unit 142 to adjust the orientation of the photographic camera 141, the photographing magnification (zoom amount), the camera's photographing conditions, etc. Image data acquired by the photographic camera 141 can be transmitted to a memory unit of the drone 100 itself, the controller 200, the server 300, etc.

[0080] (A-1-2-6. Communication unit 150) The communication unit 150 is capable of radio wave communication via the communication network 400 and includes, for example, a radio wave communication module. The communication unit 150 is capable of communication with the controller 200 and the like via the communication network 400 (including the wireless base station 800).

[0081] (A-1-3. Controller 200) (A-1-3-1. Overview of the Controller 200) FIG. 4 is a simplified front view of the exterior of the controller 200 of this embodiment. FIG. 5 is a functional configuration diagram of the controller 200 of this embodiment. The controller 200 is a mobile information terminal that controls the drone 100 through operation by the operator and displays information received from the drone 100 (e.g., position, altitude, remaining battery level, camera footage, etc.). Note that in this embodiment, the flight status (altitude, attitude, etc.) of the drone 100 may be remotely controlled by the controller 200, or may be autonomously controlled by the drone 100. For example, when the operator transmits a flight command to the drone 100 via the controller 200, the drone 100 performs autonomous flight. Furthermore, manual operation may be possible during basic operations such as takeoff and return, and in emergencies.

[0082] As shown in Fig. 4, controller 200 includes, as hardware components, a display unit 201 and an input unit 202. Display unit 201 and input unit 202 are connected to each other so that they can communicate with each other via wire or wirelessly. Display unit 201 may be configured as a touch panel or LCD monitor that is integrally incorporated into controller 200, or may be configured as a display device such as an LCD monitor, tablet terminal, or smartphone that is connected to controller 200 via wire or wirelessly. Display unit 201 as hardware components may be configured as a touch panel display by integrally incorporating an element that accepts input such as touch.

[0083] The input unit 202 is a mechanism through which the operator inputs operational commands such as flight direction and takeoff / landing when operating the drone 100. As shown in FIG. 4(a), the input unit 202 includes a left slider 326L, a right slider 326R, a left input stick 327L, a right input stick 327R, a power button 328, and a return button 329. The left slider 326L and the right slider 326R are controls that accept, for example, 0 / 1 input or one-dimensional, stepless or stepwise information input. The operator performs input by sliding the sliders with their left and right index fingers while holding the controller 200 in their hands. The left input stick 327L and the right input stick 327R are controls that accept multi-dimensional, stepless or stepwise information input, and are, for example, so-called joysticks. The left input stick 327L and the right input stick 327R may also accept 0 / 1 input by pressing them down. The power button 328 and the return button 329 are controls that accept pressing down and are configured, for example, by mechanical switches.

[0084] The left input stick 327L and the right input stick 327R accept input operations that instruct three-dimensional flight operations of the drone 100, including takeoff, landing, ascending, descending, turning right, turning left, moving forward, moving backward, moving left, and moving right. FIG. 4(b) is a schematic diagram showing the movement direction or turning direction of the drone 100 corresponding to each input of the left input stick 327L and the right input stick 327R shown in FIG. 4(a). Note that this correspondence is an example.

[0085] As shown in FIG. 5, the controller 200 includes a computing device such as a CPU for executing information processing, and storage devices such as RAM and ROM, which constitute the software configuration, primarily functional blocks of a display control unit 210, an input control unit 220, and a communication unit 240.

[0086] (A-1-3-2. Display control unit 210) The display control unit 210 displays to the operator status information of the drone 100 and the like acquired from the drone 100 or the server 300. The display control unit 210 can display images relating to various information such as the field to be photographed, permitted / prohibited flight areas, flight geofences, map information, current position information of the drone 100, attitude information (directional information), speed information, acceleration information, and remaining battery power. The "current position information" referred to here is sufficient to include information on the horizontal position of the current position of the drone 100 (i.e., latitude and longitude), and does not have to include altitude information (absolute altitude or relative altitude).

[0087] The display control unit 210 has a mode display unit 211 and a shooting state display unit 212 .

[0088] The mode display unit 211 is a functional unit that displays at least the state to which the drone 100 belongs, i.e., the mode, on the display unit 201. The mode to which the drone 100 belongs is, for example, the flight mode shown in Fig. 9, but instead of or in addition to this, the display unit 201 may display the aircraft state shown in Fig. 10, the aircraft action state shown in Fig. 11, the game state shown in Fig. 12, or the offensive and defensive states shown in Fig. 13.

[0089] As shown in Figure 19, the screen G1 displayed on the display unit 201 displays, for example, a display field G21 of the flight mode to which the drone 100 belongs, as well as a status display field G22 showing the aircraft status, aircraft behavior status, game status, and offensive and defensive status.

[0090] 5 is a functional unit that displays on the display unit 201 the video captured by the imaging camera 141 mounted on the drone 100. As shown in FIG. 19, on the screen G1 displayed on the display unit 201, for example, a video field G40 is displayed in which the image being captured by the drone 100 is displayed. The screen G1 and each mode will be described in detail later.

[0091] (A-1-3-3. Input control unit 220) The input control unit 220 shown in FIG. 5 receives various inputs from a user such as a pilot. The input control unit 220 of this embodiment mainly has the following functional units: an aircraft position operation unit 221, an aircraft attitude operation unit 222, a camera attitude operation unit 223, a camera zoom operation unit 224, a flight mode switching unit 225, a target position reception unit 226, a power input unit 227, and a return input unit 228.

[0092] The aircraft position operation unit 221 includes an up / down movement input unit 221a and a left / right movement input unit 221b. The aircraft attitude operation unit 222 includes a forward / backward movement input unit 222a and a yaw rotation input unit 222b.

[0093] The up / down movement input unit 221a is an input unit for allowing the pilot to move the drone 100 up and down, and acquires an input to the right input stick 327R. That is, when the right input stick 327R is moved upward (toward the rear when held in the hand), the drone 100 rises, and when the right input stick 327R is moved downward (toward the front when held in the hand), the drone 100 descends. The left / right movement input unit 221b is an input unit for allowing the pilot to move the drone 100 left and right, and acquires an input to the right input stick 327R. That is, when the right input stick 327R is moved to the right, the drone 100 moves right, and when the right input stick 327R is moved to the left, the drone 100 moves left.

[0094] The forward / backward movement input unit 222a is an input unit used by the pilot to move the drone 100 forward or backward, and acquires input to the left input stick 327L. That is, when the left input stick 327L is moved upward (toward the rear when held in the hand), the drone 100 moves forward, and when the left input stick 327L is moved downward (toward the front when held in the hand), the drone 100 moves backward. The yaw rotation input unit 222b is an input unit used by the pilot to yaw rotate the drone 100, and acquires input to the left input stick 327L. That is, when the left input stick 327L is moved to the right, the drone 100 turns right, and when the left input stick 327L is moved to the left, the drone 100 turns left.

[0095] The camera attitude operation unit 223 is an input unit for operating the camera holding unit 142 via the shooting control unit 143 and for operating the orientation of the shooting camera 141 relative to the housing 101 of the drone 100. The camera attitude operation unit 223 acquires input to the right slider 326R. The camera attitude operation unit 223 accepts operation of either or both of the pitch angle and yaw angle of the shooting camera 141 relative to the housing 101.

[0096] Camera zoom operation section 224 is an input section for operating the imaging magnification of imaging camera 141, and acquires input to left slider 326L.

[0097] The flight mode switching unit 225 is an input unit for switching the flight mode. The flight modes selectable by the flight mode switching unit 225 include at least, for example, an outer edge flight mode M102 (see FIG. 9), an inside court flight mode M105 (see FIG. 9), and a fixed position flight mode M103 or M107 (see FIG. 9). The flight mode switching unit 225 accepts switching of the flight mode via, for example, a touch panel display integrated with the display unit 201.

[0098] The target position receiving unit 226 is a functional unit that receives input of a target shooting position to which the drone 100 should head. The target position receiving unit 226 receives input of a point on the stadium F. For example, in a state in which at least a portion of an image or schematic diagram of the stadium F is displayed on the display unit 201, the target position receiving unit 226 may receive input of the target shooting position via a touch panel display that is configured integrally with the display unit 201. In cases in which a point that can be selected as the target shooting position, i.e., a shooting position, is specified in advance, the target position receiving unit 226 may receive input of the selection of the target shooting position.

[0099] Flight mode Here, the types of flight modes that can be set in the drone 100 and examples of their state transitions will be described. As shown in FIG. 9, the flight modes of the drone 100 mainly include a pre-preparation mode M100, an off-court takeoff and landing mode M101, an outer edge flight mode M102, an off-court fixed position flight mode M103, an on-court entry mode M104, an on-court flight mode M105, an off-court exit mode M106, an on-court fixed position flight mode M107, and an on-court takeoff and landing mode M108.

[0100] The advance preparation mode M100 is a mode in which advance settings such as geofence settings are performed. The advance preparation mode M100 transitions to an off-court takeoff and landing mode M101. In this off-court takeoff and landing mode M101, the drone 100 takes off from point L101g (see FIG. 15). Note that in the off-court takeoff and landing mode M101, the drone 100 may take off from a point outside the court F100 that is different from point L101g.

[0101] The off-court takeoff and landing mode M101 is the mode to which the drone 100 belongs when control starts or ends. The drone 100 transitions from the off-court takeoff and landing mode M101 to the perimeter flight mode M102.

[0102] The outer edge flight mode M102 is a mode in which the drone flies above the outer edge along part or all of the outer edge of the court F100 to photograph the playing field F, and more specifically, is a mode in which the drone flies at one of the photographing positions L101 to L105 (FIG. 7). In the main embodiment in this description, the outer edge flight mode M102 is a mode in which the drone flies above the touch line F111b. However, the "outer edge" in which the drone flies in the outer edge flight mode M102 is a concept that includes not only directly above the touch line F111b but also slightly outside the court F100.

[0103] In outer edge flight mode M102, a user instruction is received via target position receiving unit 226 of controller 200, and the drone 100 flies at one of the specified image capture positions L101 to L105. The image capture angle may be manually controlled in response to a user instruction, or may be fixed at a predetermined angle. In outer edge flight mode M102, the drone 100 may change its image capture position while keeping the image capture angle fixed, a so-called dolly shot, in which a specific player is followed and captured.

[0104] The outer edge flight mode M102 can transition to an outside court takeoff and landing mode M101, an outside court fixed position flight mode M103, or an inside court approach mode M104.

[0105] The off-court fixed position flight mode M103 is a mode in which the drone 100 flies in a fixed position outside the area of ​​the court F100. The off-court fixed position flight mode M103 can transition to the outer edge flight mode M102. The on-court entry mode M104 is a mode in which the drone 100 performs a series of processes required to enter the area of ​​the court F100. The drone 100 transitions to the on-court flight mode M105 via the on-court entry mode M104.

[0106] The in-court flight mode M105 is a mode in which the drone flies above the court F100 to photograph the playing field F, and more specifically, flies at one of the photographing positions L206 to L215 (FIG. 7). In the in-court flight mode M105, as in the outer edge flight mode M102, a user command to select a photographing position is received via the target position receiving unit 226 of the controller 200, and the drone flies at one of the specified photographing positions L206 to L215. The photographing direction (photographing angle) may be manually controlled according to the user's instructions, or may be fixed at a predetermined angle.

[0107] The in-court flight mode M105 can transition to an out-of-court exit mode M106, an in-court fixed position flight mode M107, or an in-court takeoff and landing mode M108.

[0108] The court exit mode M106 is a mode in which the drone 100 performs a series of processes required for the drone 100 to exit the area of ​​the court F100. The drone 100 transitions to the perimeter flight mode M102 via the court exit mode M106. Note that the court exit mode M106 and the court entry mode M104 can transition to each other.

[0109] The in-court fixed position flight mode M107 is a mode in which the drone flies in a fixed position within the area of ​​the court F100. The in-court fixed position flight mode M107 can transition to the in-court flight mode M105. The in-court takeoff and landing mode M108 is a mode in which the drone takes off and lands within the area of ​​the court F100, and is a mode to which the drone transitions mainly when a command to land on the spot is issued by manual intervention. A drone that takes off in the in-court takeoff and landing mode M108 transitions to the in-court flight mode M105.

[0110] Returning to Figure 5, the power input unit 227 is a functional unit that receives power on / off commands for the controller 200 via the power button 328.

[0111] The return input unit 228 is a functional unit that receives, via the return button 329, a command to return the drone 100 located in the stadium F (FIG. 7) to the target landing point L101g (see FIG. 15).

[0112] Furthermore, instead of or in addition to the above configuration, the input control unit 220 may be configured to receive touch input to the display unit 201 and transmit a control command to the drone 100 in response to the input. More specifically, for example, when the user performs a selection operation on appropriate information such as a map or a schematic diagram displayed on the display unit 201, a route to the selected point may be automatically generated, and the drone 100 may fly autonomously.

[0113] (A-1-3-4. Communication unit 240) The communication unit 240 is a functional unit that transmits and receives signals between the controller 200 and appropriate components included in the system 1. The controller 200 has a communication function that performs wireless communication with the drone 100 using Wi-Fi, 2.4 GHz, and frequency bands of 5.6 to 5.8 GHz. The controller 200 also has a wireless communication function that enables communication with the server 300 via a communication network 400 using a communication standard such as LTE (Long Term Evolution). The communication unit 240 transmits, for example, various input signals from a user such as a pilot to the drone 100, the server 300, or the like. The communication unit 240 also receives signals from the drone 100, the server 300, or the like.

[0114] (A-1-4. Server 300) (A-1-4-1. Overview of Server 300) 6 is a functional configuration diagram of the server 300 of this embodiment. The server 300 manages or controls the flight and photography of the drone 100. The server 300 includes an input / output unit (not shown) for inputting or outputting various types of information (image output, audio output).

[0115] The server 300 may be a general-purpose computer such as a workstation or personal computer, or may be logically realized by cloud computing.

[0116] The server 300 is equipped with an arithmetic unit such as a CPU for executing information processing, and storage devices such as RAM and ROM, and as a software configuration, it mainly comprises functional blocks of a pre-setting unit 310, an event detection unit 320, a flight mode switching unit 330, an outer edge flight control unit 340, an in-court flight control unit 350, a fixed position flight control unit 360, a communication unit 370, and a memory unit 380.

[0117] (A-1-4-2. Presetting unit 310) The pre-setting unit 310 is a functional unit that performs settings necessary for the flight of the drone 100 before the drone 100 flies over the field to be photographed. The presetting unit 310 mainly includes a geofence setting unit 311 .

[0118] The geofence setting unit 311 is a functional unit that sets a geofence for the drone 100. The geofence includes information on the planar direction and the height direction.

[0119] The geofence setting unit 311 sets a geofence according to the flight mode. That is, for example, the geofence setting unit 311 activates the geofence G100 in the outer perimeter flight mode M102 (see FIG. 9). The geofence setting unit 311 also activates the geofence G200 in the inner court flight mode M105 (see FIG. 9). The geofence setting unit 311 also sets a geofence different from the geofence G100 or the geofence G200, a so-called third geofence, in intermediate modes that intervene in the transition between the outer perimeter flight mode M102 and the inner court flight mode M105, i.e., the outer court exit mode M106 and the inner court entry mode M104. The outer perimeter flight mode M102 and the inner court flight mode M105 are examples of the first or second flight mode in the claims. The outer court exit mode M106 and the inner court entry mode M104 are examples of the third flight mode in the claims.

[0120] When the geofence in the first flight mode and the geofence in the second flight mode, which is the flight mode at the destination, at least partially overlap, the third geofence is a geofence that covers an integrated area that combines the first area defined by the geofence in the first flight mode and the second area defined by the geofence in the second flight mode. In this embodiment, the geofence G100 in the outer flight mode M102 and the geofence G200 in the inner flight mode M105 overlap. Therefore, the third geofence is a geofence that divides the area that combines the geofences G100 and G200.

[0121] Furthermore, if the geofence for the first flight mode and the geofence for the second flight mode do not overlap, the third geofence may be a geofence that covers an area that combines a first area defined by the first geofence corresponding to the first flight mode, a second area defined by the second geofence corresponding to the second flight mode, and a gap between the first and second areas. With this configuration, even when transitioning between flight modes where the geofences do not overlap, the vehicle will not deviate from the geofence during the mode transition, ensuring safety. The geofence setting unit 311 stores information about the set geofence in the storage unit 380.

[0122] (A-1-4-3. Event detection unit 320) The event detection unit 320 is a functional unit that detects the state of the subject to be photographed or the drone 100. The event detection unit 320 detects an event based on the camera image of the photographing camera 141 or an input from the external system 700. The detection criteria for each event are stored in, for example, the storage unit 380, and the event detection unit 320 detects an event by referring to the storage unit 380. The event detection unit 320 may also detect an event by analysis using a neural network. The detection process by the event detection unit 320 can be performed using any known appropriate image analysis technology.

[0123] The event detection unit 320 detects events that trigger a change in the flight mode or shooting conditions of the drone 100, in particular. The event detection unit 320 mainly includes a robot state acquisition unit 321 , a robot action state acquisition unit 322 , a game state acquisition unit 323 , and an offensive / defensive state acquisition unit 324 .

[0124] The aircraft status acquisition unit 321 is a functional unit that acquires the aircraft status of the drone 100. 10 is a diagram showing the state transition of the aircraft state of the drone 100. The aircraft state is broadly divided into, for example, a normal operation flight mode M200, a detection and judgment mode M210, and an action mode M220. When the drone 100 starts flying, the drone 100 transitions to the normal operation flight mode M200.

[0125] When any detection or reception occurs in the normal operation flight mode M200, the aircraft state transitions to a detection and judgment mode M210, which includes an abnormality detection mode M211, a fault detection mode M212, a manual intervention mode M213, and a low battery mode M214.

[0126] Specifically, if an abnormality is detected in the normal operation flight mode M200, the mode transitions to the abnormality detection mode M211. This abnormality is a transient, reversible disturbance such as a decrease in radio wave intensity or strong wind. If the abnormality is resolved in the abnormality detection mode M211, the mode transitions to the normal operation flight mode M200.

[0127] In the normal operation flight mode M200, if a failure of the aircraft or system is detected, the drone 100 transitions to a failure detection mode M212. If a manual control command is received, the drone 100 transitions to a manual intervention mode M213, and if it is detected that the remaining battery charge is lower than a predetermined value, the drone 100 transitions to a low battery mode M214. Furthermore, if a manual control command is received in the anomaly detection mode M211, the failure detection mode M212, or the low battery mode M214, the drone 100 transitions to the manual intervention mode M213. The drone 100 transitions to an action mode M220 corresponding to the detection and determination mode M210.

[0128] The action mode M220 is a state in which the drone 100 performs a series of actions preset for each state. The action mode M220 includes a landing at an evacuation point mode M221, an emergency stop mode M222, a landing in place mode M223, a return mode M224, and a fixed position flight mode M225.

[0129] The landing at evacuation point mode M221 is set to fly the drone 100 to the evacuation point H200 and land it. The landing at evacuation point mode M221 is entered when the abnormality is not resolved in the abnormality detection mode M211.

[0130] The emergency stop mode M222 is set to stop the propellers 122 immediately. In the emergency stop mode M222, the drone 100 falls freely. The emergency stop mode M222 can be selected in the manual intervention mode M213 when the propellers 122 are about to come into contact with a person or object.

[0131] The landing mode M223 is set to perform a soft landing on the spot, and the return mode M224 is set to perform a return to the takeoff and landing point.

[0132] The fixed position flight mode M225 is a state in which the drone 100 flies at a fixed position, and can transition to the normal operation flight mode M200 based on a user operation. The user operation is input, for example, by selecting a button displayed on the display unit 201. In addition, when an event that can cause a transition from the normal operation flight mode M200 to the detection and determination mode M210, i.e., an abnormality, a malfunction, manual intervention, or a low battery, is detected in the fixed position flight mode M225, the drone 100 transitions from the fixed position flight mode M225 to the detection and determination mode M210 via the normal operation flight mode M200. In addition, the drone 100 in the fixed position flight mode M225 can transition to the return mode M224 based on a user operation.

[0133] The drone 100 in the anomaly detection mode M211 and the failure detection mode M212 transitions to the landing at evacuation point mode M221. The drone 100 in the manual intervention mode M213 transitions to one of the landing at evacuation point mode M221, emergency stop mode M222, landing in place mode M223, return mode M224, and fixed position flight mode M225, depending on the input command. The drone 100 in the low battery mode M214 transitions to the return mode M224.

[0134] The drone 100 in the normal operation flight mode M200 can also transition to the return mode M224 based on a user operation. The user operation is input by selecting a button displayed on the display unit 201, for example.

[0135] The aircraft behavior state acquisition unit 322 is a functional unit that acquires the aircraft behavior state of the drone 100. Each mode of the aircraft behavior state is a submode of the aircraft state that is performed to realize a transition of the aircraft state. 11 is a diagram showing state transitions of the aircraft's behavioral states. The aircraft's behavioral states are broadly divided into, for example, takeoff mode M300, escape mode M310, normal mode M320, and landing mode M340.

[0136] The takeoff mode M300 is a mode in which the drone 100 takes off. The state transition of the aircraft's behavioral state starts from the takeoff mode M300. When the drone 100 takes off, the aircraft's behavioral state transitions from the takeoff mode M300 to the evacuation mode M310 or the normal mode M320. The evacuation mode M310 mainly includes an evacuation point arrival stationary mode M311 and an evacuation moving mode M312. The normal mode M320 also includes a point arrival stationary mode M321 and a moving mode M322. The evacuation mode M310 and the normal mode M320 can transition to each other via the pause mode M330. Note that this is just one example.

[0137] When transitioning from takeoff mode M300 to evacuation mode M310, the drone 100 in takeoff mode M300 transitions to evacuation point arrival stationary mode M311. The evacuation point arrival stationary mode M311 is a mode in which the drone moves to the evacuation point H200 and remains stationary there, i.e., hovers. When moving from the evacuation point H200 to another location for the purpose of evacuation, the drone 100 in the evacuation point arrival stationary mode M311 transitions to evacuation moving mode M312. When the drone 100 reaches the specified destination or there is no movement command due to manual intervention, the aircraft behavior state transitions from the evacuation point arrival stationary mode M311 or the evacuation moving mode M312 to the pause mode M330.

[0138] Furthermore, when transitioning from takeoff mode M300 to normal mode M320, the drone 100 in takeoff mode M300 transitions to point arrival stationary mode M321. Point arrival stationary mode M321 is a mode in which the drone moves to a predetermined destination and remains stationary, i.e., hovers. When moving to another location in normal use, the drone 100 in point arrival stationary mode M321 transitions to moving mode M322. When the drone 100 reaches the predetermined destination or no movement command is issued by manual intervention, the aircraft behavior state transitions from point arrival stationary mode M321 or moving mode M322 to pause mode M330.

[0139] The drone 100 in the evacuation point arrival stationary mode M311, the point arrival stationary mode M321, the moving mode M322, and the pause mode M330 can transition to the landing mode M340. The aircraft's behavior state ends processing in the landing mode M340.

[0140] The game status acquisition unit 323 shown in FIG. 6 is a functional unit that acquires the game status of a competition held at stadium F. The game status acquisition unit 323 detects the game status by processing captured images. The game status acquisition unit 323 may also acquire the game status based on decision-related information input by a referee to the external input device 600 or a referee support system, which is an example of the external system 700. Furthermore, the game status acquisition unit 323 may acquire the game status based on information input from the external input device 600 held by a team member, for example, a manager or coach.

[0141] FIG. 12 is a diagram showing an example of a state transition of a game state. The diagram shows an example of a soccer game state. The game state includes a pre-game state M400, a normal play state M410, and an end-of-game state M460. The state transition starts from the pre-game state M400, and transitions from the pre-game state M400 to the normal play state M410. The normal play state M410 is a state in which the game is progressing. When the game ends, transition occurs from the normal play state M410 to the end-of-game state M460. Note that a transition from the normal play state M410 to the end-of-game state M460 may occur not only at the end of the game, but also during breaks during the game, such as halftime.

[0142] The game state also includes a play suspended without a foul play state M420 and a play suspended with a foul play state M440. When play is suspended without a foul play in the normal play state M410, the state transitions to the play suspended without a foul play state M420. The play suspended without a foul play state M420 is entered, for example, when the ball crosses the goal line F110a, F110b or the touchline F111a, F111b and goes out of the court. The play suspended without a foul play state M420 transitions to a throw-in state M421, a goal kick state M422, or a corner kick state M423 in accordance with events that occur according to the rules of the game, such as the type of line the ball crossed or the affiliation of the player who sent the ball out of the court. The throw-in state M421, the goal kick state M422, or the corner kick state M423 transitions to the normal play state M410 when the event in each state ends.

[0143] In the normal play state M410, if a foul occurs, or more precisely, if the referee recognizes that a foul has occurred, the state transitions to a foul state M431. Furthermore, if an offside occurs or is recognized by the referee, the state transitions to an offside state M432. Then, from the foul state M431 or the offside state M432, the state transitions to a foul play suspended state M440. In the foul play suspended state M440, the state transitions to a free kick state M441 or a penalty kick state M442 depending on the location of the foul and the event that occurred. In the free kick state M441, a so-called indirect free kick may be taken instead of a free kick. The free kick state M441 may also be subdivided into a free kick state for the attacking team and a free kick state for the defending team. In the free kick state M441 and the penalty kick state M442, once the events in each state are completed, the game resumes and the game state transitions to the normal play state M410.

[0144] When the match ends, the normal play state M410 transitions to an after-match state M460, and the state transition of the match state ends.

[0145] Furthermore, the normal play state M410 can transition to a penalty shootout state M443. Although not shown in the figures, the penalty shootout state M443 may transition to an end-of-match state M460, thereby ending the state transition.

[0146] 12, some of the game states may trigger a change in flight mode, while other game states may not. For example, the flight mode may be changed based on a transition to the shaded states in the figure, i.e., the pre-game state M400, goal kick state M422, corner kick state M423, free kick state M441, penalty kick state M442, player substitution state M450, and end-of-game state M460. Note that when a transition occurs from another game state to the normal play state M410, the flight mode may be changed to a mode appropriate for the offensive or defensive state.

[0147] The offensive and defensive status acquisition unit 324 is a functional unit that acquires the offensive and defensive status of the teams in the match held at stadium F. The offensive and defensive status acquisition unit 324 detects the offensive and defensive status by processing the captured images. The offensive and defensive status acquisition unit 324 may also acquire the offensive and defensive status based on decision-related information input by the umpire to the external input device 600 or a referee support system, which is an example of external system 700. Furthermore, the offensive and defensive status acquisition unit 324 may acquire the offensive and defensive status based on information input from the external input device 600 held by a team member, for example, a manager or coach.

[0148] 13 is a diagram showing an example of a state transition between an offensive and defensive state. The figure shows an example of an offensive and defensive state in soccer. When a match starts, the offensive and defensive state transitions to an offensive state M510 or a defensive state M520. The offensive state M510 and the defensive state M520 also transition to each other via an offensive / defensive change state M530 or an offensive / defensive uncertainty state M540.

[0149] The offensive state M510 is a state in which one pre-designated team (hereinafter also referred to as "Team A") is on the offensive. The offensive state is, for example, a state in which Team A is in possession of the ball and is advancing toward the other team (hereinafter also referred to as "Team B"), but is not limited to this and may be a predetermined state determined by any determination criterion stored in advance in the storage unit 380.

[0150] The attack state M510 includes an A team offensive (own side) state M511, an A team offensive (enemy side) state M512, and an A team fast attack state M513. A transition is possible between the A team offensive (own side) state M511 and the A team offensive (enemy side) state M512, and between the A team offensive (own side) state M511 and the A team fast attack state M513. A transition is also possible from the A team fast attack state M513 to the A team offensive (enemy side) state M512.

[0151] The defensive state M520 includes a team A defensive (opponent's territory) state M521, a team A defensive (owner's territory) state M522, and a team B fast attack state M523. Transitions are possible between the team A defensive (owner's territory) state M521 and the team A defensive (owner's territory) state M522, and between the team A defensive (opponent's territory) state M521 and the team B fast attack state M523. Also, a transition is possible from the team B fast attack state M523 to the team A defensive (owner's territory) state M522.

[0152] The offense / defense change state M530 and the offense / defense uncertain state M540 can be transitioned to from any of the following states: Team A offensive (home side) state M511, Team A offensive (enemy side) state M512, Team A fast attack state M513, Team A defensive (enemy side) state M521, Team A defensive (home side) state M522, and Team B fast attack state M523.

[0153] Note that instead of or in addition to the above-described acquisition units 321 to 324, the event detection unit 320 may determine an event based on input information from the external system 700. For example, the event detection unit 320 may determine a disturbance such as a strong wind as an event based on input information from a weather information system, which is an example of the external system 700. The event detection unit 320 may also determine an event based on input information from a court facility system, which is another example of the external system 700, or facility information input by a person involved with the court facility.

[0154] (A-1-4-4. Flight mode switching unit 330) The flight mode switching unit 330 is a functional unit that switches the flight mode depending on the detection result by the event detection unit 320. The flight mode switching unit 330 mainly includes a mode switching determination unit 331, a flight-permitted area switching unit 332, a geofence switching unit 333, and a flight path generation unit 334.

[0155] The mode switching determination unit 331 is a functional unit that determines whether or not switching of the flight mode is necessary. The mode switching determination unit 331 determines whether or not switching of the flight mode is necessary in accordance with an event detected by the event detection unit 320. The mode switching determination unit 331 references a table T1 (see FIG. 14 ) that associates the state detected by the event detection unit 320 with the flight mode in that state, and determines that switching of the flight mode is necessary if the flight modes are different.

[0156] 14, the event-flight mode table T1 is a table in which events detected as game states are associated with flight modes to be selected for those events. More specifically, for example, in the goal kick state M422 and the defending free kick state M441, the outer perimeter flight mode M102 is associated to avoid the risk of the ball hitting the drone 100. In addition, in the penalty kick state M442 and the attacking free kick state M441, the inside court flight mode M105 is associated to capture the goal scene from close up. In the pre-match state M400, the penalty shootout state M443, and the player substitution state M450, the movement of the target to be focused on is small or nonexistent, so the fixed position flight mode M103 or M107 is associated. Whether to use the off-court fixed position flight mode M103 or the on-court fixed position flight mode M107 may be determined depending on the flight mode at the time of transition to the pre-match state M400, the penalty shootout state M443, or the player substitution state M450. In the post-match state M460, the on-court flight mode M105 is associated in order to capture close-up shots of the players' expressions.

[0157] According to the above-described configuration, a competition or event taking place in the target area F can be appropriately photographed using a flight mode suited to the event.

[0158] The flight-permitted area switching unit 332 is a functional unit that switches the flight-permitted area in response to switching of the flight mode.

[0159] The geofence switching unit 333 is a functional unit that switches geofences in response to switching of flight modes. For example, when the flight mode is the outer perimeter flight mode M102, the geofence switching unit 333 sets a geofence G100. When the flight mode is the inner court flight mode M105, the geofence switching unit 333 sets a geofence G200. In addition, in the intermediate modes that intervene in the transition between the outer perimeter flight mode M102 and the inner court flight mode M105, i.e., the inner court entry mode M104 and the outer court exit mode M106, a third geofence different from the geofences G100 and G200 in the outer perimeter flight mode M102 and the inner court flight mode M105 is set.

[0160] The flight path generation unit 334 is a functional unit that generates a flight path of the drone 100 when moving with a flight mode switch. The flight path generation unit 334 determines, for example, a shooting position at which the drone transitions to an in-court entry mode M104 or an out-of-court exit mode M106. The flight path generation unit 334 also determines a shooting position at which the drone transitions from the in-court entry mode M104 to an in-court flight mode M105, or a shooting position at which the drone transitions from the out-of-court exit mode M106 to an outer edge flight mode M102. The flight path generation unit 334 also generates a specific flight path for the in-court entry mode M104 or the out-of-court exit mode M106. This flight path is generally generated within the area of ​​the third geofence.

[0161] (A-1-4-5. Outer edge flight control unit 340) The outer edge flight control unit 340 is a functional unit that controls the flight of the drone 100 in the outer edge flight mode. The outer edge flight control unit 340 has a target position acquisition unit 341 and a flight path generation unit 342.

[0162] The target position acquisition unit 341 is a functional unit that acquires a target position to which the drone 100 should head. The target position acquisition unit 341 acquires a target position that is located within the flyable area in the outer flight mode M102. The target position acquisition unit 341 may acquire a target position that is input by a user and that is received via the target position receiving unit 226 of the controller 200, for example.

[0163] The flight path generation unit 342 is a functional unit that generates a flight path along which the drone 100 moves in the outer flight mode M102. In other words, the flight path generation unit 342 generates a flight path within the flight area available in the outer flight mode M102. When the drone 100 belongs to the flight area available in the outer flight mode M102 and the acquired target position also belongs to the flight area available in the outer flight mode M102, the flight path generation unit 342 generates a flight path from the current location to the target position. Furthermore, when the drone 100 moves across the outer flight mode M102 and the on-court flight mode M105, the flight path generation unit 342 may generate a flight path within the flight range in the outer flight mode M102. The flight path generation unit 342 moves the moving object along the touch line F111b, for example.

[0164] When the ball crosses the touchline F111b where the drone 100 is flying and enters the off-court area F200, the flight path generation unit 342 moves the drone 100 to a position outside the touchline F111b in the off-court area F200. The flight path generation unit 342 may also cause the drone 100 to capture images directly below the drone 100 or from that point toward the court F100. This configuration enables tracking and capturing images of the ball even when the ball rolls into the off-court area F200. The geofence G100 for the outer flight mode M102 is preferably set in advance to extend beyond the touchline F111b to the outside of the court F100. This configuration allows the drone 100 to reliably remain within the geofence G100, even when the drone 100 flies slightly outside the touchline F111b to track the ball, as described above.

[0165] If the flight path generation unit 342 detects an obstacle on the flight path or near the drone 100, it regenerates a flight path that detours around the obstacle inside the court F100. Alternatively, if the flight path generation unit 342 detects an obstacle, it may hover for a predetermined period of time and then cause the drone 100 to move along the originally generated flight path. This is because, while the safety of the drone 100 in the area outside the court F200 of the stadium F is not ensured, the safety of the drone 100 inside the court F100 is highly likely to be ensured. The obstacle may be detected, for example, by the obstacle detection unit 130 of the drone 100, or may be detected using information from an external system 700, etc. Furthermore, when an obstacle is detected, the outer flight control unit 340 may switch to manual control after having the flight path generation unit 342 hover for a predetermined time. Furthermore, when an obstacle is detected, the outer flight control unit 340 may have the flight path generation unit 342 hover for a predetermined time, and then display a message via the display control unit 210 prompting the user to re-input the target position.

[0166] (A-1-4-6. In-court flight control unit 350) The in-court flight control unit 350 is a functional unit that controls the flight of the drone 100 in the in-court flight mode M105. The in-court flight control unit 350 has a target position acquisition unit 351 and a flight path generation unit 352. The target position acquisition unit 351 is a functional unit that acquires a target position located within the range of the flight area in the in-court flight mode M105.

[0167] The flight path generation unit 352 generates a flight path for the drone 100 to travel in the intra-court flight mode M105. That is, the flight path generation unit 352 generates a flight path within a flyable area in the intra-court flight mode M105. More specifically, the flight path generation unit 352 generates a flight path in the intra-court flight mode M105 by connecting multiple preset shooting positions. Like the flight path generation unit 342 of the outer edge flight control unit 340, the flight path generation unit 352 may generate a flight path within a flight range in the intra-court flight mode M105 when the current location and the target location belong to flyable areas of different flight modes.

[0168] If the flight path generation unit 352 detects an obstacle on the flight path or near the drone 100, it regenerates a flight path that bypasses the obstacle. The obstacle is detected, for example, by the obstacle detection unit 130. The flight path generation unit 352 may regenerate the flight path by changing the connection between multiple preset shooting positions, or may change the flight path to a higher altitude while maintaining the flight path on a plane. Furthermore, if the flight path generation unit 352 detects an obstacle, it may hover for a predetermined period of time and then resume movement along the originally generated flight path. Furthermore, if the intra-court flight control unit 350 detects an obstacle, it may switch to manual control after the flight path generation unit 352 has hovered for a predetermined period of time. Furthermore, if the intra-court flight control unit 350 detects an obstacle, it may display a message via the display control unit 210 prompting the user to re-input the target position after the flight path generation unit 352 has hovered for a predetermined period of time. The obstacle may be, for example, a bird, fixed equipment, or a player. The obstacles also include balls.

[0169] In this embodiment, flight control in the outer flight mode M102 is performed by the outer flight control unit 340, and flight control in the inner court flight mode M105 is performed by the inner court flight control unit 350. The specified shooting positions for each of the outer flight mode M102 and inner court flight mode M105 are presented as options, and a flight path to the selected target shooting position is generated. However, the technical scope of the present invention is not limited to this. The controller 200 may control the shooting position and orientation of the drone 100 to fly freely within any location within the geofences G100 and G200 set for each flight mode.

[0170] Furthermore, the functional configuration of the flight path generation units 334, 342, and 352 is an example, and for example, a single flight path generation unit may generate the flight path without subdivision.

[0171] 15 is a diagram schematically illustrating the routes that the drone 100 can take, defined by the image capture positions L101 to L105, L206 to L215, and the evacuation point H200. Point L101g on the ground at image capture position L101 is the takeoff and landing point for the drone 100. The position transition of the drone 100 begins with the step of taking off from point L101g and arriving at image capture position L101. The drone 100 also descends at image capture position L101 and lands at point L101g, completing image capture.

[0172] At each of the image capture positions L101 to L105 and L206 to L215, the drone 100 may only be able to transition to an adjacent image capture position. For example, the point to which the drone 100 at image capture position L105 can transition while maintaining the outer edge flight mode is image capture position L104. Furthermore, the points to which the drone 100 at image capture position L105 can transition after switching to the inside court flight mode M105 are image capture positions L106 and L107. The flight path generation unit 334, 342, or 352 (see FIG. 6) generates a flight path for the drone 100 by referring to the transitionable path.

[0173] When the target position receiving unit 226 receives the selection of the photographing position, the drone 100 transitions to the selected photographing position via the photographing positions to which it can be transferred. For example, when the drone 100 is at photographing position L105 and photographing position L215 is selected, the flight path generating unit 334, 342, or 352 (see FIG. 6) generates a flight path that transitions through photographing positions L105, L207, L208, L213, L212, and L215 in this order, and the drone 100 flies along this flight path.

[0174] Note that the drone 100 may transition to an adjacent image capture position when the transition of the image capture position involves a flight mode switch, and may transition directly to a non-adjacent image capture position when the transition does not involve a flight mode switch. For example, when the drone moves from image capture position L105 to image capture position L215, it may transition from image capture position L105 to image capture position L207 with a mode switch, and then move linearly within the area within the geofence G200 from image capture position L107 to image capture position L215.

[0175] The outer edge flight control unit 340 and the inner court flight control unit 350 autonomously fly the drone 100 in each flight area according to the flight mode. For example, the outer edge flight control unit 340 and the inner court flight control unit 350 may perform dolly shooting within each flight area, i.e., the drone 100 may automatically track and shoot a specific object such as a ball or a designated player. The outer edge flight control unit 340 and the inner court flight control unit 350 may also automatically control the direction of the nose of the drone 100 or the direction of the shooting camera 141, i.e., the shooting direction. The outer edge flight control unit 340 and the inner court flight control unit 350 may also automatically control the flight height of the drone 100. The autonomous flight mode may differ depending on the flight mode. For example, dolly shooting may be performed when controlled by the outer edge flight control unit 340, while automatic tracking shooting with only the shooting direction fixed and shooting position, or automatic tracking shooting with both the position and shooting direction may be performed when controlled by the inner court flight control unit 350. In addition, the outer edge flight control unit 340 and the in-court flight control unit 350 may generate a flight path within the court (within the stadium F) for moving the drone 100 to a target position specified by the user in each flight area.

[0176] (A-1-4-7. Fixed position flight control unit 360) The fixed position flight control unit 360 is a functional unit that controls the flight of the drone 100 in the off-court fixed position flight mode M103 and the on-court fixed position flight mode M107. In the fixed position flight mode, the fixed position flight control unit 360 hovers at a predetermined position and controls the direction of the nose or the direction of the shooting camera 141 to follow a specific player or the ball and take automatic photos. Note that the above-mentioned "control of direction" is a concept that includes control not only in the left-right direction (so-called "pan direction") but also in the up-down direction (so-called "tilt direction").

[0177] The fixed position flight control unit 360 includes a mode switching permission determination unit 361. The mode switching permission determination unit 361 is a functional unit that determines whether or not switching of the flight mode is permitted in the fixed position flight mode M103 or M107.

[0178] (A-1-4-8. Communication unit 370) The communication unit 370 has a modem or the like (not shown) and is capable of communicating with the drone 100, the controller 200, and the like via the communication network 400. The communication unit 370 may, for example, monitor the state of the drone 100 and its surroundings and notify the controller 200 of the state.

[0179] (A-1-4-9. Storage section 380) The storage unit 380 is a functional unit, such as a database, that stores information related to flight control of the drone 100. The storage unit 380 stores, for example, the coordinates of multiple shooting positions L101-L105, L210-L215 in the stadium F. These coordinates may be two-dimensional coordinates on a plane or three-dimensional coordinates that include information in the height direction.

[0180] The storage unit 380 also stores an event-flight mode table T1 shown in FIG. 14. Furthermore, the storage unit 380 may store strong wind conditions detected as events in association with fixed position flight modes M103 or M107. This is because in the case of strong winds, the drone 100 may not be able to fly in the intended direction, so it is safer not to move the drone 100. The event-flight mode table T1 is recorded in a rewritable manner. Furthermore, multiple event-flight mode tables T1 may be stored.

[0181] ●Flowchart Fig. 16 is a flowchart showing the overall flow of aerial photography control in this embodiment. Fig. 17 is a subroutine of the flight restriction process S1002 in Fig. 16. Fig. 18 is a subroutine of the flight mode switching process S1010 in Fig. 16.

[0182] The control described in the flowchart shown in Fig. 16 is executed in a regular loop. As shown in Fig. 16, if it is detected that the drone 100 is approaching the vicinity of the geofences G100 and G200 while flying (YES in step S1001), the process proceeds to flight restriction processing in step S1002. The subroutine of the flight restriction processing S1002 will be described in Fig. 17.

[0183] If it is not detected in step S1001 that the drone is approaching the vicinity of the geofences G100 and G200 (NO in step S1001), the presence or absence of an obstacle is detected in the path or near the drone 100 (step S1003).If an obstacle is detected in step S1003 (YES in step S1003), the drone 100 is caused to hover or a detour route is generated, and the flight route of the drone 100 is changed to the detour route (step S1004).

[0184] If no obstacle is detected in step S1003 (NO in step S1003), the presence or absence of an action determination of the aircraft state is detected (step S1005). If an action determination of the aircraft state is detected in step S1005 (YES in step S1005), the process proceeds to step S1006, where an event type determination process is executed (step S1006).

[0185] If no action is detected in step S1005 (NO in step S1005), it is determined whether or not there is an input from the controller 200 by the user (step S1007). If an input from the controller 200 is detected in step S1007 (YES in step S1007), a command based on the input is executed (step S1008).

[0186] If no input from the controller 200 is detected in step S1007 (NO in step S1007), the presence or absence of an event is determined (step S1009). If an event is detected (YES in step S1009), the process proceeds to flight mode switching processing in step S1010 (step S1010). If no event is detected in step S1009 (NO in step S1009), the process returns to step S1001 and steps S1001 to S1009 are repeated.

[0187] As shown in FIG. 16, the overall processing of aerial photography control is performed in the following order: geofence restriction, obstacle detection, control based on aircraft status, control based on user input, and control based on in-game events such as the game status or offensive / defensive status. In other words, each control is executed before control based on in-game events. This order is determined by the priority of performing safe control processing. This configuration can more reliably ensure the safety of the drone 100 flight.

[0188] 17, first, the flight control unit 123 of the drone 100 issues an operation command to restrict the drone 100 from advancing outside the geofence (step S1101). In step S1101, a restriction is set on the flight target position so that the drone 100 does not advance outside the geofence even when the drone 100 is manually operated.

[0189] Next, if the drone 100 does not advance outside the geofence (NO in step S1102), the processing ends.

[0190] If the drone 100 advances outside the geofence (YES in step S1102), proceed to step S1103. Possible causes of this situation include, for example, the wind being too strong and causing the drone to be swept away, or a malfunction of the drone 100 preventing it from flying in the intended direction. In step S1103, the flight control unit 123 issues an operation command to return to the geofence. More specifically, in step S1103, a flight target position command to return to the geofence, that is, an operation command to set a predetermined point inside the geofence as the flight target position, is issued to the drone 100.

[0191] Next, information measured by the measurement unit 110 of the drone 100, such as information on position, orientation, altitude, or speed, is referenced to determine whether the drone 100 is approaching the inside of the geofence (step S1104). In this case, step S1104 is executed a predetermined time after step S1103. Note that in step S1104, it is sufficient if the drone 100 is closer to the inside of the geofence than at the time of step S1103, and it is not necessary to determine whether the drone 100 is located inside the geofence.

[0192] In step S1104, if it is determined that the drone 100 is not approaching the geofence (NO in step S1104), it is determined that the above operation command is ineffective, and the flight control unit 123 forces the drone 100 to land (step S1105).

[0193] If the drone 100 is approaching the geofence in step S1104 (YES in step S1104), the process proceeds to step S1106. In step S1106, the process refers to information measured by the measurement unit 110 of the drone 100, such as the position and altitude, to determine whether the drone 100 is located within the geofence. If the drone 100 is located within the geofence (YES in S1106), the process ends. If the drone 100 is not located within the geofence (NO in S1106), the process returns to step 1104, and the process continues operating based on the operation command to return to the geofence until the drone 100 returns within the geofence.

[0194] Figure 18 is a diagram showing an example of a switching process flow for switching flight modes. Figure 18 shows a process flow involving switching from outer edge flight mode M102 to inner court flight mode M105, and in particular, shows a process flow including switching flight modes and generating a flight path when moving from photographing position L101 to photographing position L215. Note that the following explanation assumes that automatic switching of flight modes has occurred based on event detection, but the flight mode switching process itself is the same when the flight mode switch is input through manual intervention.

[0195] 18, first, the flight mode switching unit 330 switches the flight mode from the outer edge flight mode M102 to the inner court entry mode M104, and the geofence switching unit 333 activates the geofence corresponding to the inner court entry mode M104 (step S1201). The geofence corresponding to the inner court entry mode M104 is an integrated geofence defined by an area combining the geofence G100 of the outer edge flight mode M102 and the geofence G200 of the inner court flight mode M105.

[0196] Next, the flight path generation unit 334 generates a flight path from the photographing position L101 to the photographing position L215 (step S1202). The flight path may be a flight path that transitions between adjacent photographing positions along the path shown in Fig. 15, or may be a flight path that linearly connects the photographing position L101 to the photographing position L215. Note that the flight path may be generated by the flight path generation unit 342 or 352 instead of or in addition to the flight path generation unit 334.

[0197] Next, it is determined whether the generated flight path is generated inside the integrated geofence (step S1203), and if at least a portion of the flight path is not generated inside the integrated geofence (NO in step S1203), the flight path generation unit 334 corrects the flight path (step S1204), and the process returns to step S1203. This configuration prevents the drone 100 from going outside the geofence while moving, ensuring a high level of safety. If the flight path is generated inside the integrated geofence (YES in step S1203), proceed to step S1205.

[0198] Next, a command to move to the photographing position L215 is sent (step S1205), and the drone 100 starts moving.

[0199] During movement, the system 1 determines whether or not there is a command from the controller 200 to exit the court F100 (step S1206). If no exit command is received (NO in step S1206), step S1206 is continued until the movement into the court F100 is completed (NO in step S1207). When the movement of the drone 100 into the court F100 is completed (YES in step S1207), the flight mode switching unit 330 changes the flight mode to the in-court flight mode M105, and the geofence switching unit 333 switches the geofence to the geofence G200 corresponding to the in-court flight mode M105 (step S1208). The drone 100 continues flying, and when it reaches the destination, the shooting position L215 (YES in step S1220), the processing ends.

[0200] In step S1206, if an exit command to outside the court F100 is received from the controller 200 (YES in step S1206), the process proceeds to step S1209. Next, the flight path generation unit 334 generates a flight path to the input destination outside the court F100 (step S1209). Next, the flight control unit 123 starts moving the drone 100 along that flight path. The movement continues until the movement to the destination outside the court F100 is complete (NO in step S1210), and when the movement to the destination is complete (YES in step S1210), the process proceeds to step S1211.

[0201] In step S1211, the flight mode switching unit 330 changes the flight mode to the outer edge flight mode M102, and the geofence switching unit 333 switches the geofence to the geofence G100 corresponding to the outer edge flight mode M102 (step S1211). Next, the drone 100 continues moving within the geofence G100, and when the movement to the destination is completed, the processing ends (step S1220).

[0202] ●Example of display on display unit 201 19 and 20 are examples of screens G1 and G2 displayed on the display unit 201 of the controller 200 by the display control unit 210. FIG.

[0203] The screen G1 shown in FIG. 19 displays a field map G10 that shows a bird's-eye view of the stadium F and shooting positions L101-L215, an icon G11 indicating the position information of the drone 100, a shooting range G12 captured by the shooting camera 141, a display field G21 for the flight mode to which the drone 100 belongs, a status display field G22 that displays the status detected by the event detection unit, such as the aircraft status, aircraft behavior status, game status, and offensive / defense status, a mode switch button G31 that accepts an operation to switch between flight modes, a control switch button G32 that accepts an operation to switch between automatic and manual control, and a video field G40 that displays images captured by the drone 100. In the example of FIG. 19, the drone is flying in perimeter flight mode at shooting position L101. The mode switch button G31 displays switchable modes, and in FIG. 19, it is indicated that switchable to in-court flight mode is possible. The control switch button G32 also displays whether switchable mode is automatic or manual. In FIG. 19, it is shown that switching to manual intervention is possible.

[0204] The position and shooting direction of the drone 100 may be controlled manually, or automatic tracking control of the ball or a specific player may be performed. When automatic tracking control is performed, information about the ball or specific player to be tracked may be displayed on the screen G1.

[0205] FIG. 20 shows an example of the screen G2 displayed by the display control unit 210 when the drone 100 is moving. FIG. 20 shows that the drone 100 is flying from the shooting position L208 to the shooting position L206 in the in-court flight mode M105. Screen G2 also displays information about the destination, i.e., the target position, shooting position L206. More specifically, the drone 100 is flying between the shooting positions L206 and L208, and an arrow extending from the drone 100 toward the destination shooting position L206 is displayed. Furthermore, the target position, shooting position L206, is displayed in a different manner from the other shooting positions, for example, highlighted. More specifically, shooting position L206 is displayed in bold or a large font. Furthermore, the status display field G22 indicates that the aircraft's behavioral status is "moving."

[0206] The nose direction of the drone 100 does not necessarily have to be the direction of travel of the drone 100, but may be directed in any direction. Furthermore, the nose direction of the drone 100 does not have to be constant while moving, and for example, the drone 100 may move while photographing a player or the ball by yaw rotation.

[0207] [A-2. Effects of this embodiment] According to this embodiment, safety in aerial photography can be ensured. More specifically, when switching flight modes in which different geofences are set, a third geofence different from the geofences before and after the switching is set at the time of switching, thereby ensuring that the drone flies within the geofence even during switching, preventing interference between the geofence and the flight path, and ensuring safe switching of flight modes.

[0208] The present invention is not limited to the above-described embodiment, and it goes without saying that various configurations can be adopted based on the contents of the present specification.

[0209] The series of processes described in connection with the above embodiment may be implemented using software, hardware, or a combination of software and hardware. A computer program for implementing each function of the server 300 according to this embodiment may be created and installed on a PC or the like. A computer-readable recording medium storing such a computer program may also be provided. Examples of the recording medium include a magnetic disk, an optical disk, a magneto-optical disk, and a flash memory. The computer program may also be distributed, for example, via a communication network 400 without using a recording medium.

[0210] The flowcharts used in the above embodiments do not necessarily have to be executed in the order shown in the drawings. Some processing steps may be executed in parallel. In addition, additional processing steps may be employed, and some processing steps may be omitted. [Explanation of symbols]

[0211] 1. Aerial photography system 100 Drones (Mobile) 141 Photography camera 200 Controller 220 Input control section 300 servers 320 Event detection unit 330 Flight mode switch 331 Mode switching determination unit 380 Storage section F Stadium F100 Court F200 Outside the court M102 Edge Flight Mode M105 In-court flight mode

Claims

1. A moving object flying over a target area; an event detection unit that detects an event based on an image acquired by a camera that photographs the target area or an input from an external system; a mode switching determination unit that determines whether or not switching of a flight mode is necessary, the flight area of ​​which is determined for at least the moving object; Equipped with The flight mode is an edge flight mode in which the robot flies above the outer edge along a part or all of the outer edge of a court configured within the target area; an in-court flight mode in which the drone flies above the court; At least The mode switching determination unit determines whether or not the flight mode needs to be switched in response to the event detected by the event detection unit. Aerial photography system.

2. The mode switching determination unit switches the flight mode in accordance with an event related to a game being held in the target area, the event being detected from an image captured by the camera.

2. The aerial photography system according to claim 1.

3. In the in-court flight mode, an in-court flight control unit generates, within the court, a flight path for the moving body to automatically follow a specific player or a ball in the target area, or a flight path for moving the moving body to a target position designated by a user.

2. The aerial photography system according to claim 1.

4. a display control unit that displays information about the moving object on an operation screen; the display control unit displays the specific player or the ball to be followed on the operation screen when the ball is flying while being automatically followed.

4. The aerial photography system according to claim 3.

5. The in-court flight control unit generates the flight path by connecting a plurality of preset shooting positions in the in-court flight mode.

4. The aerial photography system according to claim 3.

6. a display control unit that displays information about the moving object on an operation screen; the display control unit displays information about the destination shooting position on the operation screen while flying between the shooting positions in the intra-court flight mode.

6. The aerial photography system according to claim 5.

7. the target area is made up of the court and an off-court area defined by the connection of a pair of goal lines facing each other and a pair of touch lines facing each other, The system further includes an outer edge flight control unit that controls the flight of the moving body in the outer edge flight mode, and the outer edge flight control unit moves the moving body along the touch line in the outer edge flight mode and causes the moving body to follow a specific player or the ball to perform automatic photography.

2. The aerial photography system according to claim 1.

8. When the ball goes beyond the touch line on which the moving object is flying and enters the outside court area, the outer edge flight control unit moves the moving object to the outside court area and beyond the touch line, and takes a photograph directly below the moving object or toward the court.

8. The aerial photography system according to claim 7.

9. The flight modes further include a fixed-position flight mode in which the moving body is flown at a fixed position; A fixed position flight control unit controls the operation of the moving body in the fixed position flight mode, In the fixed position flight mode, the fixed position flight control unit hovers at a predetermined position, controls the nose direction or the direction of the camera, and automatically photographs a specific player or ball by following it.

2. The aerial photography system according to claim 1.

10. When the in-court flight control unit detects an obstacle on the flight path or near the moving body in the in-court flight mode, it performs at least one of the following operations: changing the connection of the shooting positions to regenerate a flight path that bypasses the obstacle; determining flight at a higher altitude than the flight path; starting movement of the moving body on the flight path after hovering for a predetermined time; switching to manual control after hovering for a predetermined time; and displaying a message prompting the user to re-input a target position after hovering for a predetermined time.

6. The aerial photography system according to claim 5.

11. The outer edge flight control unit generates a flight path for the moving body in the outer edge flight mode, and when an obstacle is detected on the flight path or near the moving body, performs at least one of the following operations: regenerates a flight path that detours around the obstacle to the inside of the court; determines flight at an altitude higher than the flight path; starts moving the moving body on the flight path after hovering for a predetermined time; switches to manual control after hovering for a predetermined time; and displays a message prompting the user to re-input the target position after hovering for a predetermined time.

8. The aerial photography system according to claim 7.

12. an event detection step of detecting an event based on an image acquired by a camera photographing an area of ​​interest or an input from an external system; a mode switching determination step for determining whether or not switching of a flight mode is necessary, in which a flight area in which a moving object flying in at least the target area is defined; Equipped with The flight mode is an edge flight mode in which the robot flies above the outer edge along a part or all of the outer edge of a court configured within the target area; an in-court flight mode in which the drone flies above the court; At least In the mode switching determination step, it is determined whether or not the flight mode needs to be switched depending on the event detected in the event detection step. Aerial photography method.

13. An event detection instruction for detecting an event based on an image acquired by a camera photographing an area of ​​interest or an input from an external system; A mode switching determination command for determining whether or not switching of a flight mode is necessary, in which a flight area in which a moving object flying in at least the target area is defined is determined; on the computer, The flight mode is an edge flight mode in which the robot flies above the outer edge along a part or all of the outer edge of a court configured within the target area; an in-court flight mode in which the drone flies above the court; At least The mode switching determination command determines whether or not the flight mode needs to be switched depending on the event detected by the event detection command. Aerial photography program.

Images