Aerial imaging system, aerial imaging method, and aerial imaging program

Abstract

[Problem] To ensure safety in aerial imaging. [Solution] An aerial imaging system 1 comprises a mobile object 100 that flies over a target area F, a camera 141 mounted on the mobile object to capture an image of the target area, an event detection unit 320 that detects an event on the basis of the image captured by the camera or an input from an external system, and an imaging condition determination unit 325 that determines an imaging condition, including at least one of the target imaging position and target imaging direction of the mobile object, according to the detected event.

Description

Aerial photography system, aerial photography method, and aerial photography program

[0001] The present invention relates to an aerial photography system, an aerial photography method, and an aerial photography program.

[0002] Patent Literature 1 discloses a camera viewpoint display system that detects the aircraft position and nose direction of an aircraft, detects the pan angle and tilt angle of a camera device mounted on the aircraft, calculates the camera viewpoint from these pieces of data, and displays the viewpoint on a map on a monitor screen. In this system, an operator controls the aircraft's position and attitude, the camera's shooting direction, etc., while grasping the aircraft's position and heading from a ground station.

[0003] Patent Document 2 discloses a technique for automatically controlling the positions and photographing directions of multiple UAVs so as to track and photograph a specific object.

[0004] JP 2006-281830 A JP 2020-115642 A

[0005] To effectively capture sports and other events, it is necessary to capture images that are suited to the details of the subject, such as the game situation. More specifically, since the range to be captured changes depending on the game situation, it is desirable to be able to appropriately control the shooting position and direction according to the situation.

[0006] In this regard, the system described in Patent Document 1 is complicated because it is necessary to control the camera and the flying object simultaneously, while the system described in Patent Document 2 is inconvenient because it only tracks the shooting position and shooting direction of the object.

[0007] The present invention has been made in consideration of the above-mentioned problems, and aims to provide an aerial photography system that reduces the labor required for photography and enables appropriate photography according to the situation of the subject being photographed.

[0008] In order to achieve the above-mentioned object, an aerial photography system according to one aspect of the present invention comprises a mobile body that flies over a target area, a camera mounted on the mobile body that photographs the target area, an event detection unit that detects an event based on an image acquired by the camera or input from an external system, and a photography condition determination unit that determines photography conditions including at least one of a target photography position and a target photography direction of the mobile body according to the detected event.

[0009] The photographing condition determination unit may determine the photographing conditions according to the type of the detected event.

[0010] The target shooting direction may be achieved by controlling at least one of the nose direction of the moving body and the angle of the camera relative to the moving body.

[0011] The photographing conditions may include a target zoom amount of the camera.

[0012] When the shooting condition determination unit receives input of the shooting conditions from a controller of the moving body or the external system, it may determine the shooting conditions based on the operation received via the controller even if the event detection unit has detected the event.

[0013] The camera may further include a controller that accepts input of the shooting conditions by a user, and the shooting condition determination unit may determine the shooting conditions based on the input via the controller if the event detection unit has not detected the event, and may determine the shooting conditions based on the event if the event detection unit has detected the event.

[0014] The aerial photography system may include a plurality of moving bodies, and photograph a single target area by flying the plurality of moving bodies simultaneously over the target area, and the photography condition determination unit may determine different photography conditions for each of the plurality of moving bodies.

[0015] The shooting condition determination unit may set, for multiple moving bodies flying simultaneously, shooting conditions that photograph the same shooting range from different target shooting positions, or shooting conditions that photograph an area including the same shooting range with different zoom amounts.

[0016] The photographing condition determination unit may determine the photographing conditions in accordance with a predicted result of a ball trajectory predicted by the event detection unit as a detection result of the event.

[0017] The shooting condition determination unit may determine the shooting conditions so that the shooting range is around the ball used in the game or the position of the referee of the game when an event indicating a foul has occurred in a game held in the target area is detected.

[0018] The aerial photography system may include a plurality of moving bodies, and photograph a single target area by flying the plurality of moving bodies simultaneously over the target area, and the photography condition determination unit may determine the photography conditions, when an event indicating that a foul has occurred in the competition, such that the plurality of moving bodies have different target photography positions, target photography directions, or zoom amounts to photograph the area around the ball or the referee of the competition.

[0019] The system may further include a flight path generation unit that generates a flight path of the moving body, and the flight path generation unit may automatically generate the flight path to the target shooting position that is determined based on the event detected from the captured image.

[0020] The flight path generation unit may generate the flight path within a court configured within the target area, and the flight path generation unit may connect a plurality of pre-set shooting positions to generate the flight path to the target shooting position, and may change the shooting positions to be connected depending on the detection status of the event.

[0021] In order to achieve the above-mentioned object, an aerial photography method according to another aspect of the present invention includes an event detection step of detecting an event based on an image acquired by a camera photographing a target area or input from an external system, and a photography condition determination step of determining photography conditions including at least one of a target photography position and a target photography direction of a moving body equipped with the camera according to the detected event.

[0022] To achieve the above object, according to yet another aspect of the present invention, an aerial photography program causes a computer to execute an event detection command that detects an event based on an image captured by a camera photographing a target area or an input from an external system, and an photography condition determination command that determines photography conditions including at least one of a target photography position and a target photography direction of a mobile body equipped with the camera in accordance with the detected event. Note that the computer program can be provided by being stored on various data-readable recording media or by being downloadable via a network such as the Internet.

[0023] According to the present invention, it is possible to reduce the labor required for photographing and to photograph an object appropriately according to the situation of the object.

[0024] 1 is a diagram illustrating the overall configuration of an aerial photography system according to an embodiment of the present invention. It is an external perspective view that simply illustrates a drone according to the embodiment. It is a diagram illustrating the functional configuration of the drone according to the embodiment. (a) is an external front view that simply illustrates a control device according to the embodiment, and (b) is a schematic diagram illustrating the direction in which the drone moves or turns in response to input from the control device. It is a diagram illustrating the functional configuration of the control device according to the embodiment. It is a diagram illustrating the functional configuration of the server according to the embodiment. It is a schematic diagram illustrating an example of a photography position of the drone that is set in advance in a photography target field where the drone flies. It is a schematic state transition diagram illustrating the transition of flight modes of the drone. It is a schematic state transition diagram illustrating the transition of states of the drone according to the aircraft state of the drone. It is a schematic state transition diagram illustrating the transition of states of the drone according to the aircraft behavior state of the drone. It is a schematic state transition diagram illustrating the transition of states of a game state in a stadium as an example of a photography target field. It is a schematic state transition diagram illustrating the transition of states of offensive and defensive states in the stadium. It is a table illustrating an example of the correspondence between game states in the stadium and the photography range, camera position, photography direction, and zoom amount photographed by the drone. It is a schematic diagram illustrating photography positions and flight paths that can be transitioned from the photography position. 15A and 15B are flowcharts of control performed during flight of the drone. FIG. 15B is a flowchart of control of flight restrictions in the drone (details of S1002 in FIG. 15). FIG. 15C is a flowchart of control of flight mode switching in the drone (details of S1010 in FIG. 15). FIG. 15C is a diagram of a first example of a screen displayed on a terminal of the aerial photography system. FIG. 15D is a diagram of a second example of a screen displayed on a terminal of the aerial photography system. FIG. 15E is a diagram of a third example of a screen displayed on a terminal of the aerial photography system. FIG. 15F is a diagram of a fourth example of a screen displayed on a terminal of the aerial photography system.

[0025] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted. Furthermore, the embodiments described below are merely examples, and other known elements or alternative means may be adopted depending on the application, purpose, scale, etc.

[0026] <A. One Embodiment> [A-1. Configuration] (A-1-1. Overall Configuration) Fig. 1 is a diagram showing the overall configuration of an aerial photography system 1 (hereinafter also referred to as "system 1") according to one embodiment of the present invention. System 1 uses a drone 100 (an example of a moving object) to take aerial photographs of a competition taking place at a stadium F (Fig. 7) (an example of a target area) or an event taking place at an event venue. Stadium F is an example of a target area. A single system 1 may include multiple drones 100. In this case, system 1 can photograph a single stadium F by flying multiple drones simultaneously over the stadium F.

[0027] In the following explanation, the system 1 that photographs soccer as needed will be used as an example, but the system 1 can also be applied to sports and events other than soccer.

[0028] As shown in Figure 1, in addition to the drone 100, the system 1 mainly includes a controller 200 that allows the pilot to operate the drone 100, a server 300 that manages the flight and photography of the drone 100, an external input device 600, and an external system 700.

[0029] The drone 100 and the controller 200 are connected to each other via wireless communication (which may include communication via a base station 800). The controller 200 and the server 300 are connected to each other via a communication network 400 such as an internet line. The drone 100 acquires satellite signals from an artificial satellite 500 to determine its own location, etc.

[0030] The external input device 660 is a device separate from the controller 200 that can send and receive information to and from the system 1, and is configured, for example, as a mobile device such as a smartphone or tablet. The external input device 660 can be operated, for example, by a manager, coach, bench player, referee, or court facility personnel of a game being played at the stadium F. The external input device 660 has a function for receiving, for example, an emergency command to stop filming, and the drone 100 performs emergency evacuation based on the command. The external input device 660 may also receive an input to switch the flight mode of the drone 100. Furthermore, the external input device 660 may include a display device that may display information similar to that displayed on the display unit 201 of the controller 200. In particular, the external input device 660 may acquire event information occurring during the game. The event information is referenced when a user of the external input device 660 inputs to switch the flight mode of the drone 100.

[0031] The external system 700 may be any system configured separately from the system 1. For example, systems such as a court facility system, a match management system, and a referee support system may be applied as systems deployed for a competition held at the stadium F, as well as systems such as a weather observation system or an earthquake observation system deployed independently of the competition. Multiple external systems 700 may be connected to the system 1. The system 1 may receive an emergency command to stop filming or a command to switch the flight mode of the drone 100 from various external systems 700. Furthermore, the various external systems 700 may acquire event information occurring during the competition.

[0032] A court facilities system as an example of the external system 700 may, for example, obtain the brightness of the captured image from the system 1 and control the illuminance adjustment or blinking of the lighting in the stadium F. The court facilities system may also receive a request for lighting illuminance from the system 1 and control the illuminance adjustment or blinking.

[0033] The configuration of system 1 is not limited to that shown in FIG. 1 , and the drone 100, controller 200, server 300, and base station 800 may be connected to each other so that they can communicate with each other via a communication network 400 such as an internet line. In this case, the drone 100 may communicate wirelessly directly with the communication network 400 using a communication method such as LTE without going through the controller 200. Therefore, the drone 100, controller 200, and base station 800 do not need to communicate wirelessly directly; they only need to be connected to the communication network 400 in remote locations. Therefore, this system configuration is suitable for cases where the drone 100 and controller 200 are located in remote locations (for example, when a pilot remotely controls the drone).

[0034] In addition, in system 1, the drone 100, the controller 200, the base station 800, and the server 300 are each connected to each other so that they can communicate with each other via a communication network 400 such as an Internet line, and the drone 100 and the base station 800 may be communicatively connected to the communication network 400 via satellite communication via an artificial satellite 500.

[0035] Furthermore, the system 1 may be configured redundantly, with multiple servers 300 connected to one drone 100 via multiple communication networks 400. In this case, even if an abnormality occurs in the server 300 or the communication network 400, the operation of the system 1, and therefore image capture by the drone 100, can be continued by the other redundant servers 300 and communication networks 400, thereby improving the reliability of the system 1. Note that, in the above two configurations, the drone 100 and the controller 200 can be controlled even when they are remotely located, and therefore the configuration is suitable for remote control, but is not limited to this and can also be applied to visual flight, in which the pilot manually controls the drone 100 while watching it.

[0036] The device described in the above embodiment may be realized as a single device, or may be realized by a plurality of devices (e.g., drone 100, controller 200, cloud server 300) that are partly or entirely connected via communication network 400. For example, each functional unit and memory unit of server 300 may be implemented in a different server 300, drone 100, and controller 200 that are connected to each other via communication network 400.

[0037] (A-1-2. Drone 100) (A-1-2-1. Overview of the drone 100) Fig. 2 is a simplified perspective view of the exterior of the drone 100 of this embodiment. Fig. 3 is a functional configuration diagram of the drone 100 of this embodiment. As described above, the drone 100 takes aerial photographs of competitions taking place at the stadium F (Fig. 7), events taking place at event venues, and the like.

[0038] In this specification, the term "drone" refers to any flying object that has the ability to autonomously control its attitude, regardless of the power source (electricity, prime mover, etc.), the control method (wireless or wired, fully autonomous flight type or partially manual flight type, etc.), and whether it is manned or unmanned. Drones are also sometimes called unmanned aerial vehicles (UAVs), flying objects, multicopters, RPAS (remote piloted aircraft systems), or UAS (unmanned aircraft systems), etc.

[0039] As shown in FIG. 2 , the exterior of the drone 100 is primarily composed of a housing 101 and multiple propellers 122. The housing 101 is, for example, a substantially rectangular parallelepiped, but may have any shape. Laterally extending rod-shaped connecting parts 102 are connected to the left and right sides of the housing 101. A propeller 122 and a motor 121 that rotates each propeller 122 are connected to the other end of each connecting part 102. The motor 121 is, for example, an electric motor. Note that, in this embodiment, four connecting parts 102, four propellers 122, and four motors 121 are provided, but the number is not limited thereto. The propeller 122 may be a single propeller or multiple propellers arranged coaxially. The number and shape of the blades of each propeller are not particularly limited.

[0040] Additionally, a propeller guard (not shown) may be provided on the outside of the propeller 122 to prevent the propeller from interfering with obstacles.

[0041] The housing 101 holds, for example, a photographing camera 141 by a camera holder 142 below the housing 101. An obstacle detection camera 131 is also provided on the front surface of the housing 101. In this embodiment, the obstacle detection camera 131 is a so-called dual camera made up of two cameras that form a pair. The obstacle detection camera 131 is provided so as to capture an image in front of the drone 100. Note that the obstacle detection cameras 131 may be provided not only on the front surface but also on all surfaces of the housing 101, for example, on six surfaces in the case of a housing 101 that is a substantially rectangular parallelepiped.

[0042] The drone 100 is equipped with an alarm device 250 that alerts people around the drone 100 to the presence of the drone 100. The alarm device 250 has, for example, a warning light 251 and a speaker 252. The warning light 251 is provided for each propeller 122 or motor 121, and is disposed, for example, on each side of multiple motors 121. The warning light 251 may be disposed along the cylindrical side surface of the motor 121 so that it can be seen from all directions, including the front. The speaker 252 outputs an alarm sound and is provided in the housing 101 of the drone 100. The speaker 252 is provided, for example, on the underside of the housing 101, and transmits the alarm sound downward from the drone 100.

[0043] (A-1-2-2. Functional blocks of drone 100) As shown in Figure 3, the drone 100 is equipped with an arithmetic unit such as a CPU (Central Processing Unit) for executing information processing, and storage devices such as a RAM (Random Access Memory) and a ROM (Read Only Memory), and thereby has the following functional blocks: a measurement unit 110, a flight function unit 120, an obstacle detection unit 130, an imaging unit 140, and a communication unit 150.

[0044] The measurement unit 110 is a functional unit that measures information related to the drone 100 or its surroundings. The measurement unit 110 includes, for example, a position measurement unit 111, a direction measurement unit 112, an altitude measurement unit 113, and a speed measurement unit 114. In addition to these, the measurement unit 110 may also include various sensors that acquire information such as temperature, air pressure, wind speed, and acceleration.

[0045] The position measurement unit 111 receives signals from the artificial satellites 500 and measures the position (absolute position) of the aircraft based on the signals. The position measurement unit 111 measures its current position using, for example, a Global Navigation Satellite System (GNSS) or a Global Positioning System (GPS), although this is not particularly limited. As a method for measuring its position, for example, a Real Time Kinematic - Global Navigation Satellite System (RTK-GNSS) can also be used. The position information includes at least two-dimensional coordinate information (e.g., latitude and longitude) in a planar view, and preferably includes three-dimensional coordinate information including altitude information.

[0046] Furthermore, the base station 800, which provides information on the reference point of a fixed station used for relative positioning such as RTK, is wirelessly connected to the drone 100 and the controller 200, thereby enabling measurement of the position of the drone 100 with higher accuracy. When RTK measurement is performed using a virtual reference point method with a VRS (Virtual Reference Station), the base station 800 can be omitted, or the accuracy of the position coordinate estimation of the base station 800 or the drone 100 can be further improved.

[0047] The orientation measurement unit 112 measures the orientation (nose direction, heading direction) of the aircraft. The orientation measurement unit 112 is composed of a geomagnetic sensor that measures the nose orientation (heading direction) of the aircraft of the drone 100 by measuring geomagnetism, a compass, etc.

[0048] The altitude measurement unit 113 measures the altitude above ground (hereinafter also referred to as "flight altitude") as the distance from the ground below the drone 100 (vertically downward).

[0049] The speed measurement unit 114 detects the flight speed of the drone 100. The speed measurement unit 114 may measure the speed using a known sensor such as a gyro sensor.

[0050] (A-1-2-3. Flight function unit 120) The flight function unit 120 is a mechanism and functional unit that causes the drone 100 to fly, and generates thrust in the airframe to lift the drone 100 and move it in a desired direction. As shown in Figures 2 and 3, the flight function unit 120 has multiple motors 121, multiple propellers 122, and a flight control unit 123.

[0051] The flight control unit 123 independently controls the multiple motors 121 to rotate each propeller 122, causing the drone 100 to perform various operations such as taking off, moving forward, turning, and landing, and controls the attitude angle control and flight operations of the drone 100 from takeoff to flight and landing.

[0052] The flight control unit 123 includes a processing unit, also referred to as a flight controller. The processing unit may include one or more processors, such as a programmable processor (e.g., a central processing unit (CPU), an MPU, or a DSP). The processing unit has access to a memory (storage unit). The memory stores logic, code, and / or program instructions that the processing unit can execute to perform one or more steps. The memory may include a separable medium, such as an SD card or RAM, or an external storage device. Various data acquired by the measurement unit 110 or video or still image data captured by the imaging camera 141 may be directly transmitted to and stored in the memory. Note that each data may also be recorded in an external memory.

[0053] The processing unit includes a control module configured to control the airframe state of the drone 100. For example, the control module controls the flight function unit 120 (thrust generating unit) of the drone 100 to adjust the spatial configuration, attitude angle, angular velocity, angular acceleration, angular jerk rate, and / or acceleration of the drone 100, which has six degrees of freedom (translational motion x, y, and z, and rotational motion θx, θy, and θz).

[0054] The flight control unit 123 can control the flight of the drone 100 based on control signals from the controller 200 or based on a preset autonomous flight program. The flight control unit 123 can also control the flight of the drone 100 by controlling the motor 121 based on various information such as the field to be photographed, flight permitted / prohibited areas, information on the corresponding flight geofences, map information including two-dimensional or three-dimensional map data, current position information of the drone 100, attitude information (heading information), speed information, and acceleration information, or any combination of these.

[0055] Examples of Shooting Target Field and Shooting Position In this specification, "shooting target field" or "target area" refers to a two-dimensional location (for example, stadium F) that is the subject of shooting.

[0056] 7 is a schematic diagram showing an example of a stadium F, which is an example of a field to be photographed by a drone, viewed from above. The stadium F is composed of a court F100, which is generally rectangular and defined by a linear outer edge, and an outer court area F200, which is a predetermined area that covers the outer edge of the court F100. The outer edge of the court F100 is formed by mutually opposing goal lines F110a, F110b and mutually opposing touch lines F111a, F111b, which are connected at approximately right angles. The connection points between the goal lines F110a, F110b and the touch lines F111a, F111b are corners F112a, F113a, F112b, and F113b.

[0057] Goals F120a and F120b are provided approximately in the center of the pair of goal lines F110a and F110b, respectively. Penalty areas F130a and F130b are defined in predetermined areas within the court F100 adjacent to the goals F120a and F120b, and penalty lines F140a and F140b are drawn on the outer edges of the penalty areas.

[0058] A halfway line F150 is drawn in the center of the court F100, connecting the midpoints of the pair of touchlines F111a and F111b to divide the court F100 into approximately equal parts. The halfway line F150 is approximately parallel to the goal lines F110a and F110b.

[0059] Note that the goal lines F110a, F110b, touchlines F111a, F111b, penalty lines F140a, F140b, and halfway line F150 are required by the rules for players to play the game, and therefore, all of these lines are generally drawn in a visible manner, but the technical scope of the present invention is not limited to this. Also, while this explanation uses a soccer stadium as an example, the games photographed by the system of the present invention are not limited to soccer and include any other game, such as tennis. Furthermore, the subject of photography is not limited to sports, and the system can also be applied to other events (concerts, ceremonies, etc.).

[0060] A plurality of predetermined shooting positions L101 to L105 and L206 to L215 are defined in the stadium F. The shooting positions L101 to L105 and L206 to L215 may be two-dimensional coordinates on a plane, or may be three-dimensional coordinate information that also defines the height at each position. The flight height of the drone 100 may be manually controlled based on input from the controller 200.

[0061] Photographing positions L101 to L105 are defined, for example, on the touchline F111b, at approximately equal intervals along the touchline F111b. For example, photographing position L101 is a point located in a range including the intersection of the halfway line F150 and the touchline F111b and a point slightly outside the court F100. Photographing positions L103 and L105 are points near the corners F112a or F112b on either side of the touchline F111b. Photographing positions L102 and L104 are points between photographing positions L103 and L105 and photographing position L101. Note that the above-mentioned positions are merely examples and are not limited to these, and any appropriate positions may be used.

[0062] Photographing positions L206 to L215 are points defined within the court F100. For example, photographing positions L206 and L211 are points near the center of the penalty lines F140a and F140b on a line parallel to the goal lines F110a and F110b, so-called goal-front photographing positions. Photographing positions L207 and L212 are photographing positions closer to the touchline or F111a or F111b and closer to the halfway line F150 than photographing positions L206 and L215. More specifically, photographing positions L207 and L212 are points on an imaginary line connecting photographing position L101 and goals F120a and F120b, such as points approximately in the center of the imaginary line. Photographing positions L209 and L215 are points symmetrical to photographing positions L207 and L212. Photographing position L208 is a point between photographing position L207 and the halfway line F150, photographing position L210 is a point between photographing position L209 and the halfway line F150, photographing position L213 is a point between photographing position L212 and the halfway line F150, and photographing position L214 is a point between photographing position L215 and the halfway line F150.

[0063] In the off-court area F200, an evacuation point H200 is set to which the drone 100 is to be evacuated if an abnormality or malfunction of the drone 100 or the system 1 is detected. The abnormality referred to here refers to an abnormality related to the stability of the drone 100's aerial movement. The abnormality may include, for example, a case where the computational load associated with the operation control (behavior control, shooting control, etc.) of the drone 100 exceeds a load threshold. Alternatively, the abnormality may include a transient abnormality related to the environment, such as a case where a measured value of the drone 100's behavior control value (e.g., speed) exceeds an allowable value due to the influence of strong winds or the like.

[0064] The evacuation point H200 is set at a location different from the shooting positions L101 to L105 and L206 to L215, and in this embodiment, it is set outside the touchline F111a and along the touchline F111a. There may be multiple evacuation points H200; in this embodiment, there are three evacuation points. The evacuation point H220 is set near an extension of the halfway line F150. The evacuation points H210 and H230 are set closer to the goals F120a and F120b than the shooting positions L206 and L211. The evacuation points H210 and H230 are set, for example, at the ends of the area defined by the geofence G200 described below. At the evacuation point H200, for example, the drone 100 is replaced or the battery installed in the drone 100 is replaced.

[0065] In the stadium F, multiple geofences G100, G200 are set, each encompassing at least the respective shooting positions L101 to L105 and L206 to L215. A geofence indicates a virtual boundary line that divides an area, and in particular, the geofence in this embodiment indicates a fence that is the boundary line between a permitted flight area where the drone 100 is permitted to fly or move and a prohibited flight area. A geofence is a boundary line that divides an area that extends three-dimensionally, including in plan and height. If a mobile object such as the drone 100 comes into contact with a geofence, flight or movement is restricted to prevent the aircraft from flying outside the permitted flight area.

[0066] The boundary line of the geofence in the height direction may include an upper limit and a lower limit. In this embodiment, the geofences G100 and G200 that are applied to flight permission and prohibition are switched in accordance with the control of the system 1 while the drone 100 is flying. Although the number of geofences G100 and G200 depicted in the figure is two, the number is arbitrary, and specifically may be three or more.

[0067] The geofence G100 is an area that encompasses the shooting positions L101 to L105 and defines an area that encompasses the area on and near the touchline F111b. In other words, the geofence G100 is defined near the outer edge of the court F100, with a portion extending into the outer court area F200. The geofence G200 is a geofence that is primarily applied in the outer edge flight mode M102, which will be described later. The geofence G200 is an area that encompasses the shooting positions L206 to L215 and is set at least inside the court F100. This geofence G200 is a geofence that is primarily applied in the inner court flight mode M105, which will be described later.

[0068] The areas defined by the multiple geofences G100, G200 at least partially contact or overlap with each other. The areas defined by the multiple geofences G100, G200 also overlap in the height direction. The heights of the multiple geofences G100, G200 may be different from each other. Specifically, the lower limit of the altitude of the geofence G200 set inside the stadium F is set higher than the lower limit of the altitude of the geofence G100 set on the outer edge of the stadium F. Because there is a high probability that subjects to be photographed, such as players, are present inside the stadium F, setting the lower limit of the altitude of the geofence high and flying the drone 100 at a sufficient height can prevent interference with the movements of the subjects to be photographed and collisions with the subjects to be photographed or the ball.

[0069] (A-1-2-4. Obstacle detection unit 130) Returning to the explanation of Figure 3, the obstacle detection unit 130 is a functional unit that detects obstacles around the drone 100. Obstacles may include, for example, people, players, objects, animals such as birds, fixed equipment, and the ball. The obstacle detection unit 130 measures the position, velocity vector, etc. of obstacles located below the drone 100, etc., based on the acquired image.

[0070] The obstacle detection unit 130 includes, for example, an obstacle detection camera 131, a ToF (Time of Flight) sensor 132, and a laser sensor 133. The ToF sensor 132 measures the time it takes for a laser pulse emitted from the sensor to return to a light-receiving element within the sensor, and converts this time into distance to measure the distance to an object. The laser sensor 133 uses, for example, a LiDAR (Light Detection and Ranging) method to irradiate an object with light such as near-infrared light, visible light, or ultraviolet light, and captures the reflected light with an optical sensor to measure the distance.

[0071] In FIG. 2, in this embodiment, the obstacle detection camera 131 is shown to be positioned facing forward, but the type, position and number of the camera 131, the ToF sensor 132 and the laser sensor 133 are arbitrary, and the ToF sensor 132 or the laser sensor 133 may be positioned instead of the camera 131, or the ToF sensor 132 or the laser sensor 133 may be provided on all six surfaces of the housing 101, i.e., the front, back, top, bottom and both side surfaces.

[0072] (A-1-2-5. Filming Unit 140) The filming unit 140 is a functional unit that captures video of competitions at the stadium F (FIG. 7) or events at the event venue, and includes a filming camera 141, a camera holding unit 142, and a filming control unit 143. As shown in FIG. 2, the filming camera 141 (imaging device) is disposed at the bottom of the main body of the drone 100, and outputs image data relating to peripheral images captured around the drone 100. The filming camera 141 is a video camera (color camera) that captures video. The video may include audio data acquired by a microphone (not shown). Additionally or alternatively, the filming camera 141 may also capture still images.

[0073] The orientation of the photographic camera 141 (the attitude of the photographic camera 141 relative to the housing 101 of the drone 100) can be adjusted by a camera actuator (not shown) incorporated in the camera holding unit 142. The photographic camera 141 may have an automatic control function for parameters such as exposure, contrast, or ISO. The camera holding unit 142 may have a so-called gimbal control mechanism that suppresses transmission of shaking or vibration of the aircraft to the photographic camera 141. The photographing control unit 143 controls the photographic camera 141 and the camera holding unit 142 to adjust the orientation of the photographic camera 141, the photographing magnification (zoom amount), the camera's photographing conditions, etc. Image data acquired by the photographic camera 141 can be transmitted to a memory unit of the drone 100 itself, the controller 200, the server 300, etc.

[0074] (A-1-2-6. Communication Unit 150) The communication unit 150 is capable of radio wave communication via the communication network 400 and includes, for example, a radio wave communication module. The communication unit 150 is capable of communication with the controller 200 and the like via the communication network 400 (including the wireless base station 800).

[0075] (A-1-3. Controller 200) (A-1-3-1. Overview of Controller 200) FIG. 4 is a simplified front view of the exterior of the controller 200 of this embodiment. FIG. 5 is a functional configuration diagram of the controller 200 of this embodiment. The controller 200 is a mobile information terminal that controls the drone 100 through operation by the operator and displays information received from the drone 100 (e.g., position, altitude, remaining battery level, camera footage, etc.). Note that in this embodiment, the flight status (altitude, attitude, etc.) of the drone 100 may be remotely controlled by the controller 200, or may be autonomously controlled by the drone 100. For example, when the operator transmits a flight command to the drone 100 via the controller 200, the drone 100 performs autonomous flight. Furthermore, manual operation may be possible during basic operations such as takeoff and return, and in emergencies.

[0076] As shown in Fig. 4, controller 200 includes, as its hardware configuration, a display unit 201 and an input unit 202. Display unit 201 and input unit 202 are connected to each other so that they can communicate with each other via wire or wirelessly. Display unit 201 may be configured as a touch panel or LCD monitor that is integrally incorporated into controller 200, or may be configured as a display device such as an LCD monitor, tablet terminal, or smartphone that is connected to controller 200 via wire or wirelessly. Display unit 201 as a hardware configuration may be configured as a touch panel display by integrally incorporating an element that accepts input such as touch.

[0077] The input unit 202 is a mechanism through which the pilot inputs operational commands such as flight direction and takeoff / landing when piloting the drone 100. As shown in FIG. 4A , the input unit 202 includes a left slider 326L, a right slider 326R, a left input stick 327L, a right input stick 327R, a power button 328, and a return button 329. The left slider 326L and the right slider 326R are controls that accept, for example, 0 / 1 input or one-dimensional, stepless or gradual information input. The pilot performs input by sliding the sliders with their left and right index fingers while holding the controller 200 in their hands. The left input stick 327L and the right input stick 327R are controls that accept multi-dimensional, stepless or gradual information input, such as joysticks. The left input stick 327L and the right input stick 327R may also accept 0 / 1 input by pressing them. The power button 328 and the feedback button 329 are operators that are pressed down, and are configured by mechanical switches or the like.

[0078] The left input stick 327L and the right input stick 327R accept input operations that instruct three-dimensional flight operations of the drone 100, including takeoff, landing, ascent, descent, right turn, left turn, forward movement, backward movement, left movement, and right movement. Fig. 4(b) is a schematic diagram showing the movement direction or rotation direction of the drone 100 corresponding to each input of the left input stick 327L and the right input stick 327R shown in Fig. 4(a). Note that this correspondence is an example.

[0079] As shown in FIG. 5, the controller 200 includes a computing device such as a CPU for executing information processing, and storage devices such as RAM and ROM, which constitute the software configuration, primarily functional blocks of a display control unit 210, an input control unit 220, and a communication unit 240.

[0080] (A-1-3-2. Display control unit 210) The display control unit 210 displays to the pilot status information of the drone 100 and the like obtained from the drone 100 or the server 300. The display control unit 210 can display images relating to various information such as the field to be photographed, permitted / prohibited flight areas, flight geofence, map information, current position information of the drone 100, attitude information (directional information), speed information, acceleration information, and remaining battery power. The "current position information" referred to here is sufficient to include information on the horizontal position of the current position of the drone 100 (i.e., latitude and longitude), and does not have to include altitude information (absolute altitude or relative altitude).

[0081] The display control unit 210 has a mode display unit 211 and a shooting state display unit 212 .

[0082] The mode display unit 211 is a functional unit that displays at least the state, i.e., the mode, to which the drone 100 belongs on the display unit 201. The mode to which the drone 100 belongs is, for example, the flight mode shown in Fig. 8, but instead of or in addition to this, the display unit 201 may display the aircraft state shown in Fig. 9, the aircraft action state shown in Fig. 10, the game state shown in Fig. 11, or the offensive and defensive states shown in Fig. 12.

[0083] As shown in Figure 18, the screen G1 displayed on the display unit 201 displays, for example, a display field G21 for the flight mode to which the drone 100 belongs, as well as a status display field G22 showing the aircraft status, aircraft behavior status, game status, and offensive and defensive status.

[0084] The shooting status display unit 212 shown in Fig. 5 is a functional unit that displays, on the display unit 201, images captured by the shooting camera 141 mounted on the drone 100. As shown in Fig. 18, the screen G1 displayed on the display unit 201 displays, for example, an image field G40 in which images being captured by the drone 100 are displayed. Details of the screen G1 and each state will be described later.

[0085] 5 accepts various inputs from a user such as a pilot. The input control unit 220 of this embodiment mainly has the following functional units: an aircraft position operation unit 221, an aircraft attitude operation unit 222, a camera attitude operation unit 223, a camera zoom operation unit 224, a flight mode switching unit 225, a target position acceptance unit 226, a power supply input unit 227, and a return input unit 228.

[0086] The aircraft position operation unit 221 includes an up / down movement input unit 221a and a left / right movement input unit 221b. The aircraft attitude operation unit 222 includes a forward / backward movement input unit 222a and a yaw rotation input unit 222b.

[0087] The up / down movement input unit 221a is an input unit used by the pilot to move the drone 100 up and down, and acquires input to the right input stick 327R. That is, when the right input stick 327R is moved upward (toward the back when held in the hand), the drone 100 rises, and when the right input stick 327R is moved downward (toward the front when held in the hand), the drone 100 descends. The left / right movement input unit 221b is an input unit used by the pilot to move the drone 100 left and right, and acquires input to the right input stick 327R. That is, when the right input stick 327R is moved to the right, the drone 100 moves right, and when the right input stick 327R is moved to the left, the drone 100 moves left.

[0088] The forward / backward movement input unit 222a is an input unit used by the pilot to move the drone 100 forward or backward, and acquires input to the left input stick 327L. That is, when the left input stick 327L is moved upward (toward the rear when held in the hand), the drone 100 moves forward, and when the left input stick 327L is moved downward (toward the front when held in the hand), the drone 100 moves backward. The yaw rotation input unit 222b is an input unit used by the pilot to yaw rotate the drone 100, and acquires input to the left input stick 327L. That is, when the left input stick 327L is moved to the right, the drone 100 turns right, and when the left input stick 327L is moved to the left, the drone 100 turns left.

[0089] The camera attitude operation unit 223 is an input unit for operating the camera holding unit 142 via the photography control unit 143 and for operating the orientation of the photography camera 141 relative to the housing 101 of the drone 100. The camera attitude operation unit 223 acquires input to the right slider 326R. The camera attitude operation unit 223 accepts operation of either or both of the pitch angle and yaw angle of the photography camera 141 relative to the housing 101.

[0090] The camera zoom operation section 224 is an input section for operating the imaging magnification of the imaging camera 141, that is, the zoom amount, and acquires input to the left slider 326L.

[0091] The flight mode switching unit 225 is an input unit for switching flight modes. Flight modes selectable by the flight mode switching unit 225 include at least, for example, an outer edge flight mode M102 (see FIG. 8 ), an inside court flight mode M105 (see FIG. 8 ), and a fixed position flight mode M103 or M107 (see FIG. 8 ). The flight mode switching unit 225 accepts switching of the flight mode via, for example, a touch panel display integrated with the display unit 201.

[0092] The target position receiving unit 226 is a functional unit that receives input of a target shooting position to which the drone 100 should head. The target position receiving unit 226 receives input of a point on the stadium F. For example, in a state in which at least a portion of an image or schematic diagram of the stadium F is displayed on the display unit 201, the target position receiving unit 226 may receive input of the target shooting position via a touch panel display that is configured integrally with the display unit 201. In cases in which a point that can be selected as the target shooting position, i.e., a shooting position, is specified in advance, the target position receiving unit 226 may receive input of the selection of the target shooting position.

[0093] Flight Modes Here, we will explain the types of flight modes that can be set in drone 100 and examples of their state transitions. As shown in Fig. 8, the flight modes of drone 100 mainly include advance preparation mode M100, off-court takeoff and landing mode M101, outer edge flight mode M102, off-court fixed position flight mode M103, on-court entry mode M104, on-court flight mode M105, off-court exit mode M106, on-court fixed position flight mode M107, and on-court takeoff and landing mode M108.

[0094] The advance preparation mode M100 is a mode in which advance settings such as geofence setting are performed. The advance preparation mode M100 transitions to an off-court takeoff and landing mode M101. In this off-court takeoff and landing mode M101, the drone 100 takes off from point L101g (see FIG. 14 ). Note that in the off-court takeoff and landing mode M101, the drone 100 may take off from a point outside the court F100 that is different from point L101g.

[0095] The off-court takeoff and landing mode M101 is the mode to which the drone 100 belongs when control starts or ends. The drone 100 transitions from the off-court takeoff and landing mode M101 to the perimeter flight mode M102.

[0096] The outer edge flight mode M102 is a mode in which the drone flies above the outer edge along part or all of the outer edge of the court F100 to photograph the playing field F, and more specifically, is a mode in which the drone flies at one of the photographing positions L101 to L105 (FIG. 14). In the main embodiment in this description, the outer edge flight mode M102 is a mode in which the drone flies above the touch line F111b. However, the "outer edge" in which the drone flies in the outer edge flight mode M102 is a concept that includes not only directly above the touch line F111b but also slightly outside the court F100.

[0097] In outer flight mode M102, the drone 100 receives user instructions via the target position receiving unit 226 of the controller 200 and flies at one of the specified image capture positions L101 to L105. The image capture direction may be manually controlled in response to user instructions, or may be fixed at a predetermined angle. In outer flight mode M102, the drone 100 may change its image capture position while keeping the image capture direction fixed, a technique known as dolly image capture, in which the drone 100 follows and captures a specific player.

[0098] The outer edge flight mode M102 can transition to an outside court takeoff and landing mode M101, an outside court fixed position flight mode M103, or an inside court approach mode M104.

[0099] The off-court fixed position flight mode M103 is a mode in which the drone 100 flies in a fixed position outside the area of ​​the court F100. The off-court fixed position flight mode M103 can transition to the outer edge flight mode M102. The on-court entry mode M104 is a mode in which the drone 100 performs a series of processes required for entering the area of ​​the court F100. The drone 100 transitions to the on-court flight mode M105 via the on-court entry mode M104.

[0100] The in-court flight mode M105 is a mode in which the drone flies above the court F100 to photograph the playing field F, specifically, in which the drone flies at one of the photographing positions L206 to L215 (FIG. 7). In the in-court flight mode M105, as in the outer edge flight mode M102, the drone receives a user command to select a photographing position via the target position receiving unit 226 of the controller 200, and flies at one of the specified photographing positions L206 to L215. The photographing direction may be manually controlled in response to a user instruction, or may be fixed at a predetermined angle.

[0101] The in-court flight mode M105 can transition to an out-of-court exit mode M106, an in-court fixed position flight mode M107, or an in-court takeoff and landing mode M108.

[0102] The court exit mode M106 is a mode in which the drone 100 performs a series of processes required for the drone 100 to exit the area of ​​the court F100. The drone 100 transitions to the outer perimeter flight mode M102 via the court exit mode M106. Note that the court exit mode M106 and the court entry mode M104 can transition to each other.

[0103] The on-court fixed position flight mode M107 is a mode in which the drone flies in a fixed position within the area of ​​the court F100. The on-court fixed position flight mode M107 can transition to the on-court flight mode M105. The on-court takeoff and landing mode M108 is a mode in which the drone takes off and lands within the area of ​​the court F100, and is a mode to which the drone transitions mainly when a command to land on the spot is issued by manual intervention. A drone that takes off in the on-court takeoff and landing mode M108 transitions to the on-court flight mode M105.

[0104] Returning to Figure 5, the power input unit 227 is a functional unit that receives a command to turn the power of the controller 200 on or off via a power button 328.

[0105] The return input unit 228 is a functional unit that receives a command via the return button 329 to return the drone 100 located in the stadium F (Figure 7) to the target landing point L101g (see Figure 14).

[0106] Furthermore, instead of or in addition to the above configuration, the input control unit 220 may be configured to receive touch input to the display unit 201 and transmit a control command to the drone 100 in response to the input. More specifically, for example, when the user performs a selection operation on appropriate information such as a map or a schematic diagram displayed on the display unit 201, a route to the selected point may be automatically generated, causing the drone 100 to fly autonomously.

[0107] (A-1-3-4. Communication Unit 240) The communication unit 240 is a functional unit that transmits and receives signals between the controller 200 and appropriate components included in the system 1. The controller 200 has a communication function that performs wireless communication with the drone 100 using Wi-Fi, 2.4 GHz, and frequency bands of 5.6 to 5.8 GHz. The controller 200 also has a wireless communication function that enables communication with the server 300 via the communication network 400 using a communication standard such as LTE (Long Term Evolution). The communication unit 240 transmits, for example, various input signals from a user such as a pilot to the drone 100, the server 300, or the like. The communication unit 240 also receives signals from the drone 100, the server 300, or the like.

[0108] (A-1-4. Server 300) (A-1-4-1. Overview of Server 300) Fig. 6 is a functional configuration diagram of the server 300 of this embodiment. The server 300 manages or controls the flight and photography of the drone 100. The server 300 has an input / output unit (not shown) for inputting or outputting various types of information (image output, audio output).

[0109] The server 300 may be a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing.

[0110] The server 300 is equipped with an arithmetic unit such as a CPU for executing information processing, and storage devices such as RAM and ROM, and as a software configuration, it mainly comprises functional blocks such as a pre-setting unit 310, an event detection unit 320, a shooting condition determination unit 325, a flight mode switching unit 330, an outer edge flight control unit 340, an in-court flight control unit 350, a fixed position flight control unit 360, a communication unit 370, and a memory unit 380.

[0111] (A-1-4-2. Pre-setting unit 310) The pre-setting unit 310 is a functional unit that performs settings necessary for the flight of the drone 100 before the drone 100 flies over the field to be photographed. The pre-setting unit 310 mainly has a geofence setting unit 311.

[0112] The geofence setting unit 311 is a functional unit that sets a geofence for the drone 100. The geofence includes information on the planar direction and the height direction.

[0113] The geofence setting unit 311 sets a geofence according to the flight mode. That is, for example, the geofence setting unit 311 activates the geofence G100 (see FIG. 7) in the outer edge flight mode M102 (see FIG. 8). The geofence setting unit 311 also activates the geofence G200 (see FIG. 7) in the inner court flight mode M105 (see FIG. 8). The geofence setting unit 311 also sets a geofence different from the geofence G100 or the geofence G200, a so-called third geofence, in intermediate modes that intervene in the transition between the outer edge flight mode M102 and the inner court flight mode M105, i.e., the outer court exit mode M106 and the inner court entry mode M104.

[0114] When the geofence in the first flight mode and the geofence in the second flight mode, which is the flight mode at the destination, at least partially overlap, the third geofence is a geofence that covers an integrated area that combines the first area defined by the geofence in the first flight mode and the second area defined by the geofence in the second flight mode. In this embodiment, the geofence G100 in the outer edge flight mode M102 and the geofence G200 in the inner court flight mode M105 overlap. Therefore, the third geofence is a geofence that defines the area that combines the geofence G100 and the geofence G200.

[0115] Furthermore, if the geofence for the first flight mode and the geofence for the second flight mode do not overlap, the third geofence may be a geofence covering an area that combines a first area defined by the first geofence corresponding to the first flight mode, a second area defined by the second geofence corresponding to the second flight mode, and a gap between the first and second areas. This configuration ensures safety by preventing deviation from the geofence during mode transitions, even when transitioning between flight modes in which the geofences do not overlap. The geofence setting unit 311 stores information about the set geofence in the storage unit 380.

[0116] (A-1-4-3. Event detection unit 320) The event detection unit 320 is a functional unit that detects the state of the subject to be photographed or the drone 100. The event detection unit 320 detects an event based on the camera image from the photographing camera 141 or an input from the external system 700. The detection criteria for each event are stored in, for example, the storage unit 380, and the event detection unit 320 detects an event by referring to the storage unit 380. The event detection unit 320 may also detect an event by analysis using a neural network. The detection process by the event detection unit 320 can be performed using any known appropriate image analysis technology.

[0117] The event detection unit 320 particularly detects an event that triggers a change in the flight mode or shooting conditions of the drone 100. The event detection unit 320 mainly includes an aircraft state acquisition unit 321, an aircraft behavior state acquisition unit 322, a game state acquisition unit 323, and an offensive / defensive state acquisition unit 324.

[0118] The aircraft state acquisition unit 321 is a functional unit that acquires the aircraft state of the drone 100. Fig. 9 is a diagram showing the state transitions of the aircraft state of the drone 100. The aircraft states are broadly divided into, for example, a normal operation flight mode M200, a detection and judgment mode M210, and an action mode M220. When the drone 100 starts flying, the drone 100 transitions to the normal operation flight mode M200.

[0119] When any detection or reception occurs in the normal operation flight mode M200, the aircraft state transitions to a detection and judgment mode M210. The detection and judgment mode M210 includes an abnormality detection mode M211, a failure detection mode M212, a manual intervention mode M213, and a low battery mode M214.

[0120] Specifically, if an abnormality is detected in the normal operation flight mode M200, the mode transitions to the abnormality detection mode M211. This abnormality is a transient, reversible disturbance such as a decrease in radio wave intensity or strong wind. If the abnormality is resolved in the abnormality detection mode M211, the mode transitions to the normal operation flight mode M200.

[0121] In the normal operation flight mode M200, if a failure of the aircraft or system is detected, the drone 100 transitions to the failure detection mode M212. If a manual control command is received, the drone 100 transitions to the manual intervention mode M213. If it is detected that the remaining battery charge is lower than a predetermined value, the drone 100 transitions to the low battery mode M214. Furthermore, if a manual control command is received in the abnormality detection mode M211, the failure detection mode M212, or the low battery mode M214, the drone 100 transitions to the manual intervention mode M213. The drone 100 transitions to the action mode M220 corresponding to the detection and determination mode M210.

[0122] The action mode M220 is a state in which the drone 100 performs a series of operations preset for each state. The action mode M220 includes a landing at an evacuation point mode M221, an emergency stop mode M222, a landing in place mode M223, a return mode M224, and a fixed position flight mode M225.

[0123] The landing at evacuation point mode M221 is set to an operation of flying the drone 100 to the evacuation point H200 and landing it. The landing at evacuation point mode M221 is entered when the abnormality is not resolved in the abnormality detection mode M211.

[0124] The emergency stop mode M222 is set to stop the propellers 122 immediately. In the emergency stop mode M222, the drone 100 falls freely. The emergency stop mode M222 can be selected in the manual intervention mode M213 when the propellers 122 are about to come into contact with a person or object.

[0125] The on-the-spot landing mode M223 is set to perform a soft landing on the spot, and the return mode M224 is set to perform a return to the takeoff and landing point.

[0126] The fixed position flight mode M225 is a state in which the drone 100 flies at a fixed position, and can transition to the normal operation flight mode M200 based on a user operation. The user operation is input, for example, by selecting a button displayed on the display unit 201. Furthermore, in the fixed position flight mode M225, if an event that can cause a transition from the normal operation flight mode M200 to the detection and judgment mode M210, i.e., an abnormality, a malfunction, manual intervention, or a low battery, is detected, the drone 100 transitions from the fixed position flight mode M225 to the detection and judgment mode M210 via the normal operation flight mode M200. In the fixed position flight mode M225, the drone 100 can transition to the return mode M224 based on a user operation.

[0127] The drone 100 in the abnormality detection mode M211 and the failure detection mode M212 transitions to the evacuation point landing mode M221. The drone 100 in the manual intervention mode M213 transitions to one of the evacuation point landing mode M221, emergency stop mode M222, in-place landing mode M223, return mode M224, and fixed position flight mode M225, depending on the input command. The drone 100 in the low battery mode M214 transitions to the return mode M224.

[0128] The drone 100 in the normal operation flight mode M200 can also transition to the return mode M224 based on a user operation. The user operation is input by selecting a button displayed on the display unit 201, for example.

[0129] The aircraft behavior state acquisition unit 322 is a functional unit that acquires the aircraft behavior state of the drone 100. Each mode of the aircraft behavior state is a submode of the aircraft state that is performed to realize the transition of the aircraft state. Figure 10 is a diagram showing the state transition of the aircraft behavior state. The aircraft behavior state is broadly divided into, for example, a takeoff mode M300, an escape mode M310, a normal mode M320, and a landing mode M340.

[0130] Takeoff mode M300 is a mode in which the drone 100 takes off. The state transition of the aircraft's behavioral state starts from takeoff mode M300. When the drone 100 takes off, the aircraft's behavioral state transitions from takeoff mode M300 to evacuation mode M310 or normal mode M320. Evacuation mode M310 mainly includes an evacuation point arrival stationary mode M311 and an evacuation moving mode M312. Furthermore, normal mode M320 includes a point arrival stationary mode M321 and a moving mode M322. Evacuation mode M310 and normal mode M320 can transition to each other via temporary suspension mode M330. Note that this is just one example.

[0131] When transitioning from takeoff mode M300 to evacuation mode M310, the drone 100 in takeoff mode M300 transitions to evacuation point arrival stationary mode M311. The evacuation point arrival stationary mode M311 is a mode in which the drone moves to the evacuation point H200 and remains stationary there, i.e., hovers. When moving from the evacuation point H200 to another location for the purpose of evacuation, the drone 100 in the evacuation point arrival stationary mode M311 transitions to evacuation moving mode M312. When the drone 100 reaches a predetermined destination or there is no movement command due to manual intervention, the aircraft behavior state transitions from the evacuation point arrival stationary mode M311 or the evacuation moving mode M312 to the temporary suspension mode M330.

[0132] Furthermore, when transitioning from takeoff mode M300 to normal mode M320, the drone 100 in takeoff mode M300 transitions to point arrival stationary mode M321. Point arrival stationary mode M321 is a mode in which the drone moves to a predetermined destination and remains stationary, i.e., hovers. When moving to another location in normal use, the drone 100 in point arrival stationary mode M321 transitions to moving mode M322. When the drone 100 reaches the predetermined destination or no movement command is issued by manual intervention, the aircraft behavior state transitions from point arrival stationary mode M321 or moving mode M322 to pause mode M330.

[0133] The drone 100 in the evacuation point arrival stationary mode M311, the point arrival stationary mode M321, the moving mode M322, and the temporary suspension mode M330 can transition to the landing mode M340. The aircraft's behavior state ends processing in the landing mode M340.

[0134] The game status acquisition unit 323 shown in FIG. 6 is a functional unit that acquires the game status of a competition held at stadium F. The game status acquisition unit 323 detects the game status by processing captured images. The game status acquisition unit 323 may also acquire the game status based on decision-related information input by a referee to the external input device 600 or a referee support system, which is an example of the external system 700. Furthermore, the game status acquisition unit 323 may acquire the game status based on information input from the external input device 600 held by a team member, for example, a manager or coach.

[0135] FIG. 11 is a diagram showing an example of a state transition of a game state. This figure shows an example of a soccer game state. The game state includes a pre-game state M400, a normal play state M410, and an end-of-game state M460. The state transition starts from the pre-game state M400, and transitions from the pre-game state M400 to the normal play state M410. The normal play state M410 is a state in which the game is progressing. When the game ends, transition occurs from the normal play state M410 to the end-of-game state M460. Note that a transition from the normal play state M410 to the end-of-game state M460 may occur not only at the end of the game, but also during a break during the game, such as halftime.

[0136] The game state also includes a play suspended without a foul play state M420 and a play suspended with a foul play state M440. When play is suspended without a foul play in the normal play state M410, the state transitions to the play suspended without a foul play state M420. The play suspended without a foul play state M420 is entered, for example, when the ball crosses the goal line F110a, F110b or the touchline F111a, F111b and goes out of the court. The play suspended without a foul play state M420 transitions to a throw-in state M421, a goal kick state M422, or a corner kick state M423 in accordance with events that occur according to the rules of the game, such as the type of line the ball crossed or the affiliation of the player who sent the ball out of the court. The throw-in state M421, the goal kick state M422, or the corner kick state M423 transitions to the normal play state M410 when the event in each state ends.

[0137] In the normal play state M410, if a foul occurs, or more precisely, if the referee recognizes the occurrence of a foul, the state transitions to the foul state M431. Furthermore, if an offside occurs or is recognized by the referee, the state transitions to the offside state M432. Then, from the foul state M431 or the offside state M432, the state transitions to the foul play suspended state M440. In the foul play suspended state M440, the state transitions to the free kick state M441 or the penalty kick state M442 depending on the location of the foul and the event that occurred. In the free kick state M441, a so-called indirect free kick may be taken instead of a free kick. The free kick state M441 may also be subdivided into a free kick state for the attacking team and a free kick state for the defending team. In the free kick state M441 and the penalty kick state M442, once the events in each state are completed, the game resumes and the game state transitions to the normal play state M410.

[0138] When the match ends, the normal play state M410 transitions to an after-match state M460, and the state transition of the match state ends.

[0139] Furthermore, the normal play state M410 can transition to a penalty shootout state M443. Although not shown in the drawings, the penalty shootout state M443 may transition to a state M460 after the match ends, thereby ending the state transition.

[0140] 11 , some game states may trigger flight mode switching, while other game states may not. For example, flight mode switching may be performed based on transitions to the shaded states in the figure, i.e., the pre-game state M400, goal kick state M422, corner kick state M423, free kick state M441, penalty kick state M442, player substitution state M450, and end-of-game state M460. Note that when transitioning from another game state to the normal play state M410, the flight mode may be switched to one corresponding to the offensive / defensive state.

[0141] The offensive and defensive status acquisition unit 324 is a functional unit that acquires the offensive and defensive status of the teams in the match held at the stadium F. The offensive and defensive status acquisition unit 324 detects the offensive and defensive status by processing the captured images. The offensive and defensive status acquisition unit 324 may also acquire the offensive and defensive status based on decision-related information input by the umpire to the external input device 600 or a referee support system, which is an example of the external system 700. Furthermore, the offensive and defensive status acquisition unit 324 may acquire the offensive and defensive status based on information input from the external input device 600 held by a team member, for example, a manager or coach.

[0142] 12 is a diagram showing an example of a state transition between an offensive and defensive state. The figure shows an example of an offensive and defensive state in soccer. When a match starts, the offensive and defensive state transitions to an offensive state M510 or a defensive state M520. Furthermore, the offensive state M510 and the defensive state M520 transition to each other via an offensive / defensive change state M530 or an offensive / defensive uncertainty state M540.

[0143] The offensive state M510 is a state in which one pre-designated team (hereinafter also referred to as "Team A") is on the offensive. For example, the offensive state is a state in which Team A is in possession of the ball and advancing toward the other team (hereinafter also referred to as "Team B"), but is not limited to this and may be a predetermined state determined by any determination criterion stored in advance in the storage unit 380.

[0144] The offensive state M510 includes a team A offensive (own side) state M511, a team A offensive (opponent's side) state M512, and a team A quick attack state M513. The team A offensive (own side) state M511 and the team A offensive (opponent's side) state M512, and the team A offensive (own side) state M511 and the team A quick attack state M513 can transition to each other. Also, the team A quick attack state M513 can transition to the team A offensive (opponent's side) state M512.

[0145] The defensive state M520 includes a team A defensive (opponent's territory) state M521, a team A defensive (owner's territory) state M522, and a team B fast attack state M523. The team A defensive (owner's territory) state M521 and the team A defensive (owner's territory) state M522, and the team A defensive (opponent's territory) state M521 and the team B fast attack state M523 can transition to each other. In addition, the team B fast attack state M523 can transition to the team A defensive (owner's territory) state M522.

[0146] The offense / defense change state M530 and the offense / defense uncertain state M540 can be transitioned to from any of the following states: Team A offensive (home side) state M511, Team A offensive (opponent's side) state M512, Team A quick attack state M513, Team A defensive (opponent's side) state M521, Team A defensive (home side) state M522, and Team B quick attack state M523.

[0147] The offensive and defensive state acquisition unit 324 detects a transition to a fast attack state M513 or M523 when the offensive and defensive state is switched M530 or the offensive and defensive state is uncertain M540. The offensive and defensive state acquisition unit 324 analyzes, for example, the acceleration of the ball or players, fluctuations in the direction of ball movement or player orientation, the number of players in a specified area, the direction of player movement, the number of players moving in a certain direction, etc. from images acquired by the photographing camera 141. The offensive and defensive state acquisition unit 324 detects a fast attack state M513 or M523 based on the results of this analysis. The offensive and defensive state acquisition unit 324 also determines whether the team is in an A fast attack state M513 or a B fast attack state M523 based on the direction of movement of the players or the ball.

[0148] The offensive and defensive states are not limited to those described above, and any state that triggers a change in the shooting conditions may be defined. For example, a state that transitions upon detecting a long pass may be defined. Furthermore, the offensive and defensive states may be defined appropriately depending on the content of the game or event being shot.

[0149] Instead of or in addition to the above-described acquisition units 321 to 324, the event detection unit 320 may determine an event based on input information from the external system 700. For example, the event detection unit 320 may determine a disturbance such as a strong wind as an event based on input information from a weather information system, which is an example of the external system 700. The event detection unit 320 may also determine an event based on input information from a court facility system, which is another example of the external system 700, or facility information input by a person involved with the court facility.

[0150] (A-1-4-4. Photographing condition determination unit 325) The photographing condition determination unit 325 is a functional unit that determines the photographing conditions to be set in the photographing camera 141 of the drone 100. The photographing condition determination unit 325 determines the photographing conditions according to an event detected by the event detection unit 320. The photographing conditions include at least one of a target photographing position and a target photographing direction of the drone 100. The target photographing direction includes, for example, information on either or both of a pitch angle with respect to the horizontal and a yaw angle with respect to a predetermined reference direction. Furthermore, the target photographing direction may include information on a target zoom amount of the photographing camera 141. In the following explanation, the photographing direction will be described as including the pitch angle, yaw angle, and zoom amount.

[0151] The shooting conditions may also include information about the shooting range of the shooting camera 141. Note that the technical scope of the present invention is not limited to a mode in which both a target shooting position and a target shooting direction are set automatically as long as at least one of the target shooting position and the target shooting direction is set automatically depending on the event.

[0152] The target shooting direction is achieved by controlling at least one of the nose direction of the drone 100 and the shooting direction of the shooting camera 141. The nose direction of the drone 100 is controlled by the flight control unit 123 of the drone 100. The shooting direction of the shooting camera 141 is controlled, for example, by the shooting control unit 143 driving the camera holding unit 142. Note that the above-mentioned "control of the nose direction" and "control of the shooting direction" are concepts that include control not only in the left-right direction (so-called "pan direction") but also in the up-down direction (so-called "tilt direction").

[0153] The photographing condition determination unit 325 determines photographing conditions according to the type of event detected. In the normal play state M410, the photographing condition determination unit 325 permits manual control via the controller 200. When an event is detected by the event detection unit 320, the photographing condition determination unit 325 refers to the event-photography condition table T1 (see FIG. 13 ) stored in the memory unit 380 and determines photographing conditions according to the event.

[0154] As shown in FIG. 13 , the event-photography condition table T1 is a table in which events detected as game states and the photography conditions selected for those events are stored in association with each other. More specifically, the event-photography condition table T1 associates events with the photography range, the photography position where the drone 100 is located, the photography direction of the photography camera 141, and the zoom amount of the photography camera 141. For example, in the penalty kick state M442 or the penalty shootout state M443, the photography range is the penalty area F130a or F130b where the ball is located, and the photography position is the photography position L206 or L211. The photography direction is toward the goal F120a or F120b where the ball is located. The zoom amount is predefined in stages, for example, IN, Middle, and OUT, in descending order of zoom amount, and is IN in the penalty kick state M442 or the penalty shootout state M443.

[0155] In the goal kick state M422 and the defending free kick state M441, the shooting position is one of the shooting positions L101, L102, and L104 in the outer flight mode M102. This configuration reduces the risk of the ball colliding with the drone 100. In the attacking free kick state M441, the penalty kick state M442, and the penalty shootout state M443, shooting from one of the shooting positions L206 to L211 within the court F100 allows for a more visible position to capture the goal scene. In the foul state M431, shooting from the shooting positions L101, L102, or L104 along the outer edge allows for the entire court F100 to be captured. Furthermore, in the foul state M431, the shooting direction is set so that the shooting range is around the ball or the referee's position. This configuration allows for reliable capture of the game immediately after a foul, such as the referee's decision and the players' movements. In the corner kick state M423, the area in front of the goal can be photographed closely by photographing at the photographing position L207, L209, L212 or L215.

[0156] In sports, the ideal shooting direction required for different events occurring during a match, such as free kicks, fast breaks, and corner kicks, varies even for the same shooting position. When shooting sports where the situation changes rapidly, it is difficult to quickly and accurately achieve shooting from the ideal shooting direction through manual control. Operational errors or delays can result in missing important scenes. Furthermore, manual control requires the deployment of multiple camera operators. However, since the required shooting direction for each event is somewhat predetermined, the above-described configuration, which automatically controls the drone 100 to a shooting position and direction appropriate for the event, allows for appropriate shooting according to the game situation. Furthermore, it can reduce the number of camera operators, contributing to labor savings.

[0157] Furthermore, in the case of an event in which a plurality of photographing positions are stored, the photographing condition determination unit 325 may select the photographing position that is closest to the ball from among the stored photographing positions.

[0158] The event-photography condition table T1 may store different photography conditions set for the multiple drones 100. For example, in the foul play state M431, the first drone 100 may photograph the ball location with a large zoom amount, and the second drone 100 may photograph the ball location with a small zoom amount. The first drone 100 and the second drone 100 may also photograph while yaw rotating in opposite directions. Furthermore, the first drone 100 may photograph from the side, and the second drone 100 may photograph from directly above. While manually controlling multiple drones 100 to achieve appropriate photography conditions is even more difficult, the above-described automatic control configuration enables multiple drones 100 to quickly photograph from multiple angles.

[0159] Shooting with a zoom amount of "OUT" is what is known as bird's-eye view shooting. In the event-shooting condition table T1, events designated for bird's-eye view shooting may be associated with opposite shooting directions. That is, for example, a shooting direction from Team A's court toward Team B's court and a shooting direction from Team B's court toward Team A's court are associated with one event. The offensive / defensive status acquisition unit 324 detects which team is in possession of the ball, and the shooting condition determination unit 325 determines the shooting direction according to the team in possession of the ball. Specifically, the shooting condition determination unit 325 determines that the shooting direction should be the attacking direction of the team in possession of the ball. This configuration allows continuous bird's-eye view shooting of the moving ball.

[0160] The event-photography condition table T1 may also be configured to accept user correction input, with the changes stored in the storage unit 380. With this configuration, the user's photography knowledge is reflected in the event-photography condition table T1, enabling more optimal automatic photography. Multiple types of event-photography condition tables T1 may also be stored in the storage unit 380, allowing the user to select the table to apply. This is because, for example, the content to be captured differs depending on whether the purpose of photography is watching sports or coaching athletes. With this configuration, automatic photography suited to the purpose can be easily achieved. The event-photography condition table T1 shown in FIG. 13 is an example of a table set for coaching purposes, but it is merely an example, and the specific photography conditions stored in the table are arbitrary.

[0161] In the above embodiment, both the shooting positions L101 to L215 and the shooting direction are automatically set in accordance with the detected event, but instead, only the shooting positions L101 to L215 may be automatically set. In this case, the shooting direction is determined by input via the controller 200. Furthermore, when the user selects a shooting position L101 to L215 via the controller 200, the shooting condition determination unit 325 may be configured to determine the shooting direction in accordance with the detected event and the selected shooting position L101 to L215.

[0162] The photographing condition determination unit 325 may determine photographing conditions based on input to the controller 200 when the event detection unit 320 has not detected an event, and may determine photographing conditions based on the event when the event detection unit 320 has detected an event. With this configuration, even if there is a user input, photographing can be reliably performed under the specified photographing conditions when an event is detected, thereby maintaining appropriate photographing conditions for important photographic scenes. Furthermore, when no event is detected, the user's freedom of photographing can be ensured, achieving both the convenience of automatic control and the flexibility of manual control.

[0163] Furthermore, when the shooting condition determination unit 325 receives an operation from the controller 200 or the external system 700, it may determine the shooting conditions based on the operation received via the controller 200 or the external system 700, even if the event detection unit 320 has detected an event. That is, the shooting conditions input from the controller 200 are applied with priority over the shooting conditions associated with the event. With this configuration, the user only needs to focus on operations that cannot be handled by automatic control, thereby reducing the operational burden and reducing operational errors compared to a configuration in which all operations are manually controlled. That is, with this configuration, it is possible to achieve both convenience and freedom, i.e., maintaining the convenience of automatic shooting while ensuring the freedom of shooting according to the user's appropriate requests.

[0164] If the event detection unit 320 detects an event, the photographing condition determination unit 325 determines the photographing conditions based on the event, and if the event detection unit 320 does not detect an event, the photographing condition determination unit 325 may automatically follow the ball and photograph it.

[0165] When the system 1 includes multiple drones 100, the imaging condition determination unit 325 determines different imaging conditions for each of the multiple drones 100. The multiple drones 100 may take images from different imaging positions and directions. For example, one drone 100 may take an image of a player taking a shot, while the other drone 100 may take an image of the opposing team's goalkeeper.

[0166] The photographing condition determination unit 325 may also set photographing conditions for multiple drones 100 flying simultaneously such that the same photographing range is photographed from different target photographing positions. This configuration allows important scenes to be photographed from multiple angles. The photographing condition determination unit 325 may also set photographing conditions for multiple drones 100 flying simultaneously such that an area including the same photographing range is photographed with different zoom amounts. This configuration makes it possible to photograph areas of particular interest in the stadium F under multiple photographing conditions, thereby more reliably photographing important scenes.

[0167] The photographing condition determination unit 325 may analyze the photographed image, predict the photographing range to be photographed based on the analysis results, and determine the photographing conditions. For example, the photographing condition determination unit 325 may predict the moving distance of the ball after a predetermined time by analyzing the moving direction and speed or acceleration of the ball from the photographed image, and determine the photographing conditions such that the position of the ball after the predetermined time will be within the photographing range. Note that the speed of the ball may be determined based on the speed at the start of the prediction, i.e., the initial speed.

[0168] The photographing condition determination unit 325 may determine photographing conditions according to the ball trajectory predicted by the event detection unit 320 as an event detection result. The photographing condition determination unit 325 may predict the ball's trajectory, triggered by the detection of, for example, a fast attack state M513, M523, or a long pass, and determine photographing conditions according to the predicted trajectory. The photographing condition determination unit 325 may change the photographing direction toward the ball's traveling direction and determine a zoom-out magnification. Zooming out allows for a more reliable photograph of the traveling ball. Note that the ball trajectory prediction is not limited to a mode in which the event detection unit 320 detects an event when a predetermined trajectory is predicted. That is, for example, the photographing condition determination unit 325 may predict the ball's trajectory using the photographing condition determination unit 325 or another functional unit separately from the event detection unit 320, and determine photographing conditions based solely on the results of this trajectory prediction.

[0169] Furthermore, particularly when changing the shooting direction in response to a ball trajectory prediction, it is desirable to control the shooting direction of the shooting camera 141 under control of the shooting control unit 143 instead of controlling the nose direction. This is because changing the shooting direction based on a ball trajectory prediction requires changing the shooting direction with high response speed. That is, for example, when changing the shooting direction based on a ball trajectory prediction, the shooting direction of the shooting camera 141 may be changed by the shooting control unit 143, and when changing the shooting direction not based on a trajectory prediction, the shooting direction may be changed by controlling the nose direction of the drone 100.

[0170] Furthermore, when a change of offense and defense or a quick break is detected, the photographing condition determination unit 325 may determine to change the photographing direction to the direction of the ball's movement and to zoom out the photographing magnification.

[0171] (A-1-4-5. Flight mode switching unit 330) The flight mode switching unit 330 is a functional unit that switches flight modes in accordance with the detection result by the event detection unit 320. The flight mode switching unit 330 mainly has a mode switching input acquisition unit 331, a flight-permitted area switching unit 332, a geofence switching unit 333, and a flight path generation unit 334.

[0172] The mode switching input acquisition unit 331 is a functional unit that acquires input information related to switching flight modes. The flight modes are, for example, the outer edge flight mode M102, the fixed position flight mode M103 or M107, and the inside-court flight mode M105 (see FIG. 8 for all of these). Whether the off-court fixed position flight mode M103 or the inside-court fixed position flight mode M107 is selected for fixed position flight is determined based on the position of the drone 100 at the time the flight mode selection input is received. That is, when the drone 100 is in the off-court area F200, the off-court fixed position flight mode M103 is selected, and when the drone 100 is inside the court F100, the inside-court fixed position flight mode M107 is selected.

[0173] The flight permission area switching unit 332 is a functional unit that switches the flight permission area in response to switching of the flight mode.

[0174] The geofence switching unit 333 is a functional unit that switches geofences in response to flight mode switching. For example, when the flight mode is the edge flight mode M102, the geofence switching unit 333 sets a geofence G100. Furthermore, when the flight mode is the inside-court flight mode M105, the geofence switching unit 333 sets a geofence G200. Furthermore, in the intermediate modes that intervene in the transition between the edge flight mode M102 and the inside-court flight mode M105, i.e., the inside-court entry mode M104 and the outside-court exit mode M106, a third geofence different from the geofences G100 and G200 in the edge flight mode M102 and the inside-court flight mode M105 is set.

[0175] The flight path generation unit 334 is a functional unit that generates a flight path of the drone 100 when moving with a flight mode switch. The flight path generation unit 334 determines, for example, a shooting position at which the drone transitions to the in-court entry mode M104 or the out-of-court exit mode M106. The flight path generation unit 334 also determines a shooting position at which the drone transitions from the in-court entry mode M104 to the in-court flight mode M105, or a shooting position at which the drone transitions from the out-of-court exit mode M106 to the outer edge flight mode M102. The flight path generation unit 334 also generates a specific flight path for the in-court entry mode M104 or the out-of-court exit mode M106. This flight path is generally generated within the area of ​​the third geofence.

[0176] (A-1-4-6. Outer edge flight control unit 340) The outer edge flight control unit 340 is a functional unit that controls the flight of the drone 100 in outer edge flight mode. The outer edge flight control unit 340 has an imaging condition command unit 341 and a flight path generation unit 342.

[0177] The photographing condition command unit 341 is a functional unit that transmits commands related to photographing conditions to the drone 100. Photographing conditions are, for example, a target photographing position or a photographing direction. The photographing condition command unit 341 acquires a target photographing position located within the flyable area in the outer flight mode M102 from the photographing condition determination unit 325. The photographing condition command unit 341 may also acquire a target photographing position input by the user, for example, received via the target position receiving unit 226 of the controller 200.

[0178] The flight path generation unit 342 is a functional unit that generates a flight path along which the drone 100 moves in the outer flight mode M102. In other words, the flight path generation unit 342 generates a flight path within the flight area available in the outer flight mode M102. When the drone 100 belongs to the flight area available in the outer flight mode M102 and the acquired target position also belongs to the flight area available in the outer flight mode M102, the flight path generation unit 342 generates a flight path from the current location to the target position. Furthermore, when the drone 100 moves across the outer flight mode M102 and the on-court flight mode M105, the flight path generation unit 342 may generate a flight path within the flight range in the outer flight mode M102.

[0179] When the ball crosses the touchline F111b where the drone 100 is flying and enters the off-court area F200, the flight path generation unit 342 moves the drone 100 to the off-court area F200, outside the touchline F111b. The flight path generation unit 342 may also cause the drone 100 to capture images directly below the drone 100 or from that point toward the court F100. This configuration enables tracking and capturing images of the ball even when the ball rolls into the off-court area F200. The geofence G100 for the outer flight mode M102 is preferably set in advance to extend beyond the touchline F111b to the outside of the court F100. This configuration ensures that the drone 100 remains within the geofence G100, even when the drone 100 follows the ball and flies slightly outside the touchline F111b, as described above.

[0180] If the flight path generation unit 342 detects an obstacle on the flight path or near the drone 100, it regenerates a flight path that bypasses the obstacle toward the inside of the court F100. Furthermore, if the flight path generation unit 342 detects an obstacle, it may hover for a predetermined period of time and then move along the originally generated flight path. This is because, while the safety of the drone 100 is not ensured in the off-court area F200 of the stadium F, the safety of the drone 100 within the court F100 is highly likely to be ensured. Obstacles may be detected, for example, by the obstacle detection unit 130 of the drone 100 or by information from an external system 700, etc. Furthermore, if an obstacle is detected, the outer edge flight control unit 340 may switch to manual control after having the flight path generation unit 342 hover for a predetermined period of time. Furthermore, when an obstacle is detected, the outer edge flight control unit 340 may cause the flight path generation unit 342 to hover for a predetermined period of time, and then display a message via the display control unit 210 prompting the user to re-input the target position.

[0181] (A-1-4-7. Intra-court flight control unit 350) The intra-court flight control unit 350 is a functional unit that controls the flight of the drone 100 in intra-court flight mode M105. The intra-court flight control unit 350 has an imaging condition command unit 351 and a flight path generation unit 352. The imaging condition command unit 351 is a functional unit that transmits commands to the drone 100 regarding imaging conditions within the range of the flyable area in intra-court flight mode M105. The imaging conditions are, for example, a target imaging position or imaging direction.

[0182] The flight path generation unit 352 generates a flight path for the drone 100 to travel in the intra-court flight mode M105. That is, the flight path generation unit 352 generates a flight path within the flyable area in the intra-court flight mode M105. More specifically, the flight path generation unit 352 generates a flight path in the intra-court flight mode M105 by connecting multiple preset shooting positions. Like the flight path generation unit 342 of the outer edge flight control unit 340, the flight path generation unit 352 may generate a flight path within the flight range in the intra-court flight mode M105 when the current location and the target shooting position belong to flyable areas of different flight modes.

[0183] Furthermore, the flight path generation unit 352 changes the connected image capture positions depending on the event detection status. That is, when an event is detected, the flight path generation unit 352 changes the connection relationship between the image capture positions on the flight path and generates a flight path to the target image capture position.

[0184] If the flight path generation unit 352 detects an obstacle on the flight path or near the drone 100, it regenerates a flight path that bypasses the obstacle. The obstacle is detected, for example, by the obstacle detection unit 130. The flight path generation unit 352 may regenerate the flight path by changing the connection between multiple preset shooting positions, or may change the flight path to a higher altitude while maintaining the flight path on a plane. Furthermore, if the flight path generation unit 352 detects an obstacle, it may hover for a predetermined period of time and then resume movement along the originally generated flight path. Furthermore, if the intra-court flight control unit 350 detects an obstacle, it may cause the flight path generation unit 352 to hover for a predetermined period of time and then switch to manual control. Furthermore, if the intra-court flight control unit 350 detects an obstacle, it may cause the flight path generation unit 352 to hover for a predetermined period of time and then display a message via the display control unit 210 prompting the user to re-input the target position. The obstacle may be, for example, a bird, fixed equipment, or a player. The obstacles also include balls.

[0185] In this embodiment, flight control in the outer flight mode M102 is performed by the outer flight control unit 340, and flight control in the inner court flight mode M105 is performed by the inner court flight control unit 350. The specified shooting positions for each of the outer flight mode M102 and inner court flight mode M105 are presented as options, and a flight path to the selected target position is generated. However, the technical scope of the present invention is not limited to this. The pilot 200 may control the shooting position and orientation of the drone 100 to fly freely within any location within the geofences G100 and G200 set for each flight mode.

[0186] Furthermore, the functional configuration of the flight path generation units 334, 342, and 352 is an example, and for example, a single flight path generation unit may generate the flight path without subdivision.

[0187] 14 is a diagram schematically illustrating the routes that the drone 100 can take, defined by the image capture positions L101 to L105, L206 to L215, and the evacuation point H200. Point L101g on the ground at image capture position L101 is the takeoff and landing point for the drone 100. The position transition of the drone 100 begins with the step of taking off from point L101g and arriving at image capture position L101. The drone 100 also descends at image capture position L101 and lands at point L101g, completing image capture.

[0188] At each of the image capture positions L101 to L105 and L206 to L215, the drone 100 may only be able to transition to adjacent image capture positions. For example, the point to which the drone 100 at image capture position L105 can transition while maintaining the outer edge flight mode is image capture position L104. Furthermore, the points to which the drone 100 at image capture position L105 can transition after switching to the inside court flight mode M105 are image capture positions L106 and L107. The flight path generation unit 334, 342, or 352 (see FIG. 6) generates a flight path for the drone 100 by referencing the transitionable path.

[0189] When the target position receiving unit 226 receives the selection of a shooting position, the drone 100 transitions to the selected shooting position via the available shooting positions. For example, when the drone 100 is at shooting position L105 and shooting position L215 is selected, the flight path generating unit 334, 342, or 352 (see FIG. 6 ) generates a flight path that transitions through shooting positions L105, L207, L208, L213, L212, and L215 in this order, and the drone 100 flies along this flight path.

[0190] Note that drone 100 may fly a flight path that connects the current location to the target shooting location in a straight line, instead of a flight path generated by connecting shooting locations L101 to L215. Furthermore, drone 100 may transition to an adjacent shooting location when a transition of shooting locations involves a flight mode switch, and may transition directly to a non-adjacent shooting location when a transition does not involve a flight mode switch. For example, when the drone moves from shooting location L105 to shooting location L215, it may transition from shooting location L105 to shooting location L207 with a mode switch, and then move linearly within the area within geofence G200 from shooting location L107 to shooting location L215.

[0191] The outer edge flight control unit 340 and the inner court flight control unit 350 autonomously fly the drone 100 in each flight area according to the flight mode. For example, the outer edge flight control unit 340 and the inner court flight control unit 350 may perform dolly shooting within each flight area, i.e., the drone 100 may automatically track and shoot a specific object such as a ball or a designated player. The outer edge flight control unit 340 and the inner court flight control unit 350 may automatically control the flight height of the drone 100. The autonomous flight mode may differ depending on the flight mode. For example, dolly shooting may be performed when controlled by the outer edge flight control unit 340, while automatic tracking shooting of only the shooting direction with a fixed shooting position may be performed when controlled by the inner court flight control unit 350, or automatic tracking shooting of both the position and the shooting direction may be performed. The outer edge flight control unit 340 and the inner court flight control unit 350 may also generate a flight path within the court (stadium F) to move the drone 100 to a target position specified by the user in each flight area.

[0192] (A-1-4-8. Fixed position flight control unit 360) The fixed position flight control unit 360 is a functional unit that controls the flight of the drone 100 in the off-court fixed position flight mode M103 and the on-court fixed position flight mode M107. In the fixed position flight mode, the fixed position flight control unit 360 hovers at a predetermined position and controls the direction of the nose or the direction of the shooting camera 141 to follow a specific player or the ball and perform automatic shooting. Note that the above-mentioned "control of direction" is a concept that includes control not only in the left-right direction (the so-called "pan direction") but also in the up-down direction (the so-called "tilt direction").

[0193] The fixed position flight control unit 360 includes an image capture condition command unit 361. The image capture condition command unit 361 is a functional unit that transmits commands for a target position and a target image capture direction in the fixed position flight mode M103 or M107. The image capture direction may be information determined by the image capture condition determination unit 325, or may be information input by the user via the controller 200.

[0194] (A-1-4-9. Communication unit 370) The communication unit 370 has a modem or the like (not shown) and is capable of communicating with the drone 100, the controller 200, etc. via the communication network 400. The communication unit 370 may, for example, monitor the state of the drone 100 and its surroundings and notify the controller 200.

[0195] (A-1-4-10. Memory unit 380) The memory unit 380 is a functional unit that stores information related to flight control of the drone 100, and is, for example, a database. The memory unit 380 stores, for example, the coordinates of multiple shooting positions L101 to L105, L210 to L215 in the stadium F. These coordinates may be two-dimensional coordinates on a plane or three-dimensional coordinates that include information in the height direction. The memory unit 380 also stores the event-shooting condition table T1 shown in FIG. 13. As described above, the event-shooting condition table T1 is recorded in a rewritable manner. Furthermore, multiple event-shooting condition tables T1 may be stored.

[0196] Flowcharts Fig. 15 is a flowchart showing the overall flow of aerial photography control in this embodiment. Fig. 16 is a subroutine of the flight restriction process S1002 in Fig. 15. Fig. 17 is a subroutine of the photography condition switching process S1010 in Fig. 15.

[0197] The control shown in the flowchart of Fig. 15 is executed in a periodic loop. As shown in Fig. 15, if it is detected that the drone 100 is approaching the vicinity of the geofences G100 and G200 while flying (YES in step S1001), the process proceeds to flight restriction processing in step S1002. The subroutine of the flight restriction processing S1002 will be described with reference to Fig. 15.

[0198] If it is not detected in step S1001 that the drone is approaching the vicinity of the geofences G100 and G200 (NO in step S1001), it is detected whether or not there is an obstacle in the path or near the drone 100 (step S1003). If an obstacle is detected in step S1003 (YES in step S1003), the drone 100 is caused to hover or a detour route is generated, and the flight route of the drone 100 is changed to the detour route (step S1004).

[0199] If no obstacle is detected in step S1003 (NO in step S1003), it is detected whether or not an action determination has been made on the aircraft state (step S1005). If an action determination on the aircraft state has been made in step S1005 (YES in step S1005), the process proceeds to step S1006, where an event type determination process is executed (step S1006).

[0200] If no action is detected in step S1005 (NO in step S1005), it is determined whether or not there is an input from the controller 200 by the user (step S1007). If an input from the controller 200 is detected in step S1007 (YES in step S1007), a command based on the input is executed (step S1008).

[0201] If no input from the controller 200 is detected in step S1007 (NO in step S1007), it is determined whether or not an event has occurred (step S1009). If an event has been detected (YES in step S1009), the process proceeds to step S1010, where the image capturing condition is switched. If no event has been detected in step S1009 (NO in step S1009), the process returns to step S1001, and steps S1001 to S1009 are repeated.

[0202] As shown in Figure 15, the overall processing of aerial photography control is performed in the following order: geofence restriction, obstacle detection, control based on aircraft status, control based on user input, and control based on in-game events such as the game status or offensive / defensive status. In other words, each control is executed before control based on in-game events. This order is determined by the priority of performing safe control processing. This configuration can more reliably ensure the safety of the drone 100 flight.

[0203] 16, first, the flight control unit 123 of the drone 100 issues an operation command to restrict the drone 100 from advancing outside the geofence (step S1101). In step S1101, a restriction is set on the flight target position so that the drone 100 does not advance outside the geofence even when the drone 100 is manually operated.

[0204] Next, if the drone 100 does not advance outside the geofence (NO in step S1102), the processing ends.

[0205] If the drone 100 advances outside the geofence (YES in step S1102), the process proceeds to step S1103. Possible causes of this situation include, for example, the wind being too strong and causing the drone to be swept away, or a malfunction of the drone 100 preventing it from flying in the intended direction. In step S1103, the flight control unit 123 issues an operation command to return to the geofence. More specifically, in step S1103, a flight target position command to return to the geofence, i.e., an operation command to set a predetermined point inside the geofence as the flight target position, is issued to the drone 100.

[0206] Next, the information measured by the measurement unit 110 of the drone 100, such as information on position, orientation, altitude, or speed, is referenced to determine whether the drone 100 is approaching the inside of the geofence (step S1104). In this case, step S1104 is executed a predetermined time after step S1103. Note that in step S1104, it is sufficient if the drone 100 is closer to the inside of the geofence than at the time of step S1103, and it is not necessary to determine whether the drone 100 is located inside the geofence.

[0207] In step S1104, if it is determined that the drone 100 is not approaching the geofence (NO in step S1104), it is determined that the above operation command is ineffective, and the flight control unit 123 forces the drone 100 to land (step S1105).

[0208] If the drone 100 is approaching the geofence in step S1104 (YES in step S1104), the process proceeds to step S1106. In step S1106, information measured by the measurement unit 110 of the drone 100, such as the position and altitude, is referenced to determine whether the drone 100 is located within the geofence. If the drone 100 is located within the geofence (YES in S1106), the process ends. If the drone 100 is not located within the geofence (NO in S1106), the process returns to step 1104, and the drone 100 continues operating based on the operation command to return to the geofence until it returns to the geofence.

[0209] FIG. 17 shows an example of a switching process flow for switching shooting conditions. First, when a predetermined event is detected by the event detection unit 320, the shooting condition determination unit 325 determines target values ​​for the shooting position and shooting direction by referencing the event-shooting condition table T1 and transmits a control command to the drone 100 (step S1301). Next, it is determined whether an operation command due to manual intervention by the user has been received from the controller 200 (step S1302). If an operation command has not been received from the controller 200 (NO in step S1302), the drone continues flying under automatic control until it reaches the target shooting position and target shooting direction (step S1303). If an operation command has been received from the controller 200, the drone executes an operation based on the operation command due to manual intervention (step S1304).

[0210] When the target shooting position and the target shooting direction are reached in step S1303, the process proceeds to step S1305. In step S1305, the process transitions to manual control mode, and a message indicating that manual operation is permitted is displayed on the operation screen G3 (see FIG. 20).

[0211] Display Examples of the Display Unit 201 FIGS. 18 and 19 are examples of screens G1 and G2 displayed on the display unit 201 of the controller 200. FIG.

[0212] The screen G1 shown in Figure 18 displays a field map G10 that shows a bird's-eye view of the stadium F and shooting positions L101 to L215, an icon G11 indicating the position information of the drone 100, a shooting range G12 captured by the shooting camera 141, a display field G21 displaying the flight mode to which the drone 100 belongs, a status display field G22 showing the status detected by the event detection unit 320, such as the aircraft status, aircraft behavior status, game status, and offensive and defensive status, a landing button G30 for landing the drone 100, and a video field G40 displaying images captured by the drone 100. In the example of Figure 18, the display field G21 displays the control mode broadly as either automatic control mode or manual control mode. The drone is flying in the outer edge flight mode at shooting position L101.

[0213] The position and shooting direction of the drone 100 may be controlled manually, or automatic tracking control of the ball or a specific player may be performed. When automatic tracking control is performed, information about the ball or specific player to be tracked may be displayed on the screen G1.

[0214] The icon G11 representing the drone 100 displays an arrow indicating the traveling direction of the drone 100. Note that the nose direction of the drone 100 is not limited to the traveling direction of the drone 100, and may be pointing in any direction. The nose direction of the drone 100 does not have to be constant while moving, and for example, the drone 100 may move while photographing a player or the ball by yaw rotation.

[0215] Figure 19 shows an example of the display on screen G2 when multiple drones 100 are photographing one stadium F. Figure 19 particularly shows an example of the display when a free kick state M441 is detected. On screen G2, icons 11a and 11b of two drones are displayed on a field map G10. In addition, the photographing ranges G13a and G13b photographed by each of the multiple drones 100 and video fields G40a and G40b showing the photographed images are displayed in association with the icons 11a and 11b of the corresponding drones 100.

[0216] In the example of Figure 19, the first drone 100 corresponding to icon 11a is taking localized shots of the area near the goal 120a, while the second drone 100 corresponding to icon 11b is taking bird's-eye shots from a shooting position L101 on the outer edge. The shooting angles of the first drone and the second drone are different from each other. It is preferable that the shooting conditions of the first drone and the second drone are such that they complement each other's positions where they cannot shoot. In this way, a configuration in which multiple drones 100 take shots under different shooting conditions allows the stadium F to be photographed from multiple angles.

[0217] 20 is an example of a screen G3 displayed on the display unit 201 when an event has not been detected by the event detection unit 320. Because an event has not been detected, automatic control of the drone 100 is not performed, and the display field G21 indicates that the mode is "manual control mode."

[0218] 21 is an example of screen G4 when the event detection unit 320 detects an event in the state of screen G3. In this case, the drone 100 switches to automatic control in response to the detection of the event, and the display field G21 displays that it is in "automatic control mode." In addition, the status display field G22 displays that a team A quick attack state M513 has been detected as the offensive and defensive state.

[0219] When a fast attack state M513 or M523 is detected, an arrow G15 indicating the predicted trajectory of the ball and a ball G16 after movement based on the trajectory are displayed on the field map G10 in Figure 21. Also, an arrow G17 indicating the change in the shooting direction of the shooting camera 141 is displayed. With this configuration, the reason for the operation of the drone 100 is clear to the user even in automatic control mode, providing the user with a sense of security.

[0220] [A-2. Effects of this embodiment] According to this embodiment, it is possible to reduce the labor required for photographing and to photograph an object appropriately according to its condition.

[0221] The present invention is not limited to the above-described embodiment, and it goes without saying that various configurations can be adopted based on the contents of the present specification.

[0222] The series of processes described in connection with the above embodiment may be implemented using software, hardware, or a combination of software and hardware. A computer program for implementing each function of the server 300 according to this embodiment may be created and installed on a PC or the like. A computer-readable recording medium storing such a computer program may also be provided. Examples of the recording medium include a magnetic disk, an optical disk, a magneto-optical disk, and a flash memory. The computer program may also be distributed, for example, via the communication network 400 without using a recording medium.

[0223] The flowcharts used in the above embodiments do not necessarily have to be executed in the order shown in the drawings. Some processing steps may be executed in parallel. In addition, additional processing steps may be employed, and some processing steps may be omitted.

[0224] 1 Aerial photography system 100 Drone (mobile body) 141 Photography camera 200 Controller 220 Input control unit 300 Server 320 Event detection unit 330 Flight mode switching unit 331 Mode switching determination unit 380 Memory unit F Stadium F100 Court F200 Area outside the court M102 Outer edge flight mode M105 Inner court flight mode

Claims

1. A moving object flying over a target area; a camera mounted on the moving body for photographing the target area; an event detection unit that detects an event based on a captured image acquired by the camera or an input from an external system; a storage unit that stores a table that associates the events with photographing conditions including at least one of a target photographing position and a target photographing direction of the moving object that are predetermined for the target area; an imaging condition determination unit that refers to the table and determines the imaging conditions in accordance with the detected event; Equipped with Aerial photography system.

2. Further comprising a flight control unit that causes the moving body to fly, The flight control unit is capable of performing automatic tracking photography in which the target shooting direction and the target shooting position are fixed and the target shooting direction is fixed and the shooting position is changed to follow a specific object, automatic tracking photography in which the target shooting position is fixed and the shooting direction is changed to follow a specific object, or automatic tracking photography in which the shooting position and the shooting direction are changed to follow a specific object, 2. The aerial photography system according to claim 1.

3. When the event detection unit detects a predetermined event, the photographing condition determination unit determines the photographing conditions based on the event, and when the event detection unit does not detect the predetermined event, automatically follows the ball and photographs it.

2. The aerial photography system according to claim 1.

4. In the case of an event in which a plurality of target photographing positions for the event are stored in the table, the photographing condition determination unit selects the target photographing position that is closest to the ball from among the stored target photographing positions.

2. The aerial photography system according to claim 1.

5. the photographing condition determination unit determines the photographing conditions according to the type of the detected event.

2. The aerial photography system according to claim 1.

6. the target shooting direction is achieved by controlling at least one of the nose direction of the moving body and the angle of the camera relative to the moving body; 2. The aerial photography system according to claim 1.

7. the photographing conditions include a target zoom amount of the camera; 2. The aerial photography system according to claim 1.

8. When the photographing condition determination unit receives an input of the photographing conditions from a controller of the moving body or the external system, the photographing condition determination unit determines the photographing conditions based on an operation received via the controller or the external system, even if the event detection unit detects the event.

2. The aerial photography system according to claim 1.

9. further comprising a controller that accepts input of the photographing conditions by a user; the photographing condition determination unit determines the photographing conditions based on an input via the controller when the event detection unit has not detected the event, and determines the photographing conditions based on the event when the event detection unit has detected the event.

2. The aerial photography system according to claim 1.

10. an aerial photography system including a plurality of the moving bodies, the aerial photography system photographing one target area by flying the plurality of moving bodies simultaneously over the target area, the imaging condition determination unit determines different imaging conditions for each of the plurality of moving bodies, 2. The aerial photography system according to claim 1.

11. the photographing condition determination unit sets, for the plurality of moving bodies flying simultaneously, the photographing condition for photographing the same photographing range from the target photographing positions different from each other, or the photographing condition for photographing an area including the same photographing range with different zoom amounts from each other; 11. The aerial imaging system according to claim 10.

12. the photographing condition determination unit determines the photographing conditions in accordance with a prediction result of a ball trajectory predicted by the event detection unit as a detection result of the event.

2. The aerial photography system according to claim 1.

13. when an event indicating that a foul has occurred in a game held in the target area is detected, the photographing condition determination unit determines the photographing condition to set a photographing range around a ball used in the game or a position of a referee of the game.

2. The aerial photography system according to claim 1.

14. an aerial photography system including a plurality of the moving bodies, the aerial photography system photographing one target area by flying the plurality of moving bodies simultaneously over the target area, when an event indicating that the foul has occurred in the game is detected, the photographing condition determination unit determines the photographing conditions in which the plurality of moving bodies photograph the ball or the vicinity of a position of a referee of the game at different target photographing positions, target photographing directions, or zoom amounts; 14. The aerial imaging system according to claim 13.

15. a flight path generation unit that generates a flight path of the moving object; the flight path generation unit automatically generates the flight path to the target photographing position that is determined based on the event detected from the photographed image; 2. The aerial photography system according to claim 1.

16. the flight path generation unit generates the flight path within a court configured within the target area; the flight path generation unit generates the flight path to the target photography position by connecting a plurality of preset photography positions, and changes the photography positions to be connected depending on the detection status of the event.

16. The aerial imaging system according to claim 15.

17. A computer comprising: an event detection step of detecting a predetermined event based on an image captured by a camera capturing an image of a target area or an input from an external system; a photographing condition determination step of determining the photographing conditions of the moving body in accordance with the detected event by referring to a table that associates the event with photographing conditions that are predetermined for the target area and include at least one of a target photographing position and a target photographing direction of the moving body on which the camera is mounted; To execute Aerial photography method.

18. An event detection command for detecting a predetermined event based on an image captured by a camera capturing an image of a target area or an input from an external system; a photography condition determination command for determining photography conditions of the moving body in accordance with the detected event by referring to a table that associates the event with photography conditions including at least one of a target photography position and a target photography direction of the moving body on which the camera is mounted, which are predetermined for the target area; by a computer, Aerial photography program.